We can use the expression its time + subject + past verb form to refer to the present moment. To understand this let us look at:
“Gosh! Its almost midnight. Its time we went home.“
‘Its time’ with a verb in the to-infinitive form can refer to the speaker and the listener together.
“Come on. It’s time to start packing. We have to leave in two hours.” (or “It’s time we started packing.”)