The "now" belongs to the "It's time", i.e. what he's saying is that "It's now time ....." Or "now is the time" to do something. This sentence can be re-written as "It's now the time I should go home" or "It's time (right now) that I should have gone home". It is confusing but it is English.