You can use the word "town" whenever you are talking about a city affectionately, for example in the sense of it as your home. And you use it in phrases like "leaving town," "coming into town," "downtown," regardless of size. "Town" always personalizes the city, emphasizes the idea of human beings living their lives in the city. As to whether you should use it yourself, I am not sure. It is a matter of style. One could certainly use the word "city" in some introductory sentences full of statistics about size, and then shift to using "town" as you "zoom in" to describe the individual lives of people within the city. It would sound completely wrong to say "New York is a town of eight million people."

Town can sometimes have a personal feeling to it, so you can talk about NYC as a town when there is an personal connection behind it. It's like calling your dog a puppy affectionately even though it's fully grown. It would be weird to say "every morning and evening the town is" but "our town's cultural heritage" sounds fine to me. Phrases like "my city", "our city" you can swap it with town.

In preset phrases like "out of town" you say town. "Out of city" sounds...bizarre to me. But if you're just talking about it, call it a city.

In NYC (where I live) we use town to show areas, like midtown uptown or downtown (this train goes uptown, the uptown trains are on the other track, we stayed at a hotel in midtown). There's an old song from the musical On the Town, called NY NY IT's a Wonderful Town with the lyrics "NY NY a wonderful town, the Bronx is up and the Battery's down https://www.youtube.com/watch?v=x7CIgWZTdgw (see at 24 seconds) Using town with NY gives it a "small town feeling" but normally you would use the word city for NY.