"In" is the most general, and refers to something that surrounds you (e.g., in Mexico, in Mexico City, in the town square—it isn't the size, it is the fact that it is all around you 360 degrees). "At" is the most specific, and refers to a location with an address (both "at" and "address" begin with "a," if that helps you remember). Example: He lives at 123 Main Street. "By" means "near," so you can say you are by a school (you are not in the building, but you can see the building). "On" means alongside something (on the river, on the border, on the street) or actually on top of it (on the third floor, on the roof).