"I live on Oxford Street" is grammatically correct, because it gives your position as on top of or above a surface, such as the sidewalk or street of Oxford street. "In" is used for giving a position when enclosed or inside of a place, like inside the room, cafe, or house.