You can either heat up or warm it up just don't use the microwave! Heating it up may imply that you want it hot as warming may mean you just want it warm depending on what you are talking about. I would want my soup hot but my milk just warm.

You can say either. "Warm" means "a little hot." "Warm up" suggests that you are heating it a little, "heat up" suggests that you are heating it a lot. Also, to "warm up" is to prepare for a strenuous athletic effort by doing easy exercises first. Also, before a show, someone may "warm up" an audience by telling them jokes to get them to a relaxed mood.