Yes! You could also say "lukewarm" which basically means not too hot and not too cold.

Yes, that is a good description. It is probably the best description in this case and is a common thing to say in English.

Yes, people talk about room temperature water frequently. However, not always in a good way. Like "oh gross, this water is room temperature." However, I did recall seeing a water cooler that had both a cold water and room-temperature water spout. It is also commonly used as a serving instruction in the world of food and beverages. "Serve at room temperature" for example. The only time it might be awkward is if the room is dramatically cold or something. Then someone might make a joke like "not THIS room's temperature I hope...hahaha". Another good expression mentioned here is lukewarn, however in my experience that means slightly warm. Either way it implies water that is not hot and not cold. It's like medium and medium well; in both instances you are receiving meat that is pink and not red, in contrast to well-done and rare.

Yes. "Room temperature" is OK, but I might say "unchilled" instead (which is by implication also unheated). Slightly warmer and heated to some degree is "lukewarm", and also "tepid", although the latter has a slightly negative, off-putting connotation.