/khamim/. Yes, this is confusing because it's both the singular diminutive of חם (a bit hot, or warm) and the plural for חם (hot). So, since מים is plural and תה is singular:
hot tea - תה חם
warm tea - תה חמים
hot water - מים חמים
warm water - מים חמימים
But in many contexts חם is used when English uses "warm". You use the diminutive חמים only when you want to stress that it's not hot.
In the tropical heat of Brazilian summers, if a drink is served in a temperature that is not "estupidamente gelada" (slang for "ice cold"), we say it's "quente" ("hot") - for example, when a beer eventually reaches room temperature, we say it's "hot".
Is that how you described the use of חמים?
Saw now that my comment above was cut too early, left you in suspense...
There are חם, חמים and פושר, and in many contexts two or all three of them can be used, depending on what you want to stress, and whether you like or dislike the temperature.
If a beer is not sufficiently cold... you probably wouldn't use חמים, because that has a positive sound to it. You'd probably just say לא קרה or פושרת, though I guess if you're really in a complaint mood you'd go as far as חמה.
I think חמים is most often used for things that are a bit above room temperature, and you describe it as good. The classic use is for tea or soup, if it's good to drink right now - not too hot, but still above room temperature. If you're talking about water that is nice to bathe in, you'd usually use חם (the plural in case of water, confusingly חמים, see my previous comment), but then suppose they are less than ideally warm, but you want to describe them as good enough anyway, you might say המים חמימים.