OK, so first we need to know how the air got warm in the first place - the answer is that it is generally warmed by being close to the earth - mostly by energy that was absorbed from the sun and then re-radiated as heat.
As it warms, it expands and begin to rise, just like the air in a hot air balloon. The difference is that the hot air balloon takes its heat source with it, but without a balloon, hot air leaves the ground behind.
With the absence of a heat source to keep heating it, air will actually cool down as it expands. This is because it pushes against the air surrounding it, which does work and uses energy. Thus as the air rises, it cools down.
The higher you get, the more the air expands and the more it cools - so by the time you get to the top of a mountain it's pretty cold.
Interestingly if you go really really high - up into the stratosphere - it will begin to warm again as it absorbs UV energy directly from the sun.