Dissolved air, just like a carbonated drink but with much less gas. There are loads of answers on this site about how the gases come out of solution, but in this case it's probably the rise in temperature that starts off the formation of bubbles.

The glass has microscopic cracks that contain tiny amounts of air. The water poured into the glass contains dissolved air under pressure because it was under pressure in the pipes. At normal atmospheric pressure it is slightly supersaturated with that air. (Sometimes more than slightly; it might actually go cloudy with microscopic air bubbles, though they clear soon after running into the glass.)

Even at ordinary, slight, supersaturation, the excess air gradually comes out of solution, but as a rule low pressure air does so only where there is already an air bubble or air surface for it to escape into. Those microscopic bubbles in the microscopic cracks in the surface in the glass are perfect. As the air escapes into them, they swell and become visible bubbles sticking to the cracks until they become too large and float up, or get dislodged.

Even barely saturated air may come out of solution in the same way, depending on the air pressure and changes in temperature that increase the saturation in the neighbourhood of the bubbles.