The ear drum is a sensitive tissue (or it wouldn't work). What you're experiencing is a difference in pressure between the air on the outside and the air on the inside of the ear drum - this causes the drum to bow outwards. The sensitivity of the apparatus means that a very small change in pressure is apparent.

Thanks to the cunning design of the middle ear, pressure is usually equalised naturally, and quickly, by swallowing or yawning. This allows air to travel up or down the eustachian tube, which is connected to the nasopharynx. If the eustachian tube is blocked for some reason e.g. a cold or flu, then the pressure can cause considerable pain until it gets righted.

As petethebloke mentions, pressure difference bows the eardrum outwards, stretching it a little.  This extra tension causes it to have a different vibrational response to sounds.  When my ears get this way, low frequency sounds don't couple as well into the inner ear, and higher frequencies are enhanced, so everything sounds a bit tinny and shrill until I swallow and neutralize the pressure difference.