Facial clensers and the soaps and gels we regularly wash our body's with are full of detergents, little molecules that play havock with the oils (lipids) that our body makes.
Oils are made from lipids which have a unique property...they have a water loving end (hydrophillic) and a water hating end (hydrophobic). When a detergent comes into contact with a mass of lipids, it disrupts the bonds that are formed between them. Lipids don't like being in this state for too long, its not very energetically favourable so they form small balls of fally molecules called mycelles that can be easily washed away.
Experiment: Take a frying-pan which has been used to cook a fatty food. Add water and the oil will form a layer on the surface. Then add some washing up liquid and watch the effect.
Solution: The oil will move rapidly away from the detergent showing bond disruption
This is to our advantage as in this state they can easily be washed away which prevents a build up of oils on the skin that can cause problems later on.
However, this removal of oils doesn't do the skin very much good as a small layer of oil is useful to keep our skin hydrated so it is recommended that you moisturise to keep that locked into the skin.
Pores are potential trouble spots (no pun intended!). They are small indentations in your skin where you release sweat from. Because they are hard to reach, they usually suffer a build up of foreign material which can become infected forming spots. Special pore clensers use microbeads to reach the pores and clean them out using detergents in the same way as above.
To alleviate your concern, your body is a very hostile environment to the average bacteria. From the very moment it lands on your skin it is constantly under a threat of death. These bacteria and virus' are made from a genetic material called RNA whilst we use DNA. A sneaky trick we employ is to produce large amounts of the enzyme RNAse, an enzyme that breaks down RNA, and cover our skin in it. RNAse breaks down any RNA it finds on our skin, killing bacteria and virus' alike. Whilst washing does remove some RNAse from your skin, it is quickly built back up again, this reminds me of the problems I have in the lab where I work:
"No matter how many times you wash your hands and wear surgical gloves, if the slightst touch is given to an experiment involved in bacterial or viral genetics with bare skin, a years worth of work can be lost in a single afternoon"
I hope this helps!