Why pick on me?
Oh well, let's have a go!
Suppose you have a suitable polished rectangular metal bar, possibly drifting in free fall in a cabin in Spacelab.
Suppose you strike the bar near its middle with a suitable hammer.
The bar vibrates, giving off a particular note. Let us suppose (since we are doing a a lot of supposing) that the note has a frequency of 256 Hz.
Now what part of the bar was responsible for the note? If you examine the bar in vibration, you typically will find that the two ends move in one direction while the middle moves in the opposite direction and then they reverse their motion 512 times a second.
You cannot really say that any one part of the bar was responsible; it all followed from the shape of the entire bar and the material it is made of. Change the shape and you will change or abolish the note it gives off when struck, or make it give off several notes Change the material, and the note will change, or even vanish. Cotton wool for example will not give you much of a note.
Very well then; in an analogous way, a molecule of a coloured substance will either pass or generate or absorb a photon of a particular colour (frequency), depending on the structure of that molecule. As Mike and Georg variously pointed out, you have to look at the whole molecule; you cannot break the molecule apart and expect to get half of the colour from one part and half from another.
Generally a coloured molecule will contain at least two parts, joined by bonds that act as a springy hinge that permit the molecule to vibrate about that bond, much as our bar vibrated about its middle. The frequency of that vibration is what determines the colour of the molecule.
If I take two atoms of oxygen and attach them by double bonds to each side of one atom of nitrogen, giving me a molecule of NO2, then they can vibrate about the nitrogen atom in a few ways that in combination produce that handsome redbrown colour.
The vibrations of 02 and of N2 do also happen, but their colours are not visible to the human eye, so we see their gases as colourless. In much the same way, if we break that bar of ours down far enough, some of the notes it gives off will be ultrasonic and we might not be able to hear its vibrations at all.
Yet again, an atom of say helium, doesn't have a great deal of freedom to vibrate in any such mode at all; it does not have the necessary parts.
You might well ask then, why, if NO2 gives us that nice redbrown, CO2, which has roughly the same shape, does not?
It is simply that CO2 is not tuned to vibrate at any frequency that we can see, and in fact most common gas molecules are not tuned to visible colours either.
So to get back to your original question, the fact that oxygen and nitrogen do or do not vibrate in a suitable way to produce or pass light that you can see, has nothing to do with the fact that atoms of oxygen and nitrogen can be joined to make a molecule that does in fact pass redbrown light.
I hope that helps, but this is very much a handwaving description of a complex and mathematical set of principles. If part of it seems to make no sense, feel welcome to ask more, but we cannot of course promise to cover the whole matter here.