It seems that we cannot see, smell, touch or find any evidence for the mysterious yet 'theoretically necessary' dark matter. In othere spheres of science we would be applying Occam's Razor and saying that the simplest explanation is that the theory is wrong. What stops this principle applying here?

You raise an important question, as well as some more debatable points. Dark matter is not "theoretically necessary" to all theories; it just happens to be one of the more obvious approaches to explaining currently puzzling ("troublesome") observations that clash with the simplest theories that we have at present for explaining the nature of our observable universe. Some alternatives theories need not invoke any un-observable matter at all.

Secondly, there unquestionably is evidence for dark matter; that is why it has been theorised. It was not just some cosmologist musing: "Today looks like a nice day to set the cat among the pigeons by hypothesising that somewhere out there, there is a lot of a new kind of matter that we cannot see or detect. Yes, I think I'll do it today!"

As with any good science aimed at explaining the behaviour of the universe, the dark matter idea began with something more like: "That's, funny... These measurements do not match the predictions of the standard theories..."Now, there could be all sorts of reasons for that sort of thing. Our theories could be grossly wrong. Our measurements could be wrong. Our universe could be changing. There is no reason to go galloping off madly in all directions. But there is excellent reason for some really heavy, concentrated, creative, critical thought.One such class of lines of thought favours ideas such as "dark matter". There is nothing particularly sinister or naive about it.

The next question is what Occam's razor has to do with it.  To begin with, I strongly recommend that you read some rather good, plain, sensible discussions of "simplicity" and "Occam's Razor" that are freely available on the web. Some are available on Wikipedia. Some more are available at:

http://plato.stanford.edu/contents.html

They are generally good, simple, clear, constructive, and make good reading. Do yourself a favour and go through them carefully before you make any assumptions about the role of either Occam's Razor or simplicity, elegance or parsimony in science. These are crucially important concepts and well worth the trouble of getting right.Without going into a full discussion before there is any sign of interest in this blog, I simply remark that Occam's Razor is in no way in science or logic in general, a form of proof, a necessary principle, nor even unambiguous or invariably realistic or practical. It is merely a useful rule of thumb.

Note that, no matter how irreverent this may sound, nothing of the kind is intended. I regard Occam's Razor with great respect and as enormously useful in practice. One does its observation no service however by attempting to apply it inappropriately.

I hope to hear more from you, and possibly from some other blog members once you have clarified these ideas to your own satisfaction.It is an interesting field as well as important.

Go well,

Jon

GR, I must apologise for perpetrating some careless wording in a confusing matter.

When I replied in part: "Secondly, there unquestionably is evidence for dark matter; that is why it has been theorised" I should have made it clearer that this was not meant to imply that the evidence was specifically for dark matter, but for some factor causing deviations from the observations to be expected in a Newtonian/Einsteinian universe. Such evidence as we had was by its very nature evidence for any theory in that class of theories that were concistent with the observations. What it was not, was evidence for dark matter as opposed to rival theories, such as forces of types as yet unknown, or faulty observations, or parallel universes, or extrapolations of the behaviour of familiar forces, or any of the rival ideas, pedestrian or exotic, that have been proposed. For that kind of discrimination, what we need is more evidence, either of the same nature, or of a radically different type.

Does that help clarify the confusion that my wording caused?

You say: "The last sentence of the quote however seems to make my point for me. I submit that rather than there being evidence of dark matter itself, we observe cosmological phenomena that necessitate the addition of new components to make the existing theory work."

Yes, if I understand you correctly, then that is pretty well what I had intended. Bear in mind though, that in trying to aim for the "Occamest" of theories, obsessive elimination of material elements is not in itself a virtue. Sometimes we might very well multiply entities if it enables us to produce simpler theories, more robust in the face of falsification. This is true in many branches of science, and by no means only in cosmology or even astronomy. A large number of our by now most pedestrian subatomic particles and epidemiological and physiological effects were discovered by assuming entities, without which we would need more complex or less rational theories. How do we decide where to accept the line of compromise? Where the evidence leaves us. What if that situation is unsatisfactory? Hard lines! We simply must think of finding more evidence to distinguish between the alternative hypotheses. Research and all that...

You say: "Then, when we cannot (independently of the original observations - a necessary part of the scientific method) find any evidence for this wierd dark matter instead of saying 'maybe we were wrong, let's explore whether our theory might be actually wrong', we persist in the fantasy."

Yes, that seems to be compatible with what I said in an earlier paragraph. The only thing is that I am uncomfortable with the implication of anyone favouring the "fantasy" in any special way. Certainly some people do back the dark matter horse (possibly to beyond rational, let alone cogent, limits, which certainly would be naughty of them, but would not matter much as long as they neither suppressed criticism nor contrary evidence presented by dissenting parties) which is their right of course. All that they are committed to do as scientists is refrain from expecting rival parties to take their opinion or word for anything instead of checking the facts or implications independently if they so choose. (They might not so choose, of course. Not all work is worth repeating if it is convincing enough.)

You say: "Finally, I accept that I am using a populist version of Occam's (or Ockham's) Razor. How else might we treat a theory that has so far failed to produce any independently verifiable evidence for it's veracity?"

Certainly, but that diagnosis arose frim my insufficiently clear statement of what I meant by evidence for dark matter, when I should have said something more like "dark matter and rival, rational alternatives".

Sorry about that!

Cheers,

Jon