The answer lies in the head postion of swimmers as they enter the water is the key to this question. As both a competitive swimmer and a coach, I have many a time watched and even myself dived in only to find my goggles around my neck. Which I trust is where the questioners ended up too!
In the case of elite atheletes and swimmers who have mastered the skill of starting their heads are lowered during the streamlined flight and entry. The arms are tight to the head and the head faces down and slightly backwards. This angle of head on entry means that the top of the goggles are essentially shielded from any direct impact. However in less experienced swimmers the head postion in the dive is too high which not only doesnt allow the hips to rise sufficiently but also leads to a direct impact to the top of the goggles on entry and as a result, said goggles round neck effect.
See the attached picture for clarity.
Matt