Maybe it's because we are very good at distinguishing a perfect circle from an ellipse. Maybe we have that ability just for this aim - to know when someone is looking at us!
Usually we see circular objects, like the human iris, all kinds of plates, coins and so on, as ellipses because we look at them at an angle. Only when we take a closer look at them we stand in a position or hold them in a position to look at them straightly and see a circle. When this happens not by our will but accidentally, we feel like the object is "looking" at us, too. Only ball-shaped objects always show us a circle, and I think that's why they fascinate us.