There is a combination of reasons. If you could get hold of fresh commercial icing sugar, then you probably would find that the difference had more or less disappeared. The thing is that commercial icing sugar, caster sugar and granulated sugar really are simply crystalline sucrose in different degrees of fineness, and while fresh they taste the same except for slight, subtle, and possibly subjective minor differences resulting from their different behaviour in dissolving. They still can acquire differences in flavour as they grow stale however. The thing is that as a grain of sugar remains in contact with the oxygen of the air, and sometimes with various ambient aromas that cling to it, the flavour of the outside layer changes. This is partly because of smells that it has picked up, and partly because of oxidised compounds accumulating on the surface. The greater the surface exposed to the air, the larger the quantity of the altered sugar flavour will be present.

There is however another difference which probably plays a more important role. Caster sugar and granulated sugar do not tend to cake very rapidly unless there is a fairly high humidity. At a moderate humidity, they will start to cake eventually, but, their grains being large, the first traces of caking are relatively fragile, and the slightest vibration of the grains make them fall apart. Icing sugar however, has grains so small that their tendency to fall apart is very weak, and the powder cakes easily if it is not very fresh. Since commercial icing sugar has a rapid turnover, this is not much of a problem, but in normal domestic circumstances a packet of icing sugar is likely to last for months, or even years after being opened for the first time, far longer than is usually the case for granulated sugar.

This is something of an embarrassment for the sugar producers, who do not want to get a bad name as a result of the unpredictable behaviour of a domestic housewives. Accordingly, they add anti-caking agents to their icing sugar. Because the problem is less serious in caster sugar and granulated sugar, manufacturers do not generally add anti-caking agents to them.

Ideally one wants an anticaking agent to be tasteless, cheap, and harmless. It generally takes the form of an extremely fine, easily flowing, soft, insoluble, white powder. Most of those attributes are fairly easy to attain, but true tastelessness probably is impossible in any usable powder. (Possibly Teflon would do, but I have my doubts, and Teflon microparticles for human consumption probably would be a hard sell.)Typical compromise anticaking agents are calcium phosphate and plant starches, typically cornflour. As you observed, they work pretty well, but are not completely tasteless.