It's a matter of definition of words.  'Milli' is the prefix for one-thousandth.  By definition, a 'milli-second' is one-thousandth of a second.  Similarly, by definition there are 60 sexagesimal-seconds in one second (sexagesimal meaning sixtieth). 

The real question is why do we still use an ancient Babylonian system of 60 seconds in one minute, 60 minutes in one hour and 24 hours in a day.  A year (one orbit of the Earth around the Sun) is 364.25 days, but there is no such natural reason for hours or minutes.

The reason there are 60 seconds in a minute is because the Babylonians did not have any form of decimalisation in their numbering system so they divided the minute in to 60 units as 60 is the only tow digit number that divides buy 1,2,3,4 and 5 without leaving a remainder.  Although the Babylonian did have sexagesimal division of the second it required great skill to measure time accurately to such precision at the time.The measurement of time by fraction of the second did not really come into common use until much later after the adoption of decimalised thinking. So rather than divide the second in to a different unit of time as had been done before time was just measured in seconds with allowances for decimal places. And therefore time is measured imperial on the marco scale while it is measured metrically at the micro scale (with the divisions of time smaller than a second following the standing prefix naming system).