I remember an interview on the radio where the fellow told of how he had spoken to his father about a shop called (say) "Smith General Stores". His father was astonished, because the name of the shop had changed to "Bloggs' Supplies" before the interviewee was 3. He (the interviewee) took this as proof that he had been re-incarnated, because he could read the shop name long before he had been taught to read.

I assumed that he had remembered an image of the sign in his mind and read it once he was able (thus enabling me to retain my doubts about reincarnation!).

So - on the strength of that one anecdote alone - I would say that eidetic memory can probably kick in quite early.