As I prefaced in my previous comment, I said Prince Charles stated that the Queen did not particularly like her own children. Researching further, it seems the Charles meant that QE2 was 'distant' with her own children, not that she disliked them.
But I'll let his biography speak for itself.
From a biography on Prince Charles
Though parental bonds were formed,
the young Prince spent a good deal of his infant years separated from his mother and father. In the light of the failing health of his grandfather, King George VI, Princess Elizabeth was much occupied with preparing for her own accession to the Throne, and was frequently called upon to perform the stately duties that her ailing father was unable to undertake. Meanwhile the young Prince’s father, the Duke of Edinburgh, was often away at sea, as a naval officer in command of his own ship.
So frequent and prolonged were such parental absences that Prince Charles is said to have failed to recognise his mother upon one occasion of her return. 