Researchers from the Universities of Queensland and Denver have found that newborn babies preferentially look at human faces, but not human body shapes in general. This seems to suggest that face recognition might be innate in some way and might be one aspect of our genetic inheritance which promotes social interaction and allows us to develop subtle social communication skills needed for the complexity of human interaction.
A study published in 2004 suggested that this is more than just a simple preference for any face-like shape, but that newborn babies prefer attractive rather than unattractive faces. It is still unclear why this might happen, although it perhaps hints that attractive faces may seem more attractive because they more closely match a configuration passed down to us via our genes.
The excuse “Sorry honey, I was just looking to see if their face matched my genetic template of innate face shapes” is of course unlikely to get you out of trouble, regardless of your ability to describe the science behind it.