Oliver Sacks has done a wonderful TED talk on hallucinations that has just been released online. He particularly focuses on the hallucinations of Charles Bonnet syndrome where damage or decay of the retina can cause strikingly complex hallucinations of people and animals that seems to be a natural part of the visual scene.
Interestingly, the people affected by the condition are usually well aware that they are hallucinating and remain lucid throughout.
The talk is wonderful and Sacks is engaging as ever, but some of his neuroscience explanation seems a little dodgy.
He discusses the well-known role of an area in the temporal lobes called the fusiform gyrus in face recognition and relates disturbance in this area to face hallucinations:
There’s an area in the anterior part of [the fusiform gyrus] where teeth and eyes are represented and that part of the gyrus is activated when people get the deformed hallucinations [of people with big teeth and eyes].
There is another part of the brain that is especially activated when one sees cartoons. It is activated when one recognises cartoons, when one draws cartoons and when one hallucinates them…
There are other parts of the brain that are involved in the recognition and hallucination of buildings and landscapes.
Actually, all of this seems quite dodgy. I couldn’t find any evidence that part of the fusiform gyrus is specialised for teeth and eyes.
I found one study which linked the viewing of moving mouths or pair of eyes to activation on the superior temporal gyrus, but this is the other side of the temporal lobe. Also, he seems to be suggesting that specific face parts are mapped to specific areas of the fusiform gyrus, again, which I could find no evidence for.
I suspect the bit about specific parts of the brain for buildings, landscapes and cartoons comes from a misunderstanding of neuropsychology experiments as these sorts of pictures are also often used in experiments on face recognition.
One of the big debates in face perception research is whether the fusiform gyrus is dedicated to face recognition or whether it is specialised for any sort of expertise needed for fine grained visual distinction – for example, recognising car types, or birds and so on.
Hence, experiments often will test people on face recognition, but then also on building or drawings so the researchers can find out whether the problem is specific to faces or just a general visual recognition problem. For example, this exact procedure was used in this 2005 study on four people with prosopagnosia, a selective impairment in face recognition.
Apart from maybe a few minor hallucinations from Sacks himself, the talk is excellent and comes highly recommended.
Link to Oliver Sacks TED talk on hallucinations.
Mind Hacks 
He might also be thinking of the parahippocampal place area, which seems to be involved of the encoding of scene-like stimuli – landscapes, rooms etc.
There seems to be some evidence from macaques (“A face feature space in the macaque temporal lobe”, Freiwald, Tsao, & Livingstone, 2009, doi:10.1038/nn.2363) that some cells in the region comparable to human FFA that are at least somewhat feature-selective (e.g. some respond more when irises and/or hair are present than when they are not). But that’s brand new and in macaques, so it won’t be what he’s referring to, and it’s quite a leap to get from there to an explanation for hallucinations in humans!