Advances in artificial intelligence: deep learning

If you want to keep up with advances in artificial intelligence, the New York Times has an essential article on a recent step forward called deep learning.

There is a rule of thumb for following how AI is progressing: keep track of what Geoffrey Hinton is doing.

Much of the current science of artificial neural networks and machine learning stems from his work or work he has done with collaborators.

The New York Times piece riffs on the fact that Hinton and his team just won a competition to design software to help find molecules that are most likely to be good candidates for new drugs.

Hinton’s team entered late, their software didn’t include a big detailed database of prior knowledge, and they easily won by applying deep learning methods.

To understand the advance you need to know a little about how modern AI works.

Most uses abstract statistical representations. For example, a face recognition system will not use human-familiar concepts like ‘mouth’, ‘nose’ and ‘eyes’ but statistical properties derived from the image that may bear no relation to how we talk about faces.

The innovation of deep learning is that it not only arranges these properties into hierarchies – with properties and sub-properties – but it works out how many levels of hierarchy best fit the data.

If you’re a machine learning aficionado Hinton described how they won the competition in a recent interview but he also puts all his scientific papers online if you want the bare metal of the science.

Either way, while the NYT piece doesn’t go into how the new approach works, it nicely captures it’s implications for how AI is being applied.

And as many net applications now rely on communication with the cloud – think Siri or Google Maps – advances in artificial intelligence very quickly have an impact on our day-to-day tools.

Link to NYT on deep learning AI (via @hpashler)

Where is your mind?

My BBC Future column from a few days ago. The original is here. I’m donating the fee from this article to Wikipedia. Read the column and it should be obvious why. Perhaps you should too:


We like to think our intelligence is self-made; it happens inside our heads, the product of our inner thoughts alone. But the rise of Google, Wikipedia and other online tools has made many people question the impact of these technologies on our brains. Is typing in the search term, “Who has played James Bond in the movies?” the same as knowing that the answer is Sean Connery, George Lazenby, Roger Moore, Timothy Dalton, Pierce Brosnan and Daniel Craig (… plus David Niven in Casino Royale)? Can we say we know the answer to this question when what we actually know is how to rapidly access the information?

I’ve written before about whether or not the internet is rewiring our brains, but here the question is about how we seek to define intelligence itself. And the answer appears to be an intriguing one. Because when you look at the evidence from psychological studies, it suggests that much of our intelligence comes from how we coordinate ourselves with other people and our environment.

An influential theory among psychologists is that we’re cognitive misers. This is the idea that we are reluctant to do mental work unless we have to, we try to avoid thinking things though fully when a short cut is available. If you’ve ever voted for the political candidate with the most honest smile, or chosen a restaurant based on how many people are already sitting in there, then you’ve been a cognitive miser. The theory explains why we’d much rather type a zipcode into a sat-nav device or Google Maps than memorise and recall the location of a venue – it’s so much easier to do so.

Research shows that people don’t tend to rely on their memories for things they can easily access. Things like the world in front of our eyes, for example, can be changed quite radically without people noticing. Experiments have shown that buildings can somehow disappear from pictures we’re looking at, or the people we’re talking to can be switched with someone else, and often we won’t notice – a phenomenon called “change blindness”. This isn’t as an example of human stupidity – far from it, in fact – this is an example of mental efficiency. The mind relies on the world as a better record than memory, and usually that’s a good assumption.

As a result, philosophers have suggested that the mind is designed to spread itself out over the environment. So much so that, they suggest, the thinking is really happening in the environment as much as it is happening in our brains. The philosopher Andy Clark called humans “natural born cyborgs“, beings with minds that naturally incorporate new tools, ideas and abilities. From Clark’s perspective, the route to a solution is not the issue – having the right tools really does mean you know the answers, just as much as already knowing the answer.

Society wins

A memory study by Daniel Wegner of Harvard University provides a neat example of this effect. Couples were asked to come into the lab to take a memorisation test. Half the couples were kept together, and half were reassigned to pair up with someone they didn’t know. Both groups then studied a list of words in silence, and were then tested individually. The pairs that were made up of a couple in a relationship could remember more items, both overall and as individuals.

What happened, according to Wegner, was that the couples in a relationship had a good understanding of their partners. Because of this they would tacitly divide up the work between them, so that, say, one partner would remember words to do with technology, assuming the other would remember the words to do with sports. In this way, each partner could concentrate on their strengths, and so individually they outperformed people in couples where no mental division of labour was possible. Just as you rely on a search engine for answers, so you can rely on people you deal with regularly to think about certain things, developing a shared system for committing items to memory and bringing them out again, what Wegner called “transactive memory”.

Having minds that work this way is one of the great strengths of the human species. Rather than being forced to rely on our own resources for everything, we can share our knowledge and so pool our understanding. Technology keeps track of things for individuals so we don’t have to, while large systems of knowledge serve the needs of society as a whole. I don’t know how a computer works, or how to grow broccoli, but that knowledge is out there and I get to benefit. And the internet provides even more potential to share this knowledge. Wikipedia is one of the best examples – an evolving store of the world’s knowledge for which everyone can benefit from. I use Wikipedia every day, aware of all the caveats of doing so, because it supports me in all the thinking I do for things like this column.

So as well as having a physical environment – like the rooms or buildings we live or work in – we also have a mental environment. Which means that when I ask you where your mind is, you shouldn’t point toward the centre of your forehead. As research on areas like transactive memory shows, our minds are made up just as much by the people and tools around us as they are by the brain cells inside our skull.

ENDNOTE: Wikipedia is an unparalleled democratisation of knowledge, a
wonderful sharing of human intelligence that’s free to anyone to view. I’m
donating the fee for this article to help support Wikipedia’s work. If you feel you can help out please follow this link:

The relative consuming disease

The Global Mail has an amazing story about how the last treks to find cases of kuru – a cannabalism-related brain disease – have been completed.

Kuru was passed on by eating the brains of dead relatives – a long finished tradition of the Fore people in Papua New Guinea – and it infected new people through contact with prions.

Prions are misfolded proteins that cause other proteins to take on the infectious misfolding. In the case of kuru it lead to shaking, paralysis, outbursts of laughing and a host of other neurological symptoms as the brain slowly degenerated.

No-one knew prions existed or could exist before kuru. But as the article makes clear, this ‘obscure disease’ of a remote tribe revolutionised our understanding of proteins and how infections could take place.

But the story is how it was discovered is more than just lab tests and the article is a brilliant retelling of the research.

Michael Alpers has been working on the research project since the 60s and recalls some of the episodes:

After each death, he says, “I would go and talk to the family again, and say, ‘Okay?’. They had participated in cutting up bodies in the past — so that was not an unusual activity for them. We had to clear a few people — particularly the women who were wailing. But some of the women stayed. The ones involved put on masks to protect the tissue and I had gloves.

“The father, or a close relative, would hold the head, and I would take the top of the skull off with a bone handsaw. It would take maybe 20 minutes… like cutting an avocado. I would go to particular parts of the brain… take out small cubes. My assistant would hold out the bottle that was relevant, take the lid off, and I’d pop it in.

“Then I’d take the whole brain out and put it in a bucket full of formalin and cotton wool so it wouldn’t be deformed, and put the lid on. All our samples would go into an insulated box. Then I put the skull cap back on, and sewed up. Then we said goodbye… gave everyone a hug, and took off. I did this five times. It was enough.”

It’s a wonderfully written, informative piece. A long and compelling read.

Link to article ‘The Last Laughing Death’ (via @mslopatto)

Sex taboos: a brief and incomplete tour

Cultures around the world have restrictions or prohibitions on when sex is allowed which turn out to be quite amazing in their diversity.

This is a fascinating section on the wide world of sex taboos from the Encyclopedia of Sex and Gender:

In some societies, sexual activity is prohibited during certain times of day. The Cuna of Panama approve of sexual relations only at night in accordance with the laws of God. The Semang of Malaysia believe that sex during the day will cause thunderstorms and deadly lightening, leading to drowning of not only the offending couple but also of other innocent people. And the West African Bambara believe that a couple who engage in sex during the day will have an albino child.

Sometimes, sex is prohibited in certain places. The Mende of West Africa forbid sexual intercourse in the bush, while the Semang condemn sex with camp boundaries for fear that the supernatural will become angry. Among the Bambara, engaging in sexual relations out of doors will lead to the failure of crops.

Sex taboos can also apply to certain activities. Often, sex prohibitions are associated with war or economic pursuit. The Ganda of Uganda forbid sexual intercourse the night before battle if the fighting is likely to be protracted. The Lepcha prohibit sex for three months after a bear trap has been set. If the taboo is broken, no animals will be caught. The Cuna of Panama outlaw sexual intercourse during a turtle hunt, the Yapese of Oceania prohibit sex during a fishing excursion, and among the Ganda of Uganda sex is forbidden while the wood for making canoes is being processed.

Ganda women may not engage in sexual intercourse while they are mourning the dead and Kwoma men are prohibited from engaging in sexual activity after a cult ceremony has been held. The Jivaro of Ecuador refrain from engaging in sex after someone has died, after planting narcotics, when preparing a feast or after an enemy has been killed.

Of course, while these may seem exotic, our own social practices regarding sex probably seem equally as unusual to outsiders.

Imagine Jivaro anthropologists, watching Nicki Minaj videos, slapping themselves, shouting “you won’t believe what the people of North America have to do to have sex!”


Link to section from the Encyclopedia of Sex and Gender.

The road to ‘war on terror’ torture

An obscure paper called The Spokesman Review has an excellent article charting the role of psychologists in developing America’s ‘war on terror’ enhanced interrogation programme – widely condemned as torture.

The piece is fascinating because it outlines the competing tensions between those who championed the controversial physical interrogation techniques – created by reverse engineering the SERE resistance training – and those who preferred the rapport building methods.

It turns out that the division fell along inter-agency lines. The CIA used the harsh approach, the FBI relationship-based interrogation.

As is now well-known, the ‘enhanced interrogation techniques’ were developed by two formed Air Force psychologists James Mitchell and Bruce Jessen.

The article finishes with a curious snippet of information “Jessen remains [in Spokane] and was recently made the bishop of his ward in the Church of Jesus Christ of Latter-day Saints”.

That, my friend, is a novel in the making.

Link to Spokesman Review on ‘war on terror’ torture.

Letter from the mental states of America

Alistair Cooke presented the longest running radio show in history. The BBC’s Letter from America was a weekly report, where Cooke reflected on life and news in the United States. It ran for just shy of 58 years.

Despite the massive ‘psychologisation’ of society during the years Cooke was broadcasting, from 1946 to 2004 no less, Cooke rarely addressed matters of the mind and brain directly.

However, he did occasionally touch on these issues and the shows are well worth listening to.

As the BBC has just put almost the complete Letter from America archives online so I’ve collected some of the highlights.

Link: George Gallup (1901-1984) – 9 November 1984
Link: TV Violence – 13 April 1986
Link: Narcotics, interdiction and Colombian drug lords – 08 September 1989
Link: American public schools – 03 January 1992
Link: Aphasia and studying the human brain – 15 October 1993
Link: Timothy Leary (1920 -1996) – 7 June 1996
Link: New York: How are you Doing? – 3 May 2002

I’ve linked to the transcripts, but listen to the linked audio if you have the chance. Cooke had a distinctive voice and a calm style that underscored his often insightful commentary.

Link to Letter from America archives.