hormones, brain and behaviour, a not-so-simple story

There’s a simple story about sex differences in cognition, which traces these back to sex differences in early brain development, which are in turn due to hormone differences. Diagrammatically, it looks something like this:

simpleCordelia Fine’s “Delusions of Gender” (2010) accuses both scientists and popularisers of science with being too ready to believe overly simple, and biologically fixed, accounts of sex differences in cognition.

There is an undeniable sex difference in foetal testosterone in humans at around 6-8 weeks after conception. In Chapter 9 of her book, Fine introduces Simon Baron-Cohen, who seems to claim that this surge in male hormones is the reason why men are more likely to be Autistic, and why no woman had ever won the Fields Medal. So, diagrammatically:

simple_mathsThis account may appear, at first, compelling, perhaps because of its simplicity. But Fine presents us with an antidote for this initial intuition, in the form of the neurodevelopmental story of a the spinal nucleus of the bulbocavernosus (SNB), a subcortical brain area which controls muscles at the base of the penis.

Even here, the route between hormone, brain difference and behaviour is not so simple, as shown by neat experiments with rats by Celia Moore, described by Fine (p.105 in my edition). Moore showed that male rat pups are licked more by their mothers, and that this licking is due to excess testosterone in their urine. Mothers which couldn’t smell, licked male and female pups equally, and female pups injected with testosterone were licked as much as male pups. This licking had an extra developmental effect on the SNB, which could be mimicked by manual brushing of a pup’s perineum. Separate work showed that testosterone doesn’t act directly on the neurons of the SNB, but instead prevents cell death in the SNB by preserving the muscles which it connects to (in males). So, diagrammatically:

snbOne review, summarising what is known about the development of the SNB, writes ‘[There is] a life-long plasticity in even this simple system [and] evidence that adult androgens interact with social experience in order to affect the SNB system’. Not so simple!

What I love about this story is the complexity of developmental causes. Even in the rat, not the human! Even in the subcortex, not the cortex! Even in a brain area which direct controls a penis reflex. Fine’s implicit question for Baron-Cohen seems to be: If evolution creates this level of complexity for something as important for reproductive function, what is likely for the brain areas responsible for something as selectively-irrelevant as winning prizes at Mathematics?

Notice also the variety of interactions, not just the number : hormones -> body, body -> sensation in mother’s brain, brain -> behaviour, mother’s behaviour -> pup’s sensation, sensation -> cell growth. This is a developmental story which happens across hormones, brain, body, behaviour and individuals.

Against this example, sex differences in cognition due to early hormone differences look far from inevitable, and the simple hormone-brain-behaviour looks like a crude summary at best. Whether you take it to mean that sex differences in hormones have multiple routes to generate sex differences in cognition (a ‘small differences add up’ model) or that sex differences in hormones will cancel each other out, may depend on your other assumptions about development. At a minimum, the story of the SNB shows that those assumptions are worth checking.

Previously: gender brain blogging

Paper: Moore, C. L., Dou, H., & Juraska, J. M. (1992). Maternal stimulation affects the number of motor neurons in a sexually dimorphic nucleus of the lumbar spinal cord. Brain research, 572(1), 52-56.

Source for the 2009 claim by Baron-Cohen claim that foetal hormones explain why no woman has won the Fields medal: Autism test ‘could hit maths skills’.

In 2014 Maryam Mirzakhabi won the Fields medal.

Diagrams made with draw.io

A neuroscientist podcaster explains…

There’s a great ongoing podcast series called A Neuroscientist Explains that looks at some of the most important points of contact between neuroscience and the wider world.

It’s a project of The Guardian Science Weekly podcast and is hosted by brain scientist Daniel Glaser who has an interesting profile – having been a cognitive neuroscientist for many years before moving into the world of art and public engagement.

Glaser takes inspiration from culture and current affairs – which often throws up discussion about the mind or brain – and then looks at these ideas in depth, typically with one of the leading researchers in the field.

Recent episodes on empathy and music have been particularly good (although skip the first episode in the series – unusually, there’s a few clangers in it) and they manage to strike a great balance between outlining the fundamentals while debating the latest ideas and findings.

It seems you can’t link solely to the podcast but you can pick them on the page linked below.
 

Link to ‘A Neuroscientist Explains’

Why women don’t report sexual harassment

189px-milgram_experimentJulie A. Woodzicka (Washington and Lee University) and Marianne LaFrance (Yale) report an experiment reminiscent of Milgram’s famous studies of obedience to authority. Reminiscent both because it highlights the gap between how we imagine we’ll respond under pressure and how we actually do respond, and because it’s hard to imagine an ethics review board allowing it.

The study, reported in the Journal of Social Issues in 2001, involved the following (in their own words):

we devised a job interview in which a male interviewer asked female job applicants sexually harassing questions interspersed with more typical questions asked in such contexts.

The three sexually harassing questions were (1) Do you have a boyfriend? (2) Do people find you desirable? and (3) Do you think it is important for women to wear bras to work?

Participants, all women, average age 22, did not know they were in an experiment and were recruited through posters and newspaper adverts for a research assistant position.

The results illuminated what targets of harassment do not do. First, no one refused to answer: Interviewees answered every question irrespective of whether it was harassing or nonharassing. Second, among those asked the harassing questions, few responded with any form of confrontation or repudiation. Nonetheless, the responses revealed a variety of ways that respondents attempted to circumvent the situation posed by harassing questions.

Just as with the Milgram experiment, these results contrast with how participants from a companion study imagined they would respond when the scenario was described to them:

The majority (62%) anticipated that they would either ask the interviewer why he had asked the question or tell him that it was inappropriate. Further, over one quarter of the participants (28%) indicated that they would take more drastic measures by either leaving the interview or rudely confronting the interviewer. Notably, a large number of respondents (68%) indicated that they would refuse to answer at least one of the three harassing questions.

Part of the difference, the researchers argue, is that women imagining the harassing situation over-estimate the anger they will feel. When confronted with actual harassment, fear replaces anger, they claim. Women asked the harassing questions reported significantly higher rates of fear than women asked the merely surprising questions. Coding of facial expressions during the (secretly videoed) interviews revealed that the harassed women also smiled more – fake (non-Duchenne) smiles, presumably aimed at appeasing a harasser that they felt afraid of.

The research report doesn’t indicate what, if any, ethical review process the experiment was subject to.

Obviously it is an important topic, with disturbing and plausible findings. The researchers note that courts have previously interpreted inaction following harassment as indicative of some level of consent. But, despite the real-world relevance, is it a topic that is it important enough to justify employing a man to sexually harass unsuspecting women?

Reference: Woodzicka, J. A., & LaFrance, M. (2001). Real versus imagined gender harassment. Journal of Social Issues, 57(1), 15-30.

Previously: a series of Gender Brain Blogging

Much more previously: an essay I wrote arguing that moral failures are often defined by failures of imagination, not of reason: The Narrative Escape

The Social Priming Studies in “Thinking Fast and Slow” are not very replicable

train_wreck_at_montparnasse_1895In Daniel Kahneman’s “Thinking Fast and Slow” he introduces research on social priming – the idea that subtle cues in the environment may have significant, reliable effects on behaviour. In that book, published in 2011, Kahneman writes “disbelief is not an option” about these results. Since then, the evidence against the reliability of social priming research has been mounting.

In a new analysis, ‘Reconstruction of a Train Wreck: How Priming Research Went off the Rails‘, Ulrich Schimmack, Moritz Heene, and Kamini Kesavan review chapter 4 of Thinking Fast and Slow, picking out the references which provide evidence for social priming and calculating how statistically reliable they:

Their conclusion:

The results are eye-opening and jaw-dropping.  The chapter cites 12 articles and 11 of the 12 articles have an R-Index below 50.  The combined analysis of 31 studies reported in the 12 articles shows 100% significant results with average (median) observed power of 57% and an inflation rate of 43%.  …readers of… “Thinking Fast and Slow” should not consider the presented studies as scientific evidence that subtle cues in their environment can have strong effects on their behavior outside their awareness.

The argument is that the pattern of 100% significant results is near to impossible, even if the effects known were true, given the weak statistical power of the studies to detect true effects.

Remarkably, Kahneman responds in the comments:

What the blog gets absolutely right is that I placed too much faith in underpowered studies. …I have changed my views about the size of behavioral priming effects – they cannot be as large and as robust as my chapter suggested.

The original analysis, and Kahneman’s response are worth reading in full. Together they give a potted history of the replication crisis, and a summary of some of the prime causes (e.g. file draw effects), as well as showing off how mature psychological scientists can make, and respond to critique.

Original analysis: ‘Reconstruction of a Train Wreck: How Priming Research Went off the Rails‘, Ulrich Schimmack, Moritz Heene, and Kamini Kesavan. (Is it a paper? Is it a blogpost? Who knows?!)

Kahneman’s response

Sex differences in cognition are small

Lately I’ve been thinking about sex differences in brain and cognition. There are undeniable differences in the physical size of the brain, and different brain areas, even if there are no ‘female’ and ‘male’ brains categorically. These physical differences do not translate directly into commensurate differences in cognition. Indeed, there is support for a ‘gender similarities hypothesis’ which asserts that on most measures there is no difference between men and women.

Most, but maybe not all. There are a few areas of fundamental cognitive ability where gender differences seem to persist – mental rotation, vocabulary and maybe maths. But these differences are small. To see how small, I put them on the same chart with the physical differences and a few other behavioural differences for perspective.

Standardised mean differences (Cohen’s d effect size) for various gender differences in brain, behaviour and cognition:

gender_effectsReferences and calculations at the end of this post, below the fold. And if you need a primer on what is meant by standardised difference then go here.

Even with these, small, observed differences in cognition, we don’t know what proportion is due to contingent facts, such as the different experience and expectations men and women encounter in their lifetimes, and what proportion is immutable consequence of genetic difference in sex.

One possibility for why there is a mismatch between physical differences in the brain and cognitive differences is the possibility that structural differences between male and female brains may actually serve to support functional similarity, not difference.

For more, so much more, on this, see the special issue of Journal of Neuroscience Research (January/February 2017) on An Issue Whose Time Has Come: Sex/Gender Influences on Nervous System Function.

Includes: Grabowska, A. (2017). Sex on the brain: Are gender‐dependent structural and functional differences associated with behavior?. Journal of Neuroscience Research, 95(1-2), 200-212.

Previously: Gender brain blogging

Continue reading “Sex differences in cognition are small”

The gender similarities hypothesis

cubeThere is a popular notion that men and women are very different in their cognitive abilities. The evidence for this may be weaker than you expect. Janet Hyde advances what she calls the ‘gender similarities hypothesis‘, ‘which holds that males and females are similar on most, but not all, psychological variables’. In a 2016 review she states:

According to meta-analyses, however, among both children and adults, females perform equally to males on mathematics assessments. The gender difference in verbal skills is small and varies depending on the type of skill assessed (e.g., vocabulary, essay writing). The gender difference in 3D mental rotation shows a moderate advantage for males.

So from three celebrated examples of differences in ability only two actually show a moderate gender difference. Other abilities show no or negligible gender differences, Hyde concludes. Gender differences in ability may be overinflated in the popular imagination.

Worth noting is that the name of the game here isn’t to find gender differences in behaviour. That’s too easy. Women wear more make-up for example, men are more likely to wear trousers. The game is to find a measure which reflects some more fundamental aspect of mental capacity. Hence the focus on vocabulary size, mental rotation ability, maths ability and the like. These may be less subject to the vagaries of exactly what is expected of each gender, but that’s a shaky assumption. Indeed, it would be weird if different roles and expectations for men vs women didn’t produce different motivations and opportunities for practice of cognitive abilities such as these.

The real challenge is to find immutable gender differences, or to track differences in how abilities develop under different conditions. Without this evidence, we’re not going to be sure which gender differences are immutable, and which are contingent on the specific psychological history of particular men and particular women living in our particular societies.

One way of addressing this challenge is to look at how gender differences change across different socities, or across time as society changes. A 2014 study, ‘The changing face of cognitive gender differences in Europe‘ did just that, showing that less gender-restricted educational opportunities tended to decrease some gender differences but not others. In other words, increasing equality in educational attainment magnified some differences between the sexes.

You can read my take on this in this piece for The Conversation : Are women and men forever destined to think differently?

The Gender Similarities Hypothesis: Hyde, J. S. (2005). The gender similarities hypothesis. American psychologist, 60(6), 581-592

2016 update: Hyde, J. S. (2016). Sex and cognition: gender and cognitive functions. Current opinion in neurobiology, 38, 53-56.

Previously: Gender brain blogging: Sex differences in brain size, no male and female brain types.

no male and female brain types

What would it mean for there to be a “male brain” or a “female brain”? Human genitals are mostly easy to categorise just by sight as either male or female. It makes sense to talk about there being different male and female types of genitals. What would it mean for the same to be true of brains? Daphna Joel and colleagues, in a 2015 paper Sex beyond the genitalia: The human brain mosaic have a proposal on what needs to hold for us to be able to say there are distinct male and female varieties of brains:

1. particular brain features must be highly dimorphic (i.e., little overlap between the forms of these features in males and females).
and
2. those features which are dimorphic must be consistent for each brain (i.e. a brain has only “male” or only “female” features).

They analyse MRI scans of 1400 human brains and find that these conditions don’t hold. There is extensive overlap, so that categorical brains, defined like this, just don’t exist. They write:

…analyses of internal consistency reveal that brains with features that are consistently at one end of the “maleness-femaleness” continuum are rare. Rather, most brains are comprised of unique “mosaics” of features, some more common in females compared with males, some more common in males compared with females, and some common in both females and males…Our study demonstrates that, although there are sex/gender differences in the brain, human brains do not belong to one of two distinct categories: male brain/female brain.

So the easy gender categorisation we can do on the genitals doesn’t translate to the (usually-unseen) anatomy of the brain. The ‘male/female brain’ doesn’t exist in the same way as the male/female sex organs.

Context for this is that there are differences between the average male and average female brain (for overall size, at least, these differences are large). Although there may not be categorical types, a follow up analysis showed that it is possible to classify the brains used in the Joel paper as belonging to a man or a women at somewhere between 69%-77% accuracy. A related study, on a different data set, claimed 93% classification accuracy.

Paper: Joel, D., Berman, Z., Tavor, I., Wexler, N., Gaber, O., Stein, Y., … & Liem, F. (2015). Sex beyond the genitalia: The human brain mosaic. Proceedings of the National Academy of Sciences, 112(50), 15468-15473.

Responses: Del Giudice, M., Lippa, R. A., Puts, D. A., Bailey, D. H., Bailey, J. M., & Schmitt, D. P. (2016). Joel et al.’s method systematically fails to detect large, consistent sex differences. Proceedings of the National Academy of Sciences, 113(14), E1965-E1965.

Chekroud, A. M., Ward, E. J., Rosenberg, M. D., & Holmes, A. J. (2016). Patterns in the human brain mosaic discriminate males from females. Proceedings of the National Academy of Sciences, 113(14), E1968-E1968.

The responses are linked to in Debra Soh’s LA Times article Are gender feminists and transgender activists undermining science?

Betteridge’s Law

Previously: gender brain blogging

Sex differences in brain size

Next time someone asks you “Are men and women’s brains different?”, you can answer, without hesitation, “Yes”. Not only do they tend to be found in different types of bodies, but they are different sizes. Men’s are typically larger by something like 130 cubic centimeters.

Not only are they actually larger, but they are larger even once you take into account body size (i.e. men’s brains are bigger even when accounting for the fact that heavier and/or taller people will tend to have bigger heads and brains, and than men tend to be heavier and taller than women). And this is despite the fact that there is no difference in size of brain at birth – the sex difference in brain volume development seems to begin around age two. (Side note: no difference in brain volume between male and female cats).

But is this difference in brain volume a lot? There’s substantial variation between individuals, as well as across the individuals of each sex. What does ~130cc mean in the context of this variation? One way of thinking about it is in terms of standardised effect size, which measures the size of a difference between the two population averages in standard units based on the variation within those populations.

Here’s a good example – we all know that men are taller than women. Not all men are taller than all women, but men tend to be taller. With the effect size, we can precisely express this vague idea of ‘tend to be’. The (Cohen’s d) effect size statistic of the height difference between men and women is ~1.72.

What this means is that the distribution of heights in the two populations can be visualised like this:

mf_heightsWith this spread of heights, the average man is taller than 95.7% of women.

Estimates of the effect size of total brain volume vary, but a reasonable value is about ~1.3, which looks like this:

mf_brainsThis means that the average man has a larger brain, by volume, than 90% of the female population.

For reference, psychology experiments typically look at phenomena with effet sizes of the order ~0.4 , which looks like this:

mf_0p4And which means that the average of group A exceeds 65.5% of group B.

In this context, human sexual dimorphism in brain volume is an extremely large effect.

So when they ask “Are men and women’s brains different?”, you can unhesitatingly say, “yes”. And when they ask “And what does that mean for differences in how they think” you can say “Ah, now that’s a different issue”.

Link: meta-analysis of male-female differences in brain structure:

Kristoffer Magnusson’s awesome interactive effect size visualisation

Previously: gendered brain blogging

Edit 8/2/17: Andy Fugard pointed out that there are many different measures of effect size, and I only discuss/use one: the Cohen’s d effect size. I’ve edited the text to make this clearer.

Edit 2 (8/2/17): Kevin Mitchell points out this paper that claims sex differences in brain size are already apparent in neonates

How to overcome bias

How do you persuade somebody of the facts? Asking them to be fair, impartial and unbiased is not enough. To explain why, psychologist Tom Stafford analyses a classic scientific study.

One of the tricks our mind plays is to highlight evidence which confirms what we already believe. If we hear gossip about a rival we tend to think “I knew he was a nasty piece of work”; if we hear the same about our best friend we’re more likely to say “that’s just a rumour”. If you don’t trust the government then a change of policy is evidence of their weakness; if you do trust them the same change of policy can be evidence of their inherent reasonableness.

Once you learn about this mental habit – called confirmation bias – you start seeing it everywhere.

This matters when we want to make better decisions. Confirmation bias is OK as long as we’re right, but all too often we’re wrong, and we only pay attention to the deciding evidence when it’s too late.

How we should to protect our decisions from confirmation bias depends on why, psychologically, confirmation bias happens. There are, broadly, two possible accounts and a classic experiment from researchers at Princeton University pits the two against each other, revealing in the process a method for overcoming bias.

The first theory of confirmation bias is the most common. It’s the one you can detect in expressions like “You just believe what you want to believe”, or “He would say that, wouldn’t he?” or when the someone is accused of seeing things a particular way because of who they are, what their job is or which friends they have. Let’s call this the motivational theory of confirmation bias. It has a clear prescription for correcting the bias: change people’s motivations and they’ll stop being biased.

The alternative theory of confirmation bias is more subtle. The bias doesn’t exist because we only believe what we want to believe, but instead because we fail to ask the correct questions about new information and our own beliefs. This is a less neat theory, because there could be one hundred reasons why we reason incorrectly – everything from limitations of memory to inherent faults of logic. One possibility is that we simply have a blindspot in our imagination for the ways the world could be different from how we first assume it is. Under this account the way to correct confirmation bias is to give people a strategy to adjust their thinking. We assume people are already motivated to find out the truth, they just need a better method. Let’s call this the cognition theory of confirmation bias.

Thirty years ago, Charles Lord and colleagues published a classic experiment which pitted these two methods against each other. Their study used a persuasion experiment which previously had shown a kind of confirmation bias they called ‘biased assimilation’. Here, participants were recruited who had strong pro- or anti-death penalty views and were presented with evidence that seemed to support the continuation or abolition of the death penalty. Obviously, depending on what you already believe, this evidence is either confirmatory or disconfirmatory. Their original finding showed that the nature of the evidence didn’t matter as much as what people started out believing. Confirmatory evidence strengthened people’s views, as you’d expect, but so did disconfirmatory evidence. That’s right, anti-death penalty people became more anti-death penalty when shown pro-death penalty evidence (and vice versa). A clear example of biased reasoning.

For their follow-up study, Lord and colleagues re-ran the biased assimilation experiment, but testing two types of instructions for assimilating evidence about the effectiveness of the death penalty as a deterrent for murder. The motivational instructions told participants to be “as objective and unbiased as possible”, to consider themselves “as a judge or juror asked to weigh all of the evidence in a fair and impartial manner”. The alternative, cognition-focused, instructions were silent on the desired outcome of the participants’ consideration, instead focusing only on the strategy to employ: “Ask yourself at each step whether you would have made the same high or low evaluations had exactly the same study produced results on the other side of the issue.” So, for example, if presented with a piece of research that suggested the death penalty lowered murder rates, the participants were asked to analyse the study’s methodology and imagine the results pointed the opposite way.

They called this the “consider the opposite” strategy, and the results were striking. Instructed to be fair and impartial, participants showed the exact same biases when weighing the evidence as in the original experiment. Pro-death penalty participants thought the evidence supported the death penalty. Anti-death penalty participants thought it supported abolition. Wanting to make unbiased decisions wasn’t enough. The “consider the opposite” participants, on the other hand, completely overcame the biased assimilation effect – they weren’t driven to rate the studies which agreed with their preconceptions as better than the ones that disagreed, and didn’t become more extreme in their views regardless of which evidence they read.

The finding is good news for our faith in human nature. It isn’t that we don’t want to discover the truth, at least in the microcosm of reasoning tested in the experiment. All people needed was a strategy which helped them overcome the natural human short-sightedness to alternatives.

The moral for making better decisions is clear: wanting to be fair and objective alone isn’t enough. What’s needed are practical methods for correcting our limited reasoning – and a major limitation is our imagination for how else things might be. If we’re lucky, someone else will point out these alternatives, but if we’re on our own we can still take advantage of crutches for the mind like the “consider the opposite” strategy.

This is my BBC Future column from last week. You can read the original here. My ebook For argument’s sake: Evidence that reason can change minds is out now.

Can boy monkeys throw?

180px-cebus_albifrons_editAimed throwing is a gendered activity – men are typically better at it than women (by about 1 standard deviation, some studies claim). Obviously this could be due to differential practice, which is in turn due to cultural bias in what men vs women are expected to be a good at and enjoy (some say “not so” to this practice-effect explanation).

Monkeys are interesting because they are close evolutionary relatives, but don’t have human gender expectations. So we note with interest this 2000 study which claims no difference in throwing accuracy between male and female Capuchin monkeys. In fact, the female monkeys were (non-significantly) more accurate than the males (perhaps due to throwing as part of Capuchin female sexual displays?).

Elsewhere, a review of cross-species gender differences in spatial ability finds “most of the hypotheses [that male mammals have better spatial ability than females] are either logically flawed or, as yet, have no substantial support. Few of the data exclusively support or exclude any current hypotheses“.

Chimps are closer relatives to humans than monkeys, but although there is a literature on gendered differences in object use/preference among chimps, I couldn’t immediately find anything on gendered differences in throwing among chimps. Possibly because few scientists want to get near a chimp when it is flinging sh*t around.

Cite: Westergaard, G. C., Liv, C., Haynie, M. K., & Suomi, S. J. (2000). A comparative study of aimed throwing by monkeys and humans. Neuropsychologia, 38(11), 1511-1517.

Previously: gendered brain blogging

Gender brain blogging

s-l300I’ve started teaching a graduate seminar on the cognitive neuroscience of sex-differences. The ambition is to carry out a collective close-reading of Cordelia Fine’s “Delusions of Gender: The Real Science Behind Sex Differences” (US: “How Our Minds, Society, and Neurosexism Create Difference“). Week by week the class is going to extract the arguments and check the references from each chapter of Fine’s book.

I mention this to explain why there is likely to be an increase in the number of gender-themed posts by me to mindhacks.com.

Here’s Fine summarising her argument in the introduction to the 2010 book:

There are sex differences in the brain. There are also large […] sex differences in who does what and who achieves what. It would make sense if these facts were connected in some way, and perhaps they are. But when we follow the trail of contemporary science we discover a surprising number of gaps, assumptions, inconsistencies, poor methodologies and leaps of faith.

This is a book about science works and how is made to work as much as it is a book about gender. It’s the Bad Science of  cognitive neuroscience.  Essential.

The troubled friendship of Tversky and Kahneman

Daniel Kahneman, by Pat Kinsella (detail)
Daniel Kahneman, by Pat Kinsella for the Chronicle Review (detail)

Writer Michael Lewis’s new book, “The Undoing Project: The Friendship That Changed Our Minds”, is about two of the most important figures in modern psychology, Amos Tversky and Daniel Kahneman.

In this extract for the Chronicle of Higher Education, Lewis describes the emotional tension between the pair towards the end of their collaboration. It’s a compelling ‘behind the scenes’ view of the human side to the foundational work of the heuristics and biases programme in psychology, as well as being brilliantly illustrated by Pat Kinsella.

One detail that caught my eye is this response by Amos Tversky to a critique of the work he did with Kahneman. As well as being something I’ve wanted to write myself on occasion, it illustrates the forthrightness which made Tversky a productive and difficult colleague:

the objections you raised against our experimental method are simply unsupported. In essence, you engage in the practice of criticizing a procedural departure without showing how the departure might account for the results obtained. You do not present either contradictory data or a plausible alternative interpretation of our findings. Instead, you express a strong bias against our method of data collection and in favor of yours. This position is certainly understandable, yet it is hardly convincing.

Link: A Bitter Ending: Daniel Kahneman, Amos Tversky, and the limits of collaboration