Robin Hanson reports that success in marriage is quite uncorrelated with the match between your personality traits and your partner’s. Your traits matter (it pays to be happy, for example) and so do your partner’s, but the combination makes no difference. In other words, being a happy person (or an extrovert, or a stickler for detail) affects the quality of your marriage in exactly the same way whether you marry Ruth Bader Ginsberg or Lady Gaga. (This applies specifically to personality traits, not to religion, politics, wealth, intelligence, etc.)
Edited to add: The original version of this post misstated the result; I’ve changed a few words in the preceding paragraph so it’s accurate now.
From this, Robin concludes:
If you want a happy relationship, be a happy person and pick a happy partner; no need to worry about how well you match personality-wise.
NO!!!! That’s not the right conclusion at all, and it’s worth understanding why not. Suppose we lived in a world where personality matches had a huge effect on the success of marriages. In that world, why would two people with clashing personalities ever choose to marry? Presumably because there’s some special value in the match — like, say, an extraordinary mutual attraction — that overrides the personality clash.
So a survey of married couples — which is exactly the sort of evidence Robin is reporting on — is not at all a random sample of couples. Instead, it consists, for the most part, of couples with matched personalities on the one hand, and couples with mismatched personalities who are exceptionally well suited to each other for some other reason on the other hand. It’s not too surprising to find similar success rates in those two classes of couples. The third class — the couples with mismatched personalities and no redeeming match characteristics — never gets married and therefore never gets surveyed.
Conclusion: The results Robin quotes are perfectly consistent with a world where personality matching doesn’t matter — but also perfectly consistent with a world where it matters very much.
Now a more sophisticated analysis would account for the costs of search in the mating market, and the fact that people sometimes settle for imperfect mates because time is running out. My colleagues who know the search literature better than I do tell me that, depending on your assumptions about the distribution of personality traits, you might or might not be able to draw any inferences from the results Robin is quoting, but that in any event the inferences will be far less strong than you’d expect from a naive reading of the data. You’d probably want to start by thinking about a Roy model.
Moral: Never try to interpret data without thinking about how those data were selected. What’s your favorite example of a case where this moral was disastrously ignored?
A somewhat analogous situation:
Lawyers often refer to drafting manuals when preparing documents (contracts, wills, articles of incorporation, etc.) These manuals contain language that has been tested in court. Clearly, a thoughtful lawyer would want to rely on precisely this language, right?
Wrong, says legal language maven Bryan Garner. By selecting language that has been tested in courts, the compilers of these manuals systematically pick examples of CRAPPY drafting – drafting that was so unclear that parties had to ask a judge to tell them what it meant. While there are times when parties intentionally include ambiguous language in a document, generally the goal of legal drafting is to make matters sufficiently clear that no party will see any advantage to seeking a second opinion from a judge. Yet, because clear language is less likely to provoke litigation, it tends to be excluded from drafting manuals.
Perhaps this dynamic contributed to Fred Rodell’s insightful observation, “There are two things wrong with almost all legal writing. One is its style. The other is its content.”
I am surprised he didn’t survey both married couples and previously-married couples who are now divorced. This seems to me a simple way of determining the personality-match differences between successful relationships and divorces. Divorced couples were either personality matches or not matches but having some overriding good quality, at one time, but their matches failed over time for some reason. Maybe I’m missing something?
Jennifer: But even if you included both married and divorced couples in your survey, you’d still omit couples who chose not to marry in the first place, so you still wouldn’t have a random sample.
nobody.really: I love the lawyer example.
An example from Dr Phil where he explains to a couple that open marriages don’t work because in a professional setting he often sees couples in an open marriage with marriage troubles.
I rolled my eyes when I heard that.
How about:
“Atheist regimes have killed far more people than religious ones”?
Ignore the fact that population growth has been exponential, meaning that the supply of victims was far more limited historically, or that modern day tyrants also have modern day weaponry making killing 10’s of thousands of people easier than killing one with a knife.
Is the contention seriously that if only Mao had a deep personal connection to he bible or Quran, the number of people killed would have been smaller?
Does religious belief temper a tytanical maniac to stop at 5 million deaths before things get out of hand?
If the experimenters were able to control for all other relevant variables, and still establish 0 correlation, then Robin’s conclusion would be valid. You only need to pay attention to the data selection method if you don’t have the ability to account for all of the relevant variables.
Jonathan: Before you can control for variables you have to observe them, and I think it’s pretty clear that romantic chemistry is unlikely to show up in an Excel spreadsheet.
How about this classic example:
The minimum wage law is effective because McDonalds workers make more today than they did before the law was enacted. The evidence and the data are crystal clear no matter which sample of Mcdonalds workers you take…in real terms, anyone who works at McDonalds today makes more than he/she did before the minimum wage law.
“Two gas stations on opposite corners of the street charging EXACTLY the same price for gas! These companies are colluding and the government needs to step in!”
Paraphrasing some lady who was running for state office in New York a few years back.
In my Risk Management class we were discussing OSHA one day and the professor showed a graph–proudly provided by the National Safety Council–which showed the steady decline in workplace fatalities from the day OSHA was passed until the present.
I then e-mailed this link to her:
http://stossel.blogs.foxbusiness.com/2010/05/13/more-job-killing-regulation/
This shows the Safety Council’s graph, and compares it to a graph of the same data which begins about 40 years before the passing of OSHA. The second graph has an arrow indicating the year in which OSHA was passed, which is necessary because you could never have picked it out yourself if you didn’t know.
Not so much an actual study, but the example of this that sticks in my head is from Nassim Taleb and I have found it very effective when discussing such results with people.
It goes something like this. We all think dolphins are caring creatures because we hear about the stories where the a dolphin pushes a person stranded in the ocean to shore. We don’t hear about those that get pushed the other way.
I believe he used that to illustrate the point that you should read “secret to my success” books from successful people with a grain of salt because thousands of other people likely follow the same “5 rules”, but never get anywhere because they haven’t had the chance event that is truly responsible for success.
A bit like the con where you send to 24000 people a message saying “tomorrow the stock price will go up” and to 24000 people “tomorrow the price goes down”. The next day you split the correct group in half, and send the same messages to 12000. After a week you have group of 375 people convinced you bet the right way every day, and are ready to trust you with their cash.
Proof – fish oils make you smarter… (not).
An example of a very bad “trial” on whether fish oils can improve exam results. From 3000 students initially taking fish oil tablets about 800 complied. Of these about 600 were “matched” to non takers, and the exam results of the fish oil takers was higher. Demonstrating that students who are compliant and adherent to school regimes did better in exams! Full details at “bad science”
http://www.badscience.net/2008/09/dave-ford-from-durham-council-plays-at-being-a-scientist-again/#more-807
>Never try to interpret data without thinking about how those data were selected. What’s your favorite example of a case where this moral was disastrously ignored?
You’re more likely to survive a heart attack if you’re in a shopping mall than if you’re in a hospital.
I find this phenomenon especially intriguing when it promotes objectively false/deceptive conclusions.
1. One way people become successful is by taking risks. Even crazy risks. Lotteries are a lousy bet. Pursuing a career as a professional athlete/entertainer is probably an even lousier bet. Yet if you only interview the winners, you’ll come away with morals such as “You have to believe in your dream, and don’t let anyone talk you out of it,” etc., etc.
I suspect that the expected earnings for people who abandoned their dreams and instead became accountants is higher than the earnings of people who stuck to their dreams. But if you don’t interview a random sample of people, you’ll never get their stories, or the stories of the people who tried, but failed, to become movie stars.
2. Occasionally people organize investment games in which participants are given an imaginary $100 to invest for a period of a year (or whatever). The winner is assumed to have some kind of insights about investing. Now, there are a variety of strategies for maximizing expected returns from the stock market. But if your goal is instead to win this game, then you don’t want to have maximum expected returns; you want to have maximum probability that your returns exceed everyone else’s. In short, you need a high risk/high reward strategy. A strategy of investing in lottery tickets, for example, would almost certainly reduce your chance of having returns in the top 50% of participants, but might increase your chance of having returns in the top 1%.
Bottom line: The person who wins such a game may well have some insights about investing. But almost certainly the strategy used to win the game is not a strategy you’d want to use for investing.
3. What are the consequences of, say, sexism on a woman’s prospects of success in corporations? I’m under the impression that sexism imposes real impediments. Yet I also suspect that the woman who is most likely to reach the highest echelons is the woman who is NOT burdened with the idea that sexism will impose a barrier. That is, I suspect that accurate information would tend to reduce a person’s chance to achieve greatness, while false (unduly rosy) beliefs are more likely to produce extraordinary results.
4. Finally, how much lying should we expect a politician to engage in?
Imagine that the public is swayed to vote for the politician that promises the most (“offers the highest bid”). Imagine that each politician is equally capable of delivering on his promises. Imagine each politician make promises in good faith. But also imagine that politicians differ in the accuracy of their predictions, making bids that equal their abilities plus or minus a random error term. In this scenario, the politician that makes the grandest promises is almost certainly not able to deliver on those promises. The more politicians (and the larger the error term), the more likely this outcome becomes. Even if everyone acts in good faith, we would expect the public to experience buyer’s remorse.
Now inject a modicum of strategic behavior. Whichever politician might appear to be losing would seem to have a strong incentive to take ever-more-risky moves to win, including inflating the promises. If this strategy worked, then I’d expect to see all politicians making pretty extravagant promises by election day.
Ultimately, one of those politicians would win. This model of political behavior, if accurate, would suggest that virtually all successful politicians have lied to win election – and have found lying to be a successful strategy! The successful politicians might have an inkling that the unsuccessful politicians were also lying – but also might not. Thus the successful politicians might experience a kind of selection bias, concluding that lying was a uniquely beneficial strategy for winning, rather than a strategy of winners and losers alike.
Just a thought for the day!
Nobody.really: Great comment!
Nobody.really: I second that.
Third!
Steve, I disagree with your conclusion. Let’s choose a simple model for marriage: 2 people get married if their compatibility (c) exceeds some threshold T. C is equal to the sum of some other variables, which may include personality match (x), mutual attraction (y), and so on. Even if we select only data for which c > T (i.e. married couples), we should still see a positive correlation between c and x (assuming normal distributions, for example)
There are many similar situations where I imagine you would not complain. Suppose that doctors found 0 correlation between the amount of Vitamin D a pregnant mother consumed and the health of her infant child. Wouldn’t you consider that to be evidence that Vitamin D is not an important determinant of the health of an infant, despite the objection (analogous to yours above) that we are only selecting infants that were healthy enough to survive birth?
Jonathan: A better analogy would be if mothers only chose to take Vitamin D when they knew there was something wrong with their fetus. (Something, that is, that mothers are aware of but that doesn’t show up in the data.) Now we observe that children born after taking Vitamin D treatments are no healthier than others. Should we conclude that Vitamin D is ineffective?
I agree with you that the conclusion we should draw is model-dependent and, in particular, depends on assumptions about the distributions, and I agree with the conclusion you drew in your particular example. But other examples are possible.
I’m not sure I agree with what you say is “a better analogy” – wouldn’t that case be analogous to a world where people only had personality matches when they were not otherwise compatible? In either case, I suggest we should assume that all determinants (Vitamin D + other health factors; personality match + attraction) are independent. Just as there are some couples with both good personality matches and strong attraction, there are some mothers who both have no health problems and take Vitamin D.