Messy: The Power of Disorder to Transform our Lives – Tim Harford

Thoughts: I’m typing this up a while after having read the book, so I remember not too much of the experience of reading it. Lots of useful stuff, though; a good foil to Levitin’s The Organized Mind.

(The notes below are not a summary of the book, but rather raw notes - whatever I thought, at the time, might be worth remembering. I read this as an e-book, so page numbers are as they appeared in the app I used, Libby.)

Harford, Tim. 2016. Messy: The Power of Disorder to Transform our Lives. Riverhead Books.

1. Creativity

• 17: simulated annealing: maximum-finding algorithm for exploring fitness landscapes that “starts with an almost random search, willing to try any change, good or bad. Then it slowly becomes fussier and fussier about what changes it will accept, until eventually it has turned into a rigid search for small step-by-step improvements.” - helps prevent optimizing toward a local maximum, rather than the global maximum. Harford draws a comparison with Brian Eno’s creative process, where he injects “judicious [doses] of randomness” throughout the process to improve the chances of landing in an area where the local maximum is the global maximum.
  • “The combination of gradual improvements and random shocks turns out to be a very effective way to approach a host of difficult problems.”
• 20: following a transit strike in London, some commuters didn’t return to their previous paths following the resolution of the strike. “One in twenty of the computers who had switched [to a different route] then stayed with the route that they had used during the strike; presumably they had discovered that it was faster cheaper or preferable in someway to the old routine. We tend to think that commuters have their route to work honed to perfection; evidently not. A substantial minority promptly found an improvement to the journey they have been making for years. All they needed was an unexpected shock to force them to seek out something better.” -j: must do this more frequently with groceries, restaurants, etc.
• 30: “As Brian Eno says, the friend of creative work is alertness, and nothing focuses your attention like stepping onto unfamiliar ground.”
• 33: it can be useful to work on a variety of projects in parallel. “Such a network of parallel projects has four clear benefits, one of them practical and the others more psychological.”
  • “The practical benefit is that the multiple projects cross-fertilize. Knowledge gained in one enterprise provides the key to unlock another.”
  • psychological benefits:
    • first, “a fresh context is exciting…. Having several projects may seem distracting, but instead the variety grabs our attention”
    • 34: “While we’re paying close attention to one project, we may be unconsciously processing another—as with the cliché inspiration striking in the shower.”
    • “Each project in the network of enterprises provides an escape from the others. In truly original work, there will always be impasses and blind alleys. Having another project to turn to can prevent a setback from turning into a crushing experience.”

2. Collaboration

• 41: in social networks, having lots of weak connections is vital to finding new opportunities (e.g. people finding jobs), sparking creativity (e.g. fruitful collaborations).
  • “Recent data driven research… Backs up [the] claim that the vital ties are the weak ones. ¶ This seemingly paradoxical finding is obvious in retrospect. In a clique, everyone knows everyone and all your friends can tell you the same gossip. The more peripheral the contact, the more likely she is to tell you something you didn’t know.”
• 52: book mentioned: The Difference, by “complexity scientist Scott Page”.
  • quoted from Page: “There’s a lot of empirical data to show that diverse cities are more productive, diverse boards of directors make better decisions, the most innovative companies are diverse.”
  • Harford goes on: ” The logic behind these results is that when dealing with a complicated problem, even the smartest person can get stuck. Adding a new perspective or a new set of skills can unstick us, even if the perspective is off-the-wall or the skills are mediocre.”
• 53: “People think harder when they fear their views may be challenged by outsiders…. When experimental subjects are challenged to write an essay, they write better, more logical prose when told their work will be read by someone with different political beliefs rather than someone like-minded. ¶ When deliberating with a group, then, we should be seeking out people who think differently, who have different experiences and training, and who look different. Those people may bring fresh and useful ideas to the table; even if they do not, they’ll bring out the best in us—even is only by making us feel awkward and forcing us to shape up. That messy, challenging process is one we should embrace.”
• 54: found in a study evaluating how people problem solve in groups/teams composed of only friends vs. friends plus one outsider: “Members of diverse teams didn’t feel very sure that they’d gotten the right answer, and they felt socially uncomfortable. The teams made up of four friends had a more pleasant time and they also tend to be confident—wrongly—that they had found the right answer. ¶ When it comes to picking our own dream team, we seem incapable of figuring out what’s good for us. The diverse teams were more effective, but that is not how things seemed to people in those teams: team members doubted their answers, distrusted the process, and felt that the entire interaction was an awkward mess. The homogeneous teams were ineffective and complacent.”
• 57: Results of research into how people trying to establish weak connections behave: People who go to networking events tend to gravitate towards people they already know, and/or have lots of commonalties with. “Of course it’s human nature to spend time with your friends. But what’s striking about this research is that people said they intended to do exactly the opposite. People went to a networking event with the expressed intention of expanding their social networks, and they didn’t even try. Those that did meet new people encountered only friends of friends, perpetuating old cliques”
• 58: Looking at the friend networks of people who went to universities of different sizes: “On the larger campus, students were able to seek out their ideological twins; on the smaller campuses, people made friends with people very different from them. Forced by circumstance to befriend people at least somewhat different from themselves, they did so. And they made the friendships work: friendships at the smaller colleges were actually closer and lasted longer than those at the larger university. Offered a wider choice of friends, students at larger schools chose sameness.”
• 60-61: Harford suggests four lessons that we can take if we want to embrace diversity in our social circles:
  • 60: “The first, and most straightforward, is to recognize this tendency in ourselves to spend time with people who look and sound just like us.”
  • Second: “We should place great value on the people who connect together disparate teams”
  • 61: Third: “Constantly remind yourself of the benefits of tension.”
  • “A final lesson is that we have to believe the ultimate goal of the collaboration is something worth achieving, and worth the mess of dealing with awkward people.”
    • 61-62: suggests emphasizing “goal harmony” over “team harmony”
  • 63: “When you give people an important enough problem to solve together, they can put aside their differences.”

3. Workplaces

• 71-72: people tend to work better in spaces that they have control over.
  • 72: “Apparently trivial freedoms, such as the right to paint your own wall, help people to define personal space, and make people happier and more productive.”
  • 71: Useful concept from the research of Robert Sommer: “hard” vs. “soft” architectural spaces: Hard spaces cannot easily be changed/customized, whereas soft spaces can.
• 79- : Harford examines the example of Building 20 at MIT, looking at why so many innovations/discoveries came out of it.
  • layout was convoluted, meaning people would keep getting lost and running into people they didn’t yet know - serendipetous encounters. These encounters tended to happen in long hallways rather than during short elevator rides, which allowed conversations to expand beyond the “elevator pitch”
  • 80: people working on diverse projects were all sharing the space - “Into the mix were stirred machine shops for the nuclear scientists and for the electronics research lab, the photography lab, a materials lab for the anthropologists, and solar car researchers who would use the building’s corridors as driveways and parking lots. The building even hosted a piano repair shop… Right next to the office of the anti-establishment linguist Noam Chomsky. ¶ This unlikely mess made possible chance interactions among innovative researchers that paid such spectacular dividends.”
  • 81: Building 20 was built in haste during wartime, which meant people didn’t worry about changing the space: “Few modern offices boast the extreme reconfigurability of Building 20: when the atomic clock was being developed by a team lead by Jerrold Zacharias, the group simply removed a couple of floors from the laboratory to accommodate it.”
  • 82: “Building 20’s true advantage wasn’t so much that it was reconfigurable by design, but that the building’s inhabitants felt confident that they had the authority (if only by default) to make changes, even messy changes.”

4. Improvisation

• 109: “So what does it take to successfully improvise? The first element, paradoxically, is practice.”
  • “The second element is a willingness to cope with messy situations.”
  • 110: “The third crucial element is the ability to truly listen”
  • “Perhaps the most important element in successful improvisation is simply this: being willing to take risks and to let go. That is much easier when you have little to lose, but even when there is a lot on the line, improvising can be the best way forward.”

5. Winning

• general thought about this whole chapter: it leans heavily on examples of people who created messy situations and took chances and were successful (e.g. Erwin Rommel, Jeff Bezos), ignoring people who did so and weren’t successful - i.e. there’s lots of survivor bias going on. These may be useful models for explaining people who were successful, but I get the sense that people who try to implement suggestions abot creating messy situations are likely to get burned. i.e. tactics like getting inside an opponent’s OODA loop are best employed as a way to increase the odds of a plan that is already a long-shot
• 131: OODA decision-making process - “Observe-Orient-Decide-Act” - “this OODA loop of information gathering and decision making was crucial in any competitive struggle. If you could make quick decisions, that was good. If you had a strong sense of what was going on around you, that was good, too. But more profoundly, if you could disorient your opponent, forcing him to stop and figure out what was going on, you gained an advantage. And if you could do this relentlessly, your opponent would be almost paralyzed with confusion.”

6. Incentives

• 163: to avoid people gaming a system, i.e. to avoid measures becoming targets, Harford suggests “defining many rules of thumb, and deliberately leaving it ambiguous as to which will be used in any given situation.” - e.g. come up with a bunch of metrics, and then when it comes time to evaluate whether a policy/intervention has been a success, choose one of the metrics at random and evaluate the intervention based on that metric.

7. Automation

• 175: the paradox of automation: “The better the automatic systems, the more out-of-practice human operators will be, and the more unusual will be the situations they face” (especially true of systems that are designed to hand control back to humans when a situation goes beyond what the system was designed for)
• 178: one of “Weiner’s Laws” of aviation and human error: “Digital devices tune out small errors while creating opportunities for large errors.” “We might rephrase it as: ‘Automation will routinely tidy up ordinary messes, but occasionally create an extraordinary mess.’ It’s an insight that applies far beyond aviation.”
  • 188: “In the case of auto pilot and autonomous vehicles, we might add that it’s because digital devices tidily tune out small errors that they create the opportunities for large ones. Deprived of any awkward feedback, any modest challenges that might allow us to maintain our skills, when the crisis arrives we find ourselves lamentably unprepared.”
• 183-184: automation bias: humans tend to unthinkingly accept the recommendations of algorithms
  • j: on the flip side, this is related to the thing Tom Frank observed that it’s easier for people to stick to a plan (e.g. exercise schedule) when it is recommended by an expert. if someone comes up with a plan themselves, it’s easy for them to second-guess that plan, because that person is aware of all the uncertainties/decisions that went into settling on the plan, whereas if a plan is dictated by an expert/algorithm, it is harder to rationalize changing the plan.
• 184-185: Gary Klein: “As people become more dependent on algorithms, their judgement may erode, making them depend even more on the algorithms. That process sets up a vicious cycle. People get passive and less vigilant when algorithms make the decisions.”
• 185: “Decision experts such as Klein complain that many software engineers make the problem worse by deliberately designing systems to supplant human expertise by default; if we wish instead to use them to support human expertise, we need to wrestle with the system. GPS devices, for example, could provide all sorts of decision support, allowing a human driver to explore options, view maps, and alter a route. But all these functions tend to be buried deeper in the app.”
• 191: Study of drone pilots, who had to sit through long stretches of boredom while still being ready to jump in and take control of the piloting of the drone when human input was required: “Many of the more successful pilots adopted an interesting tactic. Rather than attend to their task through sheer willpower, or divide their attention, trying to do both their job and their e-mail at the same time, they distracted themselves in brief bursts. A few minutes with their back to the drone monitors, doing something completely different, would refresh them as they returned to the task.”
  • “Such behaviour suggests that when humans are asked to babysit computers, the computers themselves should be programmed to serve up occasional brief diversions. Even better might be an automated system that demanded more input, more often, from the human—even when that input wasn’t strictly needed. If you occasionally need human skill at short notice to navigate a hugely messy situation, it may make sense to artificially create smaller messages, just to keep people on their toes.”
• 193-194: traffic calming measure, the Squareabout:
  • 194: “It is precisely because the Squareabout feels so hazardous that it is safer. Drivers never quite know what is going on or where the next cyclist is coming from, and as a result they drive slowly and with the constant expectation of trouble.”

8. Resilience

• 204: physicist Cesar Hidalgo has found that “There is a strong correlation between a diversified economy, a complex economy, and a rich economy.”
• 206: two “powerful forces” that stand in the way of the creation of diverse cities:
  • “If people prefer to live near similar people—perhaps people of the same race, class, ethnicity, or income—then even quite mild preferences can lead to marked social segregation.”
  • “Bureaucratic desire for tidy, segregated cities is expressed in zoning and planning laws that are designed to prevent different aspects of city life from getting tangled up with one another.”
• 212: analysis by Steve Levitt of the effectiveness of broken windows policing (i.e. the idea that if a city looks messy/disordered, it is likely to lead to more crime, thus that crime can be prevented by strongly punishing acts like littering/vandalism): “He concluded that four factors seemed to explain the timing, extent, and geographical pattern of the fall in crime: more police; a larger prison population (this may deter crime, and will also prevent crimes because would-be criminals are locked up); the waning of an epidemic of crack use; and the legalization of abortion in the 1970s, which reduced the number of unwanted children.”

9. Life

• 224-225: study of working styles by cognitive scientist David Kirsh - two categories of people: “neats” and “scruffies”
  • “‘neats’ orient themselves with to-do lists and calendars, while ‘scruffies’ orient themselves using physical cues—The report that they were working on is lying on the desk, as is a letter that needs a reply, and receipts that must be submitted for expenses. A tidy desk conveys no information at all, and it must be bolstered with the prompt of a to-do list. Both systems can work, so we should hesitate before judging other people based on their messy desks.”
• 225-226: “filers” vs. “pilers” - “filers” maintain carefully categorized archives of documents, vs “pilers” who just let them pile up
  • 225-226: Steve Whittaker and Julia Hirschberg: “‘We predicted that filers’ attempts to evaluate and categorize incoming documents would produce smaller archives that were accessed frequently.’ But that isn’t what they found. The filers didn’t have lean archives full of useful and oft-accessed documents; They had capacious cabinets full of neatly filed paper that they never used. The filers were filing prematurely. In an effort to keep their desks clear, they would swiftly file documents that turned out to have no long-term value.”
    • j: worth thinking about while building my Obsidian vault
• 228: study by Daniel Kirschenbaum et al.: students split into three groups: control group (given simple simple tips on time management); two groups given instructions to create either daily or monthly plans (these two groups were also given the time management tips)”
  • “The researchers assumed that the planters would be more successful than the control group, and that the daily plans, with the brief, quantifiable goals, would work better than the rather Amorphis monthly plans. ¶ The researchers were in for a surprise. The daily plans were catastrophic. Students using them started by working 20 hours a week but by the end of the course they were down to about 8 hours a week. Having no plan at all was just as bad, although arguably it encouraged more consistent work effort: students began by working 15 hours a week and sagged to 10 hours a week later in the course. But the monthly plans were a tremendous success in motivating students to study—they put in 25 hours a week, and even studied slightly harder at the end of the 10-week course than at the beginning. These are huge effects—the monthly plan motivated about twice as much work as the daily plan. When did the researchers followed up a year later, the trends had continued and we’re reflected in the students’ grades: the students with monthly plans were doing better than ever, the students with no plans were treading water, and the students with daily plans were sliding even further down the scale of academic achievement.”
    • explanation? two theories: 1) daily plans took up too much time 2) students would not complete their daily plans, which led to decreased motivation. “Both of these speculations sounds plausible, but they raise the question of why students weren’t able to follow their own daily plans. ¶ The answer is the daily plans can’t adjust to unexpected events…. With a broad plan, or no plan it is easy to accommodate [obstacles such as catching a cold, having unpredictable obligations come up, etc.]”
  • 229: some people take things so far as to not schedule any appointments whatsoever - “[founder of Netscape Marc Andreessen] decided he would simply stop writing anything in his calendar. If something was important, then it could be done immediately.”
    • Harford suggests that we all could make fewer plans while leaving more flexibility in our own schedules. 230: “A plan that is too specific will soon lie in tatters. Daily plans are tidy, but life is messy.”
• 240: one successful technique for passing turing tests: insulting people. “MGonz ([One algorithm using such a technique]) would never pass a Turing test with an informed judge, but it has drawn unsuspecting humans into abusive dialogues on the Internet that last for over an hour without its ever being suspected of being a chatbot.”
• 242: “There’s a long tradition of using courageous questions to get us out of our tidy conversational habits. One list of questions was made famous by the novelist Marcel Proust, including ‘What is your most treasured possession?’ ‘What is the trait you most deplore in yourself?’ ‘What is your favourite journey?’ and ‘How would you like to die?’ All of these questions beat ‘What do you do for a living?’”
• 246: “Recent research has found a correlation between playing informal games as a child, and being creative as an adult; the opposite was true of the time spent playing formal, organized games…. In an informal game, everyone must be kept happy: if enough players stop wanting to play, the game will end. That implies the need to compromise, to empathize, and to accommodate younger, weaker, and less skillful playmates; no such need arises in formal games, where those who are having a miserable time on the losing team are obliged to keep going until the final whistle blows. As different children arrive and leave, people must switch sides to keep things interesting, evening up the numbers and the skill levels…. No wonder the skills we learn from informal games stand us in better stead for many real-life situation than the skills we learn from formal ones.”