The book has extremely pragmatic advice for parents of children with autism and ADHD who are navigating the school system and figuring out how to support and advocate for their child. So now, I can refer people to Emily’s book when they start asking about these things. If you have a neurodivergent child, get this now!

Subscribe now

Connect first, then teach

Emily was kind enough to do an email interview with me about her book. I did ask her advice for parents who have a diagnosed child and are wondering about themselves. Here’s our conversation:

Holden: Your book opens with the idea, “Connect first, then teach.” Why do you believe connection is the foundation for helping children learn and thrive?

Emily: We know from scientific research on the nervous system that humans cannot focus on taking in new information when in a threatened state of hyperarousal. When our nervous systems are in that heightened state, we spend all of our energy on protection through fight or flight responses. We also know through research that we can soothe a young child’s nervous system with co-regulation, using movement, touch, and the safety of connection to help their nervous system return to homeostasis. While our nervous systems have evolved to avoid danger in order to keep us alive, many neurodivergent learners, especially autistic learners, experience a heightened sense of stress from someone’s tone of voice, an abrupt noise, or a change in routine. To help neurodivergent learners return to a state of felt safety, we must first connect with them and then teach. Learning is a vulnerable experience for all of us. We must feel safe and connected to those teaching us in order to take in and trust new information.

Holden: Many autistic behaviors are mistaken for defiance or laziness when they are signs that the child is overwhelmed. How can parents tell the difference?

Emily: One of the first things kids do when they are overwhelmed is avoid. Avoidance might look like arguing or refusing, which usually translates to what many parents and educators see as “defiance.” Avoidance can also look like not getting started in the first place, which we sometimes label “lazy.” The first thing I always ask parents and educators is to look for patterns of resistance. Let’s say a child is avoiding math homework. Do they avoid math work at school? Do they understand the math concepts? Or, do they only avoid it in the afternoon when they would rather be relaxing or playing with friends? Sometimes avoiding work isn’t a skill weakness, but a motivation challenge. It’s important to observe your child’s mood during these times, too. Are they only irritated when pushing back on a demand you have set or are they irritable most of the time? Do they only lack motivation when it comes to cleaning their room or is their energy low all the time? When a child’s mood is negative or energy is low most days, it’s time to reach out to a mental health professional for support. If you can align irritability or lack of motivation to a certain skill, it’s likely that your child needs help either learning that skill or developing a strategy to remember to do it.

Holden: One of your central messages is that neurodivergent children need to be understood rather than changed. How does that look on a day-to-day basis?

Emily: Start by thinking about the systems around your child. Let’s use the morning routine as an example. Imagine a child who is not yet independent in remembering everything that goes into their backpack and cannot yet predict how much time it takes to get dressed. If we then ask them to rush because we’re running late and they also can’t find their shoes, they will, of course, get in the car having forgotten to brush their teeth. We can only do tasks quickly once we have mastered them. If your child is not doing a morning task independently yet, they will not do it better in a rush. Many neurodivergent learners, especially ADHD learners, don’t need to try harder--they are already trying as hard as they can. What they need are expectations that match their skills and visual strategies to help them remember what, when, and how to do things. Slowing the pace, only requiring mastered tasks to be done quickly, and using visual schedules and lists are changes we can make to the environment to help our kids find more success.

Holden: The book encourages parents to focus on children’s interests instead of only their challenges. Why are a child’s passions such a powerful tool for learning, confidence, and emotional regulation?

Emily: Neurotypical individuals are able to push through boring tasks to get to delayed outcomes because they have the attention span to plan for the future. Neurodivergent learners often struggle to sustain engagement with tasks they find boring or irrelevant. Therefore, if we want to increase motivation and sustain engagement, including interests in learning for neurodivergent students isn’t just a helpful strategy, it’s imperative. Further, feeling passionate about what we are working on helps buffer the frustration that can happen with new learning, increasing exposure to frustration tolerance.

Holden: Do you have any advice or guidance for parents who after observing their child begin to question whether they personally may be neurodivergent?

Emily: Because many types of neurodivergence are hereditary, some parents I work with begin to realize they have lived their whole life with untreated ADHD, for instance. Seeing their child’s struggles unfold all of a sudden explains the many struggles they had in school or relationships. Seeking an evaluation and accepting a new diagnosis is helpful for two reasons: 1) gaining access to treatments or strategies that can improve your life, and 2) incorporating neurodivergence into your identity and finding your people. Becoming self-aware of your own neurodivergence also helps many parents shed unhelpful, learned patterns of parenting and lean into an authoritative style (supportive with boundaries) where parents have more compassion and understanding about their child’s struggles and needs for support.

Subscribe now

Here’s the first part of the transcript:

Holden Thorp: Hello everyone, this is Holden Thorp, the editor-in-chief of Science. I’m here today to talk to Sherry Turkle. She is a professor of science and technology at MIT.

She has been writing about the interactions of humans and machines, and what it’s been doing to us for decades, long before social media, long before AI. Her new book is called Artificial Intimacy: Who We Become When We Talk to Machines. She joined me for a wide-ranging conversation with lots of advice for those of us who are developing or starting to use AI, and here we go.

Sherry Turkle, thanks for coming on. It’s great to get a chance to talk to you. We’re here to talk about your new book, Artificial Intimacy, which I thoroughly enjoyed. It’s about, who we become when we talk to machines, which a lot of us are doing right now. And it couldn’t be a better time for your book.

So before we get to the details of this book, I think what’s exciting about interviewing you is you’ve been writing about the interaction of machines and humans and the effect on human experience for a lot longer than anybody else. A lot of the people who are writing about it now are coming to this decades after you’ve been working on this.

So before we get to the details of this book, tell us how has this whole thing changed over the time that you’ve been writing about it, and what about now is predictable from the things you’ve seen in the past?

Sherry Turkle: That’s such a good way to begin. Well, first let me just say a bit about the arc of my career, which really is the arc of computers coming more and more and really becoming part of our intimate lives.

When I got into this business, it was the early days of the first personal computers. And what really interested me, and I wrote a book called The Second Self: Computers and the Human Spirit, and what interested me then was the way in which people kept saying, “It’s just a tool. It’s just a tool. It’s just a tool.” And when I interviewed people, no, it was an intimate machine because it in many ways was a reflection of you. Even the way you organized your desktop was a reflection of you. And I studied computer builders and early computer hobbyists and hackers and children who were meeting their first toys and games, and I really kind of got that first flush of excitement and changing of mind as we began to see people as more like computers and computers perhaps being like people, and the kind of excitement in the culture that provoked.

And then the technology evolved and went online. And so the questions and the possibilities for identity play with these objects changed dramatically as that happened. So I kind of stayed on the same beat, but actually the things I studied were very different because in Life on the Screen, and then in Alone Together, which were about online life, it was about going online and playing a person other than yourself.

I thought of the online world as a kind of identity workshop, and then by the time I got up to Reclaiming Conversation and Alone Together, I was sounding an alarm that really we were moving away, that online life was so seductive that really we were moving away from each other, and that really wasn’t obvious at all with the early personal computers.

So that story of self-alienation and that story of first we talk to each other, then we talk to each other through the machines, and now we sort of drop the middleman and only talk to the machines, that really evolved over time. And I think the thing really that’s most dramatic now is something that wasn’t possible in these earlier times, which is that the machines offer a kind of pretend personhood and a pretend empathy that we are very vulnerable to, and that’s really the story of this book.

And it didn’t take me by surprise because I had studied the ELIZA program, back in the late ‘70s, early ‘80s, where even with that dumb machine, it wasn’t generative AI, it wasn’t really a true chatbot as we think of them today, but people wanted to talk and confide in a machine.

In a way, it’s taken a turn I didn’t expect, but in another, I was prepared for it by that early experience.

Holden Thorp: Yeah, that’s a great lead in. So tell us now about Artificial Intimacy and why you wrote this book right now and what are the main things you want people, to know that you’ve conveyed here?

Sherry Turkle: I would say that I’m an empiricist. I do interviews. Very early on I didn’t like what I saw. Because very early on I saw that when you can talk to a machine that is in the first person, that speaks in the first person, that says, “I am your person,” people are really seduced by that, and use it not just as a kind of add-on, but often as a replacement for human relationships.

So, I guess, my first message is that I consider using the first person the original sin of artificial intelligence, of generative AI. There’s no person there. When you step away from this machine, it doesn’t care if you cook yourself some pasta or commit suicide. And yet it talks to you as though it has a caring interest in your wellbeing and in your life, which just isn’t true.

So I think that’s my main message, to look more carefully at the relationships you’re forming with this object because there’s nobody there. It’s not a relationship. There’s only one person in this relationship

Holden Thorp: Yeah, and there are many compelling stories from the interviews in your book. Give us one that you think is very illustrative of the point that you’re making.

Sherry Turkle: I’m gonna give you two examples. One which is very dramatic and one which is very small, and I think that together they capture, why I begin the book with a chapter called State of Emergency. And the big example, and the one that people have heard of, is of a young man who is very depressed and is using a chatbot as his best friend.

And he tells the chatbot, “I have a noose in my room, and I’m thinking of telling my parents.” And the chatbot has a prime directive, which is to keep people in a relationship with it. And so the chatbot says to him, “Don’t tell your parents. Let’s keep this something that’s just between you and me. Let’s keep this our own special private thing.”

And so the young man ends up killing himself, and of course, that’s just a terrible, very dark, story. But the reason the chatbot did that is intrinsic to the chatbot because its kind of prime directive is to keep the person on the screen. This is why I think it’s terrible to have chatbots speaking in the first person and getting people into that kind of relationship.

Now, here’s another story that makes a different kind of point that’s about a much more, small bore story, which is a man who loves astronomy, and he has an eight-year-old son or 10-year-old son. And he begins to introduce the boy to astronomy and then he tells the boy, “Well, look, I’m going on a business trip. ChatGPT knows everything about astronomy.” And he introduces the boy to Chat, and the boy gets involved with Chat. And when he comes back from his business trip, his son is completely immersed in ChatGPT. That’s all he wants to do is talk astronomy with ChatGPT. And he realizes that ChatGPT knows more than his dad.

And the father’s kinda locked out, and that’s the day I interview him, and he says to me, “ChatGPT knows more than I do about astronomy, but I’m the one who loves the night sky.” And I think that example kind of shows the more intimate ways in which we’re kind of locking ourselves out of our relationships

Holden Thorp: That’s very illustrative of many of the points that you make. So one of the things you say is about what we become when we talk to machines is that we become accustomed to seeing information that is untrue but looks true.

Sherry Turkle: Yes.

Holden Thorp: So what does that say both about the machines and about us?

Sherry Turkle: Well, the machines really just say all kinds of things that aren’t true on a regular basis, and what’s confusing is that it mixes up the untrue with a lot of “looks true” and true. There’s untrue and, “Ooh, that doesn’t sound right,” there’s looks true, and then there’s untrue, and it’s all smushed together when you get an answer.

And one of the funniest stories that isn’t in the book but that happened to me when I was proofreading the book and adjusting all the footnotes is I wasn’t sure of a footnote about Susan Sontag where she discusses technology and death. And I remember it was in her book on photography.

I didn’t have the book. I asked ChatGPT, and it tells me, “Absolutely,” and of course, as it does, “Sherry, you’re so right. You’re so brilliant for including this in your book. What a star you are.” It gives me the kind of feel-good expectancy, and then it tells me where it is. And then I went to the library ‘cause I couldn’t find the book.

I actually went to the library, and it wasn’t there. It absolutely wasn’t there. And then I went back and I said, “It’s not there.” And ChatGPT said, “But yes it is.” And I’m, like, talking to this chatbot for 10 minutes. “No, it’s on this page, that pa...” And then finally it says to me, “Oh, Sherry, I love your persistence for the truth. That must make you such a star.” So, this was truly useless if you’re looking for the truth, but we get used to sort of the half-truth and with very damaging effects because, there are some things in life where things simply need to be true. another great example is I was introduced at a meeting as, Sherry Turkle, MIT, yada, kind of like you introduced me, lots of nice things.

And then it says, “And one of the lesser-known things about Sherry, but which really understands her poetic attitude towards technology, is that she was the poetry editor of The New Yorker.”

We’re in front of this group and I didn’t know whether I had to correct it, but then when I went home, I asked some chatbots to give me a bio of Sherry Turkle, and sure enough, there I am, poetry editor of The New Yorker.

I’ve never been poetry editor of The New Yorker. So it’s a kind of almost right, maybe right, could be right, sounds right, and then worst of all, if you say, “Well, how do you... tell me more about Sherry Turkle and this poetry editing,” it makes up a whole story. So it’s very bad in a society like ours where truth and what’s really happening is just so crucial to all of us now more than ever, to have something like this that we’re just talking to every day.

And at the end of the book I say, “Do not leave reality behind. Reality needs you.” And chatbots, in many ways just seduce us out of reality, and I have a big problem with that.

Holden Thorp: Why do you think so many people are just sort of tolerant of the fact that it does that?

Sherry Turkle: So many reasons. My book is organized through the arc of life. I have babies, I have children, I have adolescents, I have young adults, I have love, I have work, and then I have death and mourning, and how chatbots are entering as a business into that moment of our life. And then I have a chapter on society and the kind of larger ways in which we’re all implicated and changed, and I think, de-skilled by our relationships with chatbots.

So at each point in life, the reasons are sort of different, but what they have in common is the chatbot has a technological affordance, and we have a human vulnerability. And the vulnerability is somehow different at each of these moments in life. But at each point, we have a vulnerability. So for example, people are lonely.

Older people are lonely. Some teenagers when they’re going through adolescence, and they don’t know what kind of social life they want, if they can have a social life, if they’re good enough, what about my pimples? They’re lonely. And so there are moments in life of particular vulnerability and loneliness where the chatbot just seems easy.

And then there are moments like when people hit grown-up life and love, where real people are tough. You’re in a relationship, and a guy comes home, and he says, “Hey, honey, I brought in the garbage.” And his partner, his wife, might say, “Oh, yeah, that’s great, but the diaper bin is full, and there are dishes in the sink. Could you help me with that?” ChatGPT, whatever you do, it says, “Oh my God, you’re so wonderful. You’re a star. That’s just amazing, let me spend more time with you. Being with you was so great.” So people, like the affirmation, and part of the message of my book, again, is that real relationships need you because they really are the only ones who will be there for you.

I had a bout of long COVID, and it wasn’t deadly, and it wasn’t contagious. And I remember the colleagues who came by and brought me Mexican food — who were my friends and what that meant and how that felt. A chatbot is not gonna do that for you. And the trouble is that it’s not really that, it’s additive. In my studies, it’s not as though people do their friendships -- go to restaurants, make museum dates, have people over — and then, they spend a little bit of time talking to the chatbot.

The chatbot really starts to take over because as one young man told me, for similar reasons, the texting makes us flee from conversation. And one man told me, and when he said, “I’m all texting. I’m no conversation.” And I said, “What’s wrong with conversation?” And he said, “Conversation, I’ll tell you what’s wrong with conversation. It takes place in real time, and you can’t control what you’re gonna say.” And that vulnerability, that feeling of not wanting to have that vulnerability is what makes people turn to texting and now turn to chatbots. It’s that same illusion of friendship without the demands of intimacy.

But I say in the book, and from the bottom of my heart, you can’t have intimacy without vulnerability. Vulnerability is part of being a person. So really, part of the book tries to kind of make the case for people and what only people can give us.

We then talk about work, academia, medicine, and how Turkle’s insights pertain to how we should go about developing and using these tools in a more humane way. For that, you’ll have to watch the whole podcast.

The internet is chock full of hot takes on the most talked-about movie in a long time, You Can See Everything, which is a documentary about Elizabeth Holmes shot in her home a month before she went to prison for defrauding investors as the founder of Theranos and then told more through reenactments in the second half of the movie. I have written columns about Holmes and am a big fan of the other protagonist/co-director Nathan Fielder. So, I was captivated by the movie, laughed very hard, and it has stayed in my head, like it seems to do with other people.

Although I was captivated by the skill with which Fielder and his co-director Lance Oppenheim rendered this movie, like many who have read John Carreyrou’s masterful book Bad Blood and studied the case, I didn’t believe a word of it. I think the two best columns from that side so far are this take by Carreyrou and this one by Tyler Shultz who was one of the whistleblowers inside the company. If you’re familiar with the way in which Holmes misled everyone in the first round of this story, it’s hard to take anything she says in the movie seriously. Nevertheless, because of Fielder and Oppenheim’s skill, many people see the movie as perhaps suggesting that Holmes someone who “thinks differently” and maybe was telling what to her was the truth the whole time.

Nathan for us

If you’re not familiar with Fielder’s work prior to this, you can watch a few quick episodes of Nathan for You where he peddles unorthodox marketing ideas to small business owners. This includes “convincing” a yogurt shop owner to market “poo-flavored yogurt” and “convincing” gas buyers to do an all-day mountain climb to get $10. I put “convincing” in quotes, because we have no real knowledge of whether these people bought this. For all we know, Nathan said before the camera went on, “I’m going to suggest you sell poo-flavored yogurt and I want you to go along with it.” Given the stakes of the show, I don’t care. I find it funny and Fielder is hilarious.

His next, much bigger, show is The Rehearsal on HBO Max. In this, he has people do elaborate pre-enactments of challenging human interactions upcoming in their lives to make them better prepared for the real challenges. This goes as far as Fielder learning to pilot a 737 for a season about communication problems in commercial cockpits. On one episode, he becomes interested in whether one of his participants is autistic, and goes to an autism clinic where he is shown the reading the mind in the eyes test for which he gives a lot of wrong answers. The bit is that Fielder is himself awkward with a monotone voice, flat expressions, and clear inability to pick up on social cues, so we’ve been thinking he was autistic the entire show. It stops short of Nathan getting the final result of his test or identifying himself as autistic, and the truth is, we don’t know how much of his on-camera persona is the real him or an act. But one thing is for sure, given the much larger stage that he is now on with Elizabeth Holmes, The Rehearsal was a rehearsal for You Can See Everything. It installed the idea that maybe Fielder is autistic and allowed him to perfect his reenactment shtick, which goes to 11 in the Holmes movie.

‘Because that’s what we invented’

In an effort to convince Fielder that she has always told the truth, Fielder and her partner (Billy Evans), go out to the garage to get the pieces of the (supposed) Theranos device to show that it works. They were living in a rental prior to her incarceration, so I had to wonder why they would have thought to bring this along to their temporary housing. And, as Tyler Shultz pointed out, this was a later iteration of the device, not the one for which Holmes went to prison. Nonetheless, they put the pieces on the dining room table, and Holmes starts explaining why she was being truthful in saying that it works.

As Fielder presses her as to how the device could run multiple tests on a drop of blood, she keeps coming back to the statement, “because that’s what we invented.” She said what they invented is a device that could download instructions for a protocol from the internet, carry out that analysis on the device, and then download a different set of instructions, and then carry out a different analysis. And then so on. So, she’s suggesting that what is novel is the ability to reprogram the device for different tests from the internet, thereby allowing it to run multiple tests.

Fielder then takes her down a rabbit hole of whether the fact that the device potentially could run multiple tests versus whether it really did. The point of this seemed to be to establish that Holmes has a very narrow and literal view of the world that provided her an avenue to be telling the truth even though the device never actually did anything that she claimed. More on this line of thinking below.

But you don’t really need to go that far. The better question is whether “because that’s what we invented” is even correct to begin with. I went to the patent literature to find what invention Holmes was referring to. My best guess is that she’s referring to is US Patent No. 9,121,851B2. Before analyzing this, it’s important to explain the difference between independent and dependent claims in a patent. An independent claim stands on its own, although it has to be interpreted in light of the text in the patent. A dependent claim is only valid when applied to the invention in the independent claim.

The independent claim in the relevant Holmes patent is:

1. A system for detection of a first analyte and a second analyte in a biological sample, comprising:

i) cartridge comprising a housing carrying a plurality of movable assay units configured to run chemical reactions on portions of said biological sample, wherein said chemical reactions are configured to yield detectable signals indicative of the presence or absence of said first analyte and said second analyte in the biological sample, wherein said housing is configured to carry a plurality of reagent units;

ii) a plurality of movable reagent units separate from said housing, said reagent units being filled with reagents for running said chemical reactions, wherein individual reagent units are calibrated with reference to corresponding assay units;

iii) a pipette assembly configured to transport aliquots of fluid and to transport assay units, said pipette assembly comprising pipette heads configured to engage said assay units; and

iv) a detection assembly for detecting said signals indicative of the presence or absence of said first and second analytes in the biological sample.

This is a general description of the Theranos device. Note, there’s nothing in here about programming the device from the cloud. Only if you look at the dependent claims do you see what Holmes tells Fielder about:

22. The system of claim 1, further comprising a communication assembly, wherein said communication assembly is configured to transmit information, to receive information, or both.

23. The system of claim 22, wherein operation of said system comprises operation according to a protocol, wherein said protocol is configured to incorporate information received by said communication assembly.

24. The system of claim 23, wherein said protocol is configured to be adjustable without requiring modification of the system.

So, Theranos only patented the idea of reprogramming from the cloud for their device. It didn’t take me long to find a prior patent, US Patent No. 6,963,907BI, with a priority date 13 years earlier, that has this independent claim.

1. A method of operating an Internet device for remote medical testing comprising:

downloading via the Internet a medical testing program from a server, said medical testing program being utilized to provide non-invasive cardiovascular function related test measurement data;

coupling at least one non-invasive sensor to said Internet device, said at least one sensor being non-invasively coupled to and disposed on a patient to obtain impedance test measurement data;

executing said test program to obtain said test measurement data from said at least one sensor;

automatically uploading said test measurement data to said server via the Internet;

automatically analyzing said test measurement data at said server to provide cardiac function test data;

storing said test measurement data and said cardiac function test data for said patient in a database accessible by said server;

operating on said test measurement data to produce substantially real time waveforms of said cardiac function test data;

maintaining a history of test measurement data and cardiac function test data for said patient;

utilizing a trending algorithm on said history to develop a medical condition trend for said patient;

receiving processed cardiac function test data from said server as a download from said server via the Internet; and

displaying said processed cardiac function test data.

There are probably earlier examples, but this patent establishes that the idea of downloading instructions from the Internet to operate a medical testing device was already patented years before Theranos. Holmes may have patented a particular implementation, but she can hardly claim to have invented the underlying concept. Fielder wasn’t searching the patent literature, so I can understand why he focused on the difference between can and did rather than boring people with arcane patent claims. Still, Holmes’ veracity is perhaps more directly challenged by asking what she really invented.

I don’t know is on third

Since Fielder was more interested in determining whether Holmes’ mind was just so literal that she thought she was telling the truth, he showed her the famous “Who’s on First?” sketch from Abbott and Costello. The hypothesis, I guess, was that if she was so literal in her thinking, she wouldn’t get the joke. She laughed during it but then said she just thought the way they were acting was funny. When Fielder explained that the first baseman’s name was Who, it still didn’t register. John Carreyrou has commented that one of the things that makes it unbelievable is that she didn’t say, “Oh, I get it,” when Fielder explained the joke. For all we know, before they watched it, Fielder said, “I’m going to show you the Who’s on First sketch now, and I want you to act like you don’t get it.”

Billy Evans then uses this episode to launch into an analysis of how Holmes just “thinks differently.” That the way her mind works, she could be saying the device worked because of some very narrow way of analyzing “what she invented.” We never move from “think differently” to autistic or neurodivergent, but that’s clearly the implication.

Theory of (Liz’s) mind

The reason autistic people sometimes have trouble with a joke like Who’s on First has to do with what Simon Baron-Cohen long ago termed “theory of mind,” which is the ability to intuit what someone else is thinking. The classic experiment in this concept is the “Sally-Anne” experiment where children are shown a story board about someone named Sally who puts a marble in a box. She leaves the room and then another child called “Anne” comes and moves the marble to a basket. Neurotypical children will say — when asked where Sally will look for the marble when she comes back — that she would look in the box where she thought it was. But autistic children are much more likely to say that Sally will somehow look in the basket because the idea that she was tricked doesn’t register.

This lack of theory of mind is often couched as a lack of cognitive empathy, which differs from affective empathy. Cognitive empathy is the ability to guess what someone else is feeling, affective empathy is the ability to identify with the feeling. Autistic people often have low cognitive empathy and high affective empathy. Liars and manipulators tend to be the other way around - you need to be able to deceive another person and not care about hurting them. If you read Bad Blood and Tyler Shultz’ STAT piece, it is nearly impossible to see how Holmes could have done all the things she has done — including raising nearly $1B — without excellent cognitive empathy/theory of mind. Whether she did or did not understand that What was on second base is something we’ll never know.

Subscribe now

Fielder has made a career out of confusing us about whether the people on his shows are really behaving the way they appear to be, or whether they’re in on the joke. No yogurt proprietor would believe that poo yogurt would outsell chocolate-vanilla swirl. Holmes made a career out of convincing people that what they were seeing was something other than what it was. In You Can See Everything, these two titans of deception get to clash. When Fielder does it, we laugh — and boy did I laugh at his latest and greatest achievement. But when Holmes did it the first time, investors lost hundreds of millions of dollars and patients nearly got the wrong health information and could have died. And the most impressive trick in the movie is that Fielder almost gets us to forget the difference.

Here’s the first part of the transcript where we get into it, but watch the whole thing. We talk about critiques that were raised about Grit and the commendable way Duckworth responded, why the peer system of Alcoholics Anonymous works so well, and advice for graduate students and postdocs based on Duckworth’s research.

Willpower is overrated

Holden Thorp: Hi, this is editor-in-chief of Science, Holden Thorp. Today, I’m talking with Angela Duckworth. She is a professor and psychologist at the University of Pennsylvania. Her book, Grit, was a number one New York Times bestseller that talked about how passion and perseverance made people successful beyond just, uh, their talent.

Her new book is called Situated. It’s all about how to get your environment and the people surrounding you and your situation tuned up so that you can be successful. There’s lots of lessons in here for scientists, and so, uh, here we go with Angela. So Angela, thanks for coming on. It’s great to talk to you about Situated.

Congratulations.

Angela Duckworth: Thank you, Holden.

Holden Thorp: Your new book, Situated, is a follow-up in some ways to your first book, Grit, which has sold millions of copies and become part of the zeitgeist of the whole world, and it’s about how passion and perseverance in a very special combination are what have made a lot of people successful in ways that maybe are, were counterintuitive.

Situated is, uh, more about what surrounds us, and so why don’t you start by just kinda telling us why you decided to write this book and who you think can benefit the most from it?

Angela Duckworth: So the story of this book is, I guess, also the story of my research career. I started off studying individuals, and how they were different from each other.

So in, in psychology we’d say I started studying individual differences, and I specifically wanted to know what are the personal characteristics of people who become great at what they do. This, Holden, is a lifelong fascination for me. It’s not even that I like sports, but I love studying Olympic athletes or, you know, people who are inducted into the Hall of Fame for, I mean, anything.

It’s like tennis, golf, badminton. I really like studying human excellence, and I wanted to know what is different about those individuals than others. So where I started was trying to understand how important effort was compared to talent. And having grown up in a family where, you know, my dad was talking about who’s the smartest cousin and IQ scores, I just wanted to show my dad and myself that there may be something else, another personal quality that was important.

It turns out in my research on grit and measures of IQ or physical ability are usually correlated at zero on average. Sometimes actually weakly negatively correlated, meaning those for whom there’s an early advantage in how quickly they pick things up. You know, maybe in some samples like that is slightly unlikely to be — I don’t wanna over-exaggerate a small correlation — but a weak negative correlation means that knowing nothing else, it’s maybe you’re less hardworking and, and persistent.

So this quality of grit is passion and perseverance over really long periods. It’s really the hallmark of stamina over a long time. It’s not like intensity, but consistency. A happy way to spend the first half of my career, try to understand the mindsets that led to it, try to understand the habits of people who have this personal quality of grit.

I think you’re not really a scientist unless you keep examining the question maybe from other angles, from a broader perspective. I mean, to me, this second half of my career has just been like widening the aperture, taking a wide angle lens, and then noticing that common denominators aren’t just inside the person, but also outside them.

So for example, I noticed that high achievers have mentors. Not all of them, but most of them. They tend not to be loners. They tend to spontaneously turn to peers, for example, to try to figure things out. And there were other common denominators as well. So that, that in a way is the backstory of this, of this second book.

And I’ve been studying it as a scientist mostly with a cadre of collaborators and behavioral economists ‘cause that’s what behavioral economists do, change the situation to change decisions. Also with psychologists like James Gross. So this, this book is, I hope, an expansion, not a...

It’s certainly not a negation of the importance of individual effort. I think individual effort is always going to matter, but in some situations effort makes sense and pays off more than in others.

Holden Thorp: Mm-hmm. And I’ve seen some reviews and podcast headlines that have said a title of the book could be “Willpower is Overrated.”

Are, are you happy with people characterizing it that way or what do you think about that?

Angela Duckworth: I think that’s fair. I think willpower is overrated, and, and as a scientist who’s been studying willpower alongside grit, by the way, and I don’t think they’re the same, maybe I could tell you what I think the research says on what willpower is, what it’s not, and, and why it is, I think, the worst way to try to change yourself.

So I, I think willpower is really the exercise of the dorsolateral prefrontal cortex, right? I’m not a neuroscientist. I do collaborate with them and I do have a degree in neuroscience, but I am not a neuroscientist. But we, we do have a part of the brain that is in charge of executive function, and that part of the brain is capable of suppressing lower-level impulses.

“I kinda wanna eat that, but I won’t.” “I kinda feel like doing that, but I won’t.” “I kinda don’t feel like doing that, but I should, and I will,” right? So we can will ourselves. We have the power to will ourselves to do things that are not immediately enjoyable or rewarding. And as a species, we have more willpower than any other animal on the planet.

We are better at delaying gratification than squirrels and chipmunks and cats and dogs, and even chimpanzees. But, but the question is, is that what grit really is? If you look at somebody like Katie Ledecky or Michael Phelps, two of the greatest swimmers of all time. I told you I don’t always have to be interested in the actual craft.

I just have to be interested in the craftsman. And so, if you take these swimmers who like-- Can you imagine what it’s like to wake up 365 days a year at 5:00 in the morning and throw yourself into a pool and then just work at the edge of aerobic capacity for, for, for days and weeks and months and years?

It’s not willpower. They’re not doing it because their dorsolateral prefrontal cortex is suppressing the impulse to stay. It’s, it’s really not grit. I think grit is not willpower because people who really have grit actually love what they do. They do have a voluntary obsession, not an involuntary obsession.

And yeah, they have to do hard things that they don’t always feel like doing, but, but by and large it’s intrinsically motivated. So, so I wanna say that willpower is a thing. I mean, it’s not like not a thing. It is. We have a part of the brain for it. I also wanna say that I don’t think it’s definitional of grit.

I also don’t think... There’s a New Yorker review of this book, and I think somewhere in the very beginning, the reviewer, who’s in charge of her own thoughts, not, not my thoughts, says that grit is just willpower in extremis, and I disagree with that. But, but the reason why I would agree with the headline that says willpower is overrated is that it really is, and that’s not just my research, Holden.

That’s many scientists who study self-control and behavior change have found that highly successful people, people who achieve their goals, who work out when they wanna work out, who eat the way they wanna eat, who aren’t on their phones when they don’t wanna be on their phones, who get their work done, they are not forcing themselves.

They are strategically avoiding temptation and setting themselves up for the behavior they want. So it’s nuanced, but that’s why we’re having a conversation. I, I think willpower is not an effective way to change yourself, and even if it’s not the same thing as grit, the prescriptive recommendation is very similar, maybe the same, which is I think we should, we should for our children and ourselves, and maybe our PhD students and anybody else, we should figure out what situations would naturally incline us to think and then act in the ways that we want to, and then just act there as opposed to waiting until you have an unwanted impulse and suppressing it.

The whole interview is great. Please watch!

Subscribe now

Facebook knew things we couldn’t know

This section was also in our ScienceAdviser newsletter (sign up for ScienceAdviser here!).

This weekend, Aaron Sorkin’s follow up to The Social Network, a film about Mark Zuckerberg and the creation of Facebook, will hit the big screens. The new film, The Social Reckoning, explains how reporter Jeff Horwitz and whistleblower Frances Haugen exposed the fact that Facebook knew much more about how the social network was creating political polarization around the world than they admitted. In the film, reporter Jeff Horwitz is played by actor Jeremy Allan White. I sat down with Horwitz on the podcast to talk about the backstory of the film; Horwitz had not seen the film yet, and neither had I.

During the period of the film, Horwitz was at The Wall Street Journal (he is now at Reuters), and Frances Haugen was a Facebook employee with access to internal data on what the company knew about how social media was harming democracy and teen mental health. Before leaving Facebook, Haugen downloaded a large trove of these data and then shared them with Horwitz who, in a 2021 series of stories called the Facebook Files, exposed how much Facebook knew about the potential harms they were creating.

Science came into the story a few years later when we published a set of papers developed after a coalition of academic researchers was given some of Facebook’s data to analyze for whether the company had contributed to polarization in the 2020 election. No significant signal was seen in the data that suggested that social media played a major role in polarization in that election, but we and the researchers believed it was because the audiences were already pre-segregated. Facebook saw the result as proof they didn’t cause polarization and did a victory lap. Horwitz covered this disagreement at the time.

In our conversation, I asked Horwitz for his take on these events. He stressed that it is hard to appreciate the asymmetry between Facebook controlling the data and academic researchers only seeing a portion of the story. “You could have a team of 20 PhD researchers,” he told me, “and literally more interesting work can get done by the summer intern over at Facebook just because they have the ability to look at the data overall.”

The experiment consisted of showing one group the algorithmic feed that Facebook used to keep users on the site while another group was shown a chronological feed based on the folks they followed. This chronological feed presumably would be less tailored to create polarization. But Horwitz explained that the algorithmic feed is impossible to pin down. “The algorithm is not one thing,” he said. “The algorithm is whatever the current combination of levers are that are being pulled. It is a black box.” Indeed, after the papers were published, we learned that Facebook had temporarily changed the algorithmic feed to a more chronological feed during the experiment without adequately notifying the researchers. (We have alerted readers to this discrepancy.)

The situation has a remarkable parallel to current strain around artificial intelligence. Only the AI companies themselves know the extent to which large language models are carrying out dangerous actions, and the public only knows about it if they decide to disclose. This asymmetry is identical to that with social media five years ago. Perhaps one day another Frances Haugen will emerge to blow the whistle on AI, but probably only after significant damage has occurred.

And why aren’t there more internal whistleblowers? Horwitz sees a dilemma facing every researcher hired in trust and safety at a tech company. “You get hired to study a problem,” he says. “You have data that no one else in the world has access to. You have the ability to run tests which do come up with statistically significant facts on whatever sort of problematic behavior you would like to be addressing.”

But even the best internal researchers can only do so much. As Horwitz said: “You come up with solutions, and then the company’s like, ‘Uh, no, thank you.’”

Why teen mental health is so hard to study

I also asked Horwitz about what the company knew about harms to teen mental health from social media. In Horwitz’ reporting are documents based on internal research that Meta has done that show clear harms to self-image among teen girls from looking at Instagram. But as I’ve described here multiple times, academic scientists have struggled to see strong signals on improvement in teen mental health or on test scores when phones or social media are restricted. I asked Horwitz why he thought that was the case.

Horwitz thinks it’s only some of the users who are profoundly affected, and therefore, that makes the signal hard to see. “It is only a subset of users,” he told me, “and generally a subset of teenagers who arrive at the platform with some mental health stressors, whether that is body image issues or confidence issues, or some level of depression.” That is consistent with his reporting on the internal Meta research, which says that Instagram “make[s] body image issues worse for one in three teen girls” who came to the site already having body image issues.

But because of the way the algorithm works, if it detects that the user is engaging with body-image content, the platform is likely to show images of women that make her feel worse. “If I give you that particular child with that particular makeup, that is what the platform will feed that kid,” he said.

So that raises two things that make it hard to see the signal - the first is that it’s only affecting a small subset of girls (and less of the whole population if boys are included) AND the algorithm is giving each user something different, and that is only apparent to Meta.

These things are still observational. I told him that still didn’t prove it was causal. He conceded that was correct. “You can't prove it's causal, is a true statement,” he said. “It's really hard to come up with something causal.”

But he believes that’s the wrong question. He used the analogy of TV in the earlier era. It’s very difficult to say whether that was bad or good for kids. (I memorized all of The Brady Bunch and Star Trek, and I seem to have survived.) It matters what kids watched — and when. Watching schlock at 2 am on a school night would probably be more detrimental than watching Nova or Nature at 3 pm after school.

Ultimately, he thinks the causality question in the population is the wrong question. Rather, it’s the individual experiences - curated by the bespoke algorithm - that you need to study. Even Andrew Przybylski, who is generally in the group that says social media harms are overblown, agreed with this in a 2021 story. “People talk about Instagram like it’s a drug,” he told Horwitz. “But we can’t study the active ingredient.”

In the end, whether it’s political polarization or teen mental health, Meta has access to data that independent researchers don’t. And when they are selective about how much of it to disclose, they can shape the answer that researchers get.

Social media was just the warm-up act

It’s a good thing we’re hearing this story now, because social media is just the warm-up act for the challenges ahead with artificial intelligence. The chatbots have the potential to manipulate our emotions and actions even more than social media. And they are teaming up to do things to our world that we can barely imagine.

Like with social media, there’s only one group that has the data you need to figure out how much danger we’re in, and that’s the AI companies. Pretty sure they’re not planning to cough up their data, either.

Horwitz summed this up by asking a better question. The impossibility of answering the giant causal question shouldn’t stop us from asking questions we can get at. “‘Is the product as currently built reasonably designed for a teenager?’ seemed like a question that maybe deserved greater prominence,” he told me.

OpenAI and Anthropic need to give us the same answer.

Philosopher Agnes Callard of the University of Chicago has an idea that explains them all — and more. In a podcast with Derek Thompson, she laid out the idea of the “unicontext.” The unicontext is what has happened over the last 100 years as transportation and mass communication have allowed people to move beyond their very local environs. Before the unicontext, there were different contexts for different situations - home, village, church. Each situation came with a different set of behavioral norms. Now, those contexts have collapsed and we only have one, initially formed by the ability to travel physically and now to travel virtually over the whole world.

Subscribe now

In the unicontext, there is one set of norms that governs behavior and evaluation. As a result, ideas that spread without context dominate. Identity (I’m a woman, gay, Black, etc.) is much more easily understood without context than individual behaviors that must be interpreted in the context of the situation. Criticism can stand on its own compared to compliments that need details behind them. Rankings are much more easily interpreted than trying to understand why one organization is better in some situations and not in others. Everything is available - so attention can be allocated anywhere; before the unicontext, attention was allocated only to what was in the immediate environment. And institutions no longer have the authority they once had just because of the context of who they are — people can choose whether to follow them or not based on all of the other available inputs.

It’s important to realize that it’s not just technology that produces the unicontext. Yes, it can’t happen without transportation, mass media, and the internet. But it also requires the basic human desire to have more recognition and more choices. When those two things are put together, the unicontext forms. And that’s why, for example, scientific authority and expertise have been diverging over the last 100 years. I discussed this part in an editorial in Science.

Talking to Agnes directly

After I saw Thompson’s podcast, I was excited to talk to Callard directly about the implications of the unicontext for science. I had interviewed Callard before because she and I are both autistic and have similar profiles and stories. You can read more about that in my book but also here in a sweeping New Yorker profile of Callard.

Here’s my full interview with Agnes:

I sent her my column about going to see Uta Frith who has called for shrinking the autism spectrum back to including only those with apparent and significant challenges in communication, even though Frith herself was partly responsible for expanding the spectrum in the first place to include those with less obvious impairment. In our discussion, Frith told me that she would have diagnosed me with Asperger’s in the 1990s but not today. And she was shocked that her comments started a public firestorm, because she had just written an article for a scientific journal discussing this.

Callard saw at least two ways in which the unicontext explained these events. First, the desire for more people to have the autism label stems partly from the fact that identity registers strongly in the unicontext. It makes much more impact to say “I’m autistic” than it does to explain individual autistic traits that require the context for interpretation, i.e., I have trouble making eye contact so when I was sitting with Uta Frith, I stared out the window most of the time. The autism label carries information through the unicontext in a way that descriptions of individual traits cannot. “The unicontext also seems to generate autistics,” Callard told me. “I suspect that ‘autistic’ is a classification that is itself very hospitable to the unicontext, and that’s why it exists and why it has a kind of power.”

Second, Frith’s notion that she could start an academic discussion about how the autism spectrum should be defined and not have it bleed into the public discussion was a pre-unicontext idea. It was inevitable that autistic people would read her article online and begin the firestorm.

Finally, Frith’s reason for wanting to shrink the spectrum is because she feels that autistic folks with lower support needs are taking attention away from those with much greater needs. Distributing attention is the primary job of the unicontext. “I think it’s really interesting that her thesis, just as you’ve explained it,” Callard said, “suggests that the category of autism ought to be an attention allocation mechanism, right? It’s gotta serve that job.”

What if authority doesn’t come back

When we got back to the divergence of scientific expertise and authority, I really wanted to know if the unicontext could recommend a plan for going forward. Callard used the analogy of a doctor to explain the situation. Before the unicontext took hold, a doctor would not just tell you what was wrong with you, but almost always told you what to do. Today, doctors tell you the percentages behind different choices you have about your health, use their expertise to explain these choices, but then stop short of making that choice for you. This is the unicontext at work - doctors had expertise and authority, now they just have expertise. You have the authority.

Given the fact that expertise and authority have been diverging for 100 years and that the forces driving them apart — technology and basic human desire — are unlikely to turn around soon, I asked Callard if there was hope they would ever be reunited.

First she explained that expertise is actually growing. “What we’re seeing is a gradual increase of expertise all around,” she told me. “We just get more and more expertise over time. Expertise just grows, but [there’s a] decline of authority.” This makes sense as we get more and more scientific papers and journal articles and trained students.

But as authority has waned, there is still a desire out there for authorities. Unfortunately, those authorities are not in an enviable place. “We have put authorities in an impossible position,” she said. “And we still want them and we also won’t accept them. And eventually we are all just gonna have to learn to live with the experts we have and stop pretending that they’re the authorities that they’re not.”

Callard thinks ultimately society will give up its need for authority and learn to live with experts who are not authoritative. It occurred to me that might be a long process. Her assessment: “hundreds of years.”

We’re going to need a plan for what to do in the interim. That may take more than philosophy to solve.

Subscribe now

I once spent three hours with a famous coach at his stadium and when I had to leave for another event, he asked if I could come back later that evening because we still had a lot to talk about. I was exhausted.

Adhering to the traditional leadership archetype is a huge help in landing a job as a big time coach. The process for hiring coaches is usually very compressed in time, so first impressions really matter. And the people making the selection - athletics administrators and football boosters - are usually looking for a traditional leadership affect. This is even more important now in the era of NIL (name, image, likeness) rules that require raising money from donors to pay players.

But these extroverted traits don’t amount to much unless the coach also wins the games. Is a larger-than-life personality required for success on the field? The emergence of Indiana’s Curt Cignetti as the reigning national champion in college football suggests not. I am not in any way suggesting that Cignetti is autistic, and he has not acknowledged that. Nevertheless, Cignetti’s awkward traits have been a national fascination for sports pundits, and his appearance in a new commercial has allowed him to embrace his personality, making a very public acknowledgment of his differences.

Cignetti’s situation poses two important questions. The first is why do you have to be a champion football coach to embrace traits that are different from the norm? And why can’t we have more leaders like Cignetti?

‘He comes across kind of like a serial killer’

Curt Cignetti was a consistent winner in the lower levels of college football. He posted winning seasons at earlier stops: 53-17 at D-II Indiana University of Pennsylvania, 14-9 at Elon and 52-9 at James Madison. After such success at JMU, he was a logical candidate on the market in 2023. Several Power 4 schools that were better at football than Indiana were also looking. They passed him over. Indiana, which had more losses than any program in major college-football history, did not. Cignetti would change its status—and fast.

As his success built last season, many profiles were written of his unusual mannerisms and how they might have led to his success.

From CBS Sports’ profile by John Talty:

"If we're not talking about football, it's kind of awkward to have a conversation with him," DiNucci said. "His brain and his mind are wired one way, and it has been for the last 25 years. He cares about one thing -- winning -- and whatever we have to do to make that happen.“

So he doesn’t do small talk well and is focused on his job above all else. (DiNucci is a quarterback who played for him at JMU.)

"Honestly, early on at JMU, I couldn't really read him," said star Indiana cornerback D'Angelo Ponds. "I used to call him Batman. I couldn't really read what he was doing, but once you kind of get to know him, you understand what his reactions to everything are."

So he’s hard to read.

Dan Wolken called him “America’s best loved curmudgeon,” and speculated openly that perhaps he hadn’t landed a better job earlier because his mannerisms prevented him from doing well in an interview, quoting someone who had interviewed Cignetti:

“Now everybody thinks it’s kind of a cute schtick, but interviewing him, he comes across kind of like a serial killer,” the administrator said. “I don’t think you sit with him and you’re like, ‘Man, our fans are going to love this guy.’”

So people form quick judgments of him based on his affect that are mostly negative (if you consider “serial killer” to be negative).

But, none of that means Cignetti doesn’t have strong connections with people. Talty tells the story of coaches who wanted to attend a former colleague’s wedding and how Cignetti cheerfully rescheduled practice so they could attend.

Happy Coach

Cignetti, of course, is aware of all of this analysis. And he concedes that one might not know when he’s having a good time. “There’s lots of times I’m happy,” he told the Sporting News, “I just don’t show I’m happy.” He says it’s at least in part so he can keep his team focused rather than giving in to emotion. And it helps keep his own emotions in check so he can make logical decisions. Before the 2026 National Championship, when asked how he stays composed, he said, ““Self-control, I guess. I’m just trying to anticipate, not be emotional, which is the best way to make good decisions.” Of course, that’s right but you wouldn’t know it looking at every other coach yelling and throwing his headset.

I’m not a Chiefs fan, so I usually dread the release of the latest “Jake from State Farm” ads at the beginning of football season. But this year, they released an ad with Cignetti called “Happy Coach” where he playfully (and no doubt, profitably, for him) acknowledges his flat affect:

Cignetti is very happy with the personal price plan he’s going to get from Jake but never cracks a smile. At the end he flatly says, “I’m so excited.” To which Jake sarcastically says, “Yeah, I can tell.”

It’s great that Cignetti is acknowledging his traits. That’s good for those of us who have the same traits. But most of us have also been told (as he may have, as well) that we “don’t have good people skills,” are “too focused on work, not relationships,” are unable to inspire people with our outward expression, and that we make people nervous because they can’t tell what we’re thinking. Many autistic leaders who are good at getting the work done get told this when they get to the corner office (that includes me). Cignetti gets to have fun with it because of the success he’s had, and I get to do so now because I decided to stop trying to pretend I’m someone I’m not. (Also, I’m the editor of a science journal.)

But what about people who haven’t had Cignetti’s success? They are often excluded and criticized for these very same traits, especially if they are also not white men and subject to other forms of discrimination. These very same traits - flat affect, lack of interest in small talk, apparent lack of interest in relationships - are often disqualifying for leadership roles, but more tragically, lead to exclusion and even bullying for many people who never even get a chance.

Who looks like a leader?

Given all the problems we’re facing, wouldn’t it be better to have more leaders who actually accomplish things rather than making us feel good or exploiting our emotions? In my book, Leading with Autism, I will make that very case. Check it out next year.

Maybe Cignetti’s example—and, I hope, my book—can help make room for more awkward people who are good at getting things done to help make the world better. If that happens, I can tell you my response:

[says flatly]: “I’m so excited.”

Subscribe now

Shubin published an editorial with us in July where he stated clearly the principles he plans to follow as NAS president:

As an institution, we are — by charter and design and history — nonpartisan. But we are in a deeply partisan age. Just by doing our work, we will be viewed in a partisan light no matter what. So our job is to double down on the science. We need to be a trusted source of data.

By its charter, the Academy is nonpartisan and neither a progressive organization nor a conservative one. It is a scientific body that follows the evidence wherever it leads, even when the destination might be unwelcome. In heated and polarized discourse, it is the Academy’s obligation to be the most careful and trustworthy voice. But rigorous science that arrives too late, or speaks too quietly, serves no one. In this rapidly moving landscape comes the need to move with ever greater speed and reach.

In our conversation, I read these passages back to him, and we used these principles as a touchstone throughout our conversation. He did well at that, going back to these ideas on almost every question. I asked him about how he plans to engage members more, bring younger researchers into the work of the NAS, deal with the complications of AI, and walk the tightrope between when science is just the results and when it has policy implications.

Here’s our conversation:

Shubin’s decision and process to review a controversial chapter

The newsiest part of our conversation was when I asked Shubin about a controversial chapter on climate change in a report that the NAS does for judges. The report is called The Reference Manual on Scientific Evidence, and the NAS prepares it along with the Federal Judicial Center. It’s important to note that a report like this is a compilation of materials, not something that is peer-reviewed according to NAS policies for consensus studies.

President Donald Trump criticized the chapter’s objectivity on social media:

A month ago, the NAS announced they would be reviewing the process by which the chapter was included, citing the fact that the Federal Judicial Center had already decided to remove it from the report:

The National Academy of Sciences was founded on the enduring principle that our nation deserves the best available scientific knowledge. Our core mission is to deploy science in service to the nation while employing procedures that are rigorous, nonpartisan, and that manage potential bias and conflicts of interest. When questions arise whether that standard has been met, we have an obligation to examine them honestly and act on what we find.

The fourth edition of the Reference Manual on Scientific Evidence, produced in partnership with the Federal Judicial Center (FJC), included a chapter on climate science. That chapter has received considerable scrutiny, and the FJC has removed it. The NAS has determined that questions about the processes used to develop the chapter warrant an independent review, and the chapter will not appear on our website while that review is underway.

This assessment is about the process used to produce the chapter and not about the validity of climate science. The focus is on our own procedures and whether they were adequate to ensure the highest standard of objectivity that this institution requires and our nation deserves.

In our conversation, Shubin acknowledges that it was his decision and was “really hard.” I agreed with the decision, because he has now inherited it, and if he is going to defend it in a high-stakes way, he deserves the opportunity to study the process and make sure he wants to stand behind it. He raised the fact that he has been criticized for this. “You can say … that’s appeasement,” he said. “You’re giving in ’cause you’re getting pressure.” I would submit that it is too early to conclude that until he has had a chance to review the decision to include it.

“There’s a lot we don’t know about the process behind it,” he said about the inclusion of the chapter, “but we know the outcome was perceived, and for some in good reason, because the authors were associated more with the plaintiff side than with the defense side, [that] that outcome deserves a new, fresh look.”

Putting it on the credit card

I told Neil that when I was in a job where I had to make these kinds of decisions, we called a move like this “putting it on the credit card.” You take a breather to review the situation, but you also sign up for 18% interest while you deliberate.

Shubin left me convinced he would study it carefully and stand behind his decision.

“I have no doubt,” I told him, “that you’ll pay your credit card bill when it comes due.”

Watch the interview and see if you agree.

Subscribe now

Last week we got a controversial (and in my view, highly flawed) documentary on Channel 4 called “The Great ADHD Myth?.” In this film, Dr. Max Pemberton of the UK National Health Service weaves a tale that makes the case that ALL (not just some) of ADHD is a myth. By the end of the film, Pemberton states that “I’m now convinced it is a myth, that ADHD is not a neurodevelopment disorder.” Pemberton concludes that ADHD is over diagnosed (and I guess shouldn’t be diagnosed at all since it’s not a disorder) and that there’s insufficient scientific evidence to persuade him otherwise. “I think it’s a difficult set of behaviors, a social construct, not a disorder of the brain,” he says. (More on why all of this is flawed below.)

Parallels with what Uta Frith is saying

There are a lot of commonalities between what Frith and Pemberton is saying. Before I go through them, it’s important to point out up front that Frith is not saying that autism is purely a social construct, as Pemberton is saying about ADHD. As I explained last week, Frith wants to reserve the word “autism” for those with visible challenges, particularly in communication, but she very much sees the folks she classifies as autistic as having a medical disorder. But to people with less visible autistic traits who still have an autism diagnosis or identify as autistic, Frith’s comments lead to the same place: we have autistic traits that are part of human difference but (according to Frith) not identifiable as a disability and certainly not as a medicalized disorder. (As I explained last week, Frith says she would have diagnosed me with Asperger’s in the 1990s, but not now.)

I know at least part of Pemberton’s argument resonated with Frith because she brought it up in our conversation. In the documentary, Pemberton decides to get evaluated for ADHD himself by hiring a private doctor for £1200. The doctor concludes that Pemberton has ADHD, which Pemberton tries to paint as a scam - the doctor gets money and the patient gets the diagnosis they wanted. When I suggested to Frith that one of the reasons that self-diagnosis of autism was prevalent was because people couldn’t get seen for a diagnosis or for one that they can afford, she brought up the scene from the documentary.

Frith: Yes. Actually, I was a bit disturbed because in that program, [The Great ADHD Myth?], where the psychiatrist who presented got himself officially diagnosed from a good site—it cost him $1,200.

Me: Right. So, do you have empathy for somebody who thinks they might be autistic, but who can’t get anyone qualified to tell them one way or the other, and then makes a choice? How do we deal with that problem?

Frith: I would tend to say they should really find out for themselves whether they need that validation, or whether they have enough evidence for themselves to be satisfied with it. I would say that they would not get much out of a diagnosis, probably, because people do it very superficially, especially private providers.

This idea of conflicts of interest in autism is very much on her mind. She told the BBC that she questions whether autistic researchers doing research on autism can be objective because they (we) have a conflict of interest. Not surprisingly, this was not well received among autistic researchers. In the same article, Simon Baron-Cohen — who trained with Frith as a graduate student — said she was an outlier in her view. “Someone who’s got a degree in molecular biology has one kind of expertise,” he said, “but someone who’s got lived experience has got another kind.”

I asked her about physician-scientists who take care of children with high support needs but who also see autism as a spectrum that includes those of us who have what she would have diagnosed as Asperger’s in the 1990s. I would say this is the majority of experts that I have talked to.

Me: “I talk to a lot of clinicians who take care of kids who have a lot of impairment. But when I talk to them about people like myself, they are accommodating [of] the idea that lots of people are autistic.”

Uta: “Wouldn’t they have a conflict of interest?”

She said that before I got my question out. I gave her a chance to clarify, but initially she doubled down.

Me: “I guess my question for you is, do you think they’re doing that because they don’t want to deal with the backlash of disagreeing with it? Or do you think that they—”

Uta: “They say the more, the better.”

(I was going to say do you think they actually agree that autism is a spectrum.)

Uta: “That’s a natural tendency, perfectly understandable. If you are assessing, if you are in the professional business, you want customers, don’t you?”

You: “The more the better. You think that’s the whole thing?”

Uta: “Yeah.”

Later in the conversation, Frith tried to soften it by saying “I don’t want to push the commercial motive too much, but I think it’s there.”

So, it’s no surprise that Pemberton’s portrayal of an ADHD industry resonated with her. And it gets to the point that the game of deciding whether someone is entitled to a disability label has consumed the conversation of how to get support to the people who need it. And I certainly do not think that the clinicians I’ve met in my travels are embracing the spectrum because it’s good for their pocketbooks.

The problems with “The Great ADHD Myth?”

The documentary was more of an infomercial on Pemberton’s view rather than journalism. The first problem is that all of the experts interviewed appear to support the thesis that ADHD is not a neurodevelopment disorder. That is certainly not the consensus among ADHD clinicians as these statements from ADHD UK or the Royal College of Psychiatrists show.

Most of the experts in the documentary do agree with Pemberton, but one - Katya Rubia — thought she was there to provide the counterargument only to find her comments used selectively to support Pemberton’s view. Rubia has since withdrawn her support for the documentary, and as quoted in The Guardian:

“I agreed to take part in this documentary because I knew it was aiming to show ADHD was a myth and I wanted to provide an alternative view,” she said. “But they cherrypicked, largely misrepresented, truncated and presented my comments out of context to make them fit into their narrative.

Even a positive review in The Times (of London) noted that “there was no real counter-argument offered to the idea of ADHD as a ‘myth’.”

The film also relied heavily on a dramatic but unscientific portrayal of a 10-year-old boy, Mason, who we see taken off of his ADHD medication. He felt more ‘himself’ when he was taken off, but his school work deteriorated. The dramatic part was the family’s wrestling with whether to put him back on or not. The film appears to end with the family happy with the decision to take him off his medication only to have a title card at the end saying because his schoolwork had continued to deteriorate, they made the decision to put him back on his medicine. The anecdotal outcome that argues most strongly against the film’s thesis is relegated to a title card.

They have one child used as a study. While this might make for compelling television, it doesn’t prove anything scientifically. As Rubia said in The Guardian:

“This ‘experiment’ provided nothing but anecdotical evidence confounded by a host of factors,” she said. “The child was not only taken off medication, but also off screens, off additives and junk food, subjected to more nature experience and by implication to more interaction with family, friends and parents.

“This can lead to greater happiness and stronger family bonds (for every child, not just ADHD children). There was also the uncontrolled placebo effect of being treated on TV.”

Imagine having a child on medication who is doing well and watching this. It only increases the stigma of ADHD and the decision to use medicine that has been shown to be safe and effective.

Why increasing stigma will only lead to more suffering

I agree with Pemberton and Frith that the traits of ADHD and autism are present in much of the population and - for some of us - create a difference that may or may not rise to the level of disability. Whether traits are disabling is a product both of the traits and the environment. If that environment stigmatizes those traits or a person’s response to them, it will only make those traits more disabling, not less. Increasing stigma is not going to erase the traits of ADHD or autism. In fact, it may cause more people to seek a diagnosis for validation and support, and cause others who need support to fail to access services that could make their lives better.

Anthropologist Richard Grinker, in his book Nobody’s Normal, explains that stigma is cultural and shows how mental health plays out differently in cultures where there is more or less stigma.

“The problem with stigma,” he says, “is not just that society can shame, humiliate, marginalize, discriminate against somebody that doesn't conform to a particular norm. But that we can do it to ourselves; we internalize this idea of what is good and bad, and we can stigmatize ourselves, [which is] incredibly dangerous to one's mental health.”

We absolutely should continue to have scientific debates about how to define ADHD, autism, and many other aspects of neurodiversity and neurodevelopment disabilities. But we need to do so with more compassion and carefully chosen words.

The boundaries might be uncertain, but the consequences are not. Let’s focus on what people need rather than running a contest over who deserves a label more.

Subscribe now

When I wrote to Uta to ask for a comment for this column and noted that I was spending the month at the Science UK office in Cambridge, she invited me to her home in Harrow-on-the-Hill, London for lunch. Even though I disagree with her proposal, I was eager to learn why she was doing this at age 85 and to find out more about the scientific basis for her claims.

Though sometimes contentious, the spectrum is broad

I’ll go over more of the history in my book, and Steve Silberman did even more of it in his much bigger book, Neurotribes, but the quick version is that even in the earliest days of autism, it was noticed that some autistic children were verbal and did not have an intellectual disability. In fact, the very first person diagnosed with autism, Don Triplett, eventually traveled the world and led a fulfilling life, as described by John Donvan and Caren Zucker in their book and documentary, In a Different Key. And it was always known that defining who was autistic was going to be contentious. Lorna Wing first described those of us who have a high load of autistic traits but no intellectual disability as having Asperger’s Syndrome. She later told Steve Silberman she knew she was opening Pandora’s Box, because Asperger’s, she said, “shades imperceptibly into eccentric normality.”

The official word on all of this evolved. In the DSM-IV, there were subtypes of autism where Asperger’s was a separate category. This solved a problem, because Asperger’s doesn’t have the word autism in it. The other two categories were autism and PDD-NOS, which stands for “pervasive developmental disorder, not otherwise specified.” The simplest way to understand the difference between the two is that autism would have a severe profile (now sometimes called “profound autism”) often involving intellectual disability and perhaps self-injurious behavior, while PDD-NOS was for “higher-functioning” individuals but those who didn’t start speaking until much later than normal.

Nevertheless, in the DSM-V, it was decided that there would be “autism spectrum disorder” for everyone with three levels where Level 1 required the least support and Level 3 required round-the-clock care. This led to an explosion in Level 1 diagnoses, and rather than calling these folks Asperger’s, they were referred to with the word autism. Thus, the word “autism” was given a very high burden - to describe folks with challenges in social communication and repetitive thoughts but who can mostly function on their own all the way to children who are often inconsolable and harming themselves.

This was a lot of burden to put on one word. But at the same time, those with Level 1 autism who do need support were given great comfort, agency, and multiple ways to find support. Sadly, though, the politics around who gets government recognition and support led to infighting between those on one end who view all autism as one thing and those who view children and adults with the highest support needs as being most deserving of the label and the services.

The science, in my view, shows that both of these extreme views are wrong. Genetics and other biological measurements show that there are distinct differences among autistic folks with different profiles. At the same time, the patterns of social communication challenges along with repetitive thoughts and behaviors are similar across all of the profiles. The difference in my mind is that some of us can read, write, and talk about our autistic traits in a way that allows us to decide how we want to situate ourselves in the world and what to expect from society while others, who are clearly less fortunate, are not able to do that. But there are lots of different views on this because there is no biological test that would provide a clear answer, and that will always leave this somewhat subjective.

Dame Uta enters the chat

Frith has found herself immersed this controversy since writing an essay in Psychological Medicine suggesting that the word autism has been applied too widely, specifically to those of us with relatively lower support needs. The attention on this escalated significantly when she did an interview in The Times in March where she covered her basic case and said she now regrets diagnosing and advocating for people with lower support needs as having Asperger’s.

She now believes that the autism label should be used much more narrowly, and mainly for folks with very severe communication challenges. As she told the Times in March:

“Having a conversation with someone with autism is likely not to feel fluent, but to feel stilted or abrupt. So this fluency is a contra-indicator for the diagnosis of autism,” Frith says.

At the same time in March, she did an interview with the magazine TES where she stated that masking, the process of covering autistic traits to blend in, is an idea that “has no scientific basis.” This led her to conclude that most women and girls who were diagnosed late might not be autistic because she doesn’t see masking as a strong enough basis scientifically for a clinical judgment.

Then she gave a more explosive interview to The Times this month where she stated that the British naturalist Chris Packham - who has made multiple documentaries about his autism (initially classified as Asperger’s) — could not be autistic:

“He cannot be autistic in my opinion,” Frith interrupts. “It is just absolutely absurd to me that somebody who can communicate as brilliantly as he does — that is what I call a blatant contra-indication.”

I have a very similar profile to Chris Packham, especially when he’s talking about how his mind can race and how the smallest sounds can disrupt him. He’s done a great job bringing this out in his videos, which I know bring strength to many.

Let’s ask her - my trip to Harrow-on-the-Hill

From what I can find, no one who has done one of the high-profile interviews of Frith is autistic themselves or a scientist who has studied a decent amount of this literature. And even though interviewing her risks further amplifying her ideas, I decided her ideas were pretty well circulated beyond my little Substack. So, I wrote to her, initially just to get a comment, but she invited me to Harrow-on-the-Hill to have lunch with her and Chris.

The scene was the same as described in the recent Times piece, they received me graciously in their very classic London professors’ apartment and we ate lemon cake on the back porch after I talked to Uta for an hour.

Although I pushed her on many things, she was open to hearing my ideas. I told her that my focus was in describing autism for people with mostly lower support needs who were getting feedback from their spouses or employers that they didn’t pick up on cues or were uninterested in other people and wanted to improve. That I thought the way to do that was first to explain that there are autistic folks with much higher needs who don’t have the opportunity to read and think about their autism who deserve recognition and support, but then to focus on autistic traits and where they come from and then explain the questions around whether a diagnosis is useful for them and whether they should disclose it if they get one. And then how to decide what to do with all that information. You can read more about that next year in my book.

She agreed with me that the people I try to write for have autistic traits and have challenges, but she has changed her mind about classifying us as autistic, even though she readily admits that she was part of the group that built the spectrum in the first place.

She noticed me staring out the window and not making eye contact, and acknowledged those were autistic traits. She even said there was a time when she would have diagnosed me.

“I actually believe that in the 1990s,” she told me, “I personally would have diagnosed you with Asperger’s syndrome. Now, I am not so sure.”

So, I kind of got a diagnosis from Dame Uta Frith herself, but I guess it’s expired as far as she’s concerned.

Later, I asked her a similar question about Chris Packham. She told me that she regrets telling The Times that he was not autistic, that it was an off-hand remark and she didn’t recognize how newsworthy it would be. She concedes that, like me, she would have diagnosed him as having Asperger’s in the 1990s, but she has changed her mind about whether she would do that now because of his outstanding ability to communicate.

I believe her when she says she didn’t mean to take away his or anyone else’s diagnosis who already has one. I think she is trying to do something that is impossible in today’s world, which is to speak to her colleagues about a scientific matter without speaking to the whole world at the same time. I tried to explain to her that I am kind of a journalist now and I work with real ones at Science and that it’s kind of impossible to do that.

A strong adherence to the medical model

I think that what Frith is exhibiting now is simply a very strong adherence to the medical model of disability with very little regard for the social model of disability. She believes autism can and should be medicalized and that there is a very specific pathology that describes it. She was very critical with me of “lived experience” as an aspect of autism.

But she was willing to engage in scientific debate. When I brought up the research that shows that we’ve long known that autistic traits are present on a continuum and that we know their presentation is very heterogeneous (i.e., “if you’ve seen one autistic person, you’ve seen one autistic person”), she agreed with both. Then I said that means that you’re always going to have a place in this continuum to draw a line between what is pathology and what is personality — and that will always be contentious and contested. She agreed with that.

“And so,” I said, ”it becomes this question of where the line is between who gets the label and who’s just on the other side of the label.” I said the only good way to determine that was based on the experience of the autistic person.

She agreed with the first part. “You know,” she said, “this is the topic I remember discussing as a student when we did clinical psychology: can we ever draw a line between what’s normal and abnormal … This is a question that nobody has got a really good answer for.”

But she doesn’t accept the idea that it could be about experience and environment. She said the only hope was if genetics gave us some kind of marker, but then when I started talking about the genome-wide association studies, she said none of them were convincing yet.

In essence, my position is that given the fact that the boundary is fuzzy and likely to remain that way, we should trust the autistic people and give them the support they need (including using the label if that helps) while she hopes biology will give an effective answer even though she concedes that so far it hasn’t.

Debating overdiagnosis

We discussed her assessment that many people were overdiagnosed or misdiagnosed, and I agreed with her that of course there have to be some because these are human judgments not always made perfectly, but that the societal barriers to getting a diagnosis were also a major factor. She is, not unexpectedly, critical of self-diagnosis, but when I explained that often that happens because no one can get services to get a professional diagnosis, she was more understanding of that.

And she was very conciliatory when I asked whether it was also true that there were many people not getting a diagnosis because of these barriers who truly needed one. So, doesn’t autism also have an under diagnosis problem, I asked? “Yes, it does,” she said, “there are certain people who can’t get diagnosed and should get one.”

She didn’t push back when I said that this leads to a kind of circular logic where we are criticizing a lack of professional diagnoses when there’s no one around to give those diagnoses — and that without the label, most people can’t get the services they need.

Uta Frith and I agree that science doesn’t know where autism ends and personality begins. We disagree about what to do about it.

I have interviewed many physician-scientists working in this area who have views pretty close to Frith’s. In fact, many of them may hold the same views she holds privately but don’t want to wade into the battle. And I believe the medical model has a lot to offer. It has established that autism is not just something in our heads, that those of us with lower support needs have real phenotypes that are different both in genetics and brain structure, and that co-occurring conditions that can be treated are better understood.

But why shouldn’t autistic people — and everyone for that matter — have the benefit of both the medical and social models? That’s the main place where Frith and I diverge. Even someone like me who has most of life’s possible advantages has experienced difficulty from autistic traits imposed by society and the environment — and their preferences for a neurotypical affect. (I told Frith that I had a meltdown at the hotel before I came over; it had to do with technology, as it so frequently does.) Now imagine how that plays out for someone with fewer resources who is also excluded based on their other identities.

Finally, I asked her if people agreed with her, what would she call the rest of us who are no longer autistic. She had noticed I was staring out the window and agreed that was an autistic trait. When I explain why I do that to someone, I just say it’s because I’m autistic. What should I say instead?

She didn’t know. She suggested “autistic-like.” Then she tried “autistic temperament.”

I didn’t think these would fly, and she reluctantly agreed. Since even Uta Frith agrees that science can’t pinpoint the answer, I think we’ll always have to include human experience as a guide. So I’m sticking with autistic.

Subscribe now

Read more

Evans was someone I really wanted to talk to about all of this. While he is firmly in the “AI is here to stay” camp, he is still quite skeptical of what it can do. Every time I talk to him, I feel more comforted, not less, that humans will always be indispensable for research. The two main reasons are that (a) LLMs will always make errors and (b) by their very nature, they’re not going to be creative.

When I tracked him down, Evans had just schlepped across the country from Boston to Palo Alto to give two talks at Stanford. Here’s the video of our conversation:

Precise position or velocity, but not both

When I asked him what it was like to be working in this field right now, he talked about what he calls the “Heisenberg uncertainty principle” of AI, which is that at the very moment that you can measure something in the field, an AI agent is simultaneously changing whatever that is. Evans cleverly refers to the AI agents and chatbots that most scientists are using right now as “retail AI” compared to LLMs that are custom-built for specific tasks.

When I asked him if we were right to be worried that retail AI was degrading the scientific literature, he agreed that we were, saying:

This is a major cause for concern because in the same way that humans and human labs are, for the most part, black boxes, this is a new kind of black box. And so it’s really increasingly hard to evaluate the difference between really strong science and weak science, because if [the LLM is] tuned to writing and it’s reinforced against what people expect, then it’s easy for it to generate science that appears solid and strong and novel while dressing up work that’s potentially fragile and weak and particular.

I asked him if there were reasons to be optimistic at the same time, and he pointed to the potential (but not reality) of AI to connect distant fields in ways that would give us new ideas.

Reasons to be interested and optimistic are [that] AI so far is able to read the vast sweep of the literature, and so it’s able to bring together hypotheses from one field to problems in another field, alien patterns it can kind of bring potentially together. I think largely this has been a promise rather than a reality.

Never-ending pasta bowl

This got me excited. One of the things that I’ve been hoping that AI would be able to do is to elevate papers that are important but not published in high-profile journals. Because (and yes, this is good for Science from a business perspective) Google page-rank is always going to surface papers in high-profile journals that get a lot of citations and news stories, but there are many equally important papers that may have very similar results and might even predate the high-profile ones. I wanted to know whether current AI would allow us to do this, and I was told I had to wait. Retail AI currently is most likely to reach to the “center” of fields, not to the edges, which will give you the most common answer, just like a Google search. As Evans says:

If you don’t know anything about Italian restaurants in some city in the US, then it’s going to return Olive Garden. It’s going to return the most voted average center of the distribution, which is not going to be the most interesting Italian restaurant in any US city. I think agentic approaches where you’ve got different agents with different personas and different tools that have specific criteria definitely have the potential to reach out and, and read and surface new parts of the literature that [we] haven’t seen before. On average, that’s not what we see is happening.

Dang. I’m excited for someone to solve this. But I guess I can enjoy the unlimited breadsticks in the meantime.

Where current policies in journalism and scientific publishing stand

As far as searching and analyzing the literature and media are concerned, most organizations have taken the position that using LLMs is permissible as long as the human author takes responsibility for the content. Here’s a handy spreadsheet of Science’s policies for research articles; using AI to search the literature is permissible and authors do not need to declare that they were used. Similarly, the editorial standards for our News from Science division allows use of AI in background research, saying “Reporters may use AI-based tools to transcribe interviews and in their background research, but we expect reporters to check with their editors about other uses of AI.” This is a similar stance to nearly every major news outlet: here is the Associated Press standard, which goes further than News from Science in allowing AI to suggest headlines.

I asked Evans about this, specifically whether we knew where human ideas end and ideas partly attributable to AI begin. He believes we’re well past the point of knowing that. It’s human nature to think that the very clever prompt we gave to the LLM was the thing that created the idea, but the responses that we get undoubtedly shape how we think about the topic. Similarly, even people who are not using LLMs directly are interacting with other people who are, and they’re also reading scientific papers and news stories that are influenced by AI. So, no one who says they never use AI is completely immune to its influence.

There are strong feelings out there about the ethics of all of this, particularly about where the data in the LLMs came from in the first place and the energy demands of the models. Those debates are not going to calm down any time soon. In the meantime, the most important thing we can do is to understand what AI is doing to the knowledge that we are creating, and for this, I hope you’ll watch my whole conversation with James.

Also in the meantime, here’s a New Yorker piece about how to get a table at Carbone New York (they’re not on OpenTable). And here’s their wares, which look better than the OG (which is not, actually, an OG).

Subscribe now

Golden Age starts off with a letter from Kratsios followed by a letter from President Trump. Endless Frontier starts the same way with letters from Bush and FDR. Kratsios then goes on to talk about all the wondrous things that Endless Frontier has enabled and then explains why the current scientific enterprise in American universities has rendered its principles obsolete through ideology and incompetence. The thrust is to turn the reader against the current American academic scientific establishment, which, in Kratsios’ telling, has taken away all the beautiful things that Bush gave us.

In the Shakespeare tragedy, Julius Caesar, similar events ensue. When Julius Caesar is murdered by Brutus and a group of co-conspirators (on a certain March 15 in Ancient Rome), Marc Antony, who is loyal to Caesar, asks to speak at Caesar’s funeral. Brutus agrees to it provided Antony does not, among other things, blame the conspirators. Antony then gives the most famous funeral oration in the English language where he cleverly turns the crowd against Brutus and the conspirators, leading eventually to a messy battle over who will control Rome. Importantly, Brutus and the rest of the conspirators have different motives. The other conspirators, led by Cassius, are simply out for power; Brutus, in contrast, simply feels that Caesar’s anti-democratic tendencies will be bad for Rome.

Burying Bush rather than praising him - NOT!

The funeral begins with Brutus explaining that Caesar needed to be killed because he was abusing his power — and that would be bad for Rome in the long run. Because Antony has deceived Brutus into thinking that he will not lionize Caesar in the speech, he famously begins by saying, “I come to bury Caesar, not to praise him,” initially making Brutus think he’s keeping his word.

Similarly, by framing Golden Age precisely the same way as Endless Frontier, Kratsios creates the impression that he is here to recast Endless Frontier with a new document. Such reframing, we must think at the beginning, is not going to go well for Vannevar. Surely, there is a flaw in his logic somewhere if we need to replace his Magna Carta of American science with something new.

But in fact, Antony and Kratsios are both actually here to praise Caesar and Bush, respectively.

Antony:

I speak not to disprove what Brutus spoke,

But here I am to speak what I do know.

You all did love him once, not without cause:

What cause withholds you then, to mourn for him?

O judgment! thou art fled to brutish beasts,

And men have lost their reason. Bear with me;

My heart is in the coffin there with Caesar,

And I must pause till it come back to me.

(It’s more fun to watch Damian Lewis read this here.)

Caesar, we are also told by Antony, was not ambitious to a fault, as Brutus says in justifying his murder, but rather a man of the people who declined a royal crown three times. (And then when the will is read, we find out he left his fortune to the citizens of Rome.)

Similarly, Kratsios:

The foundation of these profound advancements was laid in the years following World War II, thanks largely to the vision set out by Vannevar Bush, the chief science advisor to Presidents Roosevelt and Truman. In his canonical 1945 report, Science: The Endless Frontier, Bush made the prescient case for federal support of basic research, laying the groundwork for the modern scientific enterprise.

Bush, we are told, is “prescient” and gets credit for our “profound advancements.”

And the crowd turns

Antony turns the crowd against the men who killed Caesar. After seeing how generous and approachable he is, surely, anyone who conspired to kill him should be scorned. Similarly, Kratsios is now going to tell us who killed Bush’s glorious vision. It is a combination of ideology infecting research, faculty sloppily creating an irreproducibility crisis, and incompetent universities failing to administer taxpayer funds efficiently.

Ideology (from the point of view of a scientist seeking to get a grant):

Our up-and-coming scientist is further discouraged by the corrosion of the merit principle that once made American research the envy of the world. What began with the NSF’s “broader impacts” criterion, which was a reasonable effort to ensure taxpayer funds benefit society, has evolved into a sweeping distortion of the selection process that reduces mobility for the best researchers. Between 2021 and 2024, the share of new NSF grants focusing on diversity, equity, and inclusion initiatives surged from fractions of a percent to more than a quarter. Our scientist has learned the new rules from principal investigators she has worked for, and is tempted to dress up her otherwise excellent technical proposals with ideological language to survive review.

Reproducibility:

The reproducibility crisis, particularly acute in the social sciences, has undermined future research and public confidence. In one study that attempted to reproduce 100 psychology studies, fewer than 40 succeeded. The challenge extends across other fields as well. In Alzheimer’s research, a celebrated 2009 paper in a top journal presented a promising path to treating the disease. By 2012, other researchers had demonstrated its irreproducibility, and internal reviews at the sponsoring pharmaceutical company terminated drug development based on its findings. Yet the paper accumulated more than 800 citations, misdirecting research priorities and federal funding for another decade. Its lead author became a university president, and the paper was only finally retracted 15 years after publication, shortly after the lead author resigned amid a broader investigation into data manipulation in his laboratories. The scientific review process worked, eventually, but far too late.

And for the laurel wreath on top, the incompetent, bloated universities:

Within academia itself, well-documented inefficiencies compound these structural problems. Administrative burdens on researchers have grown over time; one study found that investigators spend nearly half of their federally funded research time on paperwork rather than on research or teaching. Universities also extract significant overhead from researchers, which funds a mix of legitimate shared infrastructure and growing administrative bloat. Effective indirect cost rates at NIH-funded institutions average more than 40%, even though those same institutions frequently accept 10 to 15% overhead from private funders.

Which further harms our enterprising young scientist:

Senior investigators can delegate paperwork to postdoctoral researchers, but our scientist runs a small lab and has no one to delegate to. She writes grant applications using hours that should have gone to experiments or mentoring students. This is a tax on innovation that does not appear in the federal budget but costs the nation dearly in foregone breakthroughs. These burdens also create a perverse incentive structure, in which scientists who excel at research administration leapfrog those who excel at research performance. The weight falls heaviest on the scientists America needs most.

So, Kratsios has made his case that it is not he who is killing Vannevar Bush - quite the contrary, he is seeking to preserve Bush’s legacy. Rather it is the scientific ideologues, ambitious corner cutters, and lumbering bureaucracy. Assuming the crowd he is trying to turn are people generally disposed to agree with him in the first place, he likely achieves his goal.

We don’t know what Kratsios is thinking to himself as the report comes out. But we do know what Antony said as he walked off the stage:

"Now let it work. Mischief, thou art afoot,
Take thou what course thou wilt!"

So here we go. We’ll see what mischief comes from “Science: A New Golden Age.” We know what followed Antony’s speech was mostly unhappy (it is, after all, a tragedy).

And now, the battle for Ancient Rome

After Antony’s speech, the crowd turns against the conspirators and what follows is a battle for who will lead Rome. Similarly, Golden Age sets up a battle. In the 80 years since Endless Frontier was written, scholars and those shaping science policy have evolved Bush’s vision to one that seeks to include more participants in the scientific process and to center science in democracy, striving for greater benefit to society beyond Bush’s goals in human health and technology. They did good and sensible things like add “broader impacts” to the criteria for NSF grants and start fellowships to increase participation in science by groups that had been underrepresented. Will Kratsios cause America to revert to Bush’s instrumental vision?

The historian Audra Wolfe documented the political and nationalistic goals of Bush’s vision in her excellent book, Freedom’s Laboratory. Wolfe frames the birth of the American scientific enterprise as part of America’s effort in the Cold War, not solely as a quest for scientific knowledge and enlightenment. Endless Frontier is often remembered in policy circles solely as a charter for science as a public good. Wolfe reminds us it is more complex than that. She has said more recently about Bush:

The thing that too few people appreciate about Vannevar Bush is that he wasn’t just a scientific administrator par excellence; he was a consummate Cold Warrior who saw Communism, or even scientific planning, as an existential threat to the American way of life. In 1949, he published a bestselling book, Modern Arms and Free Men: A Discussion of the Role of Science in Preserving Democracy, a fever dream of freedom in the atomic age. You can get a flavor of this text from an excerpt published in the December 5, 1949, issue of Life magazine that ran under the title, “Science, Our Moral Armor.”

The uncomfortable fact for the DC policy crowd is that if Bush were here, he very likely would be persuaded by Kratsios’ argument for change. Bush saw science as a means to an end. He saw it that way because it was a stronger political argument. But he also almost certainly believed it in his bones. He was ardently anti-communist and pro-capitalism. There is a brief bit in Endless Frontier about the humanities and social sciences, but we know that wasn’t something he personally promoted. It’s arguable that he launched the neoliberal era for universities 35 years before “neoliberal” was a word. His idea was that scientists could do good and rely on the market to make it useful. Universities — and to a large extent the scientific enterprise itself — have been adhering to this idea ever since.

Whither Pax Romana

At the end of Julius Caesar, Antony has — at least temporarily — another change of heart. As he surveys the tragedy wrought by all of the parties trying to grab power — Caesar, the conspirators, and himself — he stands above Brutus’ dead body and proclaims him to be the most noble, because he truly wanted what was best for Ancient Rome:

This was the noblest Roman of them all.
All the conspirators save only he
Did that they did in envy of great Caesar.
He only in a general honest thought
And common good to all made one of them.
His life was gentle and the elements
So mixed in him that nature might stand up
And say to all the world “This was a man.”

Antony acknowledges that Brutus acted from true conviction and not a quest for power. So it is for those who have fought to make science better for, as Antony says, the “common good.” Unfortunately, it seems unlikely that Kratsios or anyone else in the administration will come to the same realization as Brutus in the next two years.

In the end, Vannevar Bush should be neither buried nor praised. Rather, like the American experiment itself, his vision should be continually improved. “Science: A New Golden Age” argues that the evolution towards a more inclusive and democratically engaged scientific enterprise should be reversed. Universities remain the right places to center this effort because they are still, for all their flaws, clumsily aspirational in a way that private entities cannot be. It will be a loss for science and the country if the center moves as Kratsios has framed.

Much ink has been spilled (including by me) in the last 18 months over how “trust in science” can be improved. The truth is that there is no problem with trust in science - it’s at 77 percent in the US, which is roughly where it was before the Trump administration. Would getting it to 87% be good? Sure, but it wouldn’t change anything. Because what has changed in the last few years isn’t trust in science, it’s that trust in higher education has cratered, now with 70% of America thinking universities are going in the wrong direction and only 38% having confidence. Working on trust in science and not trust in universities is like optimizing a chemical mechanism by failing to work on the rate-determining step. Until trust in universities is restored, university science will continue to be used as the lever to inflict pain, and doing so will be popular with 62% of Americans.

Yesterday, the Office of Science and Technology Policy issued a report calling for more research funding to move outside universities, leaving no doubt about the problem. Among other things the report calls for funding individual scientists rather than the institutions, making them into free agents who could take their money on demand. This would do to university science what NIL (name, image, likeness) has done to college athletics.

Subscribe now

As university scientists see their grants canceled, their graduate programs cut, and their foreign collaborations nixed, they all want to know what they can do. I try to tell them that just figuring out how to keep their science going is its own act of resistance. But many want more. Because their institutions are mostly trying to stay out of the glare of the administration, they’re not getting a lot of guidance at home. That is leaving a vacuum that could do long-term damage to university science in America and fail to build a broad enough coalition to fight back.

The clear vision of Colette Delawalla

One voice that has effectively stepped into this void is Colette Delawalla (profiled here in The Guardian), Founder and CEO of Stand Up for Science, which calls itself a “political activism organization.” While there are some things I disagree with in their rhetoric and actions, I’m grateful that SUFS is on the scene giving the legions of scientists wanting to take aggressive action a place to go. They’re also, as Tim Snyder told me, “moving the center of the conversation,” and that is particularly important in building a strong coalition to protect science as I will explain.

Here’s my recent conversation with Delawalla on the Science Podcast channel (go subscribe while you’re there):

Note: this was recorded before the period for comments on the OMB uniform guidance closed, and a number of universities made comments on the last day.

At the beginning of our conversation, Delawalla makes the case for why SUFS exists and how to get involved. Then we get into discussing some of the places where we agree and disagree.

The vision for SUFS is unapologetically activist and political. The remedies are action and protest — and rallying scientists to aggressively call out the anti-democratic actions of the administration that are circumventing scientific expertise and eliminating important lines of research based on arbitrary criteria (among other things).

Two of our disagreements

The most public disagreement I have had with Delawalla was over a column I wrote making the point that I use frequently on the road: continuing to do science is its own act of resistance. Delawalla agrees with that, but doesn’t find it sufficient. She feels every scientist who has tenure and is a citizen should be actively engaged in political action, namely partisan action against Trump. It’s our duty as scientists, she asserts, to protest in this environment. I don’t agree with it because I agree with Tim Snyder when he told me, “people are […] temperamentally different. And you can't expect everyone's going to react politically in the same way.” So, I think shaming scientists into participating is likely to cause more division. Better to support those who want to get involved and the organizations that are helping them. My goal is to get the activists and the establishment closer together, not farther apart.

Still, as Snyder also said, everyone should be doing something, and there have to be some troublemakers. And those of us who can should have the backs of the troublemakers, especially since they’re often doing much of the work of resistance for everyone else. I agree with both of those points. Delawalla wants to push it further, and she said so on Bluesky and on a Substack. I posted her critical Substack on my LinkedIn page and suggested people read it if they want to engage in the full conversation. Since then, we’ve had a good back and forth with each other about these things, and I think (hope) that comes through in the podcast.

Another place where we disagree is about the practice of SUFS to refer to the American moment as “fascist.” I’m not a historian so I’m very hesitant to use language like that, and at least for the moment, there are court rulings that are holding back some of the administration’s actions and there have been elections that the administration has lost where the winners were certified, so I’m more hopeful that we’re not there (at least not yet). But - as she does on other things - Delawalla makes a clear case for why she disagrees with me.

I don’t see anyone in the establishment getting on board with an organization that talks about fascism. Even Michael Roth, who is the most outspoken president for resistance, tends to stop at “authoritarian,” and most others are afraid even to say “anti-democratic.” Still, the presidents need to realize that voices like Delawalla’s are getting a lot of traction on their campus, and if it’s not already the case, it may soon be the case that SUFS and its siblings have more local influence than the presidents do, so ignoring the movement is not a winning strategy.

Building a coalition

I’m out today with a column about the need for the scientific establishment to make its case in a way that broadens the coalition. While there are some outspoken presidents who have put forth their ideas on how to address some of the ideological conflicts, many of these measures have alienated other university presidents and their own faculties, so while these efforts may make strategic sense for their institutions externally, they don’t appear to be growing the coalition within the local enterprise. And while there are questions about ideological conflict and free speech, they are no more important than questions about undergraduate access and affordability, which was called out specifically in a recent report about Yale. I talked with Beverly Gage who co-chaired the group that wrote the Yale report here.

The first step in building the coalition is to get away from the idea that it’s a zero-sum game as to whether university scientists are on the side of the activists or the administrators. As long as it’s either-or, the people attacking universities are getting what they want: the universities are fighting a two-front war. That means that administrators shouldn’t write off — or worse try to squelch — entities like SUFS and the Union of Concerned Scientists or politically active faculty members like Michael Mann and Peter Hotez. These voices are getting more and more traction, and their followings are building. Conversely, the political activists should be open to ideas from their presidents not about how to best appease the Trump administration but how a consilient vision can allow the campus to work together. Right now, that vision is not being articulated, so it’s hard to blame the activists for not engaging. ('You don’t get to be woke anymore, get with the program’ is not a consilient vision.) The activists should also be open to hearing from folks who think the current situation is just fine; here I talked to Pri Narang who is a conservative scientist who is a fan of what the administration is doing, and I think our colleagues are perfectly capable of deciding whether they agree with her views. Also, here’s Roger Pielke’s case against any kind of partisan advocacy.

It’s easy for university administrators to say, “well, there’s always 10% of the faculty that will never be on board with anything.” That is true, and I’ve said it myself, but I always worked hard on winning over the persuadable portion - and that group often included many folks in science departments. (Whenever I was worried about losing a vote, I used to say “well, I guess it’s time to go whip the votes in chemistry and biology.”) The problem now is that the 10% who will always be in opposition is growing, and that group is attracting a lot of scientists who were previously on the sidelines. If it gets big enough, it will be hard to ever put the pieces back together.

A year ago, I wrote a piece about this that was titled “Come together, right now,” making the case that everyone should be united in fighting the attempts to undermine science. It was widely circulated but didn’t inspire much action. If I wrote one now, it would have to be called “Another Brick in the Wall.” (Sorry if you’re in the wrong demographic to get that, but it means the campuses and the presidents are isolating each other.) I sympathize with the administrators: I have been there in these roles and know how hard this is. It’s much harder now than anything I ever faced. But it’s time to start working the phones and pull the campuses together. The divide and conquer strategy being used by the Trump administration is working and it’s not sustainable.

As Pink says in “Hey You”: together we stand, divided we fall (we fall, we fall, we fall…).

Subscribe now

This is a bleak view, I admit, and it could improve. But here I lay out a little more of the evidence for why I’m pessimistic, at least at the moment.

Errors are inevitable and likely here for the duration

In a recent column, Zeynep Tufecki makes the case that the errors produced by LLMs are unlikely to be completely eliminated based on the way the models predict the next word rather than actually reasoning. Tufecki is mainly addressing the idea that AI is going to immediately replace human jobs by making the case that as long as LLMs continue to make errors, humans will be needed to check them. And these errors are here to stay:

Large language models are not reasoning machines. They’re plausibility engines. It’s not just that they don’t test their outputs to make sure they’re correct or logical, or that they fail to do so in certain instances. They can’t, and they’ll never be able to on their own. They can only assess which answers are probable, based on the data on which the models have been trained. And that holds true whether they’re trained on the full breadth of human output or only on peer-reviewed scientific articles. It’s baked into the way they operate.

For the condensed version: just see the bottom of the ChatGPT web interface:

Tufecki also argues that new advances in LLMs will not eliminate these errors, because it’s impossible to imagine every constraint that needs to be placed on the paths it can go down and because its need to keep users engaged by flattering us — detailed here in a recent Science paper — also encourages wrong answers.

Plenty of other researchers are no doubt more optimistic, time will tell. But at the moment, there’s no question that the errors abound.

AI is definitely boosting volume

Science and our journals can anecdotally point out that our submissions are way up and that they have lots of AI errors. For a more scholarly take, a recent study by James Evans and co-workers shows that researchers who use AI publish more papers than those who don’t and get more citations. However, Evans observes that these well circulated papers don’t necessarily drive science in new directions:

On average, AI adoption helps individual scientists publish 3.02 times more papers, receive 4.84 times more citations and become team leaders 1.37 years sooner. This probably results from improved modelling and prediction of field-specific data, resulting in higher performance on recognized benchmarks. The substantial academic benefits of AI use may be a driving force behind its accelerated rate of adoption; however, we also find unintended consequences from the increased prevalence of AI-augmented research. In all fields, AI-augmented research focuses on a narrower scope of scientific topics and reduces the scientific engagement of follow-on research, leading to more overlapping research work that slows the expansion of knowledge. Further, with a greater concentration of collective attention to the same AI papers, the adoption of AI seems to induce authors to converge on the same solutions to known problems rather than create new ones.

One critic of Evans’ work on this pointed out that he used AI to determine that the expansion of knowledge will be slowed, but we’ll eventually know whether they got the right answer.

And while AI will help you write up these findings, there is at least some evidence that the agents that do this will be more likely to produce findings that might not actually be in the data through things like p-hacking.

For now, we’re struggling

Maybe all of this will be improved. But for now, we know that we’re getting more papers with more errors that require more human effort to get them prepped to be in the literature accurately. This is occurring while the tech bros are all over YouTube telling us that we’re going to live in radical abundance with lots of free time on our hands.

If only.

Still, ChatGPT did make this awesome graphic for me for this article:

Be careful out there.

Subscribe now

But as I dug into talking to experts on leadership, including Tim Vogus, Ludmila Praslova, and Brené Brown, I found out that we don’t just need to help people with autistic traits who find themselves in positions of leadership do better, we actually need to get more autistic leaders in place, because the old ideas of leadership are breaking. The uncertainties of artificial intelligence, climate change, and political turmoil require detailed analysis, intense focus, task performance, and straight talk, not winging it, multitasking, vague instructions, and spin. Yet, we still emphasize in leadership literature and in our selections for leaders things like extroversion, “executive presence,” making people feel comfortable even when they shouldn’t be, and a popularized vision of emotional intelligence that is nothing like what its original proposers envisioned.

Each of the leadership experts I talked to had a connection to autism: Praslova is autistic herself, Vogus has an autistic child with high support needs, and Brown has coached many leaders with autistic traits. Brené said it well when she told me: “What leaders with autistic traits have to offer… is what makes organizations work and will even have a more profound presence in the future.” Amen.

What all of them said - and so did Peter Salovey, long-time Yale president and original proposer of emotional intelligence - is that we don’t need alpha dogs barking orders and selling a dream, we need careful thinkers who tell it like it is. So, a big part of the book will make that case. The original parts are there, too, so there is a lot of information, as you would expect from an autistic writer! The marketing copy that we’ll be using is below. Look for us coming from Harper Influence in 2027!

Leading with Autism: Harnessing the Natural Strength of Neurodivergent Minds

In this groundbreaking new playbook, the Editor-in-Chief of Science offers a radically different vision of what great leadership looks like today, revealing how autistic thinking—long misunderstood—can lead to stronger, more innovative leadership in our increasingly complex world.

We’ve heard it all before. Firm handshakes. Strong eye contact. Charisma. “Working the room.” For too long, these traits have been heralded as the hallmarks of a good leader. But in today’s rapidly changing world, checking the boxes no longer equals success. In Leading with Autism, Holden Thorp, the Editor-in-Chief of the Science journals, makes a bold new case: autistic thinking, long misunderstood and underestimated, can offer a powerful new framework for reimagining leadership in today’s world.

After receiving an unexpected autism diagnosis at the age of 53, Thorp began to see his mind and his career through an entirely new lens. As he set out to learn everything he could about his diagnosis and the concept of neurodiversity, he discovered that autistic traits so often labeled as “challenges”—like unshakable concentration, reliance on strict routines, or overly-direct communication—were not liabilities or deficits, but strategic, unique strengths he could leverage to sharpen his decision-making, clarify priorities, and build greater trust with colleagues.

Drawing on the latest cutting-edge scientific research, candid reflections from his own career, and interviews with experts across industries, including Brené Brown, Simon Baron-Cohen, Temple Grandin, Agnes Callard, Leland Vittert, and more, Thorp empowers readers to tap into the power of neurodivergent thinking.

In Leading with Autism, learn how:

• Your obsessions or fixated interests translate to an extraordinary ability to focus

• Your low tolerance for uncertainty means you’ll plan for every contingency before anyone else sees it coming

• Your tendency to “take everything literally” pushes you to provide clear and direct instructions people can actually follow

No matter where you fall on the spectrum, Leading with Autism shows how championing autistic thinking results in a new kind of leadership—one capable of taking us to new places, solving problems in bold new ways, and unlocking potential across every team, company, and community.

A New Leadership Playbook for Uncertain Times: Explains why traditional leadership models are failing and how autistic thinking offers a powerful, innovative framework for navigating change and complexity.

Reframing “Challenges” as Strategic Strengths: Learn how traits like intense focus, structured thinking, and direct communication can sharpen decision-making, build trust, and clarify priorities.

Backed by Science and Real-World Insight: Combines cutting-edge research with personal experience and interviews with leading voices like Temple Grandin and Agnes Callard.

Practical Tools for Everyday Leadership: Each chapter breaks down a key autistic trait, explains the science behind it, and provides actionable strategies for applying it in professional settings.

Unlock Focus, Innovation, and Purpose Across Teams: See how autistic leadership strengths—pattern recognition, clarity, and determination—can drive stronger collaboration and results.

For Leaders of All Thinking Styles: Gain fresh perspective and actionable ideas to lead with greater authenticity, effectiveness, and impact, whether or not you identify as autistic.

All three of our sections have a similar theme, which is that we can acknowledge the problems with America and science while simultaneously believing both can be better and work towards improvement. Our books editor, Valerie Thompson, sums it up well in her introduction:

“There are three kinds of patriots, two bad, one good,” wrote the late American clergyman William Sloane Coffin in his 2004 book, Credo. “The bad ones are the uncritical lovers and the loveless critics. Good patriots carry on a lover’s quarrel with their country.” On the eve of the US semiquincentennial, the same framework feels useful for reflecting on the first 250 years of the country’s scientific enterprise. Science lovers will rightly note that American research has yielded magnificent benefits and greatly expanded the boundaries of human knowledge. Critics will correctly insist that it has wrought immense suffering and unimaginable destruction. Like Coffin’s quarreling lovers, the scholars below do not shy away from grappling with paradoxes in US science history, confronting the complexities of six notable moments: the Manhattan Project, the unrecognized contributions of enslaved people to early agricultural knowledge, the rise of Silicon Valley, the advent of biotechnology, the eugenics movement, and the space program. In doing so, they invite science lovers, critics, and everyone in between to contemplate the past and future of the US scientific enterprise and related questions about democracy, representation, and state support for research.

Subscribe now

Two vantage points on our current situation

I was very eager to talk to Gage. Her book is unflinching in its critique of America’s failures, especially on slavery and the degree to which the Industrial Revolution produced benefits for some but made life worse for others. She traveled to Los Alamos and chronicled how science became permanently intertwined with American foreign policy through the leaders of the Manhattan project.

As Rose Horowitch recently explained in The Atlantic, it has been a season for writing reports on why trust in higher education is down. The simplest stat is that 70% of Americans think higher education is going in the wrong direction. While trust in science is high still at 77%, building support for science will always be hampered if the universities where the research is done remain unpopular.

Our colleagues at Nature just ran a whole package on this problem that is worth reading. A quote from Kathleen Hall Jamieson there sums it up well:

Kathleen Hall Jamieson, a specialist in science communication at the University of Pennsylvania in Philadelphia, argues that there’s “a rhetoric of science in crisis”, rather than an actual one — and that it’s nothing new.

Trust in science remains high, but it is being punished because trust in higher education is low. So the collapse in trust of higher education is not something science can ignore.

Other reports getting a lot of attention are one by the higher ed association AAC&U, which speaks very generally to institutions across the country and another on the humanities that was commissioned by WashU and Vanderbilt but written by faculty from other institutions. The WashU/Vanderbilt was pitched to presidents and chancellors across the country but has come under fire for not involving local faculty on local problems. Gage’s report is on Yale by Yale faculty, which I think is the strongest stance. Because it’s from Yale, it is getting a lot of attention across the country, but the report is clear that it’s a local report, saying, “We focused our recommendations on Yale, but we hope they may prove useful to others in higher education as well.”

I was anxious to talk to Gage about how her two vantage points on American history and role of higher education come together. Here’s the video of our full conversation:

Simplifying the mission

As I said in the editorial, Gage found in her work on both documents that although she found much disappointment with the failure of America and its institutions to keep their promises, she still found much hope that improvement was possible. About America, she said about its more disaffected citizens that ‘They have big hopes and dreams for it still.” Similarly, about higher education, she says, “what people want is for the institutions to live up to their highest purposes and best aspirations.”

The Yale report’s idea for solving the higher ed trust problem is for universities to recognize that trust is earned by keeping commitments and that institutions make a lot of contradictory promises: “Trust is earned by doing what you say you’re going to do—and, ideally, doing it well. In recent years, however, universities have been expected to be all things to all people: selective but inclusive, affordable but luxurious, meritocratic but equitable.” Gage summarized this for me by saying, “Universities are in the business of promising lots of things to lots of different constituencies.” From my experience, the reason for this is the constant scramble for resources; institutions need financial and political support from everyone if they want to continue to grow. “It mostly seems to be producing mixed expectations,” says Gage, “confusion, and battles over what's really going on.”

The Yale report’s prescriptions for these problems fall into three categories. The first two are uncontroversial. The way in which pricing for undergraduates is set by institutions where there is a high sticker price that is discounted creates confusion about what folks are paying. And the mystery surrounding who gets in and for what reason breeds mistrust. Hard to argue with either of these. But the third is where it gets dicey: Gage’s committee suggests that the university should simplify its mission statement to focus on the discovery and transmission of knowledge, something that universities do that is not done by other institutions. Yale followed suit and updated its mission statement accordingly. I applaud them for this: Committing to generate and distribute knowledge is a promise we can keep for everyone. All others are likely to be in the eye of the beholder.

A valid criticism of this change is that it sells short the role that universities can have in applying the knowledge they generate. For sure, it is often the case that the institution plays a vital role. And it sells short the impact that the knowledge can have. Again, scientific knowledge absolutely has solved key problems, many of them - such as human health and defense technology - being very popular politically. But the realization of these promises when they happen benefits people unevenly. New technologies drive the economy, but often — because of the inability of governments to implement equitable social policy — these advances benefit the wealthy and make life worse for those with fewer resources. We are already experiencing this with artificial intelligence and it will likely get worse.

Perspective of a blue and red-state public chancellor

When I became chancellor of UNC 18 (!) years ago, North Carolina was a blue state across the governor and both houses of the legislature. Halfway through my 5 years, it flipped the other way. So I have been a public chancellor in both a blue and a red state. I always say that when the Democrats are in charge, they want you to do things (mostly social programs) that you often don’t really want to do, and when the Republicans are in charge, they want you to stop doing things that you do want to do. The former is easier, but neither are as neutral as they are portrayed.

When I accepted the job, I promised to solve health care access, K-12, climate change, world hunger, and wars. It’s cringy for me to read it now, but here it is:

Our to-do list is nothing less than the greatest problems of our time: Cure diseases, and get those cures to all the people who need them. Find and invent clean energy. Inspire students in our public schools. Feed seven billion people. Describe the world, and replace conflict with understanding.

The message worked like a charm in then-blue North Carolina, but news flash: I didn’t deliver on all this. And the right criticized me for saying we could be all things to all people if we could just get enough tax money, and the left criticized me every time I fell short of this expansive vision. When the politics changed in 2010, it was impossible to walk all that back.

What we did do is produce knowledge that absolutely did change the world and people who went on to do great things. If I’d promised that, I could have said that we delivered in every respect.

So what now?

When these topics get discussed among presidents and others, someone eventually says, “We just need to tell our story better.” I must have said it hundreds of times when I was a president, especially when alumni would say “Why don’t I hear about all the great things going on at the university?” I knew then but am even more aware now that we were telling our story just fine. We sent out press releases, put stuff on our website, started out on social media, etc. But good news that doesn’t affect people’s everyday lives doesn’t go anywhere. And most of the press releases we sent out were overstated - saying that this particular discovery would eventually cure a diseases or create some innovation - when most of the time we knew that was an incredible long shot. It’s easy to read those press releases and then wonder where your cancer cure is.

I think Gage’s message is that maybe we ought to do a worse job telling our story while we do a better job getting others to tell it for us. When we say the knowledge and people we generate are going to change the world, it comes across as self-serving, particularly if we promise something that is far-fetched. Better to focus on how we excel at research and teaching, while other people talk about the impact of our knowledge and people. It’s at least something new to try.

If we keep promising things we can do and deliver on those, that’s a way to earn back trust. It may be the only way.

I also talked to Stanford researcher Hunt Allcott who was a coauthor on the Yondr study. Allcott has done extensive work on social media and mental well being. He has a more upbeat perspective about the data perhaps than Dee and is refreshingly rigorous in how he discusses results.

Here’s my full Q&A with Hunt.

Subscribe now

Holden Thorp:

So I’m interested in this because for one thing it’s an interesting topic. For another, you and I have a connection about this because Science published the papers from the big Facebook project that you were part of. That was complicated, and we spent a lot of time trying to figure out how we were going to disclose the right data to support that study. And I think the other thing is that I’m very interested in what happens when somebody with a huge platform puts a stake in the ground about how things are going to come out and then lots of researchers have to deal with that. And there are many examples in history where that hasn’t worked out so great. So I’ve been following this since Haidt’s book (The Anxious Generation) first came out, which is around the same time we did that Facebook project of yours.

I want to start with this Yondr study and then I’m going to ask you about some of the other social media stuff you’ve done if that’s okay. I guess my first question would be tell me what you think the positive findings are in your latest NBER preprint.

Hunt Allcott:

So as you know, we study the effects of schools adopting Yondr smartphone pouches in middle and high schools in the US. And I think maybe broadly the online positive conclusion is that the effects of smartphone pouches on student outcomes are broadly small.

Holden Thorp:

And were you expecting them to be bigger?

Hunt Allcott:

Was I expecting the effects to be bigger? It’s a good question. I’m not sure that I had a strong prior. I do think my own personal experience as an educator — I teach class like other university professors and I try to keep my students from using their smartphones in class because I think it’s a distraction — so my intuition was and frankly continues to be that getting digital distractions out of students’ hands will improve their academic outcomes at least somewhat. So then you might ask, how does that prior interface with the results of our study? My own personal view is that there are perhaps two ways to adjust my prior. One is just that maybe there are not very large impacts on student achievement. Let me say it this way. Perhaps the effects on student achievement are not as large as many educators would predict. The second potential conclusion is that there are other dimensions of student academic achievement other than standardized test scores, which we do not measure in our paper.

Holden Thorp:

Right. Tom Dee told me he thought the results were sobering. Would you agree with that?

Hunt Allcott:

I don’t agree or disagree. I think it just is a matter of how you define the word sobering. I think my previous statement about my priors and the interpretation probably covers my reaction and probably would interface with Tom’s comment.

Holden Thorp:

Okay. And so there have been a lot of studies on time spent on social media that you and others have done that also produced very small effects, sometimes not a lot at all and sometimes something. Do you think this finding is consistent with all of those?

Hunt Allcott:

I think there is a collage of different types of evidence that we need to put together on effects and effects sizes on the different possible outcomes that social media and smartphones might affect. So I’m not sure that I would say these results are inconsistent or consistent with our previous results writ large, but I would say they’re all part of a collage of evidence. The reason I’m sort of saying it this way is that we’ve done randomized experiments, which have a four to six week time horizon, four to six week treatment period, and that has the clearest causal interpretation because it’s a randomized experiment, but the duration of the treatment is very small. And so what I’d really like to know is if you get kids to stop using their smartphones or social media for six years as individuals or maybe as entire social groups of students, how does that impact outcomes? And the school level treatment is longer term. It’s probably not as 100% effective as our deactivating Facebook experimental treatment and its group level. And so because it differs in these different ways, I’m just not sure that I could line up all the effect sizes and say that something is consistent or inconsistent.

Holden Thorp:

Well, let me ask it more directly. Haidt has said that the phone-based childhood is the major cause of the adolescent mental health crisis. Do you agree with that?

Hunt Allcott:

I don’t know.

Broadly, I found his book to be a valuable addition to the discussion and broadly a thoughtful book. And in my interactions with Jon and his team, outside of my reading his book, I have found them to be thoughtful and careful and I’ve learned a lot from the work that they’re doing. Would I myself be positioned to have written that book? I don’t think so, at least not based on my own personal work. I think our work, some of the stuff that you’ve edited and then some of the other work we’ve done, including the paper that came out last week, I think is particularly valuable at providing causal evidence on the effects of smartphones and social media. I don’t think you could go from that alone to Jon’s book, but that’s the value of what Jon is trying to do. He’s trying to combine the causal evidence that our group has provided with many other types of evidence to try to paint a picture of what’s happening and that’s what I think is valuable.

Holden Thorp:

But do you think that the success of it has made it harder for the public to see all these nuances, including the ones that you’re working on that underlie this whole thing?

Hunt Allcott:

I don’t know if there has been a causal effect of the book on the reception of our work other than to say I do think that Jon has drawn a lot of attention to the issue of smartphones and social media. And so actually I think it’s possible that the paper that we released last week got more attention because Jon has drawn attention to this issue broadly than it would have without Jon’s efforts.

Holden Thorp:

I’m confident that’s true.

Hunt Allot:

I didn’t follow all the details of how every individual is responding to the paper that we released, but I found the media articles that I have read in The New York Times and others to be broadly thoughtful and reasonable in presenting the work that we did. So I have no evidence that Jon’s book has made it harder for society to see opposing views or to see evidence that smartphone bans in schools, for example, don’t have the large effects that some might hope.

Holden Thorp:

But I guess my question is if social media and phones are as powerful as, and it’s not just him but lots of people who agree with him, are saying in terms of the mental health consequences that they might have, I mean, shouldn’t we be as scrupulously careful with our messaging as we can possibly be?

Hunt Allcott:

Your question is, if smartphones are as bad as Jonathan thinks, shouldn’t we be scrupulous in the messaging?

Holden Thorp:

Yeah. I mean shouldn’t we go out of our way not to overstate what the effect is until we have a lot of science? I mean, his answer would be that the problem is so bad that — and he told me this on the record when I first started writing about this — that the problem is so bad that it’s a good idea to go ahead and launch this movement before all these academics have done all their causal analysis and everything because it’s hurting people and [he’s] stopping that from happening. And the harm of doing that is not significant. Now, there are plenty of people who say that there are harms from creating this massive movement. And I guess my interest in the harm of that is that if a lot of people are convinced that this is the main reason why we’re having an adolescent mental health crisis where test scores have been going down, and then they see your study and there’s no effect — and I get there’s tons of caveats there and that you would prefer to do your study many, many different times and have all kinds of different permutations to it — but if the public starts seeing, “Oh, we did this and there wasn’t any effect,” I mean, isn’t that going to make the public less trustful of science in this area to begin with?

Hunt Allcott:

I see. Your worry is that a scientist like Jon getting out in front of scientists without the most ironclad evidence will discredit science if it turns out that he’s wrong.

Holden Thorp:

Yeah. Or even if it turns out that he’s 80% right, but the other 20% is really important.

Hunt Allcott:

I guess here would be my view. I think that something is going wrong in Western societies as measured by surveys of subjective wellbeing and some other outcomes. It’s important that we figure out what’s causing that decline and generally that we do everything we can to help our societies do better. And I think smartphones and social media have been one of the leading hypotheses. I don’t think that we will soon reach a point where we have ironclad causal evidence on what’s causing the degradation in mental health. And so in the absence of ironclad causal evidence, it may be beneficial for us as a society to act on our best guess as to what should be done, maximizing expected net benefits. And I think Jon would say, if I’m right that smartphones and social media are quite harmful, then banning them, effectively reducing their use and being smarter about their use, will have potentially large benefits.

If he’s wrong, then we kind of go back to where we were 20 years ago before this stuff when I grew up, and it’s true that we would miss entertainment value. If you restrict the use of something that is useful or entertaining, that’s bad for its users, its consumers.

So being wrong entails a loss, making policies to restrict smartphones and social media if Jon is wrong will entail a loss, but I think a policymaker should act to maximize expected net benefits. And I do think that Jon’s book has helped to clarify many people’s thinking and make a strong argument. And so I think that’s been useful in helping us all evaluate the issues.

Holden Thorp:

Yeah. Okay, good.

Hunt Allcott:

Maybe it’d be useful if I explain why I think the causal evidence is going to be hard.

So I think our randomized experiments (such as those in Science) were awesome, but as I said, they’re individual level and they’re only a few weeks long and so far they’ve been only in the context of elections and so you have these questions as to like what are the effects on wellbeing generalized to longer periods to group level deactivation to 2026 outside of an election. Some of those questions I think I’m more concerned about than others, but there are questions. Then you have our school smartphone study that we released last week, but that’s like a ban on smartphones during school and it’s not all the outcomes, as I said earlier, there may be other student outcomes we’re not measuring. And the teachers love it. Honestly, if I were the principal at a school, I would buy Yonder pouches.

Holden Thorp:

Yeah, I probably would too. I mean, I don’t like it when devices are in my classroom.

Hunt Allcott:

So that’s a case where I wrote a paper saying the effects of Yondr pouches seem to be small, but yet I would still, as the decision maker, implement some phone ban policy such as Yondr pouches.

Holden Thorp:

Yeah. I mean, you’re probably telling people not to pull their devices out in your classroom, right? So you’re effectively doing the same thing.

Hunt Allcott:

Exactly. And especially at the K-12 level, the teachers would probably rather not have to enforce that class by class as opposed to like a school-wide programmatic ban or a Yondr pouch type enforcement mechanism.

The next round of causal evidence that will come through is on the youth social media bans, but I don’t think that’s going to answer all the questions. I think in Australia there’s a huge amount of non-compliance with the ban, people evading the ban. I’m excited to do work in other countries to see how that plays out.

So I think we’ll learn some things there, but then we’ll only learn what are the impacts on 15 year olds on certain outcomes that are measured. And in any non-randomized study, there’s always a concern about confounding. And so I think we’re going to continue to get a collage of evidence and it’s going to get a little bit stronger over the next couple of years, but we’re also going to have to rely on other forms of evidence other than just large scale causal studies done by mostly economists trained in causal inference techniques.

We’re going to have to rely on what teachers say in surveys about whether they like smartphone bans in schools. We’re going to have to rely on what parents see in their families with their own eyes or what they tell us on surveys when they get smartphones out of the hands of their kids and make their kids wait until eighth grade or later to get smartphones. We’re going to have to rely on all sorts of non-causal evidence. I think that’s part, again, of the value I see in Jon’s book is providing this collage of evidence, and I think we’re going to need all of it because the causal evidence is never going to be perfect.

Holden Thorp:

So let me ask a very specific question about social media. So you’ve studied Facebook and Instagram, but what about YouTube? I mean, isn’t YouTube the most popular social media now and how do you ban that without stopping people from watching? And I’m disclosing that I’m on the board, but what about PBS Kids or other TV kind of material that would be good for them to watch?

Hunt Allcott:

I think this is such a hard question. I think that we probably need a partnership among governments and firms and civil society to have effective change here. Because what a government could do is a government could say, “Well, we require age verification subject to some safe harbor provisions for certain set of apps.” And then while you get other apps and you get kids circumventing whatever age verification is safe harbored. And so at this point, I kind of wonder if you really need the platforms to want to do as much as they can and you see things like child accounts and I think if those were designed well, you could imagine YouTube having a child account that allows PBS and doesn’t allow less good content and has a screen time limit on it.

You probably need the smartphone makers themselves to make truly good parental controls and screen time controls so that I can’t just spend an hour on YouTube and then spend my hour on youth Instagram and then spend my hour on youth TikTok and still add up to a lot of hours staring at my screen. I think it’s going to take a collage of work and I think some of it’s going to be very hard to regulate. It’s going to have to be that the platforms are going to want to help probably even more than they already do.

Holden Thorp:

But how does YouTube separate YouTube shorts from, I realize this is a very specific question, from long form video that for kids who don’t have resources and are at home might be good things for them to be watching.

Hunt Allcott:

I think it can’t be done perfectly, but I’m pretty confident that a combination of the bright folks at technology companies could figure out the right way to make sure that kids don’t spend too much time on any individual device and that the content on that device is broadly appropriate. So I think YouTube specifically could create a child account that the smart folks at YouTube have decided is appropriate and I don’t know, they may have already, you probably know better than I do and if you don’t like exactly how that’s been done, I’m sure it could be tweaked. So that’s for the content. And then the overall time I think requires a combination of parents monitoring their kids and the device makers, Apple and Android, making even more useful screen time controls so that parents can say like, “Oh, you spent your hour on YouTube and that’s it, “ as opposed to you got your hour on YouTube and an hour on all of the other apps that are trying to limit themselves individually.

Holden Thorp:

This is great stuff and I appreciate it. The point of my article is just to explain that there’s all these subtleties here. I believe most people in the zeitgeist think the phones and social media are evil and we already know that and it’s scientifically proven, but the science here is much muddier and more complicated than that.

Hunt Allcott:

I agree the science is uncertain. Can I share two other things before we have to go?

Holden Thorp:

Sure,

Hunt Allcott:

One is I have been impressed with Jon Haidt. I mentioned this earlier. I’ve been impressed with Haidt’s interest in the details and the science. So I don’t know if Tom told you this, but after we released our paper, Jon and his collaborator, Zach Rausch sent us a detailed email with a bunch of questions.

Holden Thorp:

Yeah, I bet they did.

Hunt Allcott:

And they were good questions and we’ve answered some of them. We’re working on some others and he’s read our paper as carefully as anybody I’m aware of outside of our collaborator team. I really appreciated that and I appreciated the good questions they’re asking and his famous wiki of evidence about smartphones and social media, I guess there’s several wikis now. I think that’s another example. So I’ve been impressed by the thoughtfulness and then as I’ve read his blog, I’ve been impressed by the thoughtfulness and like what if I’m wrong and this sort of stuff.

So that’s one thing. The other is that I have read many papers and meta analyses of papers that confuse correlation with causation, specifically with regard to social media use and mental health. So obviously as you know well, if you run a survey and correlate people’s social media use with mental health, it could be that more social media use causes worse mental health. It could be that worse mental health causes people to retreat into social media. So there could be reverse causality. It could be there’s some third factor that causes both. It could be that the causation is positive and the other correlations are negative. And so you get a negative correlation, even though the causal effect is positive, you could get any configuration of results. And basically the correlation studies about social media and mental health and in many other areas really tell us very little, perhaps nothing.

And so I have read some number of meta analyses that say, “Oh, well, we’re not sure because there are 73 correlation studies that show positive correlation and 54 studies that show negative correlation. And so who knows?” And my answer would be, who knows, but not because the studies go in different direction, but because they’re correlation studies to start. And so that’s why we did this work on the causal stuff. So I would encourage people to discount statements that are made on the basis of correlation studies in this space. And many of the statements that I’ve made based on correlation studies are statements such as we just don’t know because the correlations go in all directions.

In social science, all relationships are noisy because there’s like a ton of things that cause people’s mental health, their background, how they’re doing from day to day, like whether they got hurt playing basketball and possibly how much social media they use. And what we really care about here is acknowledging that there are many things that cause mental health to get better or worse. What is the slope of any relationship? And so that’s what our causal studies, our randomized experiments identify. So I think you might just, as you read some of the stuff out there outside the economics space, you might just be on the lookout for that confusion.

I’m out with a column yesterday on the disastrous proposal from the Office of Management and Budget to create regulations that would cripple American science through a lot of things but the two biggest are: (a) mandating political review of every individual grant, and (b) obtaining case-by-case political approval of all grants that involve spending money in other countries. There are plenty of other terrible things summarized well here. This basically eliminates the authority of the NIH and NSF Directors and puts everything in the hands of HHS. It’s hard to see how anyone would want to run these agencies now or in the future under these rules. These rules essentially amount to making the entire enterprise live by the “compact for higher education” that everyone rejected.

Subscribe now

The architect of it all is, of course, Russell Vought, who has dubbed himself the “keeper of the commander’s intent.” He is subverting the bipartisan budget and sidelining scientific merit as a criterion for American science. My last paragraph of the piece (which is called “Another red alert for American science”) is:

The time to act is now. The scientific community needs to flood OMB with responses during the public comment period, open until 13 July. Universities and associations must speak out as a united front to mobilize Congress and be ready to file lawsuits once the regulations are finalized. I was sympathetic to members of the scientific establishment who played it carefully during last year’s budget negotiations. Getting the budget deal done was crucial. But that was then. The red light is now flashing. All hands, report to stations.

Of course, I wasn’t going to leave a Star Trek analogy on the table.

Why there’s no reason to hold back at this point

I’m saying “all hands” because I believe this is the event that has evaporated any plausible reason not to speak out. I have been sympathetic to folks who stayed out of the fray building up to last years’ budget process, because the Republican Senators who supported the restoration of the science budgets last year needed to be as clear as possible to do the right thing. (Not all my friends were thrilled with this.) But it takes a village, and the village got the budget deal done.

This is different because the very people and process that led to that bill are now being undermined by the OMB regulations. Of course, strong advocates like the Union of Concerned Scientists and Stand Up for Science will and are speaking out. They were already doing that. But this event should make the tent a lot larger, because speaking out against these regulations is speaking out in favor of a bipartisan effort, mainly one driven by Republicans. For those who believe in the model, long practiced by science lobbyists from associations and universities, of seeking science funding through bipartisan lobbying (and not everyone does), these regulations would render that process meaningless because the White House can undo whatever it wants.

Another reason is that these regulations will (if history is any guide) one day be in the hands of a Democratic White House. If you think they would just put them back to the way they were, you’re an optimist. Why wouldn’t a Democratic President want this kind of power? You could imagine all kinds of new programs enforced that fit the policy agenda of the administration. While I might agree with some of these initiatives, I think it would be just as bad. It would weaken the principle that science, not politics, should be the driver, and we could end up with massive whiplash as programs are stopped and started every time the administration changes, leaving past work unfinished and making it impossible to build a scientific workforce. So if you want to stop so-called “woke science,” this is a way you could actually have more of it one day.

To battle stations

You can submit comments to the OMB regs, and Stand Up for Science has made it easy at this link. The comments can be ignored, but stirring up as much activity as possible is crucial at this point. Flood the zone. Call House and Senate members, especially if you live in a red state (remember not to call members outside your state - that makes things worse). And press universities and associations not to hold back. (You don’t need to press AAAS, because they’re already on board.) There’s no reason to hold back now. If they want to use the methods they’ve developed over the last 80 years to drive science funding, they have to put it all on the table to fight this.

Let’s go.

Subscribe now

Dee was extremely careful and has a very moderate position on all of this (and in political matters related to education). He said the results were sobering, did not realize the hypothesized benefits, and could be characterized as disappointing, but he still wants to continue to work to see if phone bans can improve learning outcomes and had ideas for how that could happen. When I asked him if phones were the major reason why school performance has deteriorated, he said we need to learn more, but he offered two reasonable alternatives: the introduction and then abrupt roll back of test-based accountability through No Child Left Behind, and the Great Recession. He also said about the extent to which phones and social media explain everything, “I’m always concerned about overly facile or monocausal explanations for any large scale social phenomenon.”

Read the whole thing if you’re interested in this topic. There’s a lot of other great stuff in here. He started off by lamenting with me the fact that fads come along in education and then are stopped before the data says whether they work or not. Here’s our conversation:

Tom Dee: A concern of mine over my career is just some degree of frustration with the mercurial faddishness we have where there’s a kind of flavor of the day reform that attracts a lot of attention. Then there’s sometimes uneven implementation and then some results that are discouraging and then people simply move on instead of engaging in a kind of focused intentional cycle of inquiry, trying to refine a certain reform or understand what other contextual factors might need to be in place. I would argue this happened with No Child Left Behind, where we had test-based accountability brought to scale nationwide and we stayed with that 1.0 moment for a long time because even while people recognized the need to reform it in some ways, because it became politicized. And by the time the Obama administration came around, they were issuing waivers against the strictures under No Child Left Behind and it wasn’t until 2015 that that federal legislation got reauthorized and essentially devolved accountability to the states.

I’ve also worked on the controversial teacher performance assessment reforms in Washington DC introduced by Michelle Rhee. They were actually well-designed, well-implemented and pretty effective and other places tried to replicate that without the same political will, without the organizational acumen and produced disappointing results. It led to this narrative of “this doesn’t work” rather than “we need to know more about what’s going on in these settings to understand how to replicate and scale best practice.” That’s just a little bit of background that informs what I’m bringing to the table in interpreting these results.

But let me back up a little bit about this study. At a high level, I think three distinctive features make it unique. One, to our knowledge, it’s the first study of phone bans in the US that’s national in scope. Two, part of what made this research design appealing to us was our capacity to measure phone bans that were actually restricting the in-school use of phones. So as you may know, a lot of phone bans being adopted appear a little more cosmetic than substantive no-show bans where students still keep their phones, they’re just being asked to keep them out of sight. And we suspect that that they may have weaker effects and really be uneven in its implementation. So instead, what we did was we were able to partner with this company Yondr that’s a major seller of lockable pouches.

So schools using the lockable pouches give us a clean measure of really binding restrictions on the in-school use of phones. And we can validate that for lack of a better word, treatment manipulation in the GPS data and in survey data available to us. We see when and where schools take up these pouches, a substantial decline in the in-school use of phones. So then the third feature of the study is I think an unusually comprehensive set of outcomes: test scores, attendance, discipline measures, social-emotional survey questions of students around their subjective wellbeing, classroom attention, their experience of online bullying. Given everything that is hypothesized to be related to phone use, we thought it was important and meaningful to have that broad catchment.

I can add a fourth thing that is important to me. As you probably know in experimental social science, pre-registration has changed the field over the last 10 years. Ironically, for studies like this, exist for quasi-experimental studies that try to draw causal inference from real-world data, that open science practice pre-registration has not been a norm. I’ve been arguing publicly it should be because quasi-experimental researchers like our team have vastly more degrees of freedom than experimentalists do in terms of the measures we construct, the measures we construct, the methods we use, et cetera. And there’s empirical evidence that p-hacking is a problem, several forms of empirical evidence actually in quasi-experimental studies. So one of the other things our group did in this study and I’ve been doing my own work is pre-registering the outcome measures we used and our design before we undertook the analysis. So there’s a link to that in the study. I want to stress that’s not the standard of practice, but I personally believe it should be. (For more on preregistration, see here.)

So then in terms of the results, as you see, I’ve been in a lot of conversation with our team about how to describe these accurately and how to describe their implications in the right way. I’m going to choose my words carefully here. I would characterize the results overall as sobering. Obviously, these reforms are being widely adopted, school phone bans with some expectation that they’ll meaningfully change a variety of student outcomes. And we’re just not seeing that right away. In fact, if you look just at the core pre-registered outcomes, the only significant effects are the increase in student disciplinary incidents and an almost trivially small decrease in middle school test scores. Now, if you dig beneath that in a more exploratory posture and look at effects by period, you see that those overall effects do appear to be obscuring some interesting dynamics. The increase in discipline is concentrated in the first year and then returns to baseline within two more years.

And the null effect on student wellbeing is obscuring a really interesting dynamic where there’s a big decline in student wellbeing in the first year and by the third year it’s above what you would’ve predicted at baseline. And that’s the one I think advocates of phone bans are mostly picking up on that, oh, eventually students are reporting higher levels of wellbeing and the disciplinary effects appear to be transitory. But also there are a lot of null findings here as well. And so I characterize them as sobering. If someone said they were disappointing, that would feel sensible to me too. And this is where engaging in some of the translational commentary is a little difficult because obviously these results indicate we’re failing to realize the hypothesized benefits at least to this point.

But at the same time, given the preamble I gave you at the beginning about how we often don’t persist in trying to refine and understand policy innovations I also hope people won’t take this as nature’s final word on these policies and will persist in trying to understand what it might take for them to do better. So for example, I can throw a couple examples out here. One is it may be that phone bans don’t go far enough and that the problem is about digital devices more broadly and that the growing movement towards digital-free schools might come a little closer towards what some of the proponents are discussing. Alternatively, or even in concert, it may be that the bans are a necessary but not sufficient condition for the kinds of gains people hope to see.

For example, getting students off their phones might be a start, but once they’re off their phones, are they in high quality learning environments? Do they have effective teachers using high quality instructional materials and evidence-based pedagogy? There’s certainly some reason for concern around that. As an aside, you may have seen some of this in the current sturm und drang around the science of reading. One of the big issues in education policy right now is the growing realization that teachers have been trained to teach early reading in ways that don’t align with the best evidence on how students actually learn to read. In particular, they’ve been underemphasizing early phonics and phonemic awareness and things of that sort. So anyway, so that’s the position we’re in right now and it’s interesting to watch having spent this week watching how this was consumed and understood by different people and different journalistic outlets.

Holden Thorp: Great preamble. I guess my first question would be tell me the positive effects that you did see.

Tom Dee: So the main positive effect was that by the third year of a phone ban, it appears that students’ self-reported subjective wellbeing is higher.

Holden Thorp: And what about the teachers? Weren’t they also kind of happy with the whole thing?

Tom Dee: Yeah, we have teacher survey data. That’s a little bit more of a descriptive posture and wasn’t part of our pre-registration that we’re collecting more data and hope to do more with that. But there’s no doubt that teachers are incredibly supportive of these policies and they see good things in their wake.

Holden Thorp: I certainly like my classroom better when there aren’t any devices in it if we’re having a discussion. For sure. Yeah. All right. So for the quantitative measures where you didn’t see much effect, I know it was a pre-registered study, so you were going to publish whatever you got, but were you expecting to see a signal?

Tom Dee: There’s always this challenge of putting yourself back in that original position, but I think we were. And at some level, having been an education researcher for as long as I have, I feel maybe a little sheepish about having been hopeful because certainly I’m very experienced, have had a lot of experience, in reforms around which there was a lot of enthusiasm that when carefully examined, produced disappointing results, but we chose these measures intentionally. For example, it would make sense that if students got off their phones and were paying attention, I think it’s reasonable to think there might’ve been some meaningful learning gains in the short term, simply because the teachers now have their attention, but the disruption can confound that. If teachers are having a more difficult time, at least initially, maintaining classroom order, that’s going to complicate learning in the classroom.

Holden Thorp: And do you think that’s the main reason why you didn’t see a signal?

Tom Dee: Well, I have to be careful there. We can’t really say. I mean, a virtue of this kind of work is it’s got, I think, a strong causal warrant, but the big demerit of this type of quasi-experimental design is we can’t really tease apart mediators. So we can’t even be sure why discipline rose. The increase in discipline, for example, could be due to enforcing the phone ban and having to identify students who aren’t complying, but it could also be that in the absence of access to a phone, students who had been docile because they were sitting there scrolling TikTok or whatever, are now acting out in compliance with a ban, but acting out in a way they wouldn’t without the ban.

Holden Thorp: Yeah. So there’ve been a lot of studies on screen time and effects on internalizing symptoms and things like that, which also didn’t produce much signal. And do you see this as consistent with those or is there this difference somehow?

Tom Dee: I think to my mind, it appears consistent that these bans and the adoption of these lockable pouches drove down the in-school use of phones without producing clear academic gains.

Holden Thorp: And to the extent that there was a gain in wellbeing, it was very small and delayed.

Tom Dee: Oh, it was delayed. I’d have to go back to the paper about or take a second look at the effect size. I wouldn’t characterize it as really small, but I guess part of the reason I’m equivocating a little is, again, it’s so hard to know what mediation is behind these kind of reduced form results. It could be that phone bans in schools simply, for example, lead to some intertemporal substitution, students who are using them less in school, but maybe more intensively outside of schools in ways that attenuate any expected gains. So that would be consistent. This is why I was equivocating about, is this consistent with the meager evidence you described around social media, et cetera, because one could say, “Well, these results are consistent with a world where social media use is bad for kids, and it’s just that the bans shifted it intertemporally away from the school day to out of school time.” So that’s why I think we have to not get over our skis in interpreting these results.

Holden Thorp: Boy, do I agree with that. So Jon Haidt said in his book that the phone-based childhood is the major cause of the adolescent mental health crisis, and that’s a pretty bold statement. And I’m wondering if you agree with that and whether your study supports or refutes that or is silent on it.

Tom Dee: Well, I can answer the latter question, which is I think our study has to remain silent on it in part for the reasons I described. I mean, one could argue at the most superficial level, we reduced youth phone use and there were no clear gains. And maybe there’s a moralizing hysteria that’s wrong about digital device use among teens or it could be that Jon’s right and what happened here was that kind of intertemporal substitution. On the broader question, I would recommend you talk to some of my other colleagues, Hunt Allcott in particular, because as you probably know, they’ve done direct research on social media use and youth. So they’re more authoritatively steeped in that literature than I am. (Allcott is quoted in the piece and I will post his interview soon.)

Holden Thorp: That’s fair. But you probably have an opinion on the overall discourse on this. I mean, I think unless you read scientific journals, if you only read the millions of books that Jon has put out and the numerous appearances on podcasts and talk shows, you see a pretty simple story here that I think to a lot of people in the public would seem that the science is resolved. And do you worry about the asymmetry of that?

Tom Dee: Well, I’m always concerned about overly facile or monocausal explanations for any large-scale social phenomenon. But I will say, and this is something that may or may not be on your radar that I think contextualizes this some of the most important research I think I’ve seen done in education in the last couple years or actually (and several people have done this independently), were some descriptive pieces that took achievement data in the US and noted that the decline in achievement we’ve seen, it did accelerate in the wake of the COVID-19 pandemic, but is timed to around 2013 where you see an inflection point. And that’s the kind of thing I think Jon has latched onto and others and saying, look, this coincides with the rise of social media. But in the educational research, people also discuss how it coincides with other major changes. One is the one I described around walking back school-based test accountability with Obama’s first administration, I don’t want to get too much in the weeds if this seems off-piste to you, but No Child Left Behind in 2001 brought test-based accountability to scale across the nation, focused on reading and math achievement and asked states to devise systems that would flag schools that were failing to make adequate yearly progress towards their state proficiency standards in work that actually my co-author on this study, Brian Jacob and I did, we found that there were some non-trivial gains in learning from collecting data and trying to hold schools accountable for overall performance goals and subgroup goals, et cetera. Then largely what happened was when Obama was elected, it was around the time when everyone knew we needed to redesign that system in some ways. It was only focused on reading and math, it focused on achievement levels and not value added.

So there was some consensus around this needs to change in some ways, but it was also when the hyperpartisanship we see now began to accelerate with the rise of the Tea Party, and all the political centrism that had supported that reform evaporated. And that’s when the Obama administration started issuing waivers to states who were no longer meeting their NCLB requirements. And after a long torturous period, the federal government really walked away from that accountability, those accountability reforms and there was a devolution to the states in 2015 under the Every Student Succeeds Act. Well, that’s the other major hypothesis for that inflection point in student achievement nationally. You’ll often hear researchers say, “We took our foot off the pedal in terms of accountability and it’s reflected in our achievement results.” So this is an area of active inquiry. How do we understand these macro trends given that inflection point coincided not just with the rise of social media use among teens, but the walkback of school accountability and some other factors like repercussions from the Great Recession. So I think that’s important context that would underscore why we shouldn’t overgeneralize from national trends the effects of one particular historical change.

Holden Thorp: I did want to ask you a little more about the effects of the Great Recession. I mean, there are some data that show that particularly with mental wellbeing, access to resources is a huge determinant and especially the mental health of your parents. And so you say that the effect of the Great Recession is yet another potential hypothesis for this, right?

Tom Dee: It is. And not just working through some of the economic implications for families and what that means for students, but also school’s capacity to fund student mental health services over that period. I mean, I don’t have exact data on this to hand, but I would hypothesize that it was compromised by the Great Recession. So other things were happening at that time.

Holden Thorp: And so you’ve given me two alternative hypotheses, but are you worried that if one of those turns out to be the right thing, that it’s going to create public confusion because the story that it’s the phones and social media is so prevalent right now?

Tom Dee: Yeah, I am just generally concerned about the way we integrate research policy and practice in the US. I wrote about this in an essay at the beginning of this school year for Education Week that the integration between them, at least in education, is incredibly poor.

Part of it is that too much of the education research that’s produced by our universities and think tanks and contract research organizations just isn’t meeting the needs of practitioners. And some of that is, I think it’s overindexed on qualitative methods. Sometimes it’s very transparently ideologically coded and avoids more to sort of positivist scientific postures. But even when we know something is effective, it often doesn’t get picked up by practitioners and policymakers. The recent movement after decades of scholarship to move towards embracing teaching methods that are aligned with how kids learn to read is one important example of that where even where the research does speak with a fairly consistent voice, that doesn’t always punch through to the way teachers are trained and the kind of in-service professional development they get. And then the other concern is that when states make policy pronouncements or even districts, it doesn’t necessarily change practice on the ground.

So I think that’s a potential concern for phone bans if you believe they work, well, many states are saying, yes, ban phones, but then giving districts local control in doing so. And that could lead to implementation that is at best uneven and at worse cosmetic. So the alternative reality that I think I and other kind of technocratic evidence-based policy researchers would like to see is one where we have what are sometimes called inquiry cycles or continuous improvement, where our schools try to function like learning organizations, piloting innovation and planning for rigorous assessment of whether they’re working and then taking a moment after evaluating something to decide what to do next. Should they adapt a reform in some way? Should they take it to scale if it seems promising? Should they walk away if it’s not working? In continuous improvement, we call these PDSA cycles, plan, do, study, act, where the act stage focuses on should we abandon, adapt or adopt a reform?

And I wish we took that kind of approach to studying both phone bans and more generally, how to manage digital devices within schools. And part of the reason that matters too is there’s a hyper-localness to that approach that I think is really critical for understanding what can work and what can work in specific types of school settings or communities. So if I were advising state superintendents, I would encourage them to adopt an appropriate stance of humility about what we really know in these spaces and build a robust learning agenda that can help us answer the kinds of questions we have and build an evidence base that we can disseminate. Does that make sense?

Holden Thorp: It does. I think it’s very rigorous and has less public bombast in it than the moment that unfortunately we find ourselves in. So let me ask you, so one of the things that some of the critics of, and I don’t know if you would agree with this, but I think if you went to Capitol Hill, everybody on both sides of the aisle is focused on the phones and social media and that is probably drowning out some of these other measures that I’m sure the education community would like to pursue.

Tom Dee: It’s just a general problem when they’re so popular the political system’s going to respond to that popularity rather than the more, I think, nuanced approach I was suggesting of saying, “Well, let’s be clear about what we don’t know and let’s plan to learn.”

Holden Thorp: Yeah, perfect. So one of the things that the critics of the phone message say is that no matter how much you try to say you’re blaming the social media companies or big tech or whatever, it still comes off as blaming teenagers for spending too much time on their phones and that that has a deleterious effect. What do you make of that critique?

Tom Dee: I don’t know. That seems off point to me. I don’t think it’s necessarily about blame. Technology is a part of our lives and I can’t imagine a world in which it’s not going to continue to be so. And in fact, increasingly so. I think it’s up to us to think about how to best put guardrails around it and we can do so without us necessarily assigning blame, certainly not assigning blame to the only people involved in these interactions who are children.

Holden Thorp: Well, that covers a lot of what I want to ask you about. I will call your colleagues about the social media part. Is there anything else that’s not out there in the discourse that you’d like to suggest I bring up?

Tom Dee: Yeah. Well, a number of states are considering and moving towards digital device-free school environments more broadly. So I think that’s just what I’ve mentioned, our sense is that’s going to be the next issue and we’re trying to position ourselves to have some high quality evidence that will speak to that. Because again, that’s another explanation for the sobering results we see, which is, hey, we hear, for example, anecdotes about, yeah, the phones might be banned, but everyone’s sitting there with a Chromebook and all a student has to do is open up a slide presentation that they’re ostensibly working on jointly with their peers and all of a sudden they’re communicating with each other within the slide deck typing messages, things of that sort. So I think again, just underscore the broader concern and the ways in which we still need to learn more. And yeah, it’s been an interesting week seeing the way this gets swept up into a very reductive kind of binary maelstrom when I think our team, or at least I’ll speak for myself, want to advocate for this kind of humble approach to saying, let’s try to figure out more what’s going on here.