Today, I posted a podcast interview with Jeff Horwitz who is the reporter played by Jeremy Allan White in Aaron Sorkin’s new movie about Facebook, The Social Reckoning. The movie mainly talks about the contributions of Facebook to political polarization, but Horwitz also covered the question of whether Facebook and Instagram were harming teen mental health. I wrote up a summary of the political polarization stuff for our newsletter and reprint it below. Underneath that is a write up that just appears here on what he said about teens and mental health. But you could skip all that and watch or listen to the whole video; Horwitz is a great talker, I promise:
Facebook knew things we couldn’t know
This section was also in our ScienceAdviser newsletter (sign up for ScienceAdviser here!).
This weekend, Aaron Sorkin’s follow up to The Social Network, a film about Mark Zuckerberg and the creation of Facebook, will hit the big screens. The new film, The Social Reckoning, explains how reporter Jeff Horwitz and whistleblower Frances Haugen exposed the fact that Facebook knew much more about how the social network was creating political polarization around the world than they admitted. In the film, reporter Jeff Horwitz is played by actor Jeremy Allan White. I sat down with Horwitz on the podcast to talk about the backstory of the film; Horwitz had not seen the film yet, and neither had I.
During the period of the film, Horwitz was at The Wall Street Journal (he is now at Reuters), and Frances Haugen was a Facebook employee with access to internal data on what the company knew about how social media was harming democracy and teen mental health. Before leaving Facebook, Haugen downloaded a large trove of these data and then shared them with Horwitz who, in a 2021 series of stories called the Facebook Files, exposed how much Facebook knew about the potential harms they were creating.
Science came into the story a few years later when we published a set of papers developed after a coalition of academic researchers was given some of Facebook’s data to analyze for whether the company had contributed to polarization in the 2020 election. No significant signal was seen in the data that suggested that social media played a major role in polarization in that election, but we and the researchers believed it was because the audiences were already pre-segregated. Facebook saw the result as proof they didn’t cause polarization and did a victory lap. Horwitz covered this disagreement at the time.
In our conversation, I asked Horwitz for his take on these events. He stressed that it is hard to appreciate the asymmetry between Facebook controlling the data and academic researchers only seeing a portion of the story. “You could have a team of 20 PhD researchers,” he told me, “and literally more interesting work can get done by the summer intern over at Facebook just because they have the ability to look at the data overall.”
The experiment consisted of showing one group the algorithmic feed that Facebook used to keep users on the site while another group was shown a chronological feed based on the folks they followed. This chronological feed presumably would be less tailored to create polarization. But Horwitz explained that the algorithmic feed is impossible to pin down. “The algorithm is not one thing,” he said. “The algorithm is whatever the current combination of levers are that are being pulled. It is a black box.” Indeed, after the papers were published, we learned that Facebook had temporarily changed the algorithmic feed to a more chronological feed during the experiment without adequately notifying the researchers. (We have alerted readers to this discrepancy.)
The situation has a remarkable parallel to current strain around artificial intelligence. Only the AI companies themselves know the extent to which large language models are carrying out dangerous actions, and the public only knows about it if they decide to disclose. This asymmetry is identical to that with social media five years ago. Perhaps one day another Frances Haugen will emerge to blow the whistle on AI, but probably only after significant damage has occurred.
And why aren’t there more internal whistleblowers? Horwitz sees a dilemma facing every researcher hired in trust and safety at a tech company. “You get hired to study a problem,” he says. “You have data that no one else in the world has access to. You have the ability to run tests which do come up with statistically significant facts on whatever sort of problematic behavior you would like to be addressing.”
But even the best internal researchers can only do so much. As Horwitz said: “You come up with solutions, and then the company’s like, ‘Uh, no, thank you.’”
Why teen mental health is so hard to study
I also asked Horwitz about what the company knew about harms to teen mental health from social media. In Horwitz’ reporting are documents based on internal research that Meta has done that show clear harms to self-image among teen girls from looking at Instagram. But as I’ve described here multiple times, academic scientists have struggled to see strong signals on improvement in teen mental health or on test scores when phones or social media are restricted. I asked Horwitz why he thought that was the case.
Horwitz thinks it’s only some of the users who are profoundly affected, and therefore, that makes the signal hard to see. “It is only a subset of users,” he told me, “and generally a subset of teenagers who arrive at the platform with some mental health stressors, whether that is body image issues or confidence issues, or some level of depression.” That is consistent with his reporting on the internal Meta research, which says that Instagram “make[s] body image issues worse for one in three teen girls” who came to the site already having body image issues.
But because of the way the algorithm works, if it detects that the user is engaging with body-image content, the platform is likely to show images of women that make her feel worse. “If I give you that particular child with that particular makeup, that is what the platform will feed that kid,” he said.
So that raises two things that make it hard to see the signal - the first is that it’s only affecting a small subset of girls (and less of the whole population if boys are included) AND the algorithm is giving each user something different, and that is only apparent to Meta.
These things are still observational. I told him that still didn’t prove it was causal. He conceded that was correct. “You can't prove it's causal, is a true statement,” he said. “It's really hard to come up with something causal.”
But he believes that’s the wrong question. He used the analogy of TV in the earlier era. It’s very difficult to say whether that was bad or good for kids. (I memorized all of The Brady Bunch and Star Trek, and I seem to have survived.) It matters what kids watched — and when. Watching schlock at 2 am on a school night would probably be more detrimental than watching Nova or Nature at 3 pm after school.
Ultimately, he thinks the causality question in the population is the wrong question. Rather, it’s the individual experiences - curated by the bespoke algorithm - that you need to study. Even Andrew Przybylski, who is generally in the group that says social media harms are overblown, agreed with this in a 2021 story. “People talk about Instagram like it’s a drug,” he told Horwitz. “But we can’t study the active ingredient.”
In the end, whether it’s political polarization or teen mental health, Meta has access to data that independent researchers don’t. And when they are selective about how much of it to disclose, they can shape the answer that researchers get.
Social media was just the warm-up act
It’s a good thing we’re hearing this story now, because social media is just the warm-up act for the challenges ahead with artificial intelligence. The chatbots have the potential to manipulate our emotions and actions even more than social media. And they are teaming up to do things to our world that we can barely imagine.
Like with social media, there’s only one group that has the data you need to figure out how much danger we’re in, and that’s the AI companies. Pretty sure they’re not planning to cough up their data, either.
Horwitz summed this up by asking a better question. The impossibility of answering the giant causal question shouldn’t stop us from asking questions we can get at. “‘Is the product as currently built reasonably designed for a teenager?’ seemed like a question that maybe deserved greater prominence,” he told me.
OpenAI and Anthropic need to give us the same answer.