Why study "Machine Behavior"?
Or "Why Machine Behavior is a Nice Buzzword"
Artificial Intelligence (AI) algorithms have become ubiquitous. From dating apps to autonomous weapons, the boundaries of what is and what isn’t AI blurs as applications using deep neural networks and decision trees become commonplace1 and widely used across the sciences (Gao and Wang, 2024). With the rise of Generative AI, these texts, images, and videos are increasingly AI-generated. This has even changed how we speak, with words distinctively associated with ChatGPT increasing in presentations, talks, and speeches (Yakura et al., 2024).
As AI becomes ubiquitous, AI technologies have also become consequential to society. Standalone AI algorithms can be consequential by themselves; slight tweaks to criminal risk prediction algorithms have a tangible impact on the lives of incarcerated people [as these algorithms are used in court; see Larson et al. (2016)]. However, the effects of AI become even more remarkable if we consider what it enables and augments.
Some things can only exist due to advancements in AI: TikTok does not exist without recommender systems; it is not that AI “changed” TikTok; the whole idea behind it fundamentally relies on a recommender system. Without recommender systems, TikTok isn’t.
Other things changed drastically due to advancements in AI: Content moderation in social media platforms predates AI. Slashdot (1997) had content moderation. However, AI completely changed how content moderation is done in modern social media platforms like Facebook, Reddit, and Twitter, with a large swathe of content removals automatically triggered by internal classifiers (Horta Ribeiro et al., 2023).
In light of the ubiquitousness and consequentially of AI, scholars have called for a research program that studies the behavior of AI (or of “machines”) within human society.2
As Rahwan et al. (2019) put it:
In his landmark 1969 book Sciences of the Artificial, Nobel Laureate Herbert Simon wrote: “Natural science is knowledge about natural objects and phenomena. We ask whether there cannot also be ‘artificial’ science—knowledge about artificial objects and phenomena.” In line with Simon’s vision, we describe the emergence of an interdisciplinary field of scientific study. This field is concerned with the scientific study of intelligent machines, not as engineering artefacts, but as a class of actors with particular behavioural patterns and ecology. This field overlaps with, but is distinct from, computer science and robotics. It treats machine behaviour empirically. This is akin to how ethology and behavioural ecology study animal behaviour by integrating physiology and biochemistry—intrinsic properties—with the study of ecology and evolution—properties shaped by the environment. Animal and human behaviours cannot be fully understood without the study of the contexts in which behaviours occur. Machine behaviour similarly cannot be fully understood without the integrated study of algorithms and the social environments in which algorithms operate.
But it is worth asking: why do we need this new “field”? What are the shortcomings of previous research that can’t explain the brand-new world of recommender systems (and LLMs)?
Wagner et al. (2021) provide a partial answer: understanding all societal phenomena is incredibly challenging when algorithms are co-shaping society. They identify a couple of challenges with doing social science in what they call “algorithmically infused societies.” Namely, when algorithms enter the playground: #1) social science theories crumble; #2) measuring things becomes harder; #3) (mis)measuring things have consequences.
I illustrate the above three with an example from my own line of research: trying to study the impact of recommender systems on social media. As you might already have heard, there is widespread concern that recommender systems in social media platforms like YouTube or Facebook would radicalize people (Tufekci, 2018). And, indeed, a lot of people were radicalized “within” social media—that is undeniable.3 But the question remains: what is the role of the algorithm? Let’s see if points #1, #2, and #3 by Wagner et al. (2021) hold up here.
Point #1. Radicalization is well-studied within the social sciences. However, radicalization in the age of social media is different, as individuals creating “extreme” content had to engage with algorithmic feeds and algorithmic content moderation. This has led to a lot of confusion around the impact of recommender systems. The conclusion that eventually emerged after many years of debate and studies is nicely summarized by Munger and Phillips (2022):
(...) we argue for the need to study the YouTube Right systematically an advance a “supply-and-demand framework” to understand the proliferation of rightwing media on the platform. To date, journalistic and scholarly work has argued that YouTube’s recommendation algorithm has led viewers to extremist content, radicalizing them to further-right views. We believe that this conclusion is premature, and we are certain that this is not the only important research question to be asked by political scientists about right-wing content on YouTube, or YouTube more broadly.
Point #2. This confusion was primarily caused by how hard it is to measure “online radicalization” to begin with. For example, when studying radicalization, most studies [including Horta Ribeiro et al. (2020)] did the most naïve thing possible. They measured the extent to which extreme content was recommended as you randomly walked through the (dynamic) recommendation graph. However, these measurements assume no “user models”. They assume no meaningful interaction pattern between humans and machines. In subsequent work, when human interaction patterns are considered, there’s little evidence of any “algorithmic radicalization” taking place (Hosseinmardi et al. 2024).
Point #3. Mismeasurements have consequences! In this case, it led to the deprioritization of important content moderation and platform governance policies. The mismeasurement of algorithmic radicalization arguably steered people away from other, more important interventions. However, it is worth mentioning that Wagner et al.’s point here is even broader. Even correct measurements have consequences, as they can feed back into algorithms that shape human behavior. In our running example, there are huge efforts from online companies not to recommend extreme content. Therefore, this whole literature around radicalization happened while YouTube was doing its best (I guess) to recommend as little extreme content as possible. It also coincided with a tightening of content moderation practices within these platforms. Many original channels studied by Horta Ribeiro et al. (2020) were banned shortly after the study was published and got much media attention (although a causal relationship was never established).
Both Rahwan et al. (2019) and Wagner et al. (2021) were written before the AI explosion in late 2022 with the popularization of ChatGPT. Ever since, there has been increasing interest in AI alignment, in using LLMs to simulate human behavior, and even in the consequences of the deployment of AI agents into the wild.
I’d argue that machine behavior is a nice buzzword to refer to the study of these phenomena. On the one hand, if we see AI as merely an engineering problem, we risk overlooking the empirical reality that these systems shape—and are shaped by—the very social environments in which they operate. On the other hand, if we ignore the role of AI, we risk overlooking the technical underpinnings and design choices that critically shape how users behave and interact in digital environments. Only by acknowledging the messy interplay between humans and machines can we fully understand—and responsibly guide—the future of AI in society.
Another cool paper here is Tsvetkova et al. (2024).
Horta Ribeiro et al. (2020), yours truly, show how users consistently migrated from contrarian to extreme content in the late 2010s. This is not to say that the algorithms were to blame, although this was way less clear in 2019 when this was written.