School of Social and Political Science

Blog content

Generative Artificial intelligence has barely been out of the headlines in 2023. As we move into 2024, the expectation is that this is the year in which the rubber will meet the road, with hype and speculation either crystallising into a set of core use cases, business models, and designs, or alternatively collapsing entirely into another ‘AI winter’. The new ‘generative’ AI tools which were released, largely for free (for now), to an expectant public last year have been cause for much consternation among sociologists and the humanities more generally. Rather different from the more hands-on tools of the data science era, or the text mining and network analysis tools that promised new and interesting ways of doing micro-sociology (but which still needed us to interpret them), these instead seemed to strike at the heart of something much more core to our work as sociologists. The apparent ease with which large language models could mimic the generation and analysis of text has thrown many for a loop. These are being presented as a challenge to our predominance as humans in collecting and analyse qualitative data, using theory to understand the world, and - possibly most troublingly of all - our skills as writers. 

I’m developing a new class on AI and sociological research this semester for the Research Training Centre. One of the benefits of teaching research methods is that actually using novel(ish) technologies like Large Language Models in practice brings much of the hype into focus - you are immediately confronted with a much closer look at the capacities and limitations. Much like AI technologies themselves, AI in sociology is not new. A quick review finds a wealth of papers from the 80s, 90s, 2000s, 2010s about using AI in sociological research, which often read like they could have been written yesterday (Carley, 1996). Researchers in the early and mid nineties were optimistic about the potential for the AI systems being developed at the time to create realistic social agents with complex internal lives, which would enable the automation of large-scale analysis and theory-testing. More recently, the late 2000s and early 2010s ‘big data’ hype was accompanied by claims that what now seem like fairly quaint machine learning models would lead to the ‘death of theory’ (Savage, Mike, and Burrows, 2007; Kitchin, 2014)- we wouldn’t need to create frameworks for understanding the world as sociologists, instead we would simply throw the huge amounts of data generated by our day-to-day use of digital technologies into vast models, which would impute the chaotic and complex relationships hidden within and give us answers. Fortunately for those of us who teach classes in social theory, these claims have not yet been borne out.

In the most recent era (the mature ‘big data’ era) a range of different work has coalesced into two broad approaches. The first, often termed 'computational sociology’ aims to develop machine learning tools as scientific means of studying social life. The emphasis here is on the cultivation of scientifically robust approaches for modelling the complex, messy, and large data sources generated by contemporary digital life - tweets, posts, location data, and vast repositories of text. These involve a great deal of attention on model development - ensuring that appropriate models and parameters are selected, then refined for the best and most representative fit. This allows for a development of classic Chicago School approaches to bringing micro- and macro-sociological explorations together, tracing patterns in vast swathes of human life and interaction.

The second approach is more associated with 'social data science’ - in which the machine learning tools are treated through a more ethnographic lens. Here, rather than attempting to produce the most representative and accurate models reflecting a ‘ground truth’ out there in the social world, machine learning and AI tools are intended to provide powerful and messy ways of navigating complex data sources in a quasi-ethnographic and mixed-methods fashion (Elish and Boyd, 2018). Beginning with rough-and-ready models to hack large datasets like forums and social media sites into a more manageable shape, this approach is characterised by a methodology of tacking back-and-forth, first using ethnographic ’hanging out’ in digital spaces or deep reading of samples of text to get the feel of a research site, then deploying large-scale but fairly simple models to map shape, patterns, and scope, then following up leads with further ethnographic observation and interviews, then moving back to text analysis to search for signals that these local findings might be more widely observable. This struggles to generate the large-scale conclusions of computational sociology, but can provide extremely rich accounts of mid-level features of communities and spaces. As a case in point, although I teach digital methods I’m predominately a qualitative researcher these days - I mostly incorporate digital methods in a piecemeal, ethnographic, and mixed-methods way to help me scale up qualitative work, analysing large numbers of emails in a mailing list or posts in online forums in order to complement my observations and deep reading of these sources.

Into these fields have dropped a range of new technologies - Large Language Models and other ’generative’ forms of AI. These take the neural nets which underpin previous forms of text analysis and hyper-charge them with enormous datasets - often large chunks of the entire Internet and the books, articles, conversations, and other texts contained within. The structures within the ‘hidden’ or ‘middle’ layers of these neural networks represent associations between words and series of words which, at large levels of scale and training, begin to approximate semantic and conceptual relationships - or at least appear to when used to generate or classify text. A further innovation – associated with Google’s development of ‘transformer’ models in 2017, incorporates ‘self-attention’, which allows the model to, instead of simply considering the ‘next’ word in a sentence, process all the words. In a sentence, paragraph, or piece of text at once. This allows them to capture much more of the grammatical and semantic context within a piece of text, and hence produce more complex structures of association that take this into account. We can immediately and intuitively see something rather different about these new models when we use them. However, I would argue that the most profound change has not been in the admittedly impressive leap in the capability of these models for generating text or classifying and ‘analysing’ data. Rather, it is that instead of existing as tools - individual technologies, models, and methods that can be ‘picked up’, learned, reconfigured, and deployed by analysts - these new generative systems have been turned into infrastructure; a set of systems that can be used widely across many settings, which allows new systems and capacities to be built on top of them, and whose technical workings are designed to fade into the background, allowing us to take them for granted and not think or wrestle too much with how they actually work. Like all infrastructures, they also rely on large amounts of ‘hidden’ work behind the scenes which is alienated and abstracted from the end user - in this case, the work of the people who created the original training data, the people maintaining, updating, and administering the software and hardware, and the vast numbers of poorly-paid workers currently labouring around the clock to train and retrain the models for different use cases. In the next part of this blog, I will discuss what these newer models might mean for sociological research methods.

By Ben Collier

References

Carley, “Artificial intelligence within sociology, Sociological Methods and Research, 1996

Savage, Mike, and Burrows, "The coming crisis of empirical sociology." Sociology 41.5 (2007): 885-899

Kitchin, “Big Data, new epistemologies, and paradigm shifts”, Big Data and Society, 1.1 (2014)

Elish and Boyd, “Situating methods in the magic of Big Data and AI, Communication Monographs, 2018