Societal Consequences of Large Language Models

International Business Machines Corporation, known as IBM, defines large language models (LLMs) as advanced artificial intelligence systems trained on massive amounts of text to understand, summarise, translate and generate human-like language. Language models are everywhere in our modern digital life. From typing a query in Google, instructing Siri to set a timer, asking Alexa to switch on a light, or using autocompletion in WhatsApp messages, language models get the job done. To understand their societal consequences, we need to look back at history to see how the model concept emerged.

History suggests that philosophers played a part in developing LLMs. In 1953, Austrian philosopher Wittgenstein wrote in his Philosophical Investigations: “The meaning of a word is its use in the language”. The American linguist Rupert Firth rephrased this in 1957 as “You shall know a word by the company it keeps”. After seven decades, Wittgenstein and Firth’s casually appearing statements would underlie highly complex algorithms, known as language models, which now affect millions of people on earth. Language models can predict a particular word given its neighbouring ones. They draw their knowledge about words from the statistical analysis of vast amounts of textual data. Surprisingly, these models can do a lot more than generate words. They can even have conversations with us humans.

In 2022, OpenAI launched ChatGPT, an interactive LLM with a chat interface; GPT stands for Generative Pretrained Transformer. Concerns about the societal consequences of ChatGPT keep appearing in the media. For example, in higher education, the availability of ChatGPT raises substantial concerns. Students can use it to generate well-formulated texts on any topic, making ChatGPT a risk factor for plagiarism and fraud. 

Interestingly, some research on political analyses of ChatGPT reveals a predominantly democratic, liberal orientation. A study finds that ChatGPT is biased toward the Democratic Party in the United States, President Lula da Silva in Brazil and the British Labour Party. A political analysis by Leiden University in the Netherlands reveals a left-wing orientation for ChatGPT.

Transparency is a problem for LLMs. Current LLMs and LLM-based AI assistants fail to pass the transparency test and, consequently, encounter challenges in order to be considered responsible AI. In this context, transparency is about answering what an AI model does, based on which data it produces an output, and what its limitations are. This means transparency is about insight into the intended capabilities of an AI model and directly relates to responsible AI. Another issue here is explainability, which aims for a deeper, technological narrative for a specific audience such as lawmakers, judges and engineers. This can follow different paths. For example, explaining how an AI model reaches its conclusions can have many implementations, such as a deterministic sequence of algorithmic steps or following a probabilistic path. As a result, transparency of LLMs becomes challenging in light of the emergent abilities.

Language models, in general, are ‘paraphrase’-oriented, which generates text on topics without disclosing their sources. This makes them hard to trust from a factual perspective. Lack of factual checks leads to various other biases. They can be biased by their data, which is a ‘selection bias’; biased by the architecture, which is an ‘algorithmic bias’; and biased by the humans who train or fine-tune them, which is ‘training bias’. Such biases cannot always be easily detected or even prevented.

Europe has been taking the role of AI Regulator, which culminated in the European AI Act. Considering all bias-related factors, a set of regulatory principles for AI was initiated in 2021. The AI Act was adopted provisionally in December 2023 by the EU Parliament and Council and was formally adopted by the EU Council in May 2024. The modality to enforce the AI Act is based on a risk model. To this end, the Act puts forward a pyramid model of AI risks. The model incorporates four tiers of risk. At the bottom of the pyramid is ‘minimal or no risk’, which can be permitted without restrictions. Medium risk can be permitted but subject to transparency or information requirements, such as a chatbot that impersonates a human.  High risk can be permitted after compliance checks. High-risk systems, for instance, consist of potentially biased medical applications that measure health through sensors and share data with other parties. Unacceptable risk is completely prohibited under the EU AI Act.

The risk management system also needs to consider potential manipulative activities. For example, author Yuval Noah Harari, in an article titled ‘What happens when the Bots compete for your love?’ published in the New York Times (4 September, 2024), argued that chatbots can lure us into a sense of fake intimacy. Such anthropomorphism is rooted in the apparent human-like behaviour of LLMs and may be used as an instrument of manipulation. We should guard against such activity.

Above all,  we need more research into detecting the ethical guardrails implemented in LLMs, especially open-source models that are increasingly on the web for everyone to download. Also, a deeper understanding of the complex input-output relationships LLMs exhibit is paramount for better risk management. Negative societal consequences of LLMs should not be ignored by both the developed and developing worlds.
______________________________________
The writer is a UK-based academic who writes on science, philosophy and education