Electric Youth
Why LLMs are Like My 4-Year-Old
One of the biggest buzzwords of 2023 was AI, particularly Large Language Models, or LLMs. If you missed the hype (really?!), an LLM is an AI algorithm trained on large volumes of data that can understand and generate written content in the language on which it was trained. Since Chat-GPT emerged at the start of 2023 we’ve been using it to summarize and generate text and code to solve our everyday problems.
We’ve watched with awe at some of the amazing benefits that have appeared to help us with our work productivity in the case of UK civil servants, or to help us discuss life’s difficulties with Psychologist or to chat to our favourite characters like Super Mario in the case of Character.ai chatbots. Furthermore, we’ve also been horrified at the potential for them to spread harm through disinformation, or be used to create tools for criminal purposes. Even the use of articles to train these models is under scrutiny for potential violations of copyright with open cases lodged by The New York Times and authors John Grisham and George RR Martin.
My son pre-dates LLMs, having burst onto the scene just over 4 years ago. Like any 4 year old he comes up with intriguing ideas that he spouts with more confidence than I could ever imagine to have. Yet my geeky husband and I couldn’t help but notice some rather common traits between him and LLMs. In this piece, I’ll share some of these commonalities in a slightly more humorous take on LLM usage and development concepts.
Raise the Knowledge
The concepts of grounded knowledge and the application of this knowledge are similar between young children and LLMs. Both rely on observed sources of information when generating responses to the questions we ask.
Prominent psychologist Jean Piaget in his theory of cognitive development outlined the 4 key stages of cognitive development that children progress through. Through these stages, children are building their mental model of the world by building associations between objects they encounter and operations associated with them. For example, infants in the Sensorimotor Stage learn about object manipulation and object permanence through their sensory experiences. These interactions can be limited by the objects they are given to interact with, including scenarios of gender norms of providing girls and boys with different toys.
As children age they can also form incorrect assumptions, such as young family members who assumed women only drove cars because they were used to being driven around by Mum and Gran. Or connect unrelated concepts:
Octopus speaks Spanish Mummy. Bonjour!
Mini R, aged 4
LLMs can similarly be influenced, they are generating an answer to your question using the patterns they have picked up in their training. One of the reasons the aforementioned copyright cases are popping up like the game whack-a-mole is due to the sheer volume of data sources used to train them. They are easy to mislead with poisoned data. This is already possible for generative image models using tools like Nightshade, and many are confident textual models can be manipulated with providing poisoned documents for training.
Handling specific use cases or working with proprietary data the LLM is not trained on requires grounding with accurate sources. As shown in the above diagram from Google’s grounding blog, semantic search capabilities using vector search technologist allow LLMs to use specific data to apply sensible rules and sources for the generated responses.
Sponge
Children are sponges. You never know what words or phrases they will pick up as they are always listening. The panic parents feel when their child unintentionally picks up swear words, or in my case, the word ridiculous, and recites them is real. For a while afterwards, the word will pop up in unexpected contexts as children try to figure out where in their internal schema this term should fit.
Returning to Piaget’s theory of cognitive development, my 4-year-old son is currently in the Preoperational Stage. At the start of this phase children are strongly influenced by how things appear, or the things they hear, and assume other people see the world the same way as they do. How they perceive the world is strongly informed by the objects they encounter, and the language used by others around them, meaning they mimic what others say without knowledge of wider context. Or sometimes apply unexpected actions to objects.
In the heart of Pando
Mini R, aged 4
This pattern is something that LLMs can also inadvertently do based on training data. They may generate incorrect answers by forming false connections between unrelated sources. A prominent example of this comes from Google Bard’s first demo in early 2023 where it incorrectly stated the James Webb Telescope took the first picture of a planet outside our solar system, rather than the correct answer which is the Hubble Telescope.
Grounding can partially address the issue of source confusion. But for the usage of terms in nonsensical contexts, development of domain-specific LLMs such as BioBERT for the biomedical field and BloombergGPT for finance are under development. These are intended to restrict the generated responses to utilize and hopefully reduce some of the unexpected results that can be generated by general-purpose LLMs.
Don’t Stop Believin’
The first time I used ChatGPT I found the tone of the generated answers interesting. Unlike myself and many others suffering from the confidence chasm triggered by impostor syndrome, it conveys a tone of confidence even when the answer is incorrect.
LLMs don’t think. They generate responses based on the language patterns they have picked up through training. Therefore they are not able to determine like humans if they should give you the information you have asked for. Instead, applications need to intercept and potentially refuse to answer these toxic questions. For example, do you really want to tell someone how to make a deadly chemical?
Early versions of ChatGPT suffered from issues of generating false sources rather than stating their source, such as the 2023 case in the US where lawyers were fined for submitting fake citations generated by ChatGPT. This generation of false information is known as hallucination.
Of course, the wee man thinks! That is one key difference. But he does tell a good tale and can struggle to convey the date a particular event happened or be blindly convinced that something is true.
The Tangent
My son gets very easily distracted. As do many young children. The TV coming on can stop him telling the most engaging of tales mid-way, or leave me asking several times what he wants to eat. Asking him a question while speaking can trigger a whole new story altogether:
I couldn’t go to the toilet Mummy. I got distracted by the ball!
Mini R, aged 4
Just like any software system, LLMs can be hacked in various ways. This is exactly why OWASP released a dedicated Top 10 for applications using LLMs. There are various ways to hack an LLM as covered in this Infosecurity Europe piece. One particular vulnerability, known as Prompt Injection involves crafting prompts to encourage the model to leak data or perform other unintended actions. Just like that ball that interrupted the wee man’s trip to the loo by encouraging him to chase the ball.
Edge of Evolution
Young children are sponges. The rate at which they learn and retain information is absolutely astounding. I had certainly forgotten the amount of information I retained throughout my school years. Since my son started school I’ve been amazed at how quickly he has been learning to read and remember sounds.
Today we learned OO, Mum. Zoom to the moon!
Mini Richmond, aged 4
New LLMs are released all the time and are becoming more sophisticated. Since 2021 there has been an explosion of models being built and released, as shown in the generated diagram from Mooler’s LLMs Practical Guide.
The evolution of these models also means that we are seeing metrics used to evaluate them varying significantly, as highlighted in the [comparison table from a 2023 research paper](https://arxiv.org/pdf/2307.06435.pdf):
The size of training datasets has increased. The number of parameters available for tuning the model is increasing with newer models. Furthermore, the number of tokens processed is also increasing. It’s easy to assume that parameter and token counts point to better models, but the reality is that comparing these models to determine suitability for your use case is becoming more nuanced.
We in turn learn from LLMs too. Just as I’m learning phonics over the word memorization techniques I used at school thanks to the wee one, we ask LLMs to answer our questions. However, in both situations, we need to ensure we can cross-check the information and source legitimacy to prevent the spread of disinformation.
Change is Now
Here I’ve covered the similarities between my 4-year-old and LLMs. Just like in 2023, I’m expecting AI to continue to dominate our news just like it did in 2023.
One of the things I was told by everyone before my special little guy was born was how much having a child changes your life. Any parent will tell you that it completely changes your world, perspective and priorities. AI generally, and LLMs specifically, have already changed how we generate and consume information.
We’re using ChatGPT for many different things, from summarizing text to generating responses to emails. Just as my growing child will change and impact my life and work, so will AI in the coming years.
Thanks for reading!
Originally published at http://carlyrichmond.com on January 22, 2024.
