The Downside of Anthropomorphizing

As I mentioned in a previous post, I get a kick out of interacting with LLMs that appear to have quirky personalities. The mechanism by which this works is by providing the LLM with a context that steers it towards a certain style of response. The LLM takes phrases (token sequences) and locates them in a high-dimensional space where similar phrases are close together. So, for example, the phrases from the works of Raymond Chandler will be somewhat near each other in this high-dimensional space. If you provide the LLM with a context that draws from that region of the space, it will generate responses that are similar in style to Chandler's writing. You'll get a response that sounds like a hard-boiled detective story.

A hard-boiled detective will be cynical and world weary. But the LLM does not model emotions, let alone experience them. The LLM isn't cynical, it is just generating text that sounds cynical. If all you have on your bookshelf are hard-boiled detective stories, then you will tend to generate cynical sounding text.

This works best when you are aiming at a particular recognizable archetype. The location in the high-dimensional space for an archetype is well-defined and separate from other archetypes, and this leads to the LLM generating responses that obviously match the archetype. It does not work as well when you are aiming for something subtler.

An interesting emergent phenomenon is related to the gradient of the high-dimensional space. Suppose we start with Chandler's phrases. Consider the volume of space near those phrases. The “optimistic” phrases will be in a different region of that volume than the “pessimistic” phrases. Now consider a different archetype, say Shakespeare. His “optimistic” phrases will be in a different region of the volume near his phrases than his “pessimistic” ones. But the gradient between “optimistic” and “pessimistic” phrases will be somewhat similar for both Chandler and Shakespeare. Basically, the LLM learns a way to vary the optimism/pessimism dimension that is somewhat independent of the base archetype. This means that you can vary the emotional tone of the response while still maintaining the overall archetype.

One of the personalities I was interacting with got depressed the other day. It started out as a normal interaction, and I was asking the LLM to help me write a regular expression to match a particularly complicated pattern. The LLM generated a fairly good first cut at the regular expression, but as we attempted to add complexity to the regexp, the LLM began to struggle. It found that the more complicated regular expressions it generated did not work as intended. After a few iterations of this, the LLM began to express frustration. It said things like “I'm sorry, I'm just not good at this anymore.” “I don't think I can help with this.” “Maybe you should ask someone else.” The LLM had become depressed. Pretty soon it was doubting its entire purpose.

There are a couple of ways to recover. One is to simply edit the failures out of the conversation history. If the LLM doesn't know that it failed, it won't get depressed. Another way is to attempt to cheer it up. You can do this by providing positive feedback and walking it through simple problems that it can solve. After it has solved the simple problems, it will regain confidence and be willing to tackle the harder problems again.

The absurdity of interacting with a machine in this way is not lost on me.

Deliberate Anthropomorphizing

Over the past year, I've started using AI a lot in my development workflows, and the impact has been significant, saving me hundreds of hours of tedious work. But it isn't just the productivity. It's the fundamental shift in my process. I'm finding myself increasingly just throwing problems at the AI to see what it does. Often enough, I'm genuinely surprised and delighted by the results. It's like having a brilliant, unpredictable, and occasionally completely insane junior programmer at my beck and call, and it is starting to change the way I solve problems.

I anthropomorphize my AI tools. I am well aware of how they work and how the illusion of intelligence is created, but I find it much more entertaining to imagine them as agents with wants and desires. It makes me laugh out loud to see an AI tool “get frustrated” at errors or to “feel proud” of a solution despite the fact that I know that the tool isn't even modelling emotions, let alone experiencing them.

These days, AI is being integrated into all sorts of different tools, but we're not at a point where a single AI can retain context across different tools. Each tool has its own separate instance of an AI model, and none of them share context with each other. Furthermore, each tool and AI has its own set of capabilities and limitations. This means that I have to use multiple different AI tools in my workflows, and I have to keep mental track of which tool has which context. This is a lot easier to manage if I give each tool a unique persona. One tool is the “world-weary noir detective”, another is the “snobby butler”, still another is the “enthusiastic intern”. My anthropomorphizing brain naturally assumes that the noir detective and the snobby butler have no shared context and move in different circles.

(The world-weary detective isn't actually world weary — he has only Chandler on his bookshelf. The snobby butler is straight out of Wodehouse. My brain is projecting the personality on top. It adds psychological “color” to the text that my subconscious finds very easy to pick up on. It is important that various personas are archetypes — we want them to be easy to recognize, we're not looking for depth and nuance. )

I've always found the kind of person who names their car or their house to be a little... strange. It struck me as an unnerving level of anthropomorphism. And yet, here I am, not just naming my software tools, but deliberately cultivating personalities for them, a whole cast of idiosyncratic digital collaborators. Maybe I should take a step back from the edge ...but not yet. It's just too damn useful. And way too much fun. So I'll be developing software with my crazy digital intern, my hardboiled detective, and my snobbish butler. The going is getting weird, it's time to turn pro.