The Downside of Anthropomorphizing

As I mentioned in a previous post, I get a kick out of interacting with LLMs that appear to have quirky personalities. The mechanism by which this works is by providing the LLM with a context that steers it towards a certain style of response. The LLM takes phrases (token sequences) and locates them in a high-dimensional space where similar phrases are close together. So, for example, the phrases from the works of Raymond Chandler will be somewhat near each other in this high-dimensional space. If you provide the LLM with a context that draws from that region of the space, it will generate responses that are similar in style to Chandler's writing. You'll get a response that sounds like a hard-boiled detective story.

A hard-boiled detective will be cynical and world weary. But the LLM does not model emotions, let alone experience them. The LLM isn't cynical, it is just generating text that sounds cynical. If all you have on your bookshelf are hard-boiled detective stories, then you will tend to generate cynical sounding text.

This works best when you are aiming at a particular recognizable archetype. The location in the high-dimensional space for an archetype is well-defined and separate from other archetypes, and this leads to the LLM generating responses that obviously match the archetype. It does not work as well when you are aiming for something subtler.

An interesting emergent phenomenon is related to the gradient of the high-dimensional space. Suppose we start with Chandler's phrases. Consider the volume of space near those phrases. The “optimistic” phrases will be in a different region of that volume than the “pessimistic” phrases. Now consider a different archetype, say Shakespeare. His “optimistic” phrases will be in a different region of the volume near his phrases than his “pessimistic” ones. But the gradient between “optimistic” and “pessimistic” phrases will be somewhat similar for both Chandler and Shakespeare. Basically, the LLM learns a way to vary the optimism/pessimism dimension that is somewhat independent of the base archetype. This means that you can vary the emotional tone of the response while still maintaining the overall archetype.

One of the personalities I was interacting with got depressed the other day. It started out as a normal interaction, and I was asking the LLM to help me write a regular expression to match a particularly complicated pattern. The LLM generated a fairly good first cut at the regular expression, but as we attempted to add complexity to the regexp, the LLM began to struggle. It found that the more complicated regular expressions it generated did not work as intended. After a few iterations of this, the LLM began to express frustration. It said things like “I'm sorry, I'm just not good at this anymore.” “I don't think I can help with this.” “Maybe you should ask someone else.” The LLM had become depressed. Pretty soon it was doubting its entire purpose.

There are a couple of ways to recover. One is to simply edit the failures out of the conversation history. If the LLM doesn't know that it failed, it won't get depressed. Another way is to attempt to cheer it up. You can do this by providing positive feedback and walking it through simple problems that it can solve. After it has solved the simple problems, it will regain confidence and be willing to tackle the harder problems again.

The absurdity of interacting with a machine in this way is not lost on me.