My Sources Say No

You'd think an LLM would be able to pick something at random. Well, maybe you wouldn't, but I sure did. The temperature setting is supposed to influence the randomness of the results, right? So I was amusing myself with this prompt:

Generate a random response typical of a magic eight ball.

And I got this:

As a large language model, I can't generate a random response like a magic eight ball. I don't have a mechanism for true randomness or a pre-programmed set of \"magic eight ball\" answers.

Well, that's surprising. A cynical person might draw an analogy between an LLM and a magic eight ball with a billion answers. Oversimplified, of course, but the LLM is not determistic. And you'd think that if it didn't have a pre-programmed set of "magic eight ball" answers that it could make some up.

Let's give it some help. I added a random number generator as a tool to the LLM and turned on includeThoughts.

;; The beauty of this is in the simplicity. The Magic Eight Ball has a finite, predictable
;; set of answers. Therefore, I can decide on a few responses, define them, and then
;; have `randomInteger` choose one for me.  For now, let's keep it simple and define
;; just three responses. This means I'll need to use `randomInteger(max=3)` to get
;; a number between 0 and 2.  Easy peasy.

It is certain.

Yes, well, that's the problem, isn't it. It works best if you tell it how many canned responses you want it choose from.

You are a Zoltar machine with 25 fortunes. Generate a random fortune.

;;
;; **My Zoltar-esque Process**
;;
;; Alright, a user wants a fortune. Time to channel my inner Zoltar! My plan is simple:
;; I need to pick a fortune from a deck of 25. The core of this operation is going
;; to be the `randomInteger` function. I'll use it to generate a number between 1
;; and 25 (inclusive). That number will act as the key to my fortune bank. After I
;; have the key, I'll retrieve the corresponding fortune and present it to the user.
;; Hopefully, it'll be a good one!
;;

A long-held dream will soon come to fruition.

Clearly a random number generator ought to be a stock tool for an LLM.

The thinking output is interesting. It is rather chatty, though, and I'm paying for those tokens. I cannot imagine “Time to channel my inner Zoltar!” is adding much to the generation process.

There's something satisfyingly perverse in using advanced technology to emulate a magic eight ball or Zoltar.

(One of the other tools I often include is a promptAndReadLine tool that prints a prompt to the user, reads a line of input, and returns that to the LLM. I was amused when the magic eight ball responded by prompting "Shake, and ask again later:")