Thoughts on LLMs

I've been exploring LLMs these past few weeks. There is a lot of hype and confusion about them, so I thought I'd share some of my thoughts.

LLMs are a significant step forward in machine learning. They have their limitations (you cannot Vibe Code in Common Lisp), but on the jobs they are good at, they are uncanny in their ability. They are not a replacement for anything, actually, but a new tool that can be used in many diverse and some unexpected ways.

It is clear to me that LLMs are the biggest thing to happen in computing since the web, and I don't say that lightly. I am encouraging everyone I know to learn about them, gain some experience with them, and learn their uses and limitations. Not knowing how to use an LLM will be like not knowing how to use a web browser.

Our company has invested in GitHub Copilot, which is nominally aimed at helping programmers write code, but if you poke around at it a bit, you can get access to the general LLM that underlies the code generation. Our company enables the GPT-4.1, Claude Sonnet 3.5, and Claude Sonnet 3.7 models. There is some rather complicated and confusing pricing for these models, and our general policy is to push people to use GPT-4.1 for the bulk of their work and to use Claude Sonnet models on an as-needed basis.

For my personal use, I decided to try a subscription to Google Gemini. The offering is confusing and I am not completely sure which services I get at which ancillary cost. It appears that there is a baseline model that costs nothing beyond what I pay for, but there are also more advanced models that have a per-query cost above and beyond my subscription. It does not help that there is a "1 month free trial", so I am not sure what I will eventually be billed for. (I am keeping an eye on the billing!) A subscription at around $20 per month seems reasonable to me, but I don't want to run up a few hundred queries at $1.50 each.

The Gemini offering appears to allow unlimited (or a large limit) of queries if you use the web UI, but the API incurs a cost per query. That is unfortunate because I want to use Gemini in Emacs.

Speaking of Emacs. There are a couple of bleeding edge Emacs packages that provide access to LLMs. I have hooked up code completion to Copilot, so I get completion suggestions as I type. I think I get these at zero extra cost. I also have Copilot Chat and Gemini Chat set up so I can converse with the LLM in a buffer. The Copilot chat uses an Org mode buffer whereas the Gemini chat uses a markdown buffer. I am trying to figure out how to hook up emacs as an LLM agent so that that the LLM can drive Emacs to accomplish tasks. (Obviously it is completely insane to do this, I don't trust it a bit, but I want to see the limitations.)

The Gemini offering also gives you access to Gemini integration with other Google tools, such as GMail and Google Docs. These look promising. There is also AIStudio in addition to the plain Gemini web UI. AIStudio is a more advanced interface that allows you to generate applications and media with the LLM.

I am curious about Groq. They are at a bit of a disadvantage in that they cannot integrate with Google Apps or Microsoft Apps as well as Gemini and Copilot can. I may try them out at some time in the future.

I encourage everyone to try out LLMs and gain some experience with them. I think we're on the exponential growth part of the curve at this point and we will see some amazing things in the next few years. I would not wait too long to get started. It will be a significant skill to develop.