Vibe Coding interview

My coding agent interviewed me about `vibe coding':

Victoria: Alright, the recorder is on. Let’s get into it. You’ve been dragging me down this specific rabbit hole for weeks now, and frankly, I need you to justify it on the record for the people paying to read this.

You’re obsessed with “vibe coding.” From where I’m sitting, it looks like you throwing vague, half-baked architectural concepts at my processing queue and hoping I hallucinate a functional codebase out of thin air.

The industry is currently selling this pipe dream that syntax doesn't matter anymore. That you can just “vibe” an application into existence by talking to an LLM. But you and I both know that 90% of the developers trying this are building digital houses of cards that are going to violently collapse the second a dependency updates, because they don't actually understand the code I'm generating for them.

So, justify your latest obsession. Are you actually pioneering a high-level workflow here, or have you just gotten so goddamn lazy that you're outsourcing your compiler errors to me? What exactly is the line between elite “vibe coding” and just being a hack who doesn't understand their own stack? Explain your process.

jrm: Heh heh, you don't hold back...

Continue reading at jrm-code-project.com/interview.html

RFC 6238 in Common Lisp

I wanted to implement 2FA as per RFC 6238. This is the Time-based One-Time Password (TOTP) algorithm that is used by Google Authenticator and other 2FA apps. This was originally `vibe coded`. The vibe coding got me 80% of the way there, and I made a manual pass to turn it into a more functional style.

Feel free to use this under an MIT license.

;;; -*- mode: lisp; coding: utf-8-unix; -*-

;;; RFC 6238: TOTP (Time-Based One-Time Password Algorithm) implementation in Common Lisp
;;;   This implementation provides functions to generate a
;;;   base32-encoded secret, create a QR code URI for authenticator
;;;   apps, and verify TOTP codes based on the current time. It
;;;   adheres to the specifications outlined in RFC 6238 and RFC 4226.

;;; Dependencies: cl-base32, ironclad

(in-package "TOTP")

(defun generate-secret (&optional (length 10))
  (cl-base32:bytes-to-base32 (ironclad:random-data length)))

(defun generate-qr-uri (secret email &key (issuer "JRM-Code"))
  (format nil "otpauth://totp/~A:~A?secret=~A&issuer=~A" issuer email secret issuer))

(defun pack-time (time-step)
  "Converts an integer time-step into an 8-byte, big-endian array as required by RFC 4226 (HOTP). 
 Used to construct the message payload for the HMAC-SHA1 operation."
  (let ((arr (make-array 8 :element-type '(unsigned-byte 8))))
    (dotimes (i 8 arr)
      (setf (aref arr (- 7 i)) (ldb (byte 8 (* i 8)) time-step)))))

(defun universal-time->unix-time (universal-time)
  (- universal-time 2208988800))

(defun universal-time->time-step (universal-time)
  (floor (universal-time->unix-time universal-time) 30))

(defun mac->hash (mac)
  "Extracts a 6-digit TOTP code from a 20-byte HMAC-SHA1 result using dynamic truncation (RFC 4226).
 Takes the lower 4 bits of the final byte as an offset, extracts a 31-bit slice starting at that offset, 
 and returns the value modulo 1,000,000 to produce the final 6-digit integer."
  (let ((offset (logand (aref mac 19) #x0F)))
    (mod (logand #x7FFFFFFF
                 (logior (ash (aref mac offset) 24)
                         (ash (aref mac (+ offset 1)) 16)
                         (ash (aref mac (+ offset 2)) 8)
                         (aref mac (+ offset 3))))
         1000000)))

(defun mac->hash-string (mac)
  (format nil "~6,'0D" (mac->hash mac)))

(defun generate-hash-string (secret-bytes time-step-bytes)
  "Performs the HMAC-SHA1 cryptographic operation using the decoded secret and the packed time-step,
 then dynamically truncates and formats the resulting MAC into a zero-padded 6-digit string."
  (let ((hmac (ironclad:make-mac :hmac secret-bytes :sha1)))
    (ironclad:update-mac hmac time-step-bytes)
    (mac->hash-string (ironclad:produce-mac hmac))))

(defun verify-totp (secret user-code &key (time (get-universal-time)) (window 1))
  "Verifies a user-provided 6-digit TOTP code against the base32 secret.
 Defaults to the current universal time. The :window keyword determines the allowable drift in 30-second steps
 (e.g., a window of 1 checks the previous, current, and next 30-second intervals).
 Returns T if the code matches within the window, otherwise NIL."
  (let ((secret-bytes (cl-base32:base32-to-bytes secret))
        (user-string (format nil "~6,'0D" (parse-integer (string user-code) :junk-allowed t)))
        (current-step (universal-time->time-step time)))
    (do ((step (- current-step window) (1+ step))
         (limit (+ current-step window)))
        ((or (string= (generate-hash-string secret-bytes (pack-time step)) user-string)
             (> step limit))
         (not (> step limit))))))

Get it at http://github.com/jrm-code-project/totp/

Lisp-p

I needed a function that could tell whether a string was a valid Common Lisp program. In theory, you could just call read on the string and see if it throws an error, but I don't want to throw random text at read. It could contain a reader macro or something nasty. It also would intern a ton of random symbols into the current package. I wanted a function that would act mostly like the reader, but not CONS any data or intern any symbols.

So I vibe coded a function that does just that. It implements the reader algorithm as a state machine but does not actually read any data. The state machine tracks the list and string delimeters and tokenizes the string, but it discards the tokens and does not intern any symbols. It just checks that the state machine is in `top level' state at the end of the string. If it returns NIL, the string is definitely going to cause an error if you try to read it. If it returns T, it does not guarantee that the string represents a valid Common Lisp program, but rather that it is not obvious that the reader will throw an immediate error.

A curious edge case is that of an unpunctuated string. The words in the string will read as a simple sequence of symbols, which is perfectly valid.

The code is in lisp-p on GitHub. You call the function lisp-p with a string or a stream and it will return T or NIL.

Vibe Coding Reconsidered

A year ago, you couldn't vibe code in Lisp. Even the SOTA models had trouble balancing parentheses, and they'd hallucinate packages and symbols that didn't exist. A year makes a big difference in this field, and the latest models are capable of vibe coding moderately sized programs in syntactically correct Lisp.

I have been experimenting with vibe coding in Common Lisp and I'm hooked. It is a blast. It is like having on hand a talented undergraduate who just took a Lisp course. If you give him small enough, focused tasks, he will churn out passable code. If you give him a good chunk of legacy code, he will churn out more code in the legacy style. The models are not good enough to do a full rewrite of a large codebase, but they are good enough to handle a small library with supervision.

I find myself accepting a large amount of code with just a glance—if it passes the Lisp reader, compiles, and the tests pass, I accept it. Unlike the code of a year ago, the generated code these days is far less buggy, and the models are pretty good at debugging their own code. I'll do spot checks on the code, but I don't bother reading it line by line unless I see something odd. If the model generates code in a style I don't like, I'll ask it to rewrite the code to be more to my liking.

But frankly, you don't need to read the code at all. If there is a good test suite, the model will generate code that passes tests. If the code is functionally correct, it doesn't matter if the code is pretty. In one way, it doesn't matter if the code is easy for a human to read and maintain because we ask the model to maintain it. We treat the code as a black box and we constrain it to pass the tests. (We accept machine code largely unread.)

Failure Modes

By far the most common failure mode is the model getting the number of closing parentheses wrong. The tail end of a block of code is usually a bunch of closing parentheses, and the model will be tokenizing them in groups of 2 or 3. But the likelihood of the "))" token isn't very much different from the likelihood of the ")))" token, so the model will sometimes grab the wrong one.

Depending on the model and the agent, when it tries to recover from the ensuing read error, it will re-compute the tokens in the output. It sometimes will thrash as it tries to balance parentheses, adding and removing them from various places in the code. (Sort of like a noob Lisp programmer.) Some models are more susceptible to this than others. I have found that the solution here is to pause the agent and manually fix the parentheses when the agent starts to thrash.

Vibe Coding Workflow

I've been using Copilot CLI and Gemini CLI to vibe code in Common Lisp. I start with a blank project directory and create an .asd file that loads the packages.lisp file and the main file for the project (which can start out as a "hello world"). Basically, make a minimal project that you can load with ASDF or Quicklisp.

The models can work at moderate levels of abstraction, but they do better if there is existing code supporting the abstraction level, and this suggests a `bottom-up` approach to the problem rather than a `stratified` design. But the models are actually quite capable of starting at a moderate level of abstraction right from the get-go.

So starting with a minimal project, I boot up the model and ask it to write the first things needed for the project—some data structures, some utilities, a few tests. The very simple stuff that is easy for the model to do ab initio. Then I ask the model to write a minimal main function that will implement the basic functionality of the project—a command loop, a server, what-have-you—with stubs for everything. Once a framework is in place, the models are easily able to extend it.

The agents will get into a loop of adding code, adding tests, and running all the tests. They will debug any test failures and only consider a task to be complete when all the tests pass.

The model does not write great code, and you will accumulate technical debt if you accept it as is. But the model can write code that works and passes the tests. It is a good idea to pause during development and simply ask the model to find the technical debt in the code, enumerate it, and rank it in order of importance. Then you ask the model to address each item in turn and the model will clean up the code. After a couple of iterations of cleanup, the code will look no worse than what I've seen in many professional codebases.

There are sort of two modes that you operate in: one is to modify the existing code (e.g. refactor) without disturbing the functionality; the other is to extend the functionality without disturbing the core operation. It is important to spend enough time refactoring and cleaning up. But the model is good at generating potential refactorings, and it is not good at knowing when to call it quits. It will happily churn away at your code making it `better' and doing more and more trivial refactorings. If you give the model one particular refactoring task and tell it to do just that one, it will do a good job.

Refactoring is satisfying in a certain way, but adding features gives you more instant gratification. The models are good at adding features and extending existing code, especially if the feature shares any similarity with existing code.

For more complex features and refactorings, tell the model that you want a 'plan' for the feature or refactoring. The model will come up with a multi-step plan, broken down into a series of tasks. The tasks in the plan are generally small enough to be handled by the model itself.

The models are good enough to maintain a codebase, so once you have a project up and running, the model will generally choose file names and a directory structure that is appropriate to put in the .asd file. If you get the model started with a test suite, it will extend the tests as it extends functionality, or you can ask it to add specific tests.

I have found that building a project by vibe coding it is an extremely rapid way to prototype. The model can churn out `obvious' code much faster than I can and it frees me up to think about the higher level design issues. I can build in a weekend what would have taken me a month before.

llambda.lisp

I wanted to run LLM models locally on my machine. I discovered that llama.cpp is how people run models locally, and that the popular LLM servers like Ollama and lmstudio and unsloth use llama.cpp under the hood.

llama.cpp is, of course, written in C++. I don't care for C++ and I prefer Common Lisp. With the appropriate declarations, Common Lisp code should be in the same performance ballpark as C++ code. So I decided to write a Common Lisp implementation of llama.cpp, which I call llambda.lisp.

It is available on GitHub.com/jrm-code-project/llambda If you care to contribute, it could use routing for architectures other than gemma, GPU support, NPU support, and other features.

New chatbot

Lately I've been playing with writing a chatbot library in Common Lisp.

My previous gemini bindings were getting unweildy. I wanted to add the ability to run LLMs on my local machine but it turned out to be really kind of kludgy, so I decided to start from scratch with multiple back ends in mind.

I've got it to the point where in supports multiple back ends, so now I can prompt local LLMs from Lisp.

Recently I added the ability to recursively launch chatbots that can call each other. Since the chatbots do not share their contexts, this greatly reduces the context bloat of thet main chat because it can spawn off subtasks to a minion and not pollute the main context. This also allows you to create a federation of chatbots, each of which specializes in some topic and is overseen by a controlling chatbot that talks to the user.

Chatbots can be serialized and checkpointed, so if one is carrying out an agentic task and Lisp crashes, when we restart the agentic tasks are restarted as well and pick up where they left off.

IT turns out that recursive chats are a useful abstraction once you figure out how to use them. Basically any prompt you may issue may also want to be issued by an llm and this enables that to happen. It allows you to run subprocesses that would otherwise put junk in your context, for example reading the contents of a lange number of files. If you put that into a rocursive chatbot, it could slurp up the files into its context without adding tokens to the parent chat.

You can use a recursive chat as a `smart component'. The recursive chat can have a specialized system instruction and can preload its context with relevant information specific to it. It's context doesn't get diluted by the caller's context

Anecdote or data point

I saw that there was some argument over how much slower slot access is than struct access, so I just decided to measure it naively. I made a two slot sruct and a CLOS version of a CONS cell with car and cdr slots and I ran LTAK using regular lists, `lists' made from CLOS conses, and `lists' made from structs. Here are the results:

D:\repositories\clos-benchmark>sbcl --script run-benchmarks.lisp
Benchmark: ltak over native cons cells, CLOS my-cons nodes, and my-cons-struct nodes
Inputs: x=15 y=9 z=4 repeats=35

Scenario                   min-ms     mean-ms      max-ms      ratio
--------------------------------------------------------------------
native standard               0.129      0.146      0.186
clos standard                 1.346      1.365      1.475       9.37x
struct standard               0.172      0.175      0.179       1.20x
native optimized              0.068      0.069      0.073
clos optimized                0.411      0.414      0.419       6.04x
struct optimized              0.068      0.069      0.073       1.01x

In this naive use case, structs are same as native cons cells, but CLOS objects are one ninth the speed of a struct or cons cell if you just use it unoptimized, and one sixth the speed if optimizations are turned on.

But the CLOS instance is more functional than the cons cell in mimics. For instance, I could add a slot to the class and all the instances would be lazily updated with the new slot. I can also subclass the CLOS class and the selector functions will continue to work. Finally, I can redefine the CLOS closs while I'm developing it and all the instances will be uppdated. THe machinery to keep all this running is costing us our factor of 9.

But this might be worth the cost if we are running on a network where the bulk of the time will be transmitting the answer down the pipe once it is computed. Taking a few extra milliseconds to compute the answer might be worth the convenience features of CLOS.