I had no luck vibe coding Common Lisp, but I'm pretty sure I know the reasons. First, Common Lisp doesn't have as much boilerplate as other languages. When boilerplace accumulates, you write a macro to make it go away. Second, Common Lisp is not as popular language as others, so there is far less training data.
Someone made the interesting suggestion of doing this in two steps: vibe code in a popular language, then ask the LLM to translate the result into Common Lisp. That sounded like it might work, so I decided to try it out.
Again, I used "Minesweeper" as the example. I asked the LLM to vibe code Minesweeper in Golang. Golang has a lot of boilerplate (it seems to be mostly boilerplate), and there is a good body of code written in Golang.
The first problem was that the code expected assets of images of the minesweeper tiles. I asked the LLM to generate them, but it wasn't keen on doing that. It would generate a large jpeg image of a field of tiles, but not a set of .png images of the tiles.
So I asked the LLM to vibe code a program that would generate the .png files for the tiles. It took a couple of iterations (the first time, the digits in the tiles were too small to read), but it eventually generated a program which would generate the tiles.
Then I vibe coded minesweeper. As per the philosophy of vibe coding, I did not bother writing tests, examining the code, or anything. I just ran the code.
Naturally it didn't work. It took me the entire day to debug this, but there were only two problems. The first was that the LLM simply could not get the API to the image library right. It kept thinking the image library was going to return an integer error code, but the latest api returns an Error interface. I could not get it to use this correctly; it kept trying to coerce it to an integer. Eventually I simply discarded any error message for that library and prayed it would work.
The second problem was vexing. I was presented with a blank screen. The game logic seemed to work because when I clicked around on the blank screen, stdout would eventually print "Boom!". But there were no visuals. I spent a lot of time trying to figure out what was going on, adding debugging code, and so on. I finally discovered that the SDL renderer was simply not working. It wouldn't render anything. I asked the LLM to help me debug this, and I went down a rabbit hole of updating the graphics drivers, reinstalling SDL, reinstalling Ubuntu, all to no avail. Eventually I tried using the SDL2 software renderer instead of the hardware accelerated renderer and suddenly I had graphics. It took me several hours to figure this out, and several hours to back out my changes tracking down this problem.
Once I got the tiles to render, though, it was a working Minesweeper game. It didn't have a timer and mine count, but it had a playing field and you could click on the tiles to reveal them. It had the look and feel of the real game. So you can vibe code golang.
The next task was to translate the golang to Common Lisp. It didn't do as good a job. It mentioned symbols that didn't exist in packages that didn't exist. I had to make a manual pass to replace the bogus symbols with the nearest real ones. It failed to generate working code that could load the tiles. I looked at the Common Lisp code and it was a horror. Not suprisingly, it was more or less a transliteration of the golang code. It took no advantage of any Common Lisp features such as unwind-protect. Basically, each and every branch in the main function had its own duplicate copy of the cleanup code. Since the tiles were not loading, I couldn't really test the game logic. I was in no mood to debug the tile loading (it was trying to call functions that did not exist), so I left it there.
This approach, vibe in golang and then translate to Common Lisp, seems more promising, but with two phase of LLM coding, the probability of a working result gets pretty low. And you don't really get Common Lisp, you get something closer to fully parenthesized golang.
I think I am done with this experiment for now. When I have some agentic LLM that can drive Emacs, I may try it again.
No comments:
Post a Comment