I Wrote a Compiler

I was bored so I wrote a compiler. I'm lazy so I vibe coded it. It compiles Lisp to .NET IL (the byte code that the .NET runtime executes). The IL is then JIT compiled to machine code and executed. You can use the dotnet runtime from Microsoft or the open source mono runtime as the runtime for the compiled code.

The basic idea of the compiler is to map lambda expressions to .NET classes. The lexical variables are stored as fields in the class. The body of the lambda is compiled to a method in the class. We use lambda lifting to flatten any nested lambdas. We use cell conversion to handle mutable variables and we simply copy the values of immutable variables into the lifted lambdas when they are closed over.

Although I `vibe coded` the compiler, I leveraged my experience with writing compilers to break down the problem into passes that were simple enough that `vibe coding` was possible. For instance, in order to implement lambda lifting, I first wrote a pass that determined the free variables of each lambda. That's a pretty simple operation that I could easily `vibe code`. In order to emit the correct IL, I first wrote a pass that segregated the variables into arguments, lexicals, and globals. Again, that's a simple operation that I could easily `vibe code`.

The trickiest part was the code generator. I had decided to implement tail recursion by using the `tail.` prefix in the IL. This is a hint to the JIT compiler that the call is a tail call and that it can optimize it by reusing the current stack frame. However, the JIT compiler is a bit picky about when it will actually perform the tail calls, and the other parts of the code generator kept moving the tail calls around so that they were no longer in tail position. I eventually had to add a pre-pass to the code generator that tracked the continuations in order to ensure that there was enough information later on to enforce tail position on the tail calls.

It... works? It compiles a number of the Gabriel Benchmarks, and some test programs that demonstrate lexical scoping, mutable variables, and tail recursion. It is most definitely a Lisp compiler, but if you look under the hood, well, be forewarned. It isn't pretty.

The compiler itself was vibe coded. The only restriction on the output code was that it had to implement what the input code specified. It did not have to conform to any particular notion of how to implement lisp features on the .NET runtime beyond the requirement that the output was correct. Choices that are typically made by a Lisp architect, such as how to deal with integers, the implementation of the standard library, etc., were all left up to the vibe coding process. I provided a couple of runtime libraries: a cell library for implementing mutable variables, and a List library for implementing singly linked lists. These were written in C#. The vibe coding process was allowed to modify the C# code in these libraries as well and it did so in a couple of places.

I started with one a simple benchmark and got it to compile and run. From there, I added more benchmarks and each time told the compiler to fix any errors that came up. I also added some test programs that were not part of the benchmarks in order to test specific features of the compiler. As I added more and more test programs, the `vibe coding process` added more and more features to the compiler. This ended up producing more and more complex compiler output code.

I'm going to devote a few blog posts to this compiler, so if it isn't up your alley, skip ahead a few posts.