It was a while back that I got a notice from Shibuya lisp that the 100th event is coming up. It’s a (Common) Lisp/Clojure meetup in Tokyo (though since covid, online). I don’t know if it’s a common thing among lispers, but everyone there seems to at least try writing their own lisp (and talk about it) somewhere down the path.

Before I wasn’t that interested. I could do most of what I wanted to do in Clojure without too much pain. Then I tried writing a (performant) wrapper around Netty and it got a bit more painful. Things like nth calls on function argument lists started showing up on my flame charts (testing with 100 million requests) and rough edges around interop cut my hands (hello proxy and abstract classes).

So when I was listening to someone talking about their newest attempt at a lisp interpreter, my mind caught fire and went “I should try this”. A few weeks later here I am, still just wildly experimenting (though that much is expected). I decided to try writing a lisp compiler in Rust. Of course things won’t go easy.

Calling it a compiler though might be a bit of a stretch. At this point the first-stage goal is to codegen Rust instead of compile to some byte-code. I don’t know how slow or fast this will be, but Rust’s guarantees around memory safety and such are things I don’t want to throw out the window.

Goals

One thing I did downgrade to “maybe eventually” is a full REPL. It’s pretty much trivial to implement in an interpreter, but compiled? I’ll need to think about that for a few more years.

I want macros in the Clojure sense. It’s extremely powerful to be able to write code that writes code for me. I can foresee this sending me down a few rabbit holes as I implement quoting, but that’s a cost I’m willing to pay for brevity.

I want seamless and cheap Rust interop. I don’t want to write weird foreign interface (FFI? RFI?) wrappers around everything. Just call it, it should work. Or worst case there should be a macro that makes it easy to call.

Progress

Difficulties were to be expected. Rust is a prime example of a statically typed compiled language. Without reflection of any kind (why?). Just transpiling to Rust is tricky enough, but add lisp macros to the mix (with the promise of “code as data”) and you’ve got an express ticket to hell.

I haven’t yet figured out how to deal with types either. Coming from the Rust side of things, this might sound like heresy, but coming from the Clojure side like I do it’s the way I roll. I hope it’ll turn into a language where types can be a “later concern” to harden your implementation instead of an upfront cost for nothing.

I think Rust’s limitations on what’s possible will put a few limitations on the language too. At this point it smells like (at least) macros will have to have a fixed type signature or else. But other than macros (and other “call lisp from Rust” bits) it should be pretty smooth. Right? Right?!