chanson

I've talked about the LLVM Compiler Infrastructure in the past, but what I haven't talked about yet is just how easy and quickly you can build it on your own Mac running Leopard! This is a great way to get into hacking on compiler lexical analyzers and parsers, code generators, optimizers, and so on.

What's more, you can build both LLVM and the new C front-end clang very easily and in five to ten minutes.

First, create a work area to check them out into, wherever you normally create your projects.

[~]% cd /Projects
[/Projects]% mkdir LLVM
[/Projects]% cd LLVM
[/Projects/LLVM]%

Then check out LLVM itself and clang from the LLVM Subversion repository.

[/Projects/LLVM]% svn checkout http://llvm.org/svn/llvm-project/llvm/trunk llvm
[/Projects/LLVM]% cd llvm/tools
[/Projects/LLVM/llvm/tools]% svn checkout http://llvm.org/svn/llvm-project/cfe/trunk clang
[/Projects/LLVM/llvm/tools]% cd ../..
[/Projects/LLVM]%

Then edit the PARALLEL_DIRS definition in llvm/tools/Makefile to tell it about clang. Just add clang onto the end, like this:

PARALLEL_DIRS := llvm-config  \
                 opt llvm-as llvm-dis \
                 llc llvm-ranlib llvm-ar llvm-nm \
                 llvm-ld llvm-prof llvm-link \
                 lli gccas gccld llvm-extract llvm-db \
                 bugpoint llvm-bcanalyzer llvm-stub llvmc2 \
                 clang

Now create a directory to build into, next to your llvm directory, and change into it.

[/Projects/LLVM]% mkdir build
[/Projects/LLVM]% cd build
[/Projects/LLVM/build]%

This is where you'll actually run configure. This will ensure your source tree isn't polluted with build products, and that everything stays self-contained while you hack.

[/Projects/LLVM/build]% ../llvm/configure --enable-targets=host-only
# lots of logging
[/Projects/LLVM/build]%

You'll note that above I passed an argument to configure. This ensures that LLVM is only built to target the architecture I'm running on, to speed up the build process; this is generally fine for simple front-end development.

Now, to build LLVM as well as clang all I have to do is invoke make. LLVM is set up to correctly do parallel builds, so I'll pass the number of CPUs I have in my machine via make -j 4.

[/Projects/LLVM/build]% make -j 4
# lots of logging
[/Projects/LLVM/build]%

That's it! LLVM is now (hopefully) successfully built. All of the pieces are in the build directory under Debug/bin and Debug/lib and so on; see the LLVM web site for details about what the various components are.

The LLVM Compiler Infrastructure is a great technology that came out of the computer science research community and can be used to develop extensible compiler platforms. Among other things, it provides a platform-independent assembly and object code (the "low level virtual machine" that its name is taken from), and a great object-oriented compilation, linking, optimization and code-generation infrastructure that you can use to efficiently target real hardware. The main idea is that LLVM provides a comprehensive back-end that you can easily build a front-end to target.

There's a huge amount of material available on the LLVM web site, including the LLVM Assembly Language Reference Manual and LLVM Programmer's Manual, a wide variety of papers on LLVM, and a great walkthrough of the creation of Stacker, a Forth front-end that targets LLVM. It shows how the budding language creator might leverage the tools available as part of the LLVM infrastructure. I fully expect that in time, "little languages" will have no excuse not to be JIT compiled simply because targeting LLVM is actually easier than writing your own interpreter or bytecode engine! Just walk your AST and generate naïve LLVM code for what you encounter, and let the infrastructure handle the rest. (For those who aren't developer tools weenies, an Abstract Syntax Tree is the internal representation of a program's structure before it's turned into instructions to execute.)

A couple months back, the May 2007 LLVM Developers' Meeting was held at Apple. The proceedings from this meeting — the actual session content, both in slides and in video form — are available online, and I've even created an LLVM Developers' Meeting podcast (including a subscribe-directly-in-iTunes version) for easy viewing. The video may be low bit rate, but it has a 16:9 aspect ratio so you can even pretend it's HD. (I put together the podcast primarily so I could watch the sessions on my Apple TV, since I couldn't attend the meeting.)

So if you're at all interested in compilers, language design or development, optimization, or development platforms in general, you'll be very well-served by checking out LLVM. It is a seriously cool enabling technology.

Jonathan Bachrach & Keith Playford, D-Expressions: Lisp Power, Dylan Style (1999):

This paper aims to demonstrate that it is possible for a language with a rich, conventional syntax to provide Lisp-style macro power and simplicity. We describe a macro system and syntax manipulation toolkit designed for the Dylan programming language that meets, and in some areas exceeds, this standard. The debt to Lisp is great, however, since although Dylan has a conventional algebraic syntax, the approach taken to describe and represent that syntax is distinctly Lisp-like in philosophy.

This paper includes a pretty interesting overview Dylan's macro system, which is unique in that it's a Lisp-style macro system for an Algol-style language. Which makes sense, since Dylan is really a Lisp-family language. The paper goes on to describe extensions to the Dylan macro system that were used in implementing the Functional Objects Dylan system Functional Developer (formerly Harlequin Dylan).

In a short appendix it also covers Common Lisp and Scheme macros.

There's a big difference between C/C++ preprocessor macros and Lisp macros. C preprocessor macros are text substitution while Lisp macros let you define syntax. Dylans macros let you do that too, allowing you to create new syntax and then use it to improve the expressiveness of your programs.

I'm an unabashed Objective-C fan. I think it's one of the best languages around: The Smalltalk object model with complete access to C. On the other hand, the fact that it's C makes it harder to develop high-end Objective-C tools.

I'm reading Refactoring by Martin Fowler right now, and it's totally obvious how not having a preprocessor or header files can really help in creating tools like a refactoring browser. (Of course, I learned object-oriented programming with Smalltalk, so I know firsthand how useful tools like that can be.)

I want some of this stuff for Objective-C but unfortunately it's going to be a whole bunch more work for the tool vendors to ship. Not as much work as it would be for C++, but definitely more than it is for Java. (Maybe someone needs to create a new language, like Objective-C but without the C baggage like header files, machine pointers, or a preprocessor. Sort of like WebScript with classes. It could be called "Objective" or maybe "Smalltalk"...)