Why is Lisp so great?, part 1

There are many books and texts about Lisp on the net, so I’ll leave out the “common” stuff, and I’ll try to show in this article one aspect of what makes Lisp the “most hackable” language out there.

If you think that tuples in .Net4 are awesome, or that F# is innovative, then you can stop here. This article is not for you… Seriously, don’t waste your time…

If, on the other hand, you ever wondered why your favourite language does not have this and this feature, ever wanted to hack the compiler (and/or really did), or found yourself writing bunch of a very similar code and spent days thinking of a way to refactor the code despite the limitations of the language at hand, then Lisp is The language for you.

The main feature of (and, strangely, unique to) Lisp is its macro system. Don’t get this confused with the text-based preprocessor macros in C; macros in Lisp are full blown code generators – Lisp programs that generate Lisp code (or other macros). [1] That’s why Lisp is often described as “a programmable programming language”. [2]

What do I mean by that? Let me give an example. [3]

How can we avoid failing to exactly match claim-release of a mutex in a threaded program? How could we simply wrap any code in claim-release automatically? Lisp macros allow us to do this.

We want to be able to write simply (with-mutex m (do-something)) and have the (do-something) wrapped in claim-release calls for the mutex m. [4]

Well, that’s trivial to do; we create a macro that takes the name of the mutex, and any number of “forms” to execute, and wraps these forms in the claim-release pair:

(defmacro with-mutex (mutex &body body)
  `(progn
    (claim-mutex ,mutex)
    ,@body
    (release-mutex ,mutex)))

I hope it’s obvious what’s going on – if we write

(with-mutex m (do-a-thing) (do-another-thing))

we get the following code generated (slightly simplified for presentation):

(claim-mutex m)
(do-a-thing)
(do-another-thing)
(release-mutex m)

Now, of course there are some problems with the code (it’s not returning the value of the last form, but the value of the release-mutex call, and it’s not protected against errors/exceptions [5]), but I think you’ve got the idea. And as always, this is just a tip of an iceberg. [6]

If you’ve read this far, welcome to the Lisp family! You might end up learning the whole of Lisp, and never really using it [7], or you might find yourself transferring features of Lisp into your language of choice… [8] Lisp is more of an idea than a language (in a good sense). There are many excellent free and commercial implementations of Lisp to choose from, and even more exciting Lisp books. [9] The world is all yours to hack in!

Feel free to leave a comment! There’s much more planned to come that’s sitting in my head, but I might as well need an encouragement…

Footnotes:
[1] The name “Lisp”, short for “List Processing”, might seem archaic. But it hides what makes Lisp so unique: the basic data type in Lisp – list – is also used internally by Lisp to represent the Lisp code. (read it again, let it sink in) Thus, you can replace one “list”, representing the macro in the code, with another list, representing the generated code.
Consequences of this are way beyond the scope of this article, and are subject to several advanced Lisp books.
One consequence of this also is, that all Lisp’s functions for manipulation of lists you can also use to generate the code. (read it again, let it sink in)
This is also the reason for Lisp’s unusual (and uniform, and symmetrical) syntax.
[2] “Lisp is a programmable programming language.” – John Foderaro, CACM, September 1991
[3] The code presented here will be Lisp code, but you don’t need to understand the syntax to get the idea.
[4] Don’t worry about the hyphens in the function names – all symbols in Lisp may contain pretty much any character.
[5] Both of these problems can be solved at once with Lisp’s unwind-protect special operator:

(defmacro with-mutex (mutex &body body)
  `(progn 
    (claim-mutex ,mutex)
    (unwind-protect
      (progn ,@body)
      (release-mutex ,mutex))))

But Lisp’s conditions and restarts system is topic for its own book.
[6] Initially, I wanted to present a macro that would allow C’s “idiom” for (i = 0; i < N; i++) { /* do-something */ } to be written as (do-n-times n (do-something)). But, this macro has too many leaks to plug. Also, Lisp already defines this construct in its standard – dotimes.
[7] “Lisp is worth learning for a different reason – the profound enlightenment experience you will have when you finally get it. That experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot.” – from How to Become a Hacker by Eric Raymond
[8] “Greenspun’s Tenth Rule of Programming: any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp.” – Philip Greenspun
[9] For now, I’ll leave you with 2 recommendations: Practical Common Lisp by Peter Seibel, and “the classic” Structure and Interpretation of Computer Programs by Hal Abelson, Jerry Sussman and Julie Sussman.

4 responses to “Why is Lisp so great?, part 1

  1. Hi,
    Good article, but…
    How is that you say “…You might end up learning the whole of Lisp, and never really using it…” + the food note [7]…
    I don’t really get this. is Lisp good to make things happens or not?
    If is so good, why not use it to solve your programming problems?
    I am learning it just because I like it but eventually I want to programm ONLY in Lisp.
    According to the food note, Lisp is to the programmer what the sugar is for diabetics. We will love it but we can not use it… I don’t know… It doesn’t make sense to me.

  2. Hi Manuel,
    thanks for the comment – I’m really glad you liked the post, and thanks for the very good question!

    I didn’t want to go into details in the post, as it already got much longer than I planned in the first place, but here we go – there are several reasons for me to state that, among the most important ones are:

    * The one that the whole article is about: “Lisp is to programming what topology is to mathematics” (I made this up now, but it’s pretty much true) – it makes you see known and/or common patterns and applications in an entirely new light and on a new level. Learning Lisp (if you do actually learn to write Lisp, and not to write C++/C#/Java/… in Lisp) will show you a completely new layer of abstraction and of application design; this will reflect in your programs in any other language (or, will frustrate you that you cannot do the same in the other language).

    * Lisp is not what companies understand today under “industry’s best practice” – see e.g. Paul Graham’s “Revenge of the Nerds” essay for great treatment of this; see also his “If Lisp is so Great” essay; for this “reason”, it’s generally much harder to convince your boss that Lisp is the best choice for the problem at hand (I’ve hit this wall more times than it’s pleasant).

    * Lisp is also far from “universal” language – it’s much more probable that random programmer that sees your code in C will understand it, than your code in Lisp. If algorithms/ideas exchange is what you want, you’d (I guess) rather stick to the C than Lisp, unless the Lisp community is your sole target audience.

    For the above reasons (however harsh they may sound), I say that you can learn Lisp and never really use it.
    Of course, there is no power that can stop you from writing every single program and script in Lisp for yourself; but “in the wild” – at work and on the net – you’re much more restricted.

    Maybe one day Lisp will become the major programming language; maybe some of it’s babies will; but until then, Lisp is, to those uninitiated, this “obscure old language that no one really uses or ever used” (as I heard one manager once summarize it), and to do the “real” programming (in the sense of the profession, and not the challenge) you’ll have to stick, in most cases, to the more “conventional” languages.

  3. Funny that an article on why LISP is great would give an example on multi-threading, given that the Common Lisp standard does not even include threads. I.e. there are no such functions
    claim-mutex and release-mutex, and the LISP beginner will have to dig through plenty of details to find out whether and how to get such function in his particular LISP implementation, let alone how to make his code portable among several LISP implementations.

    Thus, just getting to run the example you provide with several threads may take hours to a beginner.

    And in particular you show another weakness of LISP, the tendency to reinvent the wheel rather than reusing code.
    Both Bordeaux-threads and Portable-threads define with-lock-held.
    A beginner should use those (after digging through plenty of forums and documentation to decide which of those libraries to rather use, as they of course do not share a common API, but may name functions similarly. And then brace himself for them moment he’ll discover that he should have chosen the other library, really, for his purpose.)

  4. Hi “NoName”,
    thanks for the comment; many points are valid, though it depends on what you see this article as.

    I didn’t mean this article to be a tutorial for Lisp, but more of an appetizer; it’s not supposed kick-off beginners, but to get programmers interested.
    In fact, IMO, Lisp is not a good language to “begin” with; you cannot appreciate the macro system in Lisp if you didn’t touch the bottom of “normal” languages in search of a perfect refactoring of your code. The elegance and symmetry of Lisp is most visible only after years of coding and complaining.

    Yes, CL does not standardize threads; neither does C or C++ (there’s still a 1+ years to come till the C++0x being finished, and with regards to threads – it’s late). Is that really a weakness?

    As for reinventing the wheel – show me one good coder that didn’t reinvent the wheel. In fact,one that didn’t spend most of the time coding what already exists.
    Don’t tell me you never wrote your own rasterizer, bitmap representation, http server, database access framework, linear systems solver, … anytime you write css or html you’re reinventing the wheel! I don’t think you can become a good coder if you do not write most of the “standardized” stuff yourself, just as well as you can’t know the pitfalls and dangers of any system if you do not write your own versions of it’s functionality.

    Btw, can you show me a code example that is simple enough, “practical”, and short enough to be included in a single post, and that is not “reinventing the wheel”?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s