Why code-as-data?

It means that your program code you write is also data which can be manipulated by a program. Take a simple Scheme expression like

(+ 3 (* 6 7))

You can regard it as a mathematical expression which when evaluated yields a value. But it is also a list containing three elements, namely +, 3 and (* 6 7). By quoting the list,

 '(+ 3 (* 6 7))

You tell scheme to regard it as the latter, namely just a list containing three elements. Thus, you can manipulate this list with a program and then evaluate it. The power it gives you is tremendous, and when you "get" the idea, there are some very cool tricks to be played.

Code-as-data is actually only one side of the coin. The other is data-as-code.

The possibility to embed arbitrary data in Lisp code and load and reload it on the fly makes it (the data) very convenient to handle because it can eliminate any potential impedance mismatch between the way the data is represented and the way the code works.

Let me give you an example.

Let's say you want to write some kind of computer game with various monster classes. You have basically two choices: model the monster classes within your programming language or use a data-driven approach where the class descriptions are read from, say, an XML file.

Doing the modelling within the programming language has the benefits of ease of use and simplicity (which is always a good thing). It's also easy to specify custom behaviour depending on the monster class as needed. Finally, the implementation is probably pretty optimised.

On the other hand, loading everything from data files is much more flexible. You can do multiple inheritance where the language doesn't support it; you can do dynamic typing; you can load and reload things at run-time; you can use simple, to-the-point, domain-specific syntax, and much more. But now you need to write some kind of runtime environment for the whole thing, and specifying behaviour means either splitting the data up between the data files and the game code or embedding a scripting language, which is yet another layer of incidental complexity.

Or you can do it the Lisp way: specify your own sublanguage, translate that into code, and execute it. If the programming language you're using is sufficiently dynamic and syntactically flexible, you get all the benefits from using a data-driven approach (since code is data) combined with the simplicity of keeping everything in the code (since data is code).

This isn't specific to Lisp, by the way. There are various shades of code-data-equivalence gray in between Lisp and, say, C++. Ruby, for example, makes embedding data within the application easier than Python does, and Python makes it easier than Java does. Both data-as-code and code-as-data are more of a continuum than they are either-or questions.

As a Lisp programmer you learn to think of a program source as data. It is no longer static text, but data. In some forms of Lisp the program itself is that data structure, which gets executed.

Then all the tools are oriented that way. Instead of a textual macro processor Lisp has a macro system which works over programs as data. The transformation of programs to and from text has also its tools.

Let's think about adding two elements of a vector:

(let ((v (vector 1 2 3)))
   (+ (aref v 0)
      (aref v 1)))

There is nothing unusual about it. You can compile and run it.

But you could also do this:

(let ((v (vector 1 2 3)))
   (list '+
         (list 'aref v 0)
         (list 'aref v 1)))

That returns a list with a plus symbol and two sublists. These sublists have the symbol aref, then the array value of v and the index value.

That means that the constructed program contains actually symbols, but also data. The array is really a part of the sublists. So you can construct programs and these programs are data and can contain arbitrary data.

EVAL then evaluates the program as data.

CL-USER 17 > (setf *print-circle* t)
=>  T

Above tells us that the printer should print circular data structures such that the identities are preserved when read back.

CL-USER 18 > (let ((v (vector 1 2 3)))
               (list '+
                     (list 'aref v 0)
                     (list 'aref v 1)))
=>  (+ (AREF #1=#(1 2 3) 0) (AREF #1# 1))

Now let's eval the data as a Lisp program:

CL-USER 19 > (EVAL (let ((v (vector 1 2 3)))
                     (list '+
                           (list 'aref v 0)
                           (list 'aref v 1))))

=>  3

If your compiler expects text as source one can construct these texts, but they can never reference data directly. For these text based source construction many tools have been developed, but many of these tend to work in stages. In Lisp the functionality of manipulating data can be directly applied to manipulate programs and this functionality is directly built-in and part of the evaluation process.

So Lisp gives you an additional degree of freedom and new ways to think.