Lisp Questions

For those thinking about learning CommonLisp...

Philosophic questions tend to receive good answers right here on Ward's wiki. Things like "is this possible in Lisp?", for instance. For specific technical questions such as "what piece of software would I use to do this?", it's best to look at the AluWiki.


How do we keep the source code files synchronized when doing incremental development? For example, if we're in a Lisp interpreter and we type in a few new function definitions, and play around with it it might be a while before we're happy with the code. How do we then get that new code into the source? I presume this is different from saving an image file. I hope the answer is something nicer than copy and paste into an editor running in parallel.

Typically you type your code directly into a source file in your Lisp-enabled editor. The development environments let you evaluate any individual expression in your source-files at any time, so it's still completely incremental. It's only "throw-away" code that you type directly into the shell - though copy&paste is always there on the occasions that you decide to keep some of it.


What are the most commonly used Lisp functions?

In my Lisp image, the five most commonly used of the standard Lisp functions are ERROR, FORMAT, APPEND, GENSYM, and WRITE-STRING. At least according to this not-quite-portable program:

 (defun most-common-functions (&optional (n 5) (package :cl))
   "Return the N most-referenced exported functions in PACKAGE."
   (let ((function-symbols '()))
     (do-external-symbols (symbol package)
       (when (regular-function-p symbol)
         (push symbol function-symbols)))
     (subseq (sort function-symbols #'> :key #'number-of-callers) 0 n)))

(defun regular-function-p (symbol) "Return true if SYMBOL names a regular (non-special/macro) function." (and (fboundp symbol) (not (or (special-operator-p symbol) (macro-function symbol)))))

(defun number-of-callers (function-name) "Return the number of functions that call FUNCTION-NAME. The result is cached on FUNCTION-NAME's NUMBER-OF-CALLERS property." (or (get function-name 'number-of-callers) (setf (get function-name 'number-of-callers) (length (function-callers function-name)))))

(The answer is actually wrong, since the FUNCTION-CALLERS I'm calling out to isn't very accurate. But nevermind that!)


I'm thinking of writing a small program (say 1 day/weekend worth of effort) to learn Lisp. Any good suggestions?

Tic-Tac-Toe. Hmmm. Anything more inspiring? How about a Lisp interpreter? You guys scare me. [I think this may have been a bit of an inside joke (well, HaHaOnlySerious)... you can implement Lisp in Lisp very easily- in the trivial case you can do it in one line of of code: see MetaCircularEvaluator.]


Are there any online diaries of somebody coding in Lisp detailing what they were thinking about and the kinds of problems and solutions they had?

Not an online diary, but I ParadigmsOfArtificialIntelligenceProgramming sounds like just the ticket.


Is anyone doing XP in CommonLisp?

You mean, is anyone doing XpInCommonLisp?

Lisp people sometimes talk about incremental development, resuming execution after a bug fix etc., but it seems hard to find anything written about this. How do we do this with something like CLisp or CMUCL? Are there any good references or examples?

Most of it is explained here: http://cvs2.cons.org/ftp-area/cmucl/doc/cmu-user/debugger.html#toc77. Essentially, when CMUCL (and other Lisps) hit a runtime error, the system stops. At this point, the programmer can make any changes they like, including redefining functions, and then hit 'restart' to continue execution from an appropriate stack frame.

See also: http://lemonodor.com/archives/000720.html#000720

Most stuff these days seems to require a Web interface. Is there something in Lisp similar to servlets in Java?

There are several somethings. There are three main HTTP servers: AllegroServe?, Araneida, and CL-HTTP, all of which provide some sort of API that provides facilities similar to the Servlet API. However, they are all different. Then there's mod_lisp for Apache. Finally, of course, like any language, Lisp can be run through CGI.

Just recently (circa 2003-08-20) there's been some discussion on news:comp.lang.lisp about coming up with a common API for all or some of these things.

         :      Hunchentoot is a popular choice, too. It has many features and is support in many Lisp implementations. See http://www.weitz.de/hunchentoot/

Is there anything like JUnit?

Again, there are several. This is a bit different because it's so darn easy to write a simple test framework in CommonLisp given macros, etc. -- which is not to say that it would be a bad thing to start a community effort around polishing, promulgating, and popularizing some particular framework.

How would I connect to a database like Oracle or PostgreSql?

There are libraries for talking to Oracle, PostgreSQL and MySQL. Some speak the database protocol over a socket; others use foreign function interface (FFI) to call the C library functions for talking to the database.

         :      I particularly like CL-SQL, which has good support for MySQL, PostgreSQL, Oracle, ODBC, and SQLite. See http://clsql.b9.com/platforms.html

What IDEs exist?

Most Lisp developers use Emacs which integrates quite closely with a running CommonLisp instance. (You can send code directly from Emacs to be evaluated in the CommonLisp image). The commercial Lisps (Franz's AllegroCommonLisp and Xanalys's LispWorks) also provide other kinds of IDEs. And there's a project called Jabberwocky (c.f. http://jabberwocky.sourceforge.net/) which is writing a portable Lisp IDE in Java. Much like the Emacs-based environments, it interacts with an underlying CommonLisp implementation to do the heavy Lisp lifting.

Can Lisp code be dynamically downloaded (e.g. over a network connection) and then executed?

Yes, but not in a way that makes it a viable language for applets, the way Java is. That's because no widely-used browser has a Lisp engine, and also because it is not designed to provide a sandbox the way Java and TCL do. (See below.)

How are graphics and GraphicalUserInterfaces handled?

It depends on the CommonLisp implementation. Most CommonLisps that run on Unix support CLX which is a CommonLisp equivalent of XLib, i.e. it natively speaks the X Windows protocol. But that's pretty low level. Beyond that, different implementations provide different solutions on different OSs. LispWorks has a single API called CAPI that it supports across all the platforms it runs on (Windows, Unices, Mac OS X) that's supposed to be pretty good. OpenMCL, a CommonLisp for OS X has an ObjectiveCee bridge that allows it to use the CocoaApis?.

Is there any GPL lisp for windows with a graphics library?

CLISP runs on Windows and is GPL'd. It has no built in graphics libraries for Windows. But, like almost all Common Lisp implementations, it has a ForeignFunctionInterface (FFI) which allows you to call C libraries. In theory one could use that to get at native Win32 widgetry but I don't know that anyone has actually made that work.

If you're not totally tied to the GPL, CormanLisp? (http://www.cormanlisp.com) is a relatively inexpensive CommonLisp implementation for Windows that offers tight integration with the Windows environment. But it is Windows only.

Can Lisp be made secure? (In the sense that Java applets can run on your machine but can't trash your hard drive)

The short answer here is no. There are many ways in which a sandbox might be implemented, but you're pretty much on your own for doing so.

However, if you're willing to select a very minimal set of allowed functions and variables, the problem is easy: simply traverse the untrusted code before executing it, to make sure that it has no references to unallowed things. This fails when you try to allow greater permissions, because there's no way to keep that controlled.

There's no switch defined by the language to say, run this code in a sandbox that is incapable of interacting with the environment in the following way. However the code-is-data-data-is-code model of Lisp gives you a lot better chance of writing such a sandboxed environment than you could in a language like C. For instance, you could, in theory, load code over the network and read it in a package where the normal function names are redefined with "safe" versions. Though that's one of those things to which the old saw, "In theory, practice is no harder than theory" applies with a vengeance.

Can we use threads?

Depends on the implementation. Most of the major commercial and free implementations support threads these days. Unfortunately there isn't a single API--they each do it in their own way. Also be aware that Lisp terminology, which comes from the days of the LispMachines calls threads "processes" which can be confusing if you're used to the Unix/POSIX terminology.

How quickly can an experienced programmer learn Lisp and start being productive?

In my experience (PeterSeibel) very quickly if one is willing to accept that things are probably different than what you're used to. That is, you do have to become one with the Lisp Way a bit. Also it depends on what you are trying to do--if "being productive" means writing multithreaded GUI apps that talk to a database, you'll have a slightly longer road ahead of you since you'll have to learn the ins and outs of the language and also find a combination of CommonLisp implementation and 3rd party libraries that support your needs in those areas that aren't standardized by the language spec. But if you are writing something that is fundamentally about computing, i.e. implementing algorithms and defining data structures, and writing programs that use them to do cool stuff, Lisp can be incredibly productive, very quickly.

Every language has its bad points; what are they for Lisp?

It would be nice if there were more de facto standards for certain aspects of "modern" computing, namely the ones you put your finger on: networking, threads, database connectivity, and GUIs. But there's nothing standing in the way of such standards arising other than the relatively small size of the Lisp community compared to say, the communities using Perl, Python, or Ruby.

Common Lisp also has quite a large library of functions and data structures, and the names and conventions it uses are sometimes obscure to new users, or sometimes even to fairly experienced users. Many are artifacts of certain periods in the history of computing; the life of Lisp after all spans most of that history. Nothing is more common for a new Lisp user than implementing an algorithm, only to find the next day that a more efficient implementation was already built into the language. To get the best results, it's important to read the literature and talk to other Lisp programmers.

For the previous reason, and for others as well, it can be hard for an inexperienced user to reason accurately about the performance impact of design and implementation decisions. Lisp functions often provide various options that affect various dimensions of efficiency, and it can take some time and experience to know what to think about in order to evaluate the effficiency of your code.

When you Lisp guys are writing these large complex Lisp programs do you actually use CLOS or is it more of an academic thing?

I use CLOS. I don't use it for everything, because I don't believe that objects are the appropriate abstraction for every software problem. I do use it when it's appropriate, and I am a great fan of CLOS's generic functions, as opposed to the member functions used by Smalltalk, C++, Java, and relatives. -- DanielKnapp

Another thing to keep in mind is that CLOS is not really a separate thing. Every piece of data in Lisp is an instance of some class. Now there is a distinction to be drawn between classes whose meta-class is built-in-class or structure-class vs standard-class in that the later can be extended by user classes, have slots that are accessible via SLOT-VALUE, etc. But all classes, regardless of their meta-class can be used as parameter specializers on methods defined on generic functions. So you could "use CLOS" by writing a bunch of generic functions and then writing methods that specialize on various existing classes. Or, conversely, you could write a bunch of user-defined classes but only ever use non-generic functions to deal with instances of them. (Though that later might be a bit strange.)

[CLOS-specific content moved to ClosQuestions]


I doubt that most Lisp programmers use Emacs. I have never seen any numbers on that topic. I would believe that most Lisp programmers use an Emacs-like editor (which includes FRED, Zmacs, Hemlock, LispWorks's editor, ...).


And there's a project called Jabberwocky (c.f. http://jabberwocky.sourceforge.net/) which is writing a portable Lisp IDE in Java.

Why aren't they writing it in Lisp? Are there reasons that Lisp isn't suited to writing IDEs?

Well, there isn't really a good cross-platform GUI framework that's portable both across OS's and Common Lisp implementations. By writing the GUI foo in Java the Jabberwocky guys solve that problem.

Is there a reason there isn't a good cross-platform GUI framework for Lisp?

Yes: Lisp lost a lot of popularity in the late 80s, for reasons beyond the scope of this page (see AiWinter), so although there were a couple really nice GUI things for it on various Lisp Machines, the whole push to make things cross-platform came too late for Lisp to get a lot of money to that end.

Also, it's not clear that Lisp is all that much worse off than languages other than Java. There are Common Lisp bindings for most of the popular GUI toolkits from raw XLIB up to GTK and native widgets on Windows, OS X, etc. But unlike Java, there's no one vendor who's adopted the mission of trying to build a platform to compete with Microsoft.

If, however, one simply wanted to write cross-OS GUI apps in Lisp, the LispWorks product from Xanalys (http://www.lispworks.com) offers a GUI API (CAPI) that's supposed to be pretty good for WriteOnceRunAnywhere GUIs across Windows, the flavors of Unix they support and Mac OS X (I think--I know they support OsX but I'm not sure whether their GUI stuff works there.)

CAPI works on Mac OS X, too.


What does the S in S-Expression stand for?

Symbolic. See EssExpressions.


Does CommonLisp accept argument passing by name instead of position and default arguments (both in function calls and in closures)? Not a big issue, but a big curiosity.

Yes.

 (defun f (x y &key thing (thang 123) (thong thang))
   (list x y thing thang thong))

(f 1 2) ===> (1 2 nil 123 123) (f 1 2 :thang 99) ===> (1 2 nil 99 99)

(defun f (x y &optional z (u (+ x y))) (list x y z u))

(f 1 2) ===> (1 2 nil 3) (f 1 2 10) ===> (1 2 10 3) (f 1 2 10 20) ===> (1 2 10 20)

Also, there is a way to distinguish whether a keyword parameter was actually specified by the caller, or whether the default value was established. This is done by adding a third element to the keyword parameter specifier:

  (defun f (x y &key (thang 123 thang-specified-p))
    (when (eql thang 123)
      (if thang-specified-p
        (print "123 was specified explicitly")
        (print "123 came in as default value"))))

(f 1 2 :thang 123) =output=> "123 was specified explicitly" (f 1 2) =output=> "123 came in as default value"


Is there a reason you can't specify how AssociationLists are implemented (hashtable, binary tree, etc.)?

That's sort of like asking why one can't specify how a hashtable is implemented, as an alist, binary tree, etc?) An alist is a particular data structure that happens to have similar (but not identical) semantics to hash tables.

That's not what I'm asking.

Now one might ask why there isn't a single unifying interface that provides different implementations, based on alists, hashtables, red-black trees, etc. No particular reason but there are perhaps enough differences between them that a unification isn't really that useful.

That's what I'm asking. Java has a Map interface and HashMap, TreeMap, etc. implementations. The AssociationList page makes it sound like a map, but says there is only one implementation and lookups are O(N).

Well, there's only one kind of alist but that's not the only data structure in Lisp that's capable of acting as a map. for instance hash tables are a first class data structure in CommonLisp, created with make-hash-table and come in four flavors depending on the kind of equality test you want to do. (Many specific Common Lisp implementations also provide a way for the user to define new flavors of hash tables that use a different equality-predicate/hash-function pair. This is, it's perhaps worth noting, different than Java where the equality/hashing is a function of the objects used as keys, not the HashMap itself.)

In fact, an alists is a bit of a funny creature since alists are not really first class objects. An alist is just a bunch of ConsCells arranged in a particular way and some functions that let you manipulate that structure conveniently. Now a ConsCell is a first class object (i.e. a given ConsCell has a unique object identity and there's a class). But an alist is just one of infinitely many ways of arranging ConsCells, albeit one that happens to have some support built into the language's standard library in the form of the aforementioned functions. Alists are useful in certain situations and are part of Lisp's history but are certainly not intended to be the be-all, end-all data structure for implementing maps. In general, where you would use a HashMap in Java you should use a hash table in CommonLisp. If you really need a red-black tree (what TreeMap is) you'll need to find or write one, since there isn't one built into the language or the standard library.

[This used to fit in somewhere] For instance, a hash table needs both a hash function and an equality predicate which are provided at instantiation time, while an alist only needs the equality predicate and it is passed in via the functions that use alists. Alists can also perform reverse lookups (via the function rassoc), just as efficiently as "normal" lookups. And alists can contain shadowed values: for a given key there can be multiple pairs in the alist at one time; because of the linear search the most recently added one will be found. It can later be removed, revealing the previously shadowed value. So, an interface which unified these would have strictly lesser power than either one.

So AssociationLists are more than maps. Is there a reason why this is a good thing?

And less. But to answer your question: No. On the other hand, there's no reason it's a particularly bad thing. Since Lisp is a dynamically typed language, there's less need to have everything implement some interface in order to say what you can do with it. Which is not to say that alists and hash tables are or were ever intended to be interchangeable. They serve different purposes. Which brings us to this older comment:

Of course, it would be easy to create your own classes and generic function protocol to provide just that interface; such a thing is probably not useful enough to be worth standardizing. One way or another, it's not standard.

I'm curious why a thing that looks so much like a map isn't a map.

Well, it is a map. It just happens not to have a common functional protocol with some of the other kinds of mapping data structures that exist in CommonLisp. And can't really, without pretty radically changing what it is.

Just by way of example, here's a sketch of a simple protocol for a "mapper" and two implementations, one built on top of an alist and another on top of a hash table. If you really wanted to have a unified view of these two data structures you could write your code in terms of the generic functions PUT and LOOKUP and make instances of ALIST-MAPPER and HASH-MAPPER as appropriate. Writing an implementation of RED-BLACK-TREE-MAPPER is left as an exercise for the reader. I don't, BTW, happen to think that this particular abstraction would be particularly useful; for the reasons explained above, if you don't pretty specifically want to be using an alist, with all it's quirks, you probably shouldn't be. But here's a code sketch:

  (defclass mapper ()
	((impl :initarg :impl :accessor impl)
	(test :initarg :test :initform #'eql :reader test)))

(defgeneric put (mapper key value) (:documentation "Put the given value into the mapper under the given key."))

(defgeneric lookup (mapper key) (:documentation "Get the value in the mapper under the given key or nil."))

;;; alist based implementation of mapper

(defclass alist-mapper (mapper) ((impl :initform ())))

(defmethod put ((mapper alist-mapper) key value) (let ((pair (assoc key (impl mapper) :test (test mapper)))) (if pair (setf (cdr pair) value) (cdr (push (cons key value) (impl mapper))))))

(defmethod lookup ((mapper alist-mapper) key) (let ((pair (assoc key (impl mapper) :test (test mapper)))) (if pair (values (cdr pair) t) (values nil nil))))

;;; hash table based implementation of mapper

(defclass hash-mapper (mapper) ())

(defmethod initialize-instance :after ((mapper hash-mapper) &rest (setf (impl mapper) (make-hash-table :test (test mapper))))

(defmethod put ((mapper hash-mapper) key value) (setf (gethash key (impl mapper)) value))

(defmethod lookup ((mapper hash-mapper) key) (gethash key (impl mapper)))

CategoryCommonLisp

EditText of this page (last edited July 2, 2008) or FindPage with title or text search