Domain Specific Programming

Many folk in the CommonLisp community claim that when you are programming in Lisp you are more designing a DomainSpecificProgramming language rather than writing a program. Doubtless they're right. But of course a Lisp program is just a bunch of procedures (and macros) that define and manipulate data structures. And so is a C++ program.

I've always understood one of the foundations of ObjectOrientedProgramming to be the construction of program entities whose behaviours are in some way analogous to those of the domain entities. So a C++ or Java program (a good one) also embodies a domain specific programming language, to some extent. So the question is, why does the Lisp community make this strong claim? What are the essential features of Lisp (CommonLisp or the other dialects) that result in Lisp programs being more strongly DSPs than those written in other languages? -- KeithBraithwaite

In LISP you can add new statements to the language. Yes, strictly speaking they are just function definitions, but since the LISP language notation is functional, they are just like built-in statements.

In C++ you can add a function and call it. For C++ to be like LISP you'd have to be able to add new statements like for(;;) or if(). Since C++ only has a couple of statements, people hardly notice them. But a C++ program is a list of statements, many of which are function calls. A LISP program is a function call, consisting of other function calls, some of which define new functions, and others of which use them.

Basically there is no syntax in LISP. Just LotsofIrritatingSillyParentheses. --RonJeffries

So, taking the C/C++/Java style "for", it's the '{' '}' getting in the way. "for" looks almost like a function call, but not quite, and couldn't really be made into one because the block of statements following is just a block of statements, and not a closure. It seems like this couldn't be fixed without turning C into Lisp. -- KB

A similar, but different feeling arises in Smalltalk. There, everything is an object. Like LISP, there is essentially no syntax to the language (though there is a bit more than in LISP). Everything is a message send, including defining a class or method.

Then again, we could treat "for" as a method, if we took the thing between the '{' and '}' to be an object, but that would turn C in Smalltalk. -- KB

There's something odd that happens in your mind when your language is made up of all the same things, elephants all the way down. Jagged edges seem to go away and things become smoother. I'd explain it better ... but I can't. --rj

It's very much like the difference between Chess and Go. --

Well, I've heard similar claims made about ForthLanguage - writing a Forth program consists of defining words that you combine to solve your problem. RebolLanguage seems to make similar claims, but I have even less experience with that.

I'm not sure what makes these languages more amenable to this idea - it may just be that language and culture strongly favour composing programs from blocks. Of course, in the case of Lisp at least, the environment encourages the reuse of parts of previous programs. After a while, you'd expect a DSP to arise almost naturally out of the basic parts of different applications in the domain. -- BurkhardKloss

ElephantsAllTheWayDown?... that's good. Part of it is that in C or similar, part of your text is talking to the compiler, part is talking to the execution engine, part is fixed syntax that again talks to the compiler, the function calls are invented within the program. Syntactically, function calls are not linguistically interesting or natural.

In LISP and Smalltalk and Forth, the text only talks to the execution engine. Nothing talks to the compiler (well, the parens in LISP do, as do the [:;.] in Smalltalk. So the text all and only says what to Do. That helps a lot. Inventing new units of expression looks and feels just like using previously packaged units of expression. --AlistairCockburn

Right. The concerns of execution environment, compiler and problem domain are not separated, and the confusion is expressed in the tangled syntax of "imperative" style languages. So the Lisps (and in some similar way Smalltalk) separate those concerns, then automate and hide away as much of them as possible. I think I begin to understand. Thanks guys, very enlightening. -- KeithBraithwaite

It is a bit abstract for me. I need an example that shows the separation between execution environment, compiler and problem domain. I wonder how it looks in Lisp, Smalltalk and Java.

Do you know methods/tools designed to achieve this separation? I am sure Lisp 'compiler' does not give enough guidance? Can we gain this separation in Java/C++ using the right discipline?

It appears to be the other way around to me. It is in languages like C++ that the execution environment, compiler, and problem domain are separate. So you don't have to gain it. When working with C++, it is necessary to first work in the problem domain, by drawing up elaborate UML diagrams and design documents. This is because the finished source code usually bears no resemblance to an expression of a solution to the original problem. Every time you begin to express the solution, it immediately becomes apparent that you need to go out and declare some more temporary variables, define another class or two, and allocate some more memory before continuing.

Then, it's on to the compiler, writing the code and getting the compiler to accept that code. No matter what the problem, it must be forced to fit within the object-oriented paradigm. There's always a semicolon missing somewhere, or an illegal cast, or a missing mandatory cast, etc., and once you fix those, there are mysterious header compile errors and "undefined references" to functions you swore you compiled in. When you are working with the compiler, you are not thinking about the original problem domain, so it can be said that the problem domain is well separated from the compiler. In fact, the compiler has a problem domain of its own.

When the compiler is done, you are left with a file that contains binary instructions that are very roughly equivalent to what you wrote in the source file. This binary file is undeniably and totally separate from the compiler which produced it. It is this binary file that you work with in the execution environment. Working in the execution environment is very different from working in the other two areas, no matter how you go about it.

In Lisp, all these elements are combined, not separated: The compiler, linker, execution environment, editor, and debugger are all mixed together. At least some of the elements reside in the same binary image. You don't necessarily know when compilation is taking place (only some Lisps require an explicit command to invoke the compiler). All errors seem to occur in the same domain, not umpteen different domains. The same debugger comes up when there's a compile-time error as when there's a runtime error. In a Lisp IDE, programs are compiled incrementally, as they're being written, a little bit at a time. Even while the program is executing, it is still possible to go back, change things, recompile things, and have the results be fully integrated with the running program. It is possible to create a notation that means exactly what you want to say. In Lisp, everything sort of blurs together.

See also: BottomUpProgramming, DomainSpecificLanguage, LittleLanguage, MetaProgramming

View edit of November 13, 2014 or FindPage with title or text search