Lisp In Jak And Daxter

The post-mortem for Naughty Dog's game Jak and Daxter is at

http://www.gamasutra.com/features/20020710/white_02.htm [used to require free registration with e-mail confirmation]

It's very interesting reading from a process perspective, and also because of their use of LispLanguage. It seems they went completely ToolHappy, writing their own language and compiler (!) based on Allegro CL.

This had some benefits, but also some major downsides. SmugLispWeenies should take note of the difficulties they encountered next time they think about Lisp as the MagicBullet that can give a company a huge competitive advantage.

For Crash Bandicoot we implemented GOOL (Game Oriented Object LISP), a compiled language designed specifically for object control code that addresses the limitations of traditional languages.

Having a custom language whose primitives and constructs both lend themselves to the general task (object programming), and are customizable to the specific task (a particular object) makes it much easier to write clean descriptive code very quickly. GOOL makes it possible to prototype a new creature or object in as little as 10 minutes. New things can be tried and quickly elaborated or discarded. If the object doesn't work out, it can be pulled from the game in seconds without leaving any hard to find and wasteful traces behind in the source. In addition, since GOOL is a compiled language produced by an advanced register coloring compiler with reductions, flow analysis, and simple continuations, it is at least as efficient as C, more so in many cases because of its more specific knowledge of the task at hand. The use of a custom compiler allowed us to escape many of the classic problems of C.

You can read all about it in Gamasutra: http://www.gamasutra.com although, you will have to register.

Advantages:

Disadvantages:

The disadvantages listed below are largely problems inherent in a developer-driven (rather than customer- or goal-driven) development process, but they are to different degrees also directly related to the choice of Lisp as a development language.

Other problems:

Honestly, after reading the post-mortem, and knowing that Jak and Daxter came together to be an artistic and commercial success, I'm very impressed by the levels of skill (and luck) of the Naughty Dog crew (especially the poor designers and artists!).


Mostly the stuff above is superseded already by the latest game Jak II. Naughty Dog wrote even more GOAL (= Lisp) for this game ;-) : Jak II. Jak II is about 900000 lines (!!) of GOAL (Lisp) plus some more C/C++/Common Lisp/MEL... . So, they have not given up on Lisp, but instead worked even harder to create an absolutely stunning game.


Ahh the good ol WikiNow (it's like a discussion blog, but without dates) ... Jak III is "now" the latest game. Given how hot on the heels it came after Jak II, I suspect it's largely the same engine, perhaps tuned up a smidge. Any reports on the amount of GOAL in that game?


Jak III was mostly GOAL like Jak II. In future Naughty Dog won't be using their lisp-like language and tools because the guy who wrote it all will no longer be available to support them. So they will be moving to C++ possibly using Open C++ to bend the language to their needs.


From the following thread http://lists.midnightryder.com/pipermail/sweng-gamedev-midnightryder.com/2004-November/002611.html

"After using a completely custom language for our last three PS2 projects, coming back to C++ has been a pretty rude awakening. We were hoping we could work around some of the deficiencies of C++ with a good preprocessor, ideally one that provided more than simple hygienic macro substitution; we really would like the capability to actually parse C++ so that we can write metacode that actually understands our code structure and class hierarchy. So far, the only available packages I've found that appear to fit the bill are OpenC++ and FOG (Flexible Object Generator), although there may be others." -- Scott Shumaker, Naughty Dog, Inc.
They need to stop thinking like this if they want to make Jak IV. They're going to end up rewriting Lisp in C++ and doing the Yahoo! Store thing all over again. C++ isn't Lisp and if they're committed to changing (I imagine the effort involved in porting their compiler to new consoles is significant), they need to stop thinking about how they'd solve problems in Lisp and more about how they'd solve them in C++. If your first instinct when you move back to C++ is that you need a new preprocessor, you need to stop, take some time off, and reimmerse yourself in C++.

Alternately, they need to spend a year or so getting some other people up to speel on GOAL internals and keep using that. It'll probably cost less, both in time and money, anyway.
There are certain deficiencies in C++ that GOAL addresses neatly. Simply "re-immersing" oneself in C++ doesn't make these problems go away (not to mention the fact that pretty much all the ND programmers are already extremely proficient in C++). One trivial example: GOAL permits compile-time select/inserts on a set of shared SQL tables (containing all kinds of art asset information) - the existing C++ preprocessor certainly won't let you do this.
Well, GOAL appears to be pretty tightly bound to the track record your outfit has accumulated for prompt shipment and high quality projects. Your main GOAL dude took a hike, and now you have no resident genius to support your custom language. Stick with the custom language and push it into the world as a GPL'd dialect of Lisp. Build a community around it, and you will find you have less hassle bringing the kids up to speed as you hire them on. ;) that doesn't solve your immediate problems, though... I think I'd try to work through the patch sticking with what you have spent all this time developing to date, rather than dispatch the baby with the bathwater.
There are certain deficiencies in C++ that GOAL addresses neatly. Simply "re-immersing" oneself in C++ doesn't make these problems go away (not to mention the fact that pretty much all the ND programmers are already extremely proficient in C++). One trivial example: GOAL permits compile-time select/inserts on a set of shared SQL tables (containing all kinds of art asset information) - the existing C++ preprocessor certainly won't let you do this.
Interesting article; I am a huge proponent for using the right tool for the job. If it turns out that C/C++ isn't up to the task for a given job (especially considering time to market requirements, which is pretty vital in the gaming industry), then by golly I will not use it. It is unfortunate that doing this necessarily means restricting the pool of developers you can hire, provided you want to minimize training costs. I have some experience with Lisp, and except for the ugly source code, I generally like the language. I'm also a fan of ForthLanguage as well (particularly in the early days, Forth had a large presence in the gaming industry).

What seems to me they should do is research the possibility of embeddable languages. I know that there are a variety of Scheme, Lisp, Forth, etc. embeddable language variants. And, of course, perhaps the most famously known language used for embedding, Lua.

Is there any discussion on why they didn't persue this course of action? --SamuelFalvo?

The difficult part, as I understand the situation, was coordinating assembler code for multiple processors. Porting an existing tool to that environment would have been the same amount of work or more, since the end result included a fast compile-and-test loop, lisp macros, and other goodies.
One of the prevailing themes of this story, and possibly a cautionary one against the power of Lisps: Too much proprietary technology was invented at once. Naughty Dog's team managed to ship three games because it worked, but they were unable to draw from the "WorseIsBetter community" of C++, and had also become too distant from mainstream Lisp to get help from that area. Without successors to maintain it, the language became a bottleneck. A more cautioned approach where the game code started from a existing well-supported dialect, and gradually added in the hooks, features, and optimizations necessary to ship, might have successfully spread knowledge around the team. (on the other hand, the system specs and available implementations when GOAL was started may have made this unfeasible.)


Update: 2012-02-15 From a 20-minute behind-the-scenes video of the PlayStation 3 port of the Jak and Daxter PlayStation 2 trilogy, there is a shot of commented out LISP in a C source file, at 10m37s. Naughty Dog handed off the source code and assets to developer Mass Media, while proclaiming "This is an impossible project." The non-technical discussion about porting starts at 7m10s. http://www.joystiq.com/2012/02/15/jak-and-daxter-collection-trailer-gets-deep-super-nostalgic
Haven't you heard? They're not writing C++, they're generating it from Racket. Half of everything above is irrelevant. Now see where wild speculation gets you? http://www.gameenginebook.com/gdc09-statescripting-uncharted2.pdf
CategoryHistory CategoryGame

EditText of this page (last edited October 24, 2012) or FindPage with title or text search