I posted this on comp.lang.c++.moderated a couple days ago, and would appreciate your thoughts too. The question that is relevant for ExtremeProgramming
is, what can XP do for a programmer who works on really big projects all alone?
There's a lot I can apply and already have, but the thing that pains me the most is how much I miss PairProgramming
. I was doing that years ago at WorkingSoftware?
, formerly the President of Working Software and still the majority owner, had an interesting variation on PairProgramming
at IMagic (I think, might have been a different company) back in the 80's. They'd write video games with their computers on tall tables. Everyone would stand up to program, and their computers would be so close together all their elbows would touch as they worked. During the work day they'd play music and dance in front of their machines - which you see was enabled because they programmed standing up!
Subject: Large Scale Individual Software Development?
Date: 8 Oct 2000 12:32:23 -0400
X-Original-Date: Sun, 08 Oct 2000 05:40:54 -0230
"So he decided to watch what the government was doing, scale it
down to size, and live his life that way."
- Laurie Anderson (quoted approximately from memory)
What thoughts do you have on how to carry out the development of the
largest possible software projects entirely by yourself?
I read about techniques and processes that were developed with big-team
projects and mind and I try to apply them to myself. For example, I try
very hard now to create classes that have well-defined public interfaces
and strictly private implementations, and possibly protected accessors
for use by derived classes. When I write one class I consider the user
of that class to be a potentially-abusive stranger even when I know it
will be me the next day.
One thing I don't do a lot of is up-front documented design. I design
as I go along, but my style of coding seems to actually be a way of
doing design in my head and getting it down in the source. I struggled
for quite a while to make use of UML in my work, and got nowhere with
it. What works better for me is to just write the public interfaces to
things, write unit tests and so on, and just to have learned from hard
experience the difference between good and bad code. This wouldn't work
on a team project, where I'd have to communicate a design to a different
person to implement, and it limits the scale of what I can work on.
It also makes it impossible to make time estimates for anything. I've
never had any luck at all making time estimates for my work. Now I make
it a practice to decline to give a time or cost estimate when I take on
a project - and only work with clients willing to work on those terms.
This was learned from painful experience.
If there's anything I'd like to learn, it's how to estimate the time it
will take me to write a complete program, given that the program is
likely to take several months no matter what.
On my very first programming job, we had 10 college students paid about
five bucks an hour to write a LISP interpreter in C. It was to support
the entire common LISP standard and fit in the 640kb memory of a PCXT
(this was Sapiens Software Star Sapphire Common LISP). The way this was
made possible was that the company founder (a young but fairly
experienced programmer) wrote a manual virtual memory system, so that
you could create VM data structures and full them with lisp code and
stuff, but to do so you had to manually fetch and put 64 bit conses.
We wrote just mountains of really bad code, and as the team leader I did
my programming by day and spent each night integrating the rest of the
team's code. It was a collossal failure in project management, I think
the whole thing got recoded more than once before it shipped.
My next programming job I was the only programmer at a company that used
SunOS. I would struggle for a month to write a command like program
that had maybe 500 lines (although some of them did pretty fancy stuff
in those 500 lines). It was here that I started to learn about
debugging (using a debugger - gdb and dbx, I read Robert Ward's
excellent book "Debugging C", and also started to use profiling and do
somewhat proper performance optimization).
On several jobs I've been the only programmer or been one of a pair. At
one company, Working Software, http://www.working.com
, I had
responsibility to maintain a half dozen products, some of which were
really buggy when I inherited them, and had to ship new products at the
same time. I started to get really determined to learn how to write
large programs by myself.
At Apple I worked on MacOS system 7.5.3 and 7.5.4 (the systems that were
built to support the first PCI PowerPC Macs). This was quite an inverse
experience; I did nothing but debugging and there were hundreds of
people involved. There were people with full-time jobs to do nothing
but building and integration. Managing the project was a huge
undertaking; my team leader estimated he'd attended 3000 meetings in the
two years he'd worked there, besides his debugging work.
There were huge build problems. You _could_ get the OS to build if you
had your system set up just right, but the one little bit of original
programming I did, I was completely unable to get my code patched into
the main system build. I was simply unable to configure the build
environment so that I could build the one component I was working on on
my own machine. I was able to set up a simple build for my own
purposes, but to check it into the main source base and build from there
was completely frustrating. I got help from lots of people, and after
being moved to tears I emailed my patches to another engineer who
checked them into the source base and was able to test the build
I've been reading a lot about extreme programming, mostly on the web and
usenet, also on slashdot, and a lot of the concepts appeal to me. But
some of the practice is not possible. This is because I am a
consultant, and I work by myself. I'm even on an extended trip far from
any other software developers, so I don't know any programmers in the
province I'm staying in. In my practice a client usually comes to me
and wants an entire product developed from scratch, and after 13 years
I'm able to do this - but it is a very painful process.
I would have to say that if you want to be a consultant you must have
very sound intestinal fortitude.
I've been working on one product since early december '99 and it is near
beta now. One good thing about it is that I feel it is pretty well
architected and doesn't have a lot of bugs for a beta program, I think
there are only a dozen documented bugs and no crashes. But it's been
One bit of advice of how to approach one's work I wrote up on this page:
Study Fundamentals Not APIs, Tools or OSes
For a couple of years now I'd been concentrating on just writing code,
rather than my previous practice of constantly educating myself. I got
tired of trying to keep up with the latest fads in APIs and really felt
there was too much to know. But I got inspired to start learning again
after the "C++ Answers with Bjarne Stroustrup" feature on slashdot a
One thing I would do as I was working on my current project was get some
good books on C++ and software architecture and read them for a couple
hours each day rather than just coding - and I would immediately put
into practice what I'd read. I would often go back and rearchitect
previous code because it just wasn't right, and when I had difficult
architectural issues I would really struggle to get it right rather than
just hammering something that sort of worked - which is all to often the
Industry Standard Practice.
This paid off in the later part of the project in terms of the low bug
count and being able to implement new features in the end much easier.
But getting the initial builds of the product to do anything interesting
was very slow and painful and I think my client was very anxious. And
it has taken so long it's causing them a lot of financial stress.
So if you have to write some big program by yourself, what would you do?
One solution might be "don't do it". I really would rather work in a
team but I'm trying to build a company. Someday when I have my own
software products I hope to hire programmers, but how would I go about
collaborating with other consultants?
One big help is to make extensive use of libraries. My program uses 3
libraries plus the code I wrote myself:
cross-platform app framework 3800 kb of source
Xerces XML library for DOM 5000 kb
IJG JPEG CODEC 2200 kb
My application source 640 kb
My library source 122 kb
So you see although I've labored for months to write 760k of C++ code,
the vase majority of the source in my program (not all of which is
actually used) comes from open-source libraries. The app framework is
not yet released but soon will be under the BSD license.
Addendum - January 9, 2000 - the ZooLib?
cross-platform application framework is now released at http://zoolib.sourceforge.net
, under the MIT license rather than the BSD license (in practical terms they're equivalent).
The last item in the list is a library of utilities I've written in the
process of writing the program. At almost every step of the way, if
some part of the program could be separated out and made to stand on its
own without any dependencies on the application code, I put it in the
library. In many cases this required extra effort to get the code
written, but I think it contributed to the reliability of the code
(forcing library-type components to stand on their own without
dependency on the on the app makes them easier to debug) and will
certainly pay off in the future by making it easier to write new apps
with the framework I've been using.
My library only contains dependencies on other components within itself,
or on the other libraries listed, or the standard C++ libraries.
Since learning about STL I've been making extensive use of it, and that
pays off greatly, often in unexpected ways. For example, I made a
"Rotate" functional that can be used to rotate a range of geometric
points in an STL container around a given center by some angle. But I
also found it happy to instantiate Rotate objects as members of regular
classes or to even create the on the stack in place just to rotate one
template < class T>
typedef ZBPoint< double > DFPoint; // double-precision point
DFPoint pt( 5.5, 6.7 );
DFPoint rotPt( Rotate< double, double >( DFPoint( 0, 0 ), 15.0 )( pt )
I wouldn't have written a functional just to rotate one point but once
I'd written it, why not use it?
The thing I miss the most in working by myself, and what really makes me
sad, is when I read about Extreme Programming's recommendation for pair
programming. I used to spend a lot of time pair programming with Dave
Johnson at Working Software (he's with Be, Inc. now) and I learned a lot
of what I know from the other programmers in other places I've worked.
If there were some way consultants could hook up.
What's been a big help is the fellow who wrote the cross-platform
library I've been using. He's allowed me to use it so I can help work
out the architectural problems before he releases it, and he's been
very supportive of me in the process of working with it. He helps me
with business issues. But he's on the other side of the continent and
has his own product to work on.
What would really be better is if lone consultants could work in pairs
by actually collaborating on a single product.
Well I know this is kind of long and rambling but I appreciate your
thoughts on this.
Michael D. Crawford
GoingWare Inc. - Expert Software Development and Consulting
Tilting at Windmills for a Better Tomorrow.