Large Scale Individual Software Development Large Scale Individual Software DevelopmentThere's a lot I can apply and already have, but the thing that pains me the most is how much I miss PairProgramming. I was doing that years ago at WorkingSoftware?, http://www.working.com
DaveJohnson?, formerly the President of Working Software and still the majority owner, had an interesting variation on PairProgramming at IMagic (I think, might have been a different company) back in the 80's. They'd write video games with their computers on tall tables. Everyone would stand up to program, and their computers would be so close together all their elbows would touch as they worked. During the work day they'd play music and dance in front of their machines - which you see was enabled because they programmed standing up!
MichaelCrawford GoingWare crawford@goingware.com
Subject: Large Scale Individual Software Development? Date: 8 Oct 2000 12:32:23 -0400 Newsgroups: comp.lang.c++.moderated Message-ID: <39E02C0E.AA132F87@goingware.com> X-Original-Date: Sun, 08 Oct 2000 05:40:54 -0230What thoughts do you have on how to carry out the development of the largest possible software projects entirely by yourself?"So he decided to watch what the government was doing, scale it down to size, and live his life that way." - Laurie Anderson (quoted approximately from memory)
I read about techniques and processes that were developed with big-team projects and mind and I try to apply them to myself. For example, I try very hard now to create classes that have well-defined public interfaces and strictly private implementations, and possibly protected accessors for use by derived classes. When I write one class I consider the user of that class to be a potentially-abusive stranger even when I know it will be me the next day.
One thing I don't do a lot of is up-front documented design. I design as I go along, but my style of coding seems to actually be a way of doing design in my head and getting it down in the source. I struggled for quite a while to make use of UML in my work, and got nowhere with it. What works better for me is to just write the public interfaces to things, write unit tests and so on, and just to have learned from hard experience the difference between good and bad code. This wouldn't work on a team project, where I'd have to communicate a design to a different person to implement, and it limits the scale of what I can work on.
It also makes it impossible to make time estimates for anything. I've never had any luck at all making time estimates for my work. Now I make it a practice to decline to give a time or cost estimate when I take on a project - and only work with clients willing to work on those terms. This was learned from painful experience.
If there's anything I'd like to learn, it's how to estimate the time it will take me to write a complete program, given that the program is likely to take several months no matter what.
On my very first programming job, we had 10 college students paid about five bucks an hour to write a LISP interpreter in C. It was to support the entire common LISP standard and fit in the 640kb memory of a PCXT (this was Sapiens Software Star Sapphire Common LISP). The way this was made possible was that the company founder (a young but fairly experienced programmer) wrote a manual virtual memory system, so that you could create VM data structures and full them with lisp code and stuff, but to do so you had to manually fetch and put 64 bit conses.
We wrote just mountains of really bad code, and as the team leader I did my programming by day and spent each night integrating the rest of the team's code. It was a collossal failure in project management, I think the whole thing got recoded more than once before it shipped.
My next programming job I was the only programmer at a company that used SunOS. I would struggle for a month to write a command like program that had maybe 500 lines (although some of them did pretty fancy stuff in those 500 lines). It was here that I started to learn about debugging (using a debugger - gdb and dbx, I read Robert Ward's excellent book "Debugging C", and also started to use profiling and do somewhat proper performance optimization).
On several jobs I've been the only programmer or been one of a pair. At one company, Working Software, http://www.working.com , I had responsibility to maintain a half dozen products, some of which were really buggy when I inherited them, and had to ship new products at the same time. I started to get really determined to learn how to write large programs by myself.
At Apple I worked on MacOS system 7.5.3 and 7.5.4 (the systems that were built to support the first PCI PowerPC Macs). This was quite an inverse experience; I did nothing but debugging and there were hundreds of people involved. There were people with full-time jobs to do nothing but building and integration. Managing the project was a huge undertaking; my team leader estimated he'd attended 3000 meetings in the two years he'd worked there, besides his debugging work.
There were huge build problems. You _could_ get the OS to build if you had your system set up just right, but the one little bit of original programming I did, I was completely unable to get my code patched into the main system build. I was simply unable to configure the build environment so that I could build the one component I was working on on my own machine. I was able to set up a simple build for my own purposes, but to check it into the main source base and build from there was completely frustrating. I got help from lots of people, and after being moved to tears I emailed my patches to another engineer who checked them into the source base and was able to test the build himself.
I've been reading a lot about extreme programming, mostly on the web and usenet, also on slashdot, and a lot of the concepts appeal to me. But some of the practice is not possible. This is because I am a consultant, and I work by myself. I'm even on an extended trip far from any other software developers, so I don't know any programmers in the province I'm staying in. In my practice a client usually comes to me and wants an entire product developed from scratch, and after 13 years I'm able to do this - but it is a very painful process.
I would have to say that if you want to be a consultant you must have very sound intestinal fortitude.
I've been working on one product since early december '99 and it is near beta now. One good thing about it is that I feel it is pretty well architected and doesn't have a lot of bugs for a beta program, I think there are only a dozen documented bugs and no crashes. But it's been really hard!
One bit of advice of how to approach one's work I wrote up on this page:
Study Fundamentals Not APIs, Tools or OSes http://www.goingware.com/tips/fundamentals.html
For a couple of years now I'd been concentrating on just writing code, rather than my previous practice of constantly educating myself. I got tired of trying to keep up with the latest fads in APIs and really felt there was too much to know. But I got inspired to start learning again after the "C++ Answers with Bjarne Stroustrup" feature on slashdot a while back:
http://slashdot.org/interviews/00/02/25/1034222_F.shtml
One thing I would do as I was working on my current project was get some good books on C++ and software architecture and read them for a couple hours each day rather than just coding - and I would immediately put into practice what I'd read. I would often go back and rearchitect previous code because it just wasn't right, and when I had difficult architectural issues I would really struggle to get it right rather than just hammering something that sort of worked - which is all to often the Industry Standard Practice.
This paid off in the later part of the project in terms of the low bug count and being able to implement new features in the end much easier. But getting the initial builds of the product to do anything interesting was very slow and painful and I think my client was very anxious. And it has taken so long it's causing them a lot of financial stress.
So if you have to write some big program by yourself, what would you do?
One solution might be "don't do it". I really would rather work in a team but I'm trying to build a company. Someday when I have my own software products I hope to hire programmers, but how would I go about collaborating with other consultants?
One big help is to make extensive use of libraries. My program uses 3 libraries plus the code I wrote myself:
cross-platform app framework 3800 kb of source Xerces XML library for DOM 5000 kb IJG JPEG CODEC 2200 kbSo you see although I've labored for months to write 760k of C++ code, the vase majority of the source in my program (not all of which is actually used) comes from open-source libraries. The app framework is not yet released but soon will be under the BSD license.My application source 640 kb My library source 122 kb
Addendum - January 9, 2000 - the ZooLib? cross-platform application framework is now released at http://zoolib.sourceforge.net , under the MIT license rather than the BSD license (in practical terms they're equivalent).
The last item in the list is a library of utilities I've written in the process of writing the program. At almost every step of the way, if some part of the program could be separated out and made to stand on its own without any dependencies on the application code, I put it in the library. In many cases this required extra effort to get the code written, but I think it contributed to the reliability of the code (forcing library-type components to stand on their own without dependency on the on the app makes them easier to debug) and will certainly pay off in the future by making it easier to write new apps with the framework I've been using.
My library only contains dependencies on other components within itself, or on the other libraries listed, or the standard C++ libraries.
Since learning about STL I've been making extensive use of it, and that pays off greatly, often in unexpected ways. For example, I made a "Rotate" functional that can be used to rotate a range of geometric points in an STL container around a given center by some angle. But I also found it happy to instantiate Rotate objects as members of regular classes or to even create the on the stack in place just to rotate one point:
template < class T> struct ZBPoint{
T h; T v;};
typedef ZBPoint< double > DFPoint; // double-precision point
DFPoint pt( 5.5, 6.7 ););DFPoint rotPt( Rotate< double, double >( DFPoint( 0, 0 ), 15.0 )( pt )
I wouldn't have written a functional just to rotate one point but once I'd written it, why not use it?
The thing I miss the most in working by myself, and what really makes me sad, is when I read about Extreme Programming's recommendation for pair programming. I used to spend a lot of time pair programming with Dave Johnson at Working Software (he's with Be, Inc. now) and I learned a lot of what I know from the other programmers in other places I've worked.
If there were some way consultants could hook up.
What's been a big help is the fellow who wrote the cross-platform library I've been using. He's allowed me to use it so I can help work out the architectural problems before he releases it, and he's been very supportive of me in the process of working with it. He helps me with business issues. But he's on the other side of the continent and has his own product to work on.
What would really be better is if lone consultants could work in pairs by actually collaborating on a single product.
Well I know this is kind of long and rambling but I appreciate your thoughts on this.
Mike -- Michael D. Crawford GoingWare Inc. - Expert Software Development and Consulting http://www.goingware.com crawford@goingware.comTilting at Windmills for a Better Tomorrow.
This page mirrored in ExtremeProgrammingRoadmap as of April 29, 2006