Over Engineering

I just wrote a shell script (see AdjustingTransactions) that combines a Perl script with some other graphing utilities. The part that actually does anything is about twenty (non-commentary) lines. The script itself is about two hundred! (A hundred and sixty non-commentary.)

What's the rest? Eighty lines of argument processing (there are nine different options) (all of them useful!-) Default handling. Error handling. Stuff I wouldn't bother with, if I was just going to use this myself. But this is a reusable tool, and I'd like it to survive my users' exposure to it.

(Funny; it's very much like on of my favorites of all the programs I've written, a really nice troff front end. I stole whole-heartedly from other such scripts. When I was done, you could says "proff file", and all the right preprocessors and options would magically be invoked. But I digress.)

Anyway . . . what does it mean when the ``heart'' of a program is an order of magnitude smaller than its bulk?


I don't feel good about calling this over-engineering. It just seems like "engineering" to me. User interface almost always takes most of the code. It is just that it is highly stylized code, so it can be reused or automatically generated from a smaller specification. This is pretty obvious when you look at something like VisualWorks or MacApp. But the first time I ran into this phenomenon was when I got the argument processing package (getOpt?) from Henry Spencer at the University of Toronto. It made a huge difference in my Unix C programming. I wrote less lines of code but ended up with a much more robust program.

If you don't have some kind of package for building your user interface then you'll either spend a huge amount of time at it or (if you are like me) sluff off. We get used to sluffing off, and then when we are forced to do all the work we complain about it.

My idea of an ideal engineering environment is one in which we only have to design the heart of the system, and the rest can be reused or generated for us. In the mean time, if you document your system by starting with the heart, it will become much easier to understand.

Related Stray Complaints: Are there better alternatives? Incidentally, Knuth's latest LiterateProgramming book, Stanford Graphbase, does the exact opposite.

Amazon says this book is out of print, buy you can still try ordering it: ISBN 0201542757

Hmmm... I have to agree with the previous comment. Your support code may outweigh your "kernel" code, but I don't think that's OverEngineering; that's reality. F'r instance, one of the truisms that I regularly spout to anyone doing a program that uses an OO language and a relational store is that 40% of your program won't be concerned with the "guts" of your domain -- it'll be just the grunt work of converting from objects to relations and vice versa. I again agree that the our goal should be that the "support" code should all be available in reuse libraries, allowing us to focus on that 20-50% of each appliction that is really "new".

On the other hand, there's a classic story about OverEngineering that I've got to relate. It seems that Henry Ford once asked his engineers about how the parts in the Model T (which had been in production for many years) wore out relative to each other. His engineers pored over old, discarded Model T's and found out that all the parts tended to wear out except the cotter pins. In every Model T the cotter pins still looked and worked like new. Old Henry then blasted his Engineers for spending too much time and money building Cotter Pins that were too good for the rest of the car, and then sent them back to redesign them!


While all the options in Paul's additions are "useful", it would be interesting to see the extent to which they are really used by the folks who finally get the program. Seems to me there is an 80-20 thing going here, where the core value can be provided by a much smaller investment.

I've been consulting for a project that is in deep trouble. One of the problems is that they have accepted so many requirements that they haven't ever gotten their core working. Now no one believes they'll ever get done. Building the core and then enhancing would have given folks the confidence they need.

In a world of finite resources, building for posterity can mean you don't live to see it. --RonJeffries
I agree with that, but I've also experienced the opposite. Someone gives me a useful program, and it's great, but with just a few minor tweaks it would be so much better. Sometimes the tweaks are bug fixes, sometimes they are closer to genuine options.

This Wiki is an example. Loads of people have started WikiClones and most of them have wanted to tweak it and add their pet features. I feel this is the close to the essence of what Alexander was on about with his architecture stuff adapting precisely to the user under the users control. -- DaveHarris
If you are in an environment where tweaking is difficult, you need to put in all the options you can think of in the beginning. If you are in an environment where tweaking is easy, you put in only the bare minimum, and tweak as necessary. The thing I like about ExtremeProgramming is that it realizes that objects (and particularly Smalltalk) make it easier to tweak. Furthermore it uses Refactoring to keep things easy to tweak. With that tweakability, you can leave the options until you need them. --MartinFowler

View edit of November 9, 2014 or FindPage with title or text search