Only Foresight Matters

I used to always say that the best quality for a programmer is persistence, but the best quality for a SystemAnalyst is foresight (see also GuerrillaGuideToInterviewing). Of course, if you're stuck developing a whole system by yourself, then it helps to have both. Foresight is of supreme importance because it keeps you from spending an inordinate amount of effort following AntiPatterns down the wrong path.

Unfortunately, I have found it a rather difficult task to teach somebody to have foresight. It seems more like an InnateHumanTrait?, people either have it or they don't. If you are a person with foresight, then when you are in the middle of a development project and you face a decision (a fork in the road, which way should I go from here, what's the right thing to do next?), you connect to yourself (or your clients, or whomever will maintain your software) in the future, and you find out how come they are so happy with what you have done. This will then guide you to do the right thing in the present (a structure? internal documentation? a methodology or algorithm? formal documentation? a self-contained parameter-driven external procedure?).

Another perspective is that our future causes our present. In other words, we do what we are doing today because of where we will be tomorrow. Others will argue that Life is not a goal, it is only a journey. But I believe that good software development projects are both. You (may) have a goal (see TypesOfProjects) that infuses you with direction, but along the way you also need to DoTheRightThing.


What, your foresight didn't tell you there was a But:? Ah, you must be one of those poor benighted souls without adequate foresight ...

Most developers on this here wiki have found a more rewarding way to look at development is as a kind of SimulatedAnnealing. Let's say your goal is to boil an egg. At first, all is chaos! Well, okay, it could be worse, you may have some joker who believes that so long as you foresee a boiled egg you're as good as fed ...

But let's say you're lucky and all you have is chaos. Wonderful! We nail part of the chaos down. First thing is, where are we going to boil our egg? In the kitchen! Great, we have a kitchen, there is a step away from chaos. Now, where can we find eggs? In the refrigerator! Ah, very comforting to have a well stocked refrigerator, there's another piece of the puzzle done. But: maybe we're out of eggs - we just discovered we need to go to the store first! Damn, why didn't we foresee that before going all the way to the kitchen? Heck, perhaps it's because we're puny humans and the task of boiling an egg has far too many variables for anyone but BigOmega to foresee ...

You understand where this goes of course. In each iteration of a development you need to reduce your uncertainty empirically. If you can't do that, all the foresight in the world won't feed you. That's the trouble with BigDesignUpFront - all foresight, no empiricism. --PeterMerel

Here are a few observations:

What can we learn from these hard facts? You could draw the conclusion that we need to stop and think ahead more, imagine more possibilities before acting, and criticize less.

But here's another observation:

So have you considered drawing the opposite conclusion? Since foresight doesn't work, don't rely on it. Go for early hindsight. Just try stuff, see what you can learn, and try some more stuff. Instead of trying to do things right the first time, actively invite error. Instead of avoiding criticism, criticize everything you can, and encourage others to criticize. Keep these explorations as quick and cheap as you can, and criticize in a way that encourages people to continue to invite error.

These strategies are techniques for rapid learning. They work by getting reality itself you help you out. Reality has a lot more imagination than you have. Depending on foresight, you're stuck inside your own picayune imagination, and you'll never be able to keep up.


A few more thoughts along this line:


Ben, the advice above seems helpful in some situations, but aren't you lacking some balance here? You say "Things never turn out as you imagine." [emphasis mine]. On the contrary, for most of my waking hours, things turn out mostly to exactly as I imagined. The techniques you advocate have a special niche for me in the area of breaking a mental impasse, but not in the majority of my activities.

I don't think of "foresight" as knowledge of the future; I think of it as the process of extending knowledge of the past to likely futures. As such, it is a most teachable technique, and I teach it to my young children by asking them questions like "What's going to happen first? Then what?" and so on. The knowledge thusly mined is not magic, but it's also not obvious until you've developed the habit. And it's extremely helpful, especially when the parallel sense of uncertainty of prediction and shift to higher level is incorporated. To use the chess example, I would never try to plan out a series of actual moves ahead of time; that's too specific and brittle. But I would make a plan that sounds like this "In this game, I'm going to use my knights extensively in the opening." That's partly a learning strategy, but why not plan to learn?

There are two ways to go seriously awry. One is to rely too exclusively on the scanty knowledge already in mind to solve a problem that is largely foreign. The other is to neglect large bodies of past relevant experience and approach a mostly familiar problem as though it were not. But I'm preaching to the choir here; you explained this stuff so nicely in your Practical Requirements book (or was that a different BenKovitz?). I'd like to see more balanced analysis of these related phenomena.

-- WaldenMathews
Foresight is difficult: Those who use crystal balls eat broken glass.

Can insight replace it?

Sometimes you want a special kind of hindsight. You need the ability to pause, see that you missed something and go back to it. Once you see it it becomes obvious. There you are hurtling along on the freeway. You are making great progress! And you miss the turn off. EdwardDeBono argues that this error is wired into the way our brain works.

Perhaps this is where PairProgramming can make a big difference?

Warren Buffet's rule : "Predicting rain does not count. Building an ark does."

A very commonly heard phrase in project reviews is : "I said so a year ago !". That's a useless remark. If you said so but did not act on it, then you were just wasting your breath.

-- MaxT

You know what, MaxT? Sometimes you can say so, write memos, publish proposals, jump up and down during meetings, and generally make as much of a pain in the neck of yourself as you want, and guess what? Jack. If you can't affect a decision made at a higher <ahem> management level then you're screwdificated. This doesn't mean that the effort was wasted, just that the actions required were not taken. If you can rub the proper noses in the proper filth as a result of them not taking your advice then sometimes it's worth it. Other times it's not. -- MartySchrader
Or indeed, decisions made by obstinate programmers, despite all advice of their peers. It's their ball and you can't play with it - shame you have to interface with it. -- Jules
See also: RefactorMercilessly, IllusionOfControl, FoundingFathersDiscussion, AskTheComputer, GoldilocksSolution

View edit of November 5, 2005 or FindPage with title or text search