Great Failure Of Xp

Problem:

[STRAWMAN 1] XP appears unsustainable in commercially viable enterprises. [See Note 1.]

[STRAWMAN 2] XP in practice is no more effective than several prior methodologies. [See Note 2.]

[STRAWMAN 3] XP is a step forward, but not a final solution.
In responses to a request by a participant that "this page be restarted from a clean slate", the original text of this page now appears at OriginalGreatFailureOfXp.

[STRAWMAN 1] The GreatFailureOfXp is that it appears that it is, in practice, not sustainable in commercially viable enterprises. -- TomStambaugh

Alternative summaries, concrete examples, theories, or explanations would provide some grounds for discussion and are invited. Perhaps portions of OriginalGreatFailureOfXp might be copied here.

It appears that Tom and I do actually arrive at the same conclusion from different directions. I do not want to stick my neck out quite as far as he does, however, and claim that XP simply can't be sustained in a commercial environment. What I would like to do is challenge the Wiki community to point to current [Feb 03] products under development or on the shelf that are being developed and maintained using an XP environment. I can tell you for sure that XP does not fit into the medical device community nor into most of the industrial process machine control community.

In fact, there is only one gig I had in the past where the client could have benefited from a full-on XP implementation: a data processing system for handling credit/debit card transactions. The requirements kept changing on a weekly basis because all the banks kept changing the way they recorded and conveyed transactions. XP could have been applied properly in such an environment and probably been pretty successful. -- MartySchrader

We have used XP (modulo teleconferences, frequent off-site meetings etc. instead of OnSiteCustomer) exclusively for the last two years and more, doing strictly commercial development. There are the usual limits to what I can state in public, but in general terms, during that time we have: ported Java to a new pda platform, implemented novel UI technologies for a range of mobile phones, supported a customer's developers in writing a proprietary messaging client, produced the SDK (including example programs and self-study materials) for an industrial hand-held, and written proof-of-concept mobile web services clients. Right now we are working on a soft real-time telematics product, also with XP. We have an active work queue out past Easter, and are recruiting. How's that for "sustainable in a commercially viable enterprise"? -- KeithBraithwaite

It may be difficult to judge 'commercially viable' in these times. It will be interesting to see what happens in the next five years.

Was that the sound of a goal-post shifting?

No, I'd say the same for any attempt to judge any methodology based on a short-term sample in these times. You have noticed the trouble in the economy haven't you?

Well, there have been layoffs at the shipyard down in Barrow, it's true, but then the hotels and such weren't hit as badly by the foot-and-mouth outbreak as many people feared, and have largely recovered. Some farmers did sell up, but the restocking has....oh wait, that's just our local economy. Which economy are you talking about?

The strawman says that Xp [...] is not sustainable in commercially viable enterprises. Are you suggesting that some sort of global economic problem might cause our XP-using company to fail (which it might), so that indicates a fundamental conflict between doing XP and being commercially viable?

I'm lost. If our example (a growing company with work in hand, and a history of work successfully done with XP) doesn't falsify the claim made, then what sort of example would?

I have provided, as requested, "current [Feb 03] products under development or on the shelf that are being developed and maintained using an XP environment", lets see a reasoned response from the other side of the argument, please.-- KB

What other methodologies satisfy the criteria of being "...sustainable in commercially viable enterprises?" I personally have never worked for a private, commercial company that even pretended to follow a waterfall methodology, and during my work as a contractor, I have seen little evidence of work actually being performed following a waterfall methodology, despite CMM or ISO certifications and truck loads of documentation. The most recognizable process I have seen is: start with a 1 sentence description of a feature, have development write code based on their own opinions, have testing write tests based on their own opinions, let the two departments argue about the results until upper management decrees "ship it as is."
I've added a second strawman that reflects my growing doubts about whether XP is any more effective than several earlier methodologies. The focus of the second is not to claim that XP doesn't work, but instead that XP has not, in practice, demonstrated the compelling advantages that it seemed to originally promise. -- TomStambaugh

Please provide a list of these compelling advantages, and some indication of how the performance of XP and other methodologies with regard to them is to be compared. Thanks.

Initial Project Definition: Requirements Document versus OnsiteCustomer

Schedule Time to complete - Requirements Document: Medium to High; OnsiteCustomer: Low to Not Possible - Low if onsite customer already has an office close to developers, time to move customer onsite increases with distance, may not be possible in situations such as commercial products.

Human Resources Needed to Complete - Requirements Document: Low to Medium; OnsiteCustomer: Low

Human Resources Idle During Step - Requirements Document: Low to High; High if programmers are idle, Low if programmers can be immediately available following document approval. OnsiteCustomer: Low, programmers can start when they become available.

Completeness - Requirements Document: Medium; advantage of having multiple sources, disadvantage that the sources go away after the document is approved. OnsiteCustomer: Medium; disadvantage of having a single source, advantage is ongoing availability of the source.

Skill Level Required - Requirements Document: High; eliciting requirements is difficult. OnSiteCustomer: Low

Effort Needed to Maintain Compliance - Requirements Document: Medium; a Quality Assurance person is assigned part time to audit mapping of requirements into design, dependent upon the QA person assigned. OnsiteCustomer: Medium to High; PairProgramming is used to maintain integrity, dependent upon the particular pairs implementing individual features.

Software Design: Design Document versus TestFirstDesign

Schedule Time to complete - Design Document: Medium to High; TestFirstDesign: Low

Human Resources Needed to Complete - Design Document: Low to Medium; TestFirstDesign: Low

Human Resources Idle During Step - Design Document: Low to High; High if programmers are idle, Low if programmers can be immediately available following document approval. TestFirstDesign: Low, programmers can start when they become available.

Completeness - Design Document: Low to Medium; TestFirstDesign: Medium to High

Skill Level Required - Design Document: Medium; TestFirstDesign: High, testing philosophies not normally understood by programmers.

Effort Needed to Maintain Compliance - Design Document: Medium; a Quality Assurance person is assigned part time to audit mapping of design into development, dependent upon the QA person assigned. OnsiteCustomer: Medium to High; PairProgramming is used to ensure use and completeness of tests.

Software Integration: Integration Test Phase versus ContinuousIntegration

Suggested evaluation criteria: Schedule Time to complete, Human Resources Needed to Complete, Human Resources Idle During Step, Completeness, Feasibility, Effort Needed to Maintain Compliance

[Question: Is the above providing any value to anyone (or am I just rambling)?]
Please provide a list of these compelling advantages, and some indication of how the performance of XP and other methodologies with regard to them is to be compared. Thanks.

Here is KentBeck's summary of XP's claimed promises, taken from Kent's preface to ExtremeProgrammingExplained (p xvi):''

XP promises to reduce project risk, improve responsiveness to business changes, improve productivity throughout the life of a system, and add fun to building software in teams -- all at the same time.

On pages 11-12 of the same text, Kent suggests that XP enables a four-step strategy, as follows:

Based on these claims, it appears that there are eight dimensions along which we might contrast and compare methodologies:

Perhaps the next step is to compile a list of candidate methodologies to be contrasted and compared along these eight dimensions.
It seems very rational to me to not look at XP as the ends, but as a means by which one can improve productivity. Individual changes can provide improvement, but all those pieces together are going to provide the greatest benefit. So the success of XP should be judged by the positive change in the software project, not by whether the project was terminated or not.

What I think that people fail to realize about XP is that people are typically motivated by the things that XP heavily reduces. Guilt and fear. In addition for XP to be successful the people involved in the process must want to see the software project succeed. In every project I have been in, someone associated with the project wanted it to fail. So here is a classic example of XP working and the project failing.

A developer who shall remain nameless (me) was hired to help a failing software project for a company who shall also remain nameless (forget about it). The first thing I did when I got on the project was to asses the difference between where the project was, and where it needed to be in order to be successful. Then I drew a straight mental line between these two things, and tried to determine how to DoTheSimplestThingThatCouldPossiblyWork.

Right away, we started developing lists of features that were in the project and organizing the features based on value to the software. (seeing this was a contracting project, and the client viewed the splash screen as one of the most important parts of the project, it had a high value.) Three weeks from the beginning of this process we delivered the first iteration of the software to the client. The project had been worked on for 11 months, and not one single thing had been delivered yet! When I arrived they had passed, I believe, the 4th time their estimate on its first iteration. The first date was almost 6 months prior to the first release.

Three weeks later, we had another release with the exact features set we promised. Three weeks later we had another.

Then things started to go bad. The only person who could look at the software and evaluate the project looked at it and gave the client his belief that it would be 6 months before the software project would be ready, and provided an 8 page list of features he believed would fill the 6 months. The project was on the verge of cancellation. However, I had estimates and velocity at my disposal. Based on yesterday's weather I predicted the software would be complete with all the features he listed in 3 weeks time.

He didn't believe me, called the software pathetic. I asked him if we provided the improvements he had asked for in 3 weeks if he would be willing to make another evaluation. He called me completely unreasonable and refused to work with me ever again. (I'm not kidding folks, this is really how the conversation went.) So as the saga unfolded I found out that the software we were writing replaced the software he had written. If our project succeeded it would take away a significant amount of his value to the company. Ahhhhh, now I began to understand...but too late...

3 weeks later we turned in a new version of the software. He confirmed to the company that we had repaired everything he had asked for, and issued a new, longer list of things needing to be fixed, including changing several things back to the way they were in the previous release. We did exactly that, but now our velocity was higher and we predicted 2 weeks. Two weeks later, on the day we turned in our release, the software project was canceled before the new release was even evaluated.

The reality is if we had never switched to XP, never delivered the software, the contract likely would have continued for another few months before they canceled it. So what happened then? Did XP fail? Was it XP's fault that because we delivered on time with the features promised the project was canceled? Yes of course. But then, if you look carefully, that is in the client's Bill of Rights.

XP doesn't make everything work in a software project, it simply gives you the ability to know what is happening. Which of course is often the death of things. But tell me what it's worth to you to be able to predict -- and that accurately -- where your software project will be in 3 weeks. If it worked on a project that was 6 months overdue, just think of what it could do for a project that began with it. (Of course they might decided to cancel that one too...another failure of XP? I don't think so.)

-- Todd Edman

You know what? As long as you guys got paid based on the three week or two week turnaround, the project was successful. I worked on a web site design project where my graphics partner and I put a bunch of time into creating a design and navigation map for the site - then the project was cancelled. Since we had already submitted our billing based on phases, we got paid for the time we put into it. It was a success, even if we never got any more work out of that customer. Yeah, actually, I use this approach as part of the sales pitch whenever I am trying to land a new client. I always tell 'em that they'll get something usable at each milestone, so when they pay for work they have something in their hot little hands. Also, each milestone's work has already been completely documented so that the client can take that package and shove it into the lap of another engineering design house and they can run with it. See? See? I'm always looking out for you, Mr. Client!

I have yet to lose a job to somebody else based on a completed milestone that didn't satisfy a client's expectations. And none of them have wanted to use anybody else after seeing my fine, upstanding work. [cough]
I don't believe in XP. I only believe in the Second and Third Levels of the CapabilityMaturityModel! Harrumph! -- AugustDriveller?
[STRAWMAN 3] XP is a step forward, but not a final solution.

I think the difficulty arose when XP was presented as a nice, tidy solution as opposed to a somewhat interrelated set of improvements. XP, like any process created by human beings, is imperfect, however, that does not mask its successes. XP is not a great failure, but neither is it an absolute success. I think it is worthwhile to violate one long espoused tenet and tear XP apart and look at its individual components. Please note that XP was the culmination of a single, 11 month project; it should not be surprising that it is not a complete, general purpose solution. -- WayneMack

TestFirstDesign - I find this the most fascinating aspect of XP. It takes the long held view of programmers that the cost of upfront documentation far outweighs its value, but marries it with the traditional viewpoint that programming is inexact process and a heavy level of testing is necessary. Traditional literature typically weighted the importance of the three tasks as Documentation, then Testing, then Programming; though in practice, the weighting seemed to be Documentation, Programming, then Testing. Test First Design reverses the precedence of the latter: Testing, then Programming, then Documentation. For Test First Design to be accepted, therefore, two items need to be re-evaluated in most development organizations. It must be believed that Testing is a valuable step and not one to be pushed off to the end and done by some of the most lowly paid employees. It must also be believed that Documentation is not that valuable and does not need to be done first by some of the most highly paid employees.

OnsiteCustomer - I am very favorabably disposed to the idea of increased user interaction with the development team, but feel that the hope for a single customer voice is naive. Most companies have many semi-autonomous groups whose only point of interaction may be the development team. Thus it seems that the responsibility for the resolution of conflicting propositions should rest within the development team. This may be a lesson to be learned from C3ProjectedTerminated, as it appears there were at least two customer voices and one did not feel it was being heard. Another concern I have is the approach of bringing the user to the development team, rather than bringing the development team to the user(s). There is simply too much about a task that is difficult to communicate verbally; the developers should have a chance to experience it first hand.
A "Top 10 Dying Technologies" list that includes XP:

http://blogs.techrepublic.com.com/10things/?p=842

Various prolems with XP are listed at http://members.cox.net/cobbler/XPDangers.htm.

Listed? Read with a critical eye. Here's a gem from this: Would requirements review have saved C3? We can't say for certain, but we do know that "feature-itis," which comes at the expense of scheduling, is a predictable result of letting the programming team decide (and continuously change) the requirements priorities without informing the project sponsors. --RobMandeville


NOTES:
  1. As mentioned below, see CompaniesDoingXp. There are commercial successes here.
  2. "Several prior methodologies" also have their own individual success stories; examples are welcome.

See: CompaniesDoingXp

CategoryRant

EditText of this page (last edited August 26, 2011) or FindPage with title or text search