Why It Is So Hard To Sell Extreme Programming

In Y2K, I read two books that radically changed my thinking about software development and how it relates to business goals. The first book was ExtremeProgrammingExplained, the second was CrossingTheChasm. XPE told me HOW to EmbraceChange, CTC told me WHY. Both of these books describe a flexible, but highly disciplined process for product development. XPE from the software development point of view and CTC from the marketing point of view.

CrossingTheChasm describes the TechnologyAdoptionLifeCycle. Once I understood the PsychoGraphics underlying the TALC, I could see why so few developers have adopted XP. The Innovators who would be interested in XP are probably already involved. I'm sure that there are a few Visionaries involved (you know who you are). I know that I was attracted to XP because I was looking for an order of magnitude improvement in quality, speed, skill building, teamwork, fun, etc. Are there other Visionaries out there who haven't heard of XP yet? Or maybe they don't see how it would fit in their projects.

I really want to see XP become the mainstream software development method. Why? For purely selfish reasons. I believe it is a better way to develop software and I want to work that way. My worry is that like SA/SD and OOA/OOD, it will take an awfully long time before there is general acceptance. I think that XP is a quantum leap forward just like structured programming and object-oriented programming when they were introduced. A lot of shops paid lip service to those methods, long before they were really using them.

An even bigger concern, is how to cross the chasm to the Pragmatists. The majority of MIS managers are Pragmatists. They are not going to buy into XP en-mass until a lot of their peers are having success with XP. Yes, that is a Catch-22. No offense to anyone intended, but would a Pragmatist call C3 a success? Motorola appears to have completed a multi-national, multi-site, partial-XP project. Would a Pragmatist take a reference from them? I don't know, but I'm not confident. The fact that Pragmatists reference other Pragmatists in their MarketNiche before making buying decisions can not be over emphasized.

CrossingTheChasm says that you need a BeachHead. If you can provide a WholeProduct to that MarketNiche, you can meet their CompellingReasonToBuy?. Then, you can begin to generate the ReferenceBase? needed to convince other Pragmatists to buy. Then you go after another niche, and another, and another, until the momentum begins to feed on itself.

Why do I view XP as a high-tech product? Even though the Extremos say that all you need is a stack of index cards, that isn't going to work with Pragmatists. They want a WholeProduct. History shows that whole industries grew up around SA/SD and OOA/OOD. Dozens of applications were developed. Hundreds of books were written. Thousands of consultants appeared. Tens of thousands of people went to conferences and they all bought the tee shirt. All of this to support a simple idea. It will happen to XP as well, if it becomes a mainstream development method.

-- GaryBrown?

It would be great to have an answer to this question. Speaking as someone who has tried and failed with about ten out of ten coworkers, I'll give my tentative opinion.

Of course, it's going to be a different set of reasons for different people, but so far I've seen the following:

Maybe if you could overcome the communication barrier, you show them how much better things could be, thus overcoming the complacency. Once the complacency is dislodged, the desire to do better can overcome the fears. That's tough, though.


The communication barrier that you have encountered is the chasm. The folks that you are working with are Pragmatists. I'm guessing that you are an Innovator or Visionary with respect to XP. You have a different set of goals. See PsychoGraphics. They need other Pragmatists to tell them about their success with XP before they will be willing to try it. -- GaryBrown?


XP adoption requires significantly more involvement from the clients than traditional methodologies. Adopting it requires climbing two TechnologyAdoptionLifeCycles at the same time, as you are replacing both software creation and RequirementsAnalysis at the same time. For many business clients, a better software development process at the cost of their being frequently distracted from their other duties, which their bosses value more highly, is not a good trade off. Also, they end up more responsible for the results and avoiding responsibility for things you don't control is part of Management 101. -- MarkSwanson


I think that the customers of a software development project need to be a lot more involved. They are responsible for identifying the requirements that bring business value to a development project. Those requirements are not static in most cases, especially in new and emerging markets. To me, the genius of XP is that it places the responsibility for defining business value on the customer. It also give the customer the power and flexibility to achieve that value by steering the project.

I agree, that adopting XP requires significant change throughout an organization structure. I believe that this is a GoodThing. Traditional methods cause communication barriers to form between developers and their customers. XP asks the customer to be responsible for and control the quality, cost and scope of a project. What could a customer be doing that has more value to their boss/organization? -- GaryBrown?


Is it really hard to sell ExtremeProgramming? I had an awfully easy time out of it. Maybe because we were a startup, and our management hired me explicitly to bring in "real software engineering experience" (cough cough - I didn't tell them that software and engineering is an oxymoron).

Anyway, when I said we'll do PairProgramming, I expected the standard "no way you'll have two resources when one will do" bit, but instead I got a "why?". I told them we improve quality, couple it with CollectiveCodeOwnership and change pairs frequently to distribute knowledge around. This minimizes business risk and increases the BusNumber. They bought into it immediately.

I told them we'd do the PlanningGame, and engineering was going to be a service function towards business. They loved it.

True, most developers were hired based on their willingness to go along, but one was pre-existing, and though he hated the idea, he now thinks that it's great. (He's a smart, honest guy with an open mind.)

Anybody had similar (easy) experience in selling XP? -- AlainPicard
I had the same experience in a start up, except that I was hiring all of the developers. I gave each of them a copy of XPE after their initial interview. I only hired the ones that really wanted to work in an XP environment.

Start ups are more likely to be populated with Innovators and Visionaries. See TechnologyAdoptionLifeCycle and PsychoGraphics. It is not necessary to sell them. The Pragmatists that run most IT/MIS shops do need to be sold. And don't forget that they need to reference other Pragmatists who have had success using XP. -- GaryBrown?
I work for a small consulting/custom engineering firm, and I think they will accept XP. The problem we face is that we consult for Fortune 1000 companies. It is clear that they (well, at least the one I'm working at right now) need help in the development process. If they'd brought us in to consult on process, maybe they'd buy it, but I that's not why we're there. But as things stand, I think there are too many political groups (with great turf-protection experience) to convince, IMO. And then there is the delicate nature of us telling the customer that there current process sucks (though you'd think they know that).

If it weren't that XP started at Chrysler, I'd think it was a totally hopeless cause. -- DavidCorbin

Large corporations have many ways to absorb risk and thwart change, as they should, I suppose. XP worked at Chrysler only because all of these mechanisms had failed them at once. Once out of the Y2K bind they reverted to usual corporate behavior, only then as an even larger company. -- WardCunningham


Call me anti-social but I can't face PairProgramming. It's my biggest stumbling block.

Are you speaking from fear, or experience? If experience, tell us why you found it so painful. If fear, BeBraveLittlePiglet.

Also the fact that refactoring is not trivial in C++, VB and many other mainstream languages.

Doctor Doctor, it hurts when I do that! So don't do that.

Are you saying that one can't, or shouldn't do refactoring in C++, VB, and many other mainstream languages?


If you ask me, I think XP is still firmly in the Innovators stage, not the Visionaries stage. The graph at the top of this page shows a 3-5 year time frame, but I believe that time frame is for individual technologies such as the Walkman and the cell phone. XP is a culture, and it takes a lot longer to adopt a culture than it does to adopt a technology. The success with start-ups (above) is where XP will gain its acceptance and cross the chasm, especially start-ups where the person in charge of setting the process is an XP enthusiast. I predict (off the top of my head; and I'm no expert) it will be 2 more years +-6 months before XP moves into the Visionaries stage. From there it should move quickly through to Pragmatists (say another 3-5 years to reach the peak).

I have to disagree. The fact that there are several XP projects under way tells me that the Visionaries are in the game. The Innovators for XP are the folks who say that they are trying SoloXP. Look at pp 57 - 59 in CrossingTheChasm to get a feel for the difficulty in getting the Pragmatist IT/MIS managers to buy XP. -- GaryBrown?


Those who EmbraceChange will thrive in the MarketPlace?. Those who do not will not thrive. The MarketWillDecide? when the peak is achieved.
Part of the problem may be in the name. "Extreme Programming" may conjure up images of CowboyCoding among a less-adventuresome ItManager. See AnExtremeConversation for an example.


Just seeing ExtremeProgramming and CrossingTheChasm in the same sentence made it click for me. I think Gary is right on. But what is a good BeachHead? Startups are probably not it, because they tend not to be viewed as Pragmatists. What MarketNiche does XP fit best into? Well, clearly one in which requirements are likely to change (although my personal view is that the requirements on all projects are likely to change). I need to go home and read the book again.

This is good, and I'm excited, I'm just inarticulate.

-- PaulTevis

I actually think startups are the best way to introduce XP. You get the culture in place right from the start and 'grow' the company on XP (like fertilizer?). Culture is the biggest stumbling block XP has to handle. XP is not management-friendly on the surface (XP requires fewer managers and gives them less control). XP is a ParadigmShift, and ParadigmShifts are brought about by 'the next generation'. If you view corporations as entities (which legally they are, but what I mean is that the culture of a company is what makes it a cohesive 'individual'), then the next generation are the startups. I'm starting one up right now! :-) -- RobHarwood

Thinking about this more, two possibilities come to mind. One is along the lines we have been discussing, i.e. the mainstream acceptance of a discontinuous innovation through chasm crossing, as Gary originally proposed. The other is along the lines of the scenario outlined in The InnovatorsDilemma. In this view, XP is a "disruptive technology" which means that addresses a different set of consumer needs (the author's term is ValueChain) than does the technology it is trying to replace. As a result, most companies reject it because it does not address the needs of their market (because they want design documents, deployment plans, and other things that a heavier-weight methodology provides). Those companies that do adopt it have different needs because they address different markets (rapid deployment, responsiveness to changing requirements, etc). If XP is successful, then it migrates "up-market" as those companies who did not adopt it gradually have their markets eaten away by those who did (because as it evolves, it proves to address the needs of other markets as well). -- PaulTevis

I believe the latter will happen as you suggest. In fact, I'm banking on it. In my field of computing (multimedia), there is a huge need for any process. I believe XP will be my competitive advantage. My job now is to educate my clients. See FirstCreateTheMailbox. -- RobHarwood

Discovered over on VcapsProject
"What you get is a very good environment to apply ExtremeProgramming. Basically we are bean counting.The Problem is that the way we count the beans is very important and is dependent on exactly how the analysts are counting them...this time. Each go around will have a new set of rules and government regulations."

Perhaps financial systems are the right MarketNiche for the chasm crossing. Both VcapsProject and ChryslerComprehensiveCompensation demonstrated the effectiveness of ExtremeProgramming in this market. -- PaulTevis

Discussion about the issues and practicalities relating to OnSiteCustomers? moved to ProblemsWithOnSiteCustomer.


I have to admit that the PairProgramming is the biggest stumbling block for me too. I've tried it, and it was fun, but when I got done I felt like I'd been wasting company time because I wasn't coding myself. To me, it seems that you would have to see at least a doubling of productivity to make it worth it to have two programmers working in a pair. I have a hard time believing you get that much of a boost the majority of the time. -- EddieDeyo

You won't get a boost right away. It takes time to become accustomed to the different roles. When I started out, it was like there were two individual programmers sitting at the same machine. Later, when I realized that each programmer plays a very different part, it became more like PilotCoPilot. That's when my partner and I really saw the benefits. It helps to switch roles fairly often as well, because then you will more quickly see that each person is doing (and supposed to be doing) different things.


Geoffrey Moore's chasm crossing model asserts that selling technology initially involves romancing the visionaries and early adopters, but ultimately the mass market only cares about price and warranties.

XP is an excellent means to this end, but not an end in itself. XP could be the service that creates the product that is crossing the chasm.

The "craft" quality of XP is very effective in the early stages of product development where only 10% of projects survive to CrossTheChasm?. When a product or technology becomes fairly commoditized, then you start to see more repeatable, but less agile, software "engineering" practices start to supersede the software "craftsmanship" practices; perhaps because accountability to shareholders starts to supersede accountability to customers.

-MichaelLeach


Many projects fail due to a lack of sufficient planning and understanding of the goals. To people who keep getting hit with "We don't do enough planning.", a suggestion to do the opposite of BigDesignUpFront (instead of more big design) may come off as ridiculous and not worth further consideration.

Many projects suffer from ScopeCreep, drifting away from the spec into uncertain territory, and insufficient change control. To someone who firmly believes that "We must adhere to the spec, and stop letting the client change things, or we'll never get these projects done!", a suggestion that the problem could be solved by doing away with the spec and letting the client change things more often and more easily just sounds like blasphemy.

When a project suffers from PrematureRelease?, with insufficient QualityControl or QA (usually due to insufficient time), a suggestion to release earlier and more often seems impossible. How can you say release earlier, when we just released a month late and now have to eat the cost of all these bugs?

When processes are failing, and a team really needs a new methodology, they usually have high overhead and insufficient productivity. Pair programming, implementing test frameworks, and spending half your time writing test code just don't sound like things that would improve the bottom line. Sounds like the opposite.

So, given that the major tenets of XP sound more like extreme versions of the problems that caused someone to look for a new methodology, rather than the solutions, how do you sell the synergy? Or how do you get them to listen long enough to sell it?

Would be fair to characterize it as a HighDisciplineMethodology?

A few other people have said something along these lines but I think it bears repeating. As a programmer who has worked with other programmers, I can say that we are, on the whole, smelly and unpleasant people. Additionally, having to be given a buddy, like some kind of kindergarten student on a field trip, is the single most degrading and humiliating thing that I have ever heard asked of a skilled professional. Pair programming is such an unpleasant concept to me that I immediately lose interest in every other aspect of XP and refuse to even consider the idea. I would not only quit a job if forced to pair program, but I would quit programming altogether if it came right down to it.

Re: "I can say that we are, on the whole, smelly and unpleasant people."

Ya don't say? Time for VirtualPairProgramming? Smellavision has not been invented yet and if your partner gets too ranty, pull the plug or run his/her voice thru the Pleasant-A-Tron filter.


HowToSellGoldenHammers

EditText of this page (last edited May 7, 2013) or FindPage with title or text search