When deciding to refactor or not, you should consider the costs, risks and benefits of doing the refactoring.
This will tell you when it's worthwhile to start refactoring, and when it would be wise to stop.
Cost of Doing the Refactoring:
This is the cost of changing the code: renaming a variable; copying an expression up to an assignment statement & replacing all copies of the expression with the variable; extracting common or reusable code to a separate method; or any of the other valid refactoring moves. Typically, this cost is low, as it's not hard to make a few syntactic changes to code.
Cost of Testing the Change:
Any change you make to the code may introduce bugs, and so must be thoroughly tested. With a complete set of automated regression tests (UnitTests) that TestEverythingThatCouldPossiblyBreak, this cost will be very low because you're leveraging the testing investment made when the tests were written. However, without UnitTests, as is typical of non-XP projects, the testing costs can be significant.
Cost of Updating Tests and Documentation:
Refactoring may change class interfaces, or even introduce new classes. UnitTests for these classes, and whatever "design documentation" your project maintains for interfaces and/or classes will have to be updated/written. This can be a non-trivial cost to projects that maintain a comprehensive set of UnitTests; but having the UnitTests substantially reduces other costs. XP projects typically don't have external class/interface documentation (which would require maintenance). Projects using more traditional documentation-based (TheAlmightyThud) methodologies would have much higher costs.
Risk of Introducing Bugs (not caught by testing):
If UnitTests fall short of perfection, there's a risk that bugs introduced during the change may be released into production, causing loss of business data. On non-XP projects, this risk can be fairly high. And in an environment where one is incrementally introducing XP concepts (ExtremeProgrammingInEnemyTerritory), the political repercussions of such a slip can be severe.
On an XP project,
Cost and Risk of Refactoring tend to be low, as we have good UnitTests. Also the "size" of refactorings tends to be smaller, as the code tends to maintain good conceptual structure, even in the face of change. (Refactoring is self-reinforcing.)
Risk of regression in functionality is much higher, requiring more through manual testing to catch errors. Still, in spite of higher testing costs, risk of introducing regressive bugs will be higher than in XP projects.
Adding new features no longer corrupts the system's structure:
Optimizing the maintenance process to minimize the visible changes to the source code must, as with any optimization, compromise something else: In programming, what gets compromised is the readability, maintainability, and logical structure of the system being maintained. So, over time, a system maintained without refactoring will become unmaintainable, and must be scrapped and rewritten from scratch. Refactoring avoids the costs of working with unmaintainable code, and the eventual cost of replacing the system.
Improves programmer's understanding of the system:
Refactoring produces shorter simpler methods. It's less work to understand the smaller amount of code that needs to be changed to implement any given function.
[The benefits are proving a bit harder to quantify than the costs & risks...
Refactored code is easier to test if that was one of the goals of the refactoring.
WellFactoredCode is easier to maintain! It's:
easier to add new functionality to
easier to test
easier to find and fix bugs in
And easier translates directly into faster ... fewer man hours, costs less. Spending a little overhead refactoring all the time means an overall ProjectVelocity increase. (I'm not sure how to prove that point.)
We all fall short and find out that we've incurred a TechnicalDebt. This is when, perhaps even unknowingly, the code has become poorly factored. What are the EconomicsOfTechnicalDebt??
Best I can suggest is try to find a way to do it a little at a time. This way the customer still gets new functionality and the code gets better slowly.
Perhaps there are times when it makes sense to do it all in one big, focused effort. I don't know.