It indicates a perception that behavior and structure are separate concerns, and I sense that this is tied to an artificial separation between "functional" and "non-functional" requirements. But these are just different facets—any service has a "what it does" and a "how well it does it" (service level) aspect. A poorly designed system quickly erodes the "how well." Even when the "what" is being measured as the check box on user value delivery, the user notices, with less than delight, when system properties degrade. 

So at what point is the system degraded enough to warrant "cleaning up"? By degraded, I mean in terms of system qualities that affect user experience directly (various dimensions of performance and reliability) and indirectly (system price tag and speed of getting access to new features), as well as the business's ability to respond (development cost, time and predictability). Deferring quality impacts user experience and business agility down the road. We know that. In lean manufacturing, the factory floor is kept so clean you could eat off it. Why? Is it waste to keep the floors clean? Is it waste to ensure quality from the get-go, and at every point where quality can be improved, rather than inspecting for quality at the end? Lean manufacturing leans heavily on the total quality movement. The biggest waste of all are quality issues that cause delays at the end of the process, or worse, show up in the hands of the user.

So are we refactoring to simplify? to remove duplication? to remove complex or unnecessary dependencies? to improve performance (e.g., cleaning up a complex collaboration path)? to apply a pattern that is well-understood and easy to communicate? to make the system easier to extend? to make the system easier to debug? to make pieces of the system more useful in other contexts?   

"Yes" to some of these will show up in direct user experience, because user experience is affected by system properties (the "how well"). And some will show up in development efficiency (how much we can do with given resources) or agility (how quickly we can respond to the market). Sustaining innovations come from being able to incrementally improve the system, and a clean architecture allows for chunks of the system to be innovated on and improved at a different velocity from other chunks of the system. [The Trek architecture allows, for example, Shimano to innovate on gears independently from Bontrager on frames.] Development efficiency shows up in future product or service cost, agility in new features, which all affects business competitiveness and future customer experience.

If your system is a one-off short-term special project that's one thing. But if it is to see your business through years of evolution, don't let the need to clean up get hidden under the covers too long.   

Remember, as architects and software developers we have TWO (classes of) voices to listen to: the voice of the customer and the voice of the business. The business cares about our attention to the voice of the customer, because that is how we compete in the near-term. But the voice of the business is also about the long-term viability of our business and the systems and products that support it.

And, if none of this persuades, focus on just one thing: in business after business, industry after industry, we hear "it costs 80% of our budget just to keep the lights on" (where "the lights" are the systems that undergird the operations of the business). Maintenance or more appropriately, system evolution, matters. You can justify refactoring on the basis of getting that number down for the life of your system! And that is only one piece of the systemic economic puzzle. Structure factors!

Of course, I believe that this refactoring should be happening even before we get to code, by playing out system behavior (addressing the "what" and the "how well") through our models. Remember, architecture decisions are those that have a high cost of change (a key Booch insight), so we want to learn as much as we can, as cheaply as we can, as early as we can. That is, we want to do iterative modeling and improvement through stakeholder participation, and refactor responsibilities to achieve crisper, more balanced designs, or to address aspects or cross-cutting concerns more effectively. For larger-scale systems (where I mean code base, in this case, not transaction load), it becomes harder to significantly refactor at the code level without impacting diffuse parts of the system, and that is typically error-prone. So, model, and test our models, create a skeletal architecture, test that, and so it goes. Start agile early, with architecture.

System behavior has to do with function, the properties (a.k.a. qualities) of that function, and the interaction of qualities across the system. In a car, for example, there is an interaction between properties associated with the acceleration function, and properties associated with the braking function. If the car accelerates like a Maserati, it better brake like a Maserati, or there'll be times when the driver will face a life-threatening situation. The creators of the braking system may have nothing to do with the creators of the acceleration system (and may even be different external suppliers). But the architect has to ensure that the properties are consistent; that the architecture delivers behaviors with the intended properties and the system has integrity, so that properties of one subsystem are not inconsistent with those of another subsystem. System properties, or cross-cutting concerns, are among the factors that make architecture so crucial in complex systems.