Iterative and Incremental Development

In the May 2008 CrossTalk, Alistair Cockburn writes on Incremental and Iterative Development and concludes:

"The word increment fundamentally means add onto.

The word iterate fundamentally means to re-do.

Sadly, iterative development has come to mean either incremental or iterative, indiscriminately."

In my experience, people mean incremental and iterative (iid) when they say iterative development. In other words, iterative development, as it is used in software, may be more like the iterative method in math:

an iterative method attempts to solve a problem ... by finding successive approximations to the solution starting from an initial guess

wikipedia

Ideally, this proceeds along two dimensions: learning about the requirements by putting working increments in front of users and successively nudging the system towards user acceptance (making changes to features as misunderstanding or changing needs are uncovered, and adding features); and fixing the structure as the increments get built out and more is learned both about structural demands and opportunities to improve. Ideally. The failure modes that Cockburn describes in his case study pieces reflect mismanagement of the process. Though they start with the best intentions to do the learn/improve piece that is implied by iterative, the project team may (be asked to) give up on those intentions under schedule pressure. What we aim to do is multi-faceted—learn/improve and elaborate/add features or services. That is, we aim to take an iterative and incremental approach to evolving the system towards a viable release version. But this can be set off-kilter various ways: too many resets trying to reach feature or structural perfection; or too much weight on feature progress at the expense of structural learning and resets needed to improve structural soundness.   

When we talk of iterating in the Visual Architecting Process (VAP), we mean repeating the requirements/specification/validation cycle. That is, the actions are repeated, though the content or focus shifts as dominant uncertainties/complexities/challenges are surfaced and addressed and a new round of strategies and decisions are investigated and articulated, and tradeoffs are made. It is designedly a learn/improve cycle and we highlight this with an explicit learn/improve/validation phase of the cycle. The philosophy is to cycle quickly, and to cycle through different concerns (the architecture decision model views, as well as other stakeholder concern-view pairings), returning to views to correct, refine, elaborate, refactor, etc., as more is learned about the requirements and different architectural alternatives are envisaged and explored.

So, I think the heart of Alistair Cockburn's message is spot-on. We have two avenues we need to be more clear about in software: incrementally building out our systems; and repeatedly/systematically stepping back to figure out what we have learned and to improve our systems. But I don't think these are mutually exclusive. And I'm not worried about our terminology, but rather about our practices. Frenetic course correction in the medium of code is wasteful and demoralizing. Rather get the course more clear first. Frenetic push to feature release with mounting design debt is also wasteful and demoralizing. Good people leave in disgust. Rather pay it forward beginning with (just-enough/minimalist) architecture cycles that apply agile principles from the get-go—just loosening up the "code" requirement early on, to allow for cheaper/more flexible/adaptable learning via models in the early cycles. Cycles that move through understanding stakeholder concerns and addressing them through views, and stepping back to learn and improve, and later to validate, and repeating, with the focus of the next cycle determined at the architect's discretion. In short, drawing more of the architecture work earlier, so that the duality between system value/user experience design and structural design can be explored, opportunities to innovate through technology can be surfaced and expectations can be set realistically.      

An explicit focus on incremental development reminds me that when we talk about incremental development in software, we are talking about something more complex than just adding on boxes. We're adding to boxes, and adding on boxes, and hopefully not munging the boxes as we do so. More complex projects will need more architecture work before construction cycles begin, because starting with all the wrong boxes costs either in terms of significant rework (design to code) or in an architecture that will quickly succumb to evolutionary pressure, and bring us back to the state that caused words like mung and kludge to become intimately associated with our field.