Refactoring @ Scale
In my previous post, Refactoring for Agility, I posted an outline and some thoughts for Part I of an Overview on Refactoring. Now I'm ready to post on Part II which is about refactoring @ scale. By "at scale" I mean in the larger context of other agile practices, as well as for large projects.
PART II - REFACTORING @ SCALE
|Refactoring In-the-Small||Refactoring @ Scale|
|Small, Fast & Frequent Refactorings||Larger, Periodic & Planned Restructurings|
|Emergent Design||Incremental Design & Evolutionary Architecture|
|Deferred Refactoring||Restructuring & Technical Debt|
|Code Smells||Architecture Smells|
|Design Principles & Patterns||Software Modifiability Tactics|
|Simple/Clean Code||Supple/Domain-Driven Design|
2. Emergent Design
- Emergent Design is a fancy name for the resulting design that “emerges” from the synergy of combining Refactoring together with TDD, Continuous Integration and Automated Testing.
- Emergent Design means that ...
4. Restructuring Technical Debt
- If we accrue a non-trivial amount of technical debt, we can’t simply “refactor” it away.
- Paying it off typically requires restructuring efforts (or even reengineering) that must be planned.
- Iteration plans must accommodate specific tasks for these restructuring efforts (or even be dedicated to restructuring).
- Ignoring it, or deferring it for very long is not a viable option!
- Identifies higher-level issues (“architecture smells”) that typically represent violations of known principles of good software architecture & design.
- Periodically applies larger-scale “refactorings” and/or many small refactorings that were previously deferred.
- The goal is to “pay down technical debt” in order to limit the increasing costs of accumulated complexity.
- Typically requires a concerted effort that must be separately planned.
- Uses not only design patterns/principles, but also architectural patterns/principles, as well as software modifiability tactics.
- See Martin Fowler's description in Refactoring Malapropism
- Restructuring is often associated with absent or neglectful refactoring and/or design.
- But … Any large software project spanning multiple teams eventually needs restructuring.
- Even in the presence expert-level architecture, design & continuous refactoring
- This is just a reality of software evolution/entropy
- Therefore … Large software projects should assume that periodic restructuring will be necessary, and should plan accordingly to:
- Clean-up accumulated code-smells and apply numerous refactorings that were deferred but are now sorely needed,
- Address architecture smells by applying restructurings, patterns and modifiability tactics that have broader impact.
See Stefan Roock's and Martin Lippert's book Refactoring in Large Software Projects (There was an earlier version of their work entitled Large Refactorings").
- Smells in Dependency Graphs
- Smells in Inheritance Hierarchies
- Smells in Packages
- Smells in Subsystems
- Smells in Layers
- Localize Changes (increase cohesion)
- Prevent Ripple Effects (reduce coupling)
- Defer Binding-time (defer decision-making)
11. Evolutionary Architecture
- Software Architecture concerns infrastructure elements that must exist before you can begin execution.
- Architecture is about things that are hard to change later, it is difficult to allow an architecture to emerge.
- For large projects, this includes high-level organization of the system into functionality/elements that will be allocated to separate teams.
- Key techniques of Evolutionary Architecture include:
- Deferring Irreversible Decisions to the “Last Responsible Moment” (LRM Principle)
- Architectural “Spike”, Architecture Iteration and/or Spike Iteration
- Design evolves incrementally, iteration by iteration, based on current business priorities and discovered technical limitations.
- Incremental Design …
- Does not prohibit thinking about higher-level design.
- Does encourage planning in detail only what will be constructed soon.
- Focus is on “Just Enough, Just-In-Time” :
- Specifying too much detail too soon causes more rework later.
- But doing less now and saving the rest for later should not require significantly more work later than it would today.
- We must do Just Enough Design Initially to attain the right balance of anticipation and adaptation.
- Initial design (before coding) is still necessary.
- This use of “JEDI” was coined by Stephen Palmer as part of Feature-Driven Development (FDD)
- Basic rule of thumb to tell when “JEDI” is achieved:
- At iteration-scope, when, after one pass through the iteration backlog, modeling in small groups does not produce any new classes or associations of real significance.
- At task/TDD scope, when we have defined enough structure & interface(s) to know specifically what code to write/test, precisely where to write it, and exactly how to invoke it.
- Techniques of “the JEDI way” include:
- Domain-Driven Design (DDD) approaches modeling the core logic of the software by focusing on the domain.
- The basic idea is the design should directly reflect the core business domain and domain-logic of the problem to solve
- This helps understanding the problem as well as the implementation and increases maintainability of the software.
- DDD uses common principles and patterns as "building blocks" to model & create a "supple design”
- Supple: pliant, malleable, limber, yielding or changing readily.
- The design is firm yet flexible, with structure and intent both clearly conveyed and deeply realized by the code.
- Patterns of Supple Design include: Intention-Revealing Interfaces, Ubiquitous Language, Side-Effect-Free Functions, Assertions, Conceptual Contours, Standalone Classes, Closure of Operations, Declarative Style
Resources on Emergent Design and Evolutionary Architecture
- Neal Ford, Evolutionary Architecture and Emergent Design series of articles on IBM developerWorks:
- Emergent Design: The Evolutionary Nature of Professional Software Development, by Scott L. Bain
- Architecture in an Agile Organization by Chris Sterling (online)
- InfoQ: Michael Stal on Architecture Refactoring
- Refactoring in Large Software Projects: Performing Complex Restructurings Successfully, by Martin Lippert, Stephen Roock (also an earlier version online)
- Object-oriented Reengineering Patterns, by S. Demeyer, S. Ducasse & O. Nierstrasz (freely downloadable online)
- Technical Debt and Design Death:How to ensure you can deliver business value in the future as well as in the present; by Kane Mar and Michael James
- Design debt economics: A vocabulary for describing the causes, costs, and cures for software maintainability problems;by John Elm, IBM developerWorks, June 2009
- Managing Technical Debt and 10X Software Development - Technical Debt, by Steve McConnell
- Managing Technical Debt; by Tom Brazier
- Repaying Technical Debt; by Simon Baker
- Software Debt: Depreciation of Value for Software Assets and Technical Debt: Reducing Internal Quality for Short-term Gain, by Chris Sterling
- Software Entropy: Don’t Tolerate Broken Windows, and Zero-Tolerance Construction by Andy Hunt and Dave Thomas
- Modifiability Tactics, SEI @ CMU Technical Report CMU/SEI-2007-TR-002
- Modifiability Tactics, Chapter 5 section 3 of Software Architecture in Practice
- Understanding Architectural Patterns in Terms of Tactics and Models
- Design and Tactics, Lecture notes from an
course on Database and Software Architecture Aalborg University
- Wikipedia page on DDD
- DDD Intro Article
- DDD – a Brief Introduction
- DDD: Supple Design Patterns
- DDD Pattern Summaries
- Free mini-eBook: Domain-Driven Design Quickly
- Yet another free mini-eBook: Step-by-Step Guide to DDD
- See also domaindrivendesign.org and infoq.com/domain-driven-design