On Strike at the Software Factory
Since there has been software, there has been the dream of the big green button.
Predictions of the future usually involve elaborate thought-capturing machines that deduce the true desire of an end user and effortlessly create their ideal application, ready for orders, often in the form of a simple form and a big green button promising magic with a label like ‘Go’. This is great for the futurist daydreamer, but terrible for the software developer who lives a life of continuous improvement, watching in horror as their profession is reduced to an afterthought somewhere between waking and the business application someone dreamt up before lunch. This is right around the time that the very emotional craftsmanship debate kicks in, and the humans vs. machines themes spring to awkward, robotic life.
Since the term "software factories" was coined in the late 60’s by Hitachi, it has been linked inexorably to the manufacturing process it is meant to emulate: people fear that the automation of software components will put developers out of work, or reduce their capacity down to mere assemblers of components rather than authors of code. We hear the buzz in technical communities about the emergence of powerful new software factories, developed by Microsoft and others, and can’t help but wonder if the intended goal of these new innovations is exactly along those lines. Yet, the people most involved in the software factory movement see it as just the opposite, a way to harness factory processes simply to meet the surging global demand for software at all, let alone with chairs to spare.
"We see a capacity crisis looming. The industry continues to hand-stitch applications distributed over multiple platforms housed by multiple businesses located around the planet, automating business processes like health insurance claim processing and international currency arbitrage, using strings, integers and line by line conditional logic. Most developers build every application as though it is the first of its kind anywhere.
Without significant changes in our methods and practices, global demand for software development and maintenance will vastly exceed the pace at which the industry can deliver in the very near future." (from
Why should you care about software factories? After all they have that irksome quality of sounding like something meant to replace you. The reality is that software factories are a potential answer to a problem that you live with on a daily basis: change. When a software system changes, it creates risk which often hits your desk in the form of bugs to fix. When businesses change, they adopt new technologies, abandon others, or leverage what they already have in new ways. For developers on the ground, this means you either need to predict the future, or jump hoops in the present to keep up with this change and maintain or acquire the skills that make it work.
The future of computing, arguably, is a repeated pattern from its past: enabling technologies are introduced, adopted, and eventually reach maturity in the wake of the next round of capabilities introduced by innovators. Developers working with bleeding edge technologies knows it’s their blood. Developers working on mature or sunset systems feel a little out in the cold when they hear what’s on the horizon. The software factory is rising in popularity as a way to preempt the technology adoption cycle by introducing automation where a developer would spend time on low yield tasks, and encapsulating domain knowledge where a developer would need lengthy, dedicated time with domain experts. Today, factories are even used to allow business experts themselves to change core behavior of their software, without interrupting the software development life cycle.
You already understand the value of re-usability (think open source), maintainability (think unit testing), and replaceable architecture (think layers or seams); what’s a software factory if only more of the same on a grand, possibly unworkable scale, and what’s in it for you? Who cares?
As a developer, you should care because the goal of a software factory is to clear a path for you to get to "the good bits"; the business strategy that changes markets, the social networking feature that changes how people communicate, the custom solution that runs a small business like clockwork. In the end you’re remembered for how you solved a problem, not how separated your concerns were. The sooner you can get to writing the code that matters to your customer, the better.
Within Microsoft itself, a quiet revolution is forming in the labs and in upcoming updates to our most central development tools. Let’s look at some of the software factories that make up the future of computing on the Microsoft stack.
Factories for Business Problems
Domain Specific Language Tools ( )
A domain-specific language, or DSL, is all the rage at the moment and is likely to crop up in many new development projects in the near future. The power of a DSL lies in its ability to define a specific domain (such as the shipping business, or indoor pool maintenance) in a small utility language that is either used by a developer to quickly script up new modules for a large software package, or taught to business users so they can adapt their software to meet their changing needs. Want to change the business logic around preferred customer treatment? It’s one very human readable line in a script, as opposed to a code change, a check-in, and a redeploy across an organization. A DSL is a software factory because it automates the processes of a specific domain, and the artifacts it creates, in this case scripts for customer treatment behaviors, are as reusable as the language definition itself. Microsoft’s Domain Specific Language Tools are a collection of visual designers that let you model a DSL visually, similar to the Visual Studio Class Designer or the LINQ to SQL mapping design surface.
Destined to ship as a feature of Visual Studio 2010 and available today as a technical preview, Mg is first class support for DSL language design. Originally intended to work with other tools for defining and parsing schema, such as XML, directly to databases, Mg is useful in its own right for declarative development of DSLs and it will become a popular tool and topic for the future of DSL modelling with Microsoft technologies.
A video detailing Mg from Paul Vick, Microsoft Architect
(http://www.informit.com/podcasts/episode.aspx?e=E8D94BA1-1B77-4F4D-99CC-BFCE8CE36489)
Factories for Architecture
While a DSL would provide a factory for a generalized language, more robust needs require generalized solutions. Several evolutions of full-scale software factories exist in the Microsoft ecosystem, here are a few established and emerging technologies that will allow you to re-brand, repackage, or reconfigure code you’ve already written, for customers you will have down the line.
Microsoft Enterprise Library ( )
Microsoft’s Patterns & Practices team developed a true software factory long ago when it introduced the Microsoft Enterprise Library. Designed for composing applications out of blocks of clearly defined enterprise features, MEL reduces the number and scale of repetitive tasks that you face every day on a new software project, such as logging, messaging, services, and to some extent, even the flow and functionality of UI. This library requires a fairly low commitment in terms of education as it builds on existing Microsoft technologies but provides a unified set of repeatable features that every enterprise project needs.
Microsoft Managed Extensibility Framework (http://www.codeplex.com/MEF)
A lighter-weight library for building composite applications is available with MEF. Borrowing the term composition from the manufacturing domain, the Managed Extensibility Framework is designed to make it easy for a developer to define application features based on components and configuration rather than reinvent the spinning wheel with one-off software implementations for products that exist in many different instances to satisfy specific customer needs. Still in development, look to MEF to be the application block strategy of the future.
Revisiting the manufacturing paradigm in software factories, the end result of any factory process is a collection of artifacts. Without artifacts, there is no output and no value to use once the smokestack clears. Arguably the lynch pin behind all software factory approaches, template generation is the artifact concept that what you define at a high level can be broken down and generated into working source code at a lower level. In fact, the very same template engine behind the DSL tools developed by Microsoft is available for you, today, with Visual Studio T4 templates. After defining a T4 template, dropping them into your Visual Solution awakens them, using meta-data to create entire frameworks, if necessary. One well-known use of T4 templates is within the SubSonic 3.0 database technology (, developed by Rob Conery of Microsoft; the entire framework for database persistence is created using T4 templates dynamically built around your database’s connection string.
Prism: Composite Application Guidance for WPF and Silverlight (http://www.codeplex.com/CompositeWPF/)
Need a software factory for differentiated, rich user experiences with the latest Microsoft stack on desktops and the web? Another software factory making a big splash now and in the next few years will undoubtedly be the evolution of Prism. The future of application interfaces involves dynamically assembling behaviors, interactions, effects, and transitions together while being able to reuse each piece in new views. Composition is the buzz word for this new breed of applications that favor declaring what the UI could do, over coding what the UI must do.
Factories for Code Quality
Five years ago, the idea that code testing tools could "predict", "discover", or "experiment" with source code to find fatal flaws and defend against future quality problems would sound like science fiction, but a new breed of heuristic software factory is emerging from Microsoft Labs in the form of intelligent tools for breaking, and mending, software source code. These will be exciting, cutting-edge projects in your near future, and they’re available for preview use right now.
PeX (http://research.microsoft.com/en-us/projects/Pex)
The goal of PeX is to automate "white box" unit testing. You already know that unit testing is a safeguard, not a guarantee, of quality programs. You try very hard to achieve 100% code "coverage", or all possible paths through an application hunting for bugs both glaring and devious. PeX promises the precision of a computer with the craftiness of a human when looking for software programming flaws. It won’t be a stand-in for writing real unit tests by hand, but will help catch what you miss.
CHESS (http://research.microsoft.com/en-us/projects/Chess)
To paraphrase the immortal Wesley in The Princess Bride, "Concurrent programming is painful, Highness. Anyone who says otherwise is selling something". Well, in this case, Microsoft Research is on track to improving detecting threading complications in an automated fashion, and you can try it out for free. CHESS looks for "Heisenbugs", a pet term for uncertain, undefinable, intermittent software defects. You know these as the accounting glitch that only happens on Tuesdays, or the dynamically loaded assembly that crashes every one thousand successful operations. CHESS attempts to bring tooling to finding, and more importantly reproducing the conditions that lead to these strange bugs, so you can squash them.
Conclusion
We’re very far from the big green button, and thankfully so, but the next five to ten years of software development will bring increasing factory-mindedness to the tasks that return the least value-to-investment ratio, accelerate the time to working software iterations, and assist human innovation with computer precision. Developers should embrace anything that allows them to move from the menial to the meaningful, and should not fear the software factory. Life is too short for one developer to deliver all of the technology that his or her world demands, but software factories in the hands of skilled developers can; factories are the past, present, and future of computing.
—–
Daniel Crenna is a Microsoft MVP and creator of TweetSharp, an open-source Twitter API development library. Daniel is focused on improving the landscape for .NET developers building social software through founding www.dimebrain.com. Daniel was a contributor to the Microsoft AJAX Control Toolkit, and is a Microsoft Certified Professional Developer.
Microsoft has done a lot with software factories over the years. The concept does seem to apply mostly to large software development efforts. I put to your readers that software factories can also efficaciously scale down to small teams too. The secret is in realizing that the most important output from a software factory isn’t software at all. http://it.toolbox.com/blogs/future-of-work/software-factories-in-the-small-32410
Very bold article, the facts are laid out so well and very convincing, I hope to be working soon on the kind of factories that you mention than in the current high methodology low business value cycle I’m in.