Progress / Actional/SOA Management Products/Product Literature
A Darwinian Approach to Software Reuse
This article by Dan Foody appeared in SD Times's April 1, 2004 issue in the Editorials and Opinions section. Daniel M. Foody is the CTO of Actional, a provider of Web services management software.
For as long as software engineering has been practiced, there's been a vision that achieving the Holy Grail of software reuse was "just around the corner." Every new technology generation was supposed to have solved this problem once and for all. Functional programming, object-oriented programming, components, service-oriented architectures—the list goes on. But, if we were to survey software engineers today versus 20 years ago about how close we are to this elusive goal, we'd probably find that we aren’t really any closer today than we were back then. There’s definitely been a lot of credibility lost in our industry because of this. The quickest way to get yourself booted out of any venture capitalist's office is to say you've got an idea for a product that will solve software reusability once and for all.
There's no doubt, however, that each successive technology generation has contributed important techniques toward reusability. Functional programming ushered in encapsulation. Object-oriented programming introduced polymorphism. Components taught us that a clean separation of interface from implementation is important. And, most recently, service-oriented architectures have introduced loose coupling, allowing the pathways that connect services to be remapped without changing the services themselves—something that’s not been possible in previous technology generations.
In addition, with the introduction of modeling languages such as UML, there's now a much stronger push to think more upfront about how an application is designed. Measure twice, cut once.
So with all the progress we've made, why don't we feel any closer to achieving broad-scale reusability? Why can't I take any arbitrary service and reuse it wherever I see fit? Why? Well, unless I was lucky, what I'd probably hear would be: "It wasn't designed to be used like that," "It will take some tweaking to make it do what you want," or any number of other similar statements. On the other hand, on seeing your existing service, I might say: "I can't understand this, it's too complex," "It doesn't exactly do what I want," "It's overkill for what I need to do," or any number of other excuses. You've probably seen something like this replayed a thousand times.
Why do we continually run into these problems? Because we, as an industry, have focused primarily on ways to design for reuse upfront. Unfortunately, the challenge that you as a service developer will always face is that you can't easily anticipate all that consumers might want to do with a service. And even if you could, you couldn’t afford the time or complexity to build all of this upfront. While you can get lucky in well-controlled cases, no matter how hard you try, this barrier can never truly be broken: You just can’t see into the future.
While we may not be able to see into the future, hindsight is 20/20. So, maybe, just maybe, we’re approaching reuse backward. I certainly believe that proper design and a well-architected infrastructure are important. That said, instead of thinking about reusability as a problem to consider before we build software, what if we thought of it as a problem to consider after we build software?
To illustrate this, imagine that 10 different project teams in a company need a bond yield calculation service. Normally, the first thought through a software architect's head would be: "We need to form a cross-functional team to gather requirements and build a shared bond yield calculation service." This would work well, provided everyone knows their real requirements (not their feature-laden wish list) upfront and that all of these 10 different project teams are actually looking for a bond yield calculation service at the same time. Good luck!
Now suppose, instead, we allowed the 10 different project teams to build their own bond yield calculation service to meet their specific needs within their budgets and deadlines. Each team will certainly make different trade-offs, design choices and functionality choices.
After some time period, once all the teams have had the opportunity to understand how well their creation fits their in-production needs, the consolidation phase of the process begins: Going forward, the company will have to choose the top two bond yield calculation services out of the 10 to be used throughout the company. The teams need to choose the two that will remain, then consolidate load to those two that takes advantage of the loose coupling in an service-oriented architecture. The other eight are decommissioned. Survival of the fittest.
Of course, moving forward, project teams would need to use one of the chosen bond yield calculation services rather than build their own. This notion takes advantage of the fact that, while you might not know the right set of trade-offs upfront, after the application is created, you have the information to effectively choose from among multiple options for the best solution for addressing the overall problem.
Consolidation and decommissioning in order to get rid of redundancy in an IT infrastructure is nothing new. This is one of the most important cost-control measures driving companies toward service-oriented architectures. Consolidation, however, has never really been part of an overall software development life cycle—it has mostly been reactionary as organizations are forced to reduce costs.
In order to move from reactionary-driven consolidation into efficient and cost-effective reuse, a lot of changes need to take place. If this approach sounds like it makes sense, we as an industry should formalize it to drive success. What overall process is needed to quickly and cost effectively orchestrate the path toward the reduced set of reusable services? As a part of this, there are some challenging questions. How many redundant versions are too many? How many versions should you narrow it down to? How long should the overall process take? For the process to be effective, these questions would need to be answered upfront so that expectations can be set appropriately.
Beyond the specifics of any process, what types of organizational incentives would make sense to assure success? General Motors, for example, gives bonuses to IT staff for each system they are able to decommission. Combine this approach, perhaps, with a bonus (or additional budget) given to service owners for each consumer that chooses to use their service. Incentives have to be in place to make the teams that develop the best services want to have others consume their services. They can't be forced to support 100 consumers on the same budget with which they previously had to support only one consumer.
In many ways, a Darwinian approach to software reuse needs to be a more well-defined, controlled and measured version of what typically happens in today’s haphazard and unpredictable approach. Think about how this might apply in your organization. Is a Darwinian approach a meaningful model for reuse? There's certainly a place for it.
