Can I take back what I said about BPM and mashups?

Back what seams a very long time ago, but was actually only October last year, I wrote a post suggesting that BPM was another form of business mashup. Like-minded blogger Sandy Kemsley agreed, and bemoaned the lack of mashup understanding in the BPM community.

I've kept an eye on the BPM community looking for activity around mashups. I've seen a few comments around the edges, but nothing I would call a trend.

I was confused. Presentation mashups and BPM may have little in common, although I would suggest that Tibco, with their focus on RIA composite applications, have been playing in the presentation mashup space for a while. (Others will likely disagree, and we can have a discussion.) However, once you get 'out of the map' and start considering mashups from a business or enterprise angle, the overlap becomes pronounced.

(Note: my good friend Summer Ficarrotta coined the term 'out of the map' months ago to help people understand that Google Maps mashups at the glass weren't the only mashups on the block.)

I think I fell into the trap of thinking that because two things look the same and act the same, they should be the same. (Do you remember Papa Bear in The Big Honey Hunt?) I have a recent article by TechTarget writer Rich Seeley to thank for getting my head out of my trap. Through his insights about BPM and SOA roles and responsibilities, I now understand just how different BPM and mashups are.

In his article about the business and IT roles within SOA and BPM, he lists eight different roles involved in creating a BPM/SOA application, four each in business and IT.

Business Roles

• Business Leader: Responsible for overall business performance, compliance and governance.
• Business Professional: Manages business performance and decides on strategic and tactical needs for a specific area of responsibility.
• Business Analyst: Interprets business professional and business leader requests and documents them into process models.
• Process Analyst: Specialized business analyst who concentrates on the simulation and analysis of processes in their business environments and their interactions.

IT Roles

• IT Leader: A Business Leader responsible for delivering technology solutions that enable the business.
• IT Analyst: Interprets business analyst inputs/requirements in the context of IT capabilities, works with team on IT-based business process improvement.
• IT Architect: Defines basic operational imperatives in the provisioning of IT services with a focus on resiliency, reuse and adaptability.
• IT Developer: Follows IT architectural principles to create "building blocks" for the construction of applications.

Whew! Imagine putting a mashup together where you needed four different roles to put an idea together before tossing it over the wall to IT where four more roles did the implementation. I'm not saying this is too much for BPM. High-value and highly-complex systems need governance and discipline during their ideation and construction, regardless of whether they are implemented as a custom application built from scratch by App Dev, or as a business process built atop a BPMS.

What I'm saying is this is too much for mashups.

The premise of Serena's paper on the long tail of applications development is that there are many applications that IT never implements because they aren't high value or complex enough to merit IT involvement. This is 'The Long Tail' of Applications Development. At the time we wrote the paper I thought that BPM could be one of the answers. After all, BPM was all about empowering the business to define and build applications.

And that's where I made my bloomer. Mashups need to be easy to build, easy to deploy and easy to maintain. Mashups need some governance, as I've written about here, here, and here. (Yes, this is a subject I care about.) Just not as much as an expensive and complex App Dev initiative. They also need some lifecycle management. Again just not as much as a typical App Dev initiative.

Now I'll go on record as saying they don't need as much governance and lifecycle management as Big BPM either.

Applications Development has a long tail, a tail that can be serviced, in part, through the use of mashups. Contrary to what I've said before, BPM also has a long tail. A tail that can be serviced, in part, through the use of mashups.

What I won't say any more is that BPM is a business mashup platform. It may look the same. It may act the same, but it isn't the same at all.

Linthicum on Mashup Governance...over the top?

I usually agree with David Linthicum of ZapThink when he writes about SOA. He understands the problems faced by organizations moving to SOA-based applications development. He gets that organizations need to put governance in place to avoid SOA turning into merely A Bunch of Services. (ABOS) I'm a fan.

But I think his recent article on mashup governance in the SOA Institute newsletter was off the mark. Not completely, but I think he's making the mistake that many of us make in thinking that mashups need the same level of governance, the same rigor, as composite applications developed within IT.

Here's my take: Not only don't they need the same level of governance, they won't get it.

So let's take a look at what Linthicum has to say. He's right when he says that mashups blur the line between the web-at-large and the behind-the-firewall enterprise. In his words, "...We could find that we soon live in a world where it's difficult to determine where the enterprise stops and the web begins. Now that's scary and exciting at the same time."

I completely agree.

Next he talks about how organizations need to prepare for mashups. Linthicum says that organizations need to design their SOA with mashups in mind, and on this point we also agree. Although SOA has come to be synonymous with the WS-* standards, it need not be. In fact, early SOAs were completely custom, and even today there are a number of competing SOA standards. Yes SOAs need to be designed around standards, but not just any standards. They need standards that make the services and widgets usable by many different SOA consumers.

Let's consider an example. I happen to know of a company that created SOAP-based services to expose some of their products capabilities. However, these services were built in the image of their old API. When it came time to use these services in a BPEL orchestration, never mind within a mashup, it was extremely difficult. These services weren't built with their end use in mind. More services by techies and for techies.

When it comes time to adopt your SOA standards, keep in mind that you will have many different SOA consumers. Not just by coders in VS .NET or Eclipse, but portals, BPM, and even mashups. And when it comes time to implement your SOA, remember that you may have multiple consumers about which you know little to nothing. So architect for flexibility, scalability and security.

Linthicum goes on to discuss mashup governance in some detail, and I'm afraid here is where he and I disagree. Readers will recall I've written about mashup governance before. I believe mashup governance is necessary. Mashers need version control. Mashers need some form of feedback loop to manage defects and enhancements. Mashers need to practice safe scraping and need to be aware of the provenance of the data they use in their mashups.

However, Linthicum believes mashups should be governed with the same rigor as Big Applications developed by R&D.

"So, what do you need to do to prepare for mashups? It’s a matter of addressing the following areas: Requirements, Design, Governance, Security, Deployment, and Testing. In essence, these are core architectural activities that are required to get you to the Promised Land of mashups on top of your existing activities when you create a SOA."

Nice in theory, but impractical in practice. Mashups are to be built by subject-matter experts, not IT professionals. Think Ultra-Agile rather than Waterfall when putting together a framework for mashup governance.

Requirements? How about "build it," "try it," and, "change it." In Big Applications this may be a license for creep. The on-ramp to disaster. But for mashups, inherently smaller and more agile applications, formal Requirements Management is nothing but unnecessary overhead.

Design? Those of us with Big Applications pedigrees, myself included, will cringe, but a formal design process for mashups seems like overkill. If it works, it's probably good enough. If it stops working, then fix it. Will some mashups be inefficient? Perhaps some mashups will be slow. So what? If the mashup is going to be mission critical, if it will service a large number of transactions, then it will likely be developed by IT, not as a mashup. If it's being assembled by a subject-matter expert, it's likely that inefficiencies won't be a problem. When you bet, figure pot odds.

Governance I've already talked about, as well as security. While Linthicum and I may disagree about governance, we completely agree about security. If an organization is going to over-govern any aspect of mashups, my choice would be security. Unlike SOA's notorious performance issues, security breaches have the potential to materially harm organizations. Jail time? Wall Street disfavor? Stockholder uproar? All these have resulted from IT security problems. 'Nuf said.

Deployment is an interesting issue because it should be easy, but can be difficult. After all, what's the harm in allowing a masher to put a mashup into production? A mashup can't hurt the operational environment, right?

Clearly some governance is necessary. I suggest IT provide mashers with a secure and stable deployment platform, distinct from any mission-critical applications, and then leave mashup deployment to the mashers. IT should not attempt to require the same deployment gates for mashups that they do for SAP upgrades, changes to the financial system, or updates to eCommerce. Once the secure and stable environment is in place, leave mashers to mash.

Testing, testing, testing. I think testing falls into the same category as requirements. Should mashers test all possible configurations, scenarios and error conditions? Sure. Will they? Hardly. Putting gates in place to force mashers to test will only put mashers in an awkward position. Should they ignore the testing gates, or falsify the documentation? Darwin and sheer embarrassment will take care of quality control in the long run. That may not be optimal, but quality control will be more effective if it is instituted as a bottom-up rather than a top-down requirement.

My take-away is that mashers need to develop their own governance practices. Trying to force-feed Big Application lessons to mashers will only result in resentment and eye-rolling. Just as with Agile teams, let mashers define their own rules, and let them modify these rules as they learn, discover and create.

Hinchcliffe and Chappell Predictions for 2008 Part 2 - Mashup Governance

Welcome to Part 2 of my series on 2008 mashup predictions made by Dave Chappell and Dion Hinchcliffe. In Part 1 I discussed how both are predicting the end of Big SOA. If you are interested, read the original posts and circle back to my comments. If you aren’t interested in the death of Big SOA, but are interested in mashup governance, read on.

Both Chappell and Hinchcliffe note that organizations will start pushing for mashup governance in 2008. We all cringe at the G-word, linking it with over-governance as mandated by legalistic process Nazis. Too much governance and you bring innovation to a standstill. Not enough and you have chaos. Light-handed governance can be a great help, insuring that security is in place, that there are recovery procedures, that everyone knows the lines of responsibility.

I’ve written before about how IT and business need to work together to make sure mashers have a secure and reliable mashup platform. Now let’s take a look at what Chappell and Hinchcliffe have to say along the same lines.

Chappell believes it's only a matter of time until a business-critical mashup fails. Some feed you didn’t even know you used will go away, some 3rd party service will change endpoints, somebody will change the mashup on Friday evening and then leave to hike the Inca Trail for two weeks. What happens? As Chappell says, “…many will be scrambling to figure out who is supposed to fix it, and who to point the finger at.”

He goes on to say that by the end of the year, driven by these mashup failures, mashers and IT alike will understand that mashup governance has to be put in place. Does it need to be as rigorous as the rules and processes surrounding making changes to the SAP ERP Financial system? Probably not. But neither can it be ignored completely.

Hinchcliffe makes similar predictions, but believes the demand for governance won’t happen because of a catastrophe, but because IT will start voicing loud concerns. "...it will likely be widespread grassroots behavior outside of IT that will drive the need for these solutions within IT departments." Security will become an issue as mashers pull data from many sources behind the firewall, from SaaS applications, and even from the web-at-large leveraging Web-oriented Architecture. (WOA) Mashers should ask questions about these data. Are the data reliable? Are the data available? Are the data to be used appropriate for the audience? After all, you don't want sensitive financial data leaked to the wild by a clueless masher.

While mashers should ask these quesions, it is likely that IT will be the one doing the asking. IT will make noises to govern access to mashable data within the enterprise. Additionally, IT will demand a SOA governance infrastructure that will provide security, policy enforcement and scalability across both on premises and SaaS mashup platforms and data sources.

Unfortunately, I’m just not sure whether any SOA governance solution can put the necessary controls in place without unduly interfering with masher innovation. Let’s take it as given that in a SOA world IT will be able to control access to services. Will these same controls work in a WOA world? I doubt it, at least not without cutting off access to uncontrolled services and data. Frankly, I doubt if the business would stand for these tight restrictions.

As I've asked before, what's to be done?

While IT may not be able to enforce governance outside the firewall and known SaaS applications, it can act as an advisor, explaining to mashers why reliable data sources are important. Why it might not be a good idea to use the Unreliable News RSS feed as input to a mashup for the CEO. Why scraping content from a random web site may land him in trouble. In the end the masher will mash, but IT can at least help the masher understand the risks.

Besides the SOA Governance infrastructure Hinchcliffe believes will be put in place by IT, I believe there are some actions that IT and the business can take together to implement safe mashing in the enterprise.

• Agree up-front who owns the mashup and who is responsible for maintenance, updates and enhancement requests. Who is backup in case the responsible masher goes on holiday? The lines of responsibility should be documented before you have to send a helicopter down to the Andes to airlift your masher back to NY so he can fix the mashup.


• Define the deployment and rollback plan and rules of engagement. What is IT's involvement in a mashup deployment? What is the masher's involvement? What is the rollback plan in case a deployment fails?


• Give the masher access to a version control system. Mashers will be generating code, XML, a model, or some other artifact that defines the mashup. That code, model, etc. has to be saved so when the production copy needs to be modified or recovered, the masher doesn’t have to start from scratch. Hint to mashers: a copy sitting on your machine isn’t reliable version control.


• Work with IT to make sure your mashup platform is scalable, reliable and secure. IT should put a registry and a repository in place so mashers know what services are available, the SLAs for the services, and any security, scalability and reliability concerns for their use.


• Require the masher document the data sources before allowing mashups into production. I know, documentation sucks, but so does getting pulled back from your vacation to explain why you've been getting market sizing information from astrology.com

These recommendations won't cover all the bases, but they will provide a framework for governance, even in a world where there is, in Hinchcliffe's words, "...still too much steering committee and not enough operating governance infrastructure." And most importantly, they will help mashers understand that when they mashup a business application, it isn't just a cool science project. It's serious business.