Thursday, January 3, 2008

Mashup vendors need to work with our frenemies or the bell will be tolling for us

I’ve had a nice long break over the holidays but now it’s time to roll up my sleeves, gird my loins and otherwise prepare to get back to work. And what better subject to start off the new year than turning enemies into friends, working with foes or partnering with competitors?

Gwyneth Dwyer has posted on this very subject. Back in the 90s we called it co-opetition. The new terms are frenemies and froes, meaning working with your competition, either on a deal-by-deal basis, or more likely, on a strategic basis as partners.

I’m sure the idea has been around since The First Floridians Met the Last Mastodons, but in the Web 1.0 dot-com era it surfaced again. The web was opening up everyone's horizons. It used to be hard for customers to find niche products, and for niche vendors to find customers. The web changed all that. With the web, little guys could not only innovate on a dime, they could get the attention of customers without going through an analyst firm. These same little guys didn’t have the pull to say ‘my way or the highway.' Back when TeamShare was a small vendor selling bug tracking software, customers used to insist we work with Rational, so of course we did. One day we'd work with them to promote our ClearCase integration, the next we'd be in fierce competition against ClearQuest.

Dwyer's post on the subject takes a collaboration spin. If you end up working with a competitor, and you likely will, be sure to set expectations, define boundaries, leave turf battles at the door, and finally, communicate, communicate, communicate. Sound advice for partner collaborations of all sorts, frenemies or no.

So what does this have to do with business mashups? I'm always asking that question. Sometimes the answer is obvious, sometimes obscure. In this case it's pretty straightforward. Co-opetition is at the very heart of mashups, whether the mashup vendors want it that way or not. Mashers don't care that some of their data comes from JackBe, some from StrikeIron hosted services, some from back-end SAP systems, all displayed on a GoogleMap and pulled together with process from Serena. Mashers care that they have accurate and timely data, widgets, and process from wherever they can get them.

Imagine trying to convince a masher that they could only use technology and content from a single vendor.

Oracle: I’m sorry, Mr. Masher, but you have to use the Oracle stack, Oracle applications and Oracle services to create your mashups.

Masher: Bite me.

Marketplaces and content aggregators like StrikeIron will have the upper-hand when it comes time for mashers to search for content. Kapow's open services will draw those same mashers. Will Kapow and StrikeIron work together or will they compete? The answer is ‘Yes.’

In a mashup world we are all connected. I’m reminded of a meditation about the island and the bell by John Donne. (Sure I’m taking it out of context, but I’m a masher as well as a blogger.) “No man is an island, entire of itself; every man is a piece of the continent, a part of the main.”

We in the mashup ISV world have to change to that mindset. We not only need to make our services open and usable by mashers, but we need to work together, defining standards to make it easy to mash content. It's not enough for us to publish our APIs. As I've discussed in a previous post, we need to work with other vendors on security standards, quality metrics and semantic vocabularies so mashers can pull content from pages painlessly. If we don’t do it, somebody else will, and we will wake up some day to find that we know exactly for whom the bell is tolling.

So JackBe, ready to let me review your software yet?

Friday, December 7, 2007

Bloomberg says mashups need governance as well as an easy development environment

Jason Bloomberg, a ZapThink analyst, is planning to hold a session at SOAWorld 2008, where he will discuss the need for mashup governance when developing Service-oriented business applications. (SOBAs) I won't be at his session, or even at the conference, but I read what looks like his abstract in the press.

Bloomberg's conclusion: mashups need governance, and here's why.

Way back in the day, when PCs first made their way into the business, we developed many and varried applications. These apps were complete one-offs and at some point they became too expensive for the business to maintain. The business dropped these apps right back in ITs lap. IT proceeded to toss many of them, replace others and rewrite still others. The apps didn't have legs for long-term usefulness. Their data sources were siloed, their interfaces inconsistent and their development environments primitive.

Now we have a new wave of business-built applications, or mashup-based SOBAs in Bloomberg's terms. If we don't want to repeat our past mistakes, we need to make sure this new crop of applications is architected for the enterprise. They will need to be loosely coupled to back-end systems, employ rich interfaces akin to those now common in the consumer world, and have a development environment that will make it easy for the original business mashers to maintain them as business requirements evolve.

To this Bloomberg adds,

"...Clearly, no business would risk allowing any of its employees to assemble and reassemble business processes willy, nilly, with no controls in place to ensure that the resulting SOBAs followed corporate policies."

The problem with this statement is that employees are assembling and reassembling SOBAs willy-nilly. The lack of controls to make sure these applications follow corporate policies is one of the very things that make fast innovation possible. The ungoverned free-for-all of mashup construction is part of the mashup DNA.

So mashers are mashing, not just in the wilds of the uncontrolled web, but also behind the firewall. We don't want to stop them, and we can't trust them to self-govern. Not because they are evil, lazy or stupid, but because they should be busy innovating, not fighting a version control system. Bloomberg seems to agree, and he goes on to say that when governance is put in place the mashup environment starts looking less like a Web 2.0 environment and more like a standard development environment.

Not necessarily. The trick is to integrate light-handed governance with a mashup construction environment that looks and feels like a MS Office application, or better yet, iTunes. Certainly not like Popfly or Eclipse. But we also want mashup-based SOBAs to have legs for the long term. So we need security. We need version control. We need change management. What we don't need is to discourage our would-be mashers by making governance obstructive. Governance should be a silent side-effect of building a mashup. A Side-effect ignored right up to the point where somebody accidentally honks up one of the production mashups.

Is this a mere dream?

Yes and no. There are some mashup construction tools on the market that incorporate version control. There are some that include security. There are some that include change management. Unfortunately, there are none yet that include all the old application development disciplines. So yes, it is a dream for the moment.

However, the fact that some mashup enablement vendors are already thinking about governance tells me that the tools will evolve, and likely very quickly. So maybe this isn't a dream after all. Maybe it's just a product roadmap issue. Unlike our first go-round with business-built applications, we have some experience with what does and doesn't work. This time around IT should be on hand to provide a safety net rather than a brick wall. This time things just might be different.

If we've learned from the past we will create an environment where business-built applications will remain with the mashers, to wax and wane and ebb and flow right along with the business.

Wednesday, December 5, 2007

Business Mashups already are process aware

Today I read a post by ebiz blogger Keith Harrison-Broninski about Mashups and process. He took a look at mashups from two perspectives represented by mashup tool vendors JackBe and NexaWeb.

(As an aside, do you think JackBe allowed him to evlauate their tool? Readers will remember that JackBe declined to let me evaluate Presto, as I've noted here and here.)

I'm in the middle of evaluating Dapper, and so far I really like what I see. I had to take time out to respond to Harrison-Broninski, however, because while I agree with most of what he says, I have to take exception to his conclusion.

Here is the money quote from his post.

...there are essentially 2 types of mashup: siloed and process-aware. For now, mashup tools are siloed. However, in due course they will become process-aware via integration with HIMS technology - and once this happens, the combination of [Human Interaction Management System] HIMS processes and mashup applications will be an extremely powerful way to leverage both Web 2.0 and your legacy infrastructure.

I'm very confused by this post because we've been saying for months that true business mashups are not only human-process aware, they are human-process centric. Mashing up the data is a great step forward. Mashing up visual and data elements at the glass is a very interesting technique, perhaps more relevant to the consumer world than the business world, but a useful technique nonetheless. However, these aren't business mashups. They are presentation and data mashups.

So what?

Hinchcliffe noted in a recent and widely cited post that there aren't yet killer mashup apps. I believe that's because we've concentrated only on putting the data and presentation components together and haven't put the results in the context of a business activity. I go into more detail in my response to Hinchcliffe, so I won't repeat myself here. The bottom line is that Harrison-Broninski has it exactly right about the need for human-centric process.

What he doesn't have right is his assertion that there are no process-aware mashup tools. He notes that BPM is a possibility, but thinks processes appropriate for BPM aren't really appropriate for mashups. Read his post for his perspective on the difference between BPM processes and HIMS processes. I'm not sure I agree with that conclusion since there are BPM vendors who have strong roots in human workflow. In fact, Forrester has an entire wave for Human-centric BPM. (I'd include a summary here, but then Forrester would insist on approving my post.)

Instead, I think the reason BPM tools haven't invaded the mashup market is because BPM practitioners haven't yet made the connection. (See my post on the subject.) Until that happens the tools won't evolve to make it easy to create mashup applications within a BPM context. I'm sure that will change, but for now read the comments from long-time BPM practitioner Sandy Kemsley.

Until the BPM tools do evolve, I'd like to point out that there are at least two process-aware mashup tools already on the market. The first is ActiveGrid, which I've already reviewed. The second is Serena Mashup Composer which has been released to the public this week. Both of these tools let users create mashup applications, and both enable mashers to define human workflow to put the mashups in the context of a business activity.

Go read Harrison-Broninski's post for an interesting discussion about the difference between BPM and human-centric applications. He makes a great case for mashups of the future incorporating HIMS. But don't think you have to wait for 'the future' for a business mashup construction environment. To steal the title of a very old movie shamelessly stolen in many, many marketing taglines, "The future is now."

Friday, November 30, 2007

What's true for blogs and wikis is true for mashups as well

In a recent post, Harvard Business School professor Andrew McAfee discussed findings published earlier in Andy Dornan's Business Week article about the perceived value of Web 2.0 technologies within enterprises. I've already commented on Dornan's article, and won't do so again, but I do want to comment on McAfee's insights relating to Business Week's findings.

Those of you who have already read McAfee's post might wonder what his article has to do with mashups. McAfee's comments relate specifically to legacy Web 2.0 (yes, my tongue is well into my cheek) capabilities such as wikis and blogs. He also extends his insights to social networking sites. (SNS) Although he doesn't specifically discuss mashups, I think what he has to say applies equally well to them. Mashups intrude on application development's turf, and app dev is one of the most isolated and siloed organization within IT. In McAfee's experience, people believe "...that corporate IT departments consciously exclude outsiders and outside influences, and are concerned primarily with expanding themselves." That's true in spades for applications development.

Here's some more of what he has to say.

Enterprise 2.0 tools have no inherent respect for organizational boundaries, hierarchies, or job titles. They facilitate self-organization and emergent rather than imposed structure...They require [grizzled R&D types] to...“practice the philosophy of making it easy to correct mistakes, rather than making it difficult to make them”...They require, in short, the re-examination and often the reversal of many longstanding assumptions and practices...

Do Mashups and mashers respect organizational boundaries, hierarchies or job titles? Nope. In fact, mashers don't even respect IP. It's the essence of a mashup that the masher pulls information from where she wants in a way she wants to create the application she wants. With in-the-cloud deployment models and screen scraping tools, she's not going to care whether IT likes what she's doing. She's not going to go through a project approval cycle. She isn't going to get permission. She's just going to do it.

Do mashups facilitate emergent rather than imposed structure? You bet. I like to say that blogging, combined with Google search, lets anyone become a pundit. You don't need to work for Gartner or Forrester, or even eWeek. Eventually the bloggers with something interesting to say will get found and heard. The same is true for mashups. With business mashup technologies putting the means of production into the hands of subject matter experts, anyone can be a masher. The good mashups will be found and used, turning mashers into valued experts. These mashers don't need to work for IBM, BEA or even your IT department.

Do mashups promote the philosophy of making it easier to correct mistakes rather than harder to make mistakes? They sure do, and this is the very issue that drives IT nuts. For years IT Ops has been focused on keeping the production environment secure, even at the expense of business agility. It's a long and difficult process getting any change into the production environment. IT isn't being obstructive for no good reason. IT has a long history of having their butts firmly kicked for allowing problems to creep into the production environment. Their quarterly bonuses are probably tied to system availability and reliability, not to innovation. App dev organizations have a similar focus. We read about business disasters caused by faulty software. Software defects, found in production, are 100x more expensive to fix than when these defects are found in the requirements or design phase. Application development organizations are told to make sure the software is bullet-proof , defect-free and completely stable before it goes out the door.

Yet with mashups we are asking the business to embrace the notion that software will almost certainly be flawed, but can always be fixed. In IT's mind this philosophy is no different than the bad-old hacker days when cowboy programmers working on the enterprise's 'big apps' could bring down an entire business unit's operations. No wonder application development organizations believe end-user created mashups are a bad idea. What app dev isn't quite yet understanding is that a mashup can be fixed in minutes or hours, whereas 'big apps' take months or perhaps years to fix. Changing the 'big app' mindset will definitely require "the re-examination and often the reversal of many longstanding assumptions and practices."

McAfee concludes that IT had better wake up and start delivering innovative solutions for the business, or they will find themselves taken out by Web 2.0. I'm not quite ready to predict that level of gloom and doom. I will certainly agree that IT needs to evolve. They aren't going to block adoption of Web 2.0 capabilities, to include mashups. What they can do, as I've discussed before, is provide a secure infrastructre and behind-the-scenes mentoring so these new technologies won't harm the business.

Thursday, November 29, 2007

Coghead lets you build simple web apps, not mashups

A colleague of mine last week suggested that I review Coghead as a mashup assembly platform. I've been spending the week with them and can conclude that they are neither a good nor a bad mashup platform. In fact, they aren't a mashup platform at all.

Coghead allow its users to construct simple web applications based on a back-end database. Both the application and the database are hosted by Coghead itself. They have an on-demand pricing model that is quite reasonable, with the price of a subscription increasing with the number of database records. For simple web applications in the long tail of applications development, web applications that don’t need to mash either data or visual elements, Coghead could be a very reasonable solution.

On the surface, Coghead offers capabilities similar to ActiveGrid, but only on the surface. While ActiveGrid also lets users create a web application driven from a back-end database, unlike Coghead, ActiveGrid allow users to create mashups both by exposing services and by calling out to them.

Let's start with what Coghead can do. I found it very easy to construct web forms, create search result lists, define business logic and otherwise manipulate database records. I had a fairly sophisticated app up and running within a couple of hours of signing in. The documentation is excellent, and the tutorials instructive. You really don't need to be technical to create a Coghead application. As with ActiveGrid, it's helpful to understand how relational databases work, but not absolutely necessary as long as your application is simple and doesn't require complex relationships between tables.

There were a few things I didn't Like about Coghead. It was easy to get the web forms set up, but it wasn't easy to get them just right. I'd say that most form designers will spend 90% of their time tweaking the last 10% of the form. Form tweaking and other minor problems were just annoying. The most pressing problem with Coghead was its performance. I've got an excellent connection here at the office, and I had only a small app on Coghead, yet creating, updating, and deleting records was slow, slow slow.

All this is in the noise, however. Because Coghead is not, repeat not, a mashup construction platform, business or otherwise. The only content they aggregate, and by aggregate they actually mean replicate, is by way of a csv file. That's a good choice for a database integration conduit, but it isn't a mashup. It's replication.

You might be wondering about coglets. Surely, you might think, coglets enable mashup construction. At least you might if you were on Coghead’s site and read about coglets. And what about mashouts. Don’t they count?

Sorry. Coglets are by no means mashup fodder. Coglets merely create individual web pages to display fragments of your application's data. I can create a single page, accessible through a URL, to display a single record or to display a view. I can then embed the URL into another web page. That isn’t a mashup. That’s at best a portal and at worst just a custom web page with a lot of iframes.

Mashouts go in the other direction. Mashouts allow you to use data from the current form as variables in HTTP get requests. These requests get triggered from the web form's context menu. (Web forms with context menus...you gotta love AJAX.) The target for the request is another browser altogether. That's not even a portal. It's a link to another application.

Let’s look at an example.

Let's say I want to have an application that consolidates all the tasks I have assigned to me. Let's also say that I get tasks from a project management system that publishes a project plan to the web, my Outlook TODO list, and from an issue tracking system used by R&D to assign maintenance tasks. There are at least four ways I could interact with these different task lists. First, I could jump from app to app to app. That’s not a mashup. It isn’t even an integration, but it’s the sort of capability provided by mashouts. You can jump from a Coghead app directly to another app in another browser. It’s our old siloed applications we know and love so well.

Second, I could copy all the data from the three apps either into a single database, or designate one of the applications as the master TODO list. Then I could access all my tasks from a single repository, hopefully with two-way replication so changed TODO data can be fed back into the original app. That isn’t a mashup, that’s a replication integration, one that has all the inherent problems associated with replicated data. These sorts of integrations are enabled through Coghead’s csv import/export feature.

Third, I could create a web page where I pull the content from the three apps into the single page. In one frame I’d see the project tasks, in another my Outlook tasks, in a third my issue management tasks. This is what you get with coglets. That’s not a mashup, that’s a portal.

Finally, I could construct an application that aggregates the information from all three applications into a single unified GUI. I could reference the data but not copy it, so all three applications retain their single copy of the TODO data. I could create a unified GUI so that, unless I drilled down on a specific task, I wouldn’t even realize the data were federated across three repositories. Were I to take this approach, I’d have a mashup. Unfortunately, there are no capabilities within Coghead applications that support this approach. At least not yet. Their documentation says they will have REST services available soon, meaning that while Coghead could participate in a mashup, they still wouldn't be a mashup platform.

Bottom line: Coghead can certainly help non-technical users develop web-based applications in the long tail of applications development. For the 80% solution, Coghead makes it easy to create, deploy and even pay for those pesky applications for which there are no IT resources. It won’t be a mashup, but it could be enough for you.

Friday, November 16, 2007

Have fun with Mashup Composer, JackBe

I'm not one to hold a grudge. Hey, when JackBe said I couldn't download and try their software, I understood. I didn't like it, but I understood.

But let's play fair here.

Someone from JackBe has downloaded Serena Mashup Composer. If I were Fake Steve I might have a field day talking about how, now that they've had a chance to dig into Composer, JackBe must be crying like a girl. If I were Fake Steve I would be suggesting that the JackBe employee is actually getting ready for a job interview with Serena, and that's why he/she downloaded the software. If I were Fake Steve I might even suggest that this is JackBe's way of writing requirements for their next release. "Hey R&D, forget the requirements doc. Just make Presto do what Serena Mashups can do."

I'm not Fake Steve, so I wouldn't say any of these things.

I will say this, however. Presto and Serena Mashups solve different problems. Do you need your data mashed? Presto likely is a good choice. I wouldn't know for sure because I haven't been able to test it, but so I'm told and so I suspect based on the Presto marketing materials. Do you need this mashed data within the context of a business activity? Well, Serena Mashups may be the right way to make use of this mashed data.

The point is that JackBe and Serena should be partners, not competitors.

So JackBe, we've shown you ours. Will you show us yours?

Thursday, November 15, 2007

Who will be the business mashers?

One of the top two leading industry analyst firms has been talking for several months about how software applications are brittle, difficult to maintain and aren't architected to keep up with the fast-paced demands of today's business environment. That's not new news. It isn't even old news. What is news is that now we have analysts saying out loud what we've all known for quite a while: custom software development is broken.

I can't tell you which analyst firm has been talking about this issue because they will insist that I run this post through their vendor relations department. I don't really want to have my blog sanitized, so instead I'll just call them the Unknown Analyst. (UA)

UA has suggested some solutions to this problem. They predict more development teams will adopt agile software development practices as a way to make app dev more responsive. That will help, but agile methods will only take us so far. Even with agile development practices, multisourcing, offshoring and otherwise scaling up capacity, app dev resources are scarce and expensive. IT has been scaled to work on big-ticket and highly complex software projects, not the myriad interesting and potentially business transforming smaller applications that are backing up in app dev's queue.

Regardless of the development methodology employed by the app dev team and how many new developers they hire, IT simply isn't going to be able to satisfy the demands of the business for new and innovative applications. Especially when, as UA says, they need to change at the speed of business: constantly.

UA suggests that this gap will be filled by business analysts. The role of the business analyst will need to change to become as much someone who assembles applications as someone who simply defines them on behalf of the business. In our world this means the business analyst will turn into the business masher.

So where do business analysts come from, and will they have the necessary skills to become business mashers? What skills will the business masher need?

Good questions, both. UA says that business analysts are grown, usually from an IT staffer who is interested in business or more likely from a business expert or project manager of some sort who becomes interested in IT. Today, the most important skills business analyst posses have to do with effective communications. Can the BA communicate effectively both with the business and with IT? Does the BA know how to write, analyze a problem and mediate disputes between what can be two opposing forces?

Technical skills such as process modeling, usability engineering, and data modeling are less important. Likely because those skills can be learned, but knowing how to communicate effectively both to the business and IT isn't something that can be picked up out of an instruction manual.

But are the soft skills currently important to BAs going to be adequate when these BAs have to start turning out applications?

It's a bit of a rhetorical question because I think I have an answer already. No, these skills won't be enough. I've worked with a number of BPM tools in the past. I've also worked with a number of Web 2.0 application assembly platforms, as you will know if you've read my reviews of Intel's MashMaker, Kapow's RoboMaker, Alpha Works' QEDWiki, WaveMaker's ActiveGrid and even Serena Mashup Composer. These tools all either don't actually build applications, MashMaker, Mashup Editor, QEDWiki and RoboMaker fall into this category, or require some strong IT know-how when it comes to the deep-down nuts and bolts of web service calls.

Yes the tools will get better, but for now I can't agree with UA's assessment. Today's BAs aren't going to become tomorrow's business mashers. Most of them lack the necessary technical skills.

I want to digress a bit. A little over ten years ago there were a lot of complaints in IEEE circles because engineers would found companies only to have them run, owned and grown by MBAs. (Microsoft excepted, of course.) Many IT departments were run by MBAs working for the CFO rather than by technical people with knowledge of technical issues. We nerds put ourselves in technical silos, and then complained like crazy about the lack of technical know-how among our top executives.

IEEE put out a call and said, basically, "Quit complaining and go get MBAs." In other words, IEEE decided that if we nerds wanted leaders with strong technical knowledge, we'd have to grow these leaders from our own ranks. It was a great message and got a number of us to start worrying about things like entrepreneurialism, sales, marketing and finance in addition to whether Windows or Unix would end up ruling the world.

I think we have a similar chance to expand our horizons today. The BA role is changing. BAs today need soft skills, but the BAs of tomorrow will need more technical know-how. Most of these BAs come from the business side of the house and have learned about IT on the job. That means when these newly minted mashers start to create mashup-based business applications they will make mistakes. They won't necessarily think about version control. They won't know about consumer-side SOA Governance. They won't understand how to use XPath, call web services or in many cases how to set up a feedback system to manage changes to their applications.

Remember the last couple of times business built their own applications? Remember the Access databases from the 80s? Remember the web applications in the 90s? The business needed agility and they got it, but they also needed engineering discipline, and this they didn't get. In the end, when it became clear that these applications were business critical, they ended up back in IT's lap, and both IT and business lost. IT because they became villains. (Business: remember when we could make changes whenever we needed? Now everything takes forever.) Business because they lost their agility. (IT: those business guys don't understand that we have 400 things on our TODO list, and the capacity to do five of them.)

We don't want to repeat the mistakes of the past if we can help it. If we technical people want to keep control over our businesses, we have to learn about business. If we don't want to repeat the problems we had with business built applications in the 80s and 90s, we need to step into BA roles now so we can bring our hard-learned disciplines to bear on this new application construction paradigm.

Calling all IT application developers! Why not take the plunge and become a BA? Walk away from Eclipse and give up Java. (Although you will probably still need javascript.) Learn what makes your business tick. Read "Crossing the Chasm." Listen to a marketing podcast. Change those sneakers for loafers, throw a sports coat over that t-shirt and go apply for the BA job in sales your HR department just posted.

This time around, let's do it the right way.