Friday, November 30, 2007

What's true for blogs and wikis is true for mashups as well

In a recent post, Harvard Business School professor Andrew McAfee discussed findings published earlier in Andy Dornan's Business Week article about the perceived value of Web 2.0 technologies within enterprises. I've already commented on Dornan's article, and won't do so again, but I do want to comment on McAfee's insights relating to Business Week's findings.

Those of you who have already read McAfee's post might wonder what his article has to do with mashups. McAfee's comments relate specifically to legacy Web 2.0 (yes, my tongue is well into my cheek) capabilities such as wikis and blogs. He also extends his insights to social networking sites. (SNS) Although he doesn't specifically discuss mashups, I think what he has to say applies equally well to them. Mashups intrude on application development's turf, and app dev is one of the most isolated and siloed organization within IT. In McAfee's experience, people believe "...that corporate IT departments consciously exclude outsiders and outside influences, and are concerned primarily with expanding themselves." That's true in spades for applications development.

Here's some more of what he has to say.

Enterprise 2.0 tools have no inherent respect for organizational boundaries, hierarchies, or job titles. They facilitate self-organization and emergent rather than imposed structure...They require [grizzled R&D types] to...“practice the philosophy of making it easy to correct mistakes, rather than making it difficult to make them”...They require, in short, the re-examination and often the reversal of many longstanding assumptions and practices...

Do Mashups and mashers respect organizational boundaries, hierarchies or job titles? Nope. In fact, mashers don't even respect IP. It's the essence of a mashup that the masher pulls information from where she wants in a way she wants to create the application she wants. With in-the-cloud deployment models and screen scraping tools, she's not going to care whether IT likes what she's doing. She's not going to go through a project approval cycle. She isn't going to get permission. She's just going to do it.

Do mashups facilitate emergent rather than imposed structure? You bet. I like to say that blogging, combined with Google search, lets anyone become a pundit. You don't need to work for Gartner or Forrester, or even eWeek. Eventually the bloggers with something interesting to say will get found and heard. The same is true for mashups. With business mashup technologies putting the means of production into the hands of subject matter experts, anyone can be a masher. The good mashups will be found and used, turning mashers into valued experts. These mashers don't need to work for IBM, BEA or even your IT department.

Do mashups promote the philosophy of making it easier to correct mistakes rather than harder to make mistakes? They sure do, and this is the very issue that drives IT nuts. For years IT Ops has been focused on keeping the production environment secure, even at the expense of business agility. It's a long and difficult process getting any change into the production environment. IT isn't being obstructive for no good reason. IT has a long history of having their butts firmly kicked for allowing problems to creep into the production environment. Their quarterly bonuses are probably tied to system availability and reliability, not to innovation. App dev organizations have a similar focus. We read about business disasters caused by faulty software. Software defects, found in production, are 100x more expensive to fix than when these defects are found in the requirements or design phase. Application development organizations are told to make sure the software is bullet-proof , defect-free and completely stable before it goes out the door.

Yet with mashups we are asking the business to embrace the notion that software will almost certainly be flawed, but can always be fixed. In IT's mind this philosophy is no different than the bad-old hacker days when cowboy programmers working on the enterprise's 'big apps' could bring down an entire business unit's operations. No wonder application development organizations believe end-user created mashups are a bad idea. What app dev isn't quite yet understanding is that a mashup can be fixed in minutes or hours, whereas 'big apps' take months or perhaps years to fix. Changing the 'big app' mindset will definitely require "the re-examination and often the reversal of many longstanding assumptions and practices."

McAfee concludes that IT had better wake up and start delivering innovative solutions for the business, or they will find themselves taken out by Web 2.0. I'm not quite ready to predict that level of gloom and doom. I will certainly agree that IT needs to evolve. They aren't going to block adoption of Web 2.0 capabilities, to include mashups. What they can do, as I've discussed before, is provide a secure infrastructre and behind-the-scenes mentoring so these new technologies won't harm the business.

Thursday, November 29, 2007

Coghead lets you build simple web apps, not mashups

A colleague of mine last week suggested that I review Coghead as a mashup assembly platform. I've been spending the week with them and can conclude that they are neither a good nor a bad mashup platform. In fact, they aren't a mashup platform at all.

Coghead allow its users to construct simple web applications based on a back-end database. Both the application and the database are hosted by Coghead itself. They have an on-demand pricing model that is quite reasonable, with the price of a subscription increasing with the number of database records. For simple web applications in the long tail of applications development, web applications that don’t need to mash either data or visual elements, Coghead could be a very reasonable solution.

On the surface, Coghead offers capabilities similar to ActiveGrid, but only on the surface. While ActiveGrid also lets users create a web application driven from a back-end database, unlike Coghead, ActiveGrid allow users to create mashups both by exposing services and by calling out to them.

Let's start with what Coghead can do. I found it very easy to construct web forms, create search result lists, define business logic and otherwise manipulate database records. I had a fairly sophisticated app up and running within a couple of hours of signing in. The documentation is excellent, and the tutorials instructive. You really don't need to be technical to create a Coghead application. As with ActiveGrid, it's helpful to understand how relational databases work, but not absolutely necessary as long as your application is simple and doesn't require complex relationships between tables.

There were a few things I didn't Like about Coghead. It was easy to get the web forms set up, but it wasn't easy to get them just right. I'd say that most form designers will spend 90% of their time tweaking the last 10% of the form. Form tweaking and other minor problems were just annoying. The most pressing problem with Coghead was its performance. I've got an excellent connection here at the office, and I had only a small app on Coghead, yet creating, updating, and deleting records was slow, slow slow.

All this is in the noise, however. Because Coghead is not, repeat not, a mashup construction platform, business or otherwise. The only content they aggregate, and by aggregate they actually mean replicate, is by way of a csv file. That's a good choice for a database integration conduit, but it isn't a mashup. It's replication.

You might be wondering about coglets. Surely, you might think, coglets enable mashup construction. At least you might if you were on Coghead’s site and read about coglets. And what about mashouts. Don’t they count?

Sorry. Coglets are by no means mashup fodder. Coglets merely create individual web pages to display fragments of your application's data. I can create a single page, accessible through a URL, to display a single record or to display a view. I can then embed the URL into another web page. That isn’t a mashup. That’s at best a portal and at worst just a custom web page with a lot of iframes.

Mashouts go in the other direction. Mashouts allow you to use data from the current form as variables in HTTP get requests. These requests get triggered from the web form's context menu. (Web forms with context menus...you gotta love AJAX.) The target for the request is another browser altogether. That's not even a portal. It's a link to another application.

Let’s look at an example.

Let's say I want to have an application that consolidates all the tasks I have assigned to me. Let's also say that I get tasks from a project management system that publishes a project plan to the web, my Outlook TODO list, and from an issue tracking system used by R&D to assign maintenance tasks. There are at least four ways I could interact with these different task lists. First, I could jump from app to app to app. That’s not a mashup. It isn’t even an integration, but it’s the sort of capability provided by mashouts. You can jump from a Coghead app directly to another app in another browser. It’s our old siloed applications we know and love so well.

Second, I could copy all the data from the three apps either into a single database, or designate one of the applications as the master TODO list. Then I could access all my tasks from a single repository, hopefully with two-way replication so changed TODO data can be fed back into the original app. That isn’t a mashup, that’s a replication integration, one that has all the inherent problems associated with replicated data. These sorts of integrations are enabled through Coghead’s csv import/export feature.

Third, I could create a web page where I pull the content from the three apps into the single page. In one frame I’d see the project tasks, in another my Outlook tasks, in a third my issue management tasks. This is what you get with coglets. That’s not a mashup, that’s a portal.

Finally, I could construct an application that aggregates the information from all three applications into a single unified GUI. I could reference the data but not copy it, so all three applications retain their single copy of the TODO data. I could create a unified GUI so that, unless I drilled down on a specific task, I wouldn’t even realize the data were federated across three repositories. Were I to take this approach, I’d have a mashup. Unfortunately, there are no capabilities within Coghead applications that support this approach. At least not yet. Their documentation says they will have REST services available soon, meaning that while Coghead could participate in a mashup, they still wouldn't be a mashup platform.

Bottom line: Coghead can certainly help non-technical users develop web-based applications in the long tail of applications development. For the 80% solution, Coghead makes it easy to create, deploy and even pay for those pesky applications for which there are no IT resources. It won’t be a mashup, but it could be enough for you.

Friday, November 16, 2007

Have fun with Mashup Composer, JackBe

I'm not one to hold a grudge. Hey, when JackBe said I couldn't download and try their software, I understood. I didn't like it, but I understood.

But let's play fair here.

Someone from JackBe has downloaded Serena Mashup Composer. If I were Fake Steve I might have a field day talking about how, now that they've had a chance to dig into Composer, JackBe must be crying like a girl. If I were Fake Steve I would be suggesting that the JackBe employee is actually getting ready for a job interview with Serena, and that's why he/she downloaded the software. If I were Fake Steve I might even suggest that this is JackBe's way of writing requirements for their next release. "Hey R&D, forget the requirements doc. Just make Presto do what Serena Mashups can do."

I'm not Fake Steve, so I wouldn't say any of these things.

I will say this, however. Presto and Serena Mashups solve different problems. Do you need your data mashed? Presto likely is a good choice. I wouldn't know for sure because I haven't been able to test it, but so I'm told and so I suspect based on the Presto marketing materials. Do you need this mashed data within the context of a business activity? Well, Serena Mashups may be the right way to make use of this mashed data.

The point is that JackBe and Serena should be partners, not competitors.

So JackBe, we've shown you ours. Will you show us yours?

Thursday, November 15, 2007

Who will be the business mashers?

One of the top two leading industry analyst firms has been talking for several months about how software applications are brittle, difficult to maintain and aren't architected to keep up with the fast-paced demands of today's business environment. That's not new news. It isn't even old news. What is news is that now we have analysts saying out loud what we've all known for quite a while: custom software development is broken.

I can't tell you which analyst firm has been talking about this issue because they will insist that I run this post through their vendor relations department. I don't really want to have my blog sanitized, so instead I'll just call them the Unknown Analyst. (UA)

UA has suggested some solutions to this problem. They predict more development teams will adopt agile software development practices as a way to make app dev more responsive. That will help, but agile methods will only take us so far. Even with agile development practices, multisourcing, offshoring and otherwise scaling up capacity, app dev resources are scarce and expensive. IT has been scaled to work on big-ticket and highly complex software projects, not the myriad interesting and potentially business transforming smaller applications that are backing up in app dev's queue.

Regardless of the development methodology employed by the app dev team and how many new developers they hire, IT simply isn't going to be able to satisfy the demands of the business for new and innovative applications. Especially when, as UA says, they need to change at the speed of business: constantly.

UA suggests that this gap will be filled by business analysts. The role of the business analyst will need to change to become as much someone who assembles applications as someone who simply defines them on behalf of the business. In our world this means the business analyst will turn into the business masher.

So where do business analysts come from, and will they have the necessary skills to become business mashers? What skills will the business masher need?

Good questions, both. UA says that business analysts are grown, usually from an IT staffer who is interested in business or more likely from a business expert or project manager of some sort who becomes interested in IT. Today, the most important skills business analyst posses have to do with effective communications. Can the BA communicate effectively both with the business and with IT? Does the BA know how to write, analyze a problem and mediate disputes between what can be two opposing forces?

Technical skills such as process modeling, usability engineering, and data modeling are less important. Likely because those skills can be learned, but knowing how to communicate effectively both to the business and IT isn't something that can be picked up out of an instruction manual.

But are the soft skills currently important to BAs going to be adequate when these BAs have to start turning out applications?

It's a bit of a rhetorical question because I think I have an answer already. No, these skills won't be enough. I've worked with a number of BPM tools in the past. I've also worked with a number of Web 2.0 application assembly platforms, as you will know if you've read my reviews of Intel's MashMaker, Kapow's RoboMaker, Alpha Works' QEDWiki, WaveMaker's ActiveGrid and even Serena Mashup Composer. These tools all either don't actually build applications, MashMaker, Mashup Editor, QEDWiki and RoboMaker fall into this category, or require some strong IT know-how when it comes to the deep-down nuts and bolts of web service calls.

Yes the tools will get better, but for now I can't agree with UA's assessment. Today's BAs aren't going to become tomorrow's business mashers. Most of them lack the necessary technical skills.

I want to digress a bit. A little over ten years ago there were a lot of complaints in IEEE circles because engineers would found companies only to have them run, owned and grown by MBAs. (Microsoft excepted, of course.) Many IT departments were run by MBAs working for the CFO rather than by technical people with knowledge of technical issues. We nerds put ourselves in technical silos, and then complained like crazy about the lack of technical know-how among our top executives.

IEEE put out a call and said, basically, "Quit complaining and go get MBAs." In other words, IEEE decided that if we nerds wanted leaders with strong technical knowledge, we'd have to grow these leaders from our own ranks. It was a great message and got a number of us to start worrying about things like entrepreneurialism, sales, marketing and finance in addition to whether Windows or Unix would end up ruling the world.

I think we have a similar chance to expand our horizons today. The BA role is changing. BAs today need soft skills, but the BAs of tomorrow will need more technical know-how. Most of these BAs come from the business side of the house and have learned about IT on the job. That means when these newly minted mashers start to create mashup-based business applications they will make mistakes. They won't necessarily think about version control. They won't know about consumer-side SOA Governance. They won't understand how to use XPath, call web services or in many cases how to set up a feedback system to manage changes to their applications.

Remember the last couple of times business built their own applications? Remember the Access databases from the 80s? Remember the web applications in the 90s? The business needed agility and they got it, but they also needed engineering discipline, and this they didn't get. In the end, when it became clear that these applications were business critical, they ended up back in IT's lap, and both IT and business lost. IT because they became villains. (Business: remember when we could make changes whenever we needed? Now everything takes forever.) Business because they lost their agility. (IT: those business guys don't understand that we have 400 things on our TODO list, and the capacity to do five of them.)

We don't want to repeat the mistakes of the past if we can help it. If we technical people want to keep control over our businesses, we have to learn about business. If we don't want to repeat the problems we had with business built applications in the 80s and 90s, we need to step into BA roles now so we can bring our hard-learned disciplines to bear on this new application construction paradigm.

Calling all IT application developers! Why not take the plunge and become a BA? Walk away from Eclipse and give up Java. (Although you will probably still need javascript.) Learn what makes your business tick. Read "Crossing the Chasm." Listen to a marketing podcast. Change those sneakers for loafers, throw a sports coat over that t-shirt and go apply for the BA job in sales your HR department just posted.

This time around, let's do it the right way.

Wednesday, November 7, 2007

ActiveGrid is useful for web-based application building, but like so many mashup tools, you've got to have a fair amount of technical know-how

You'll notice from the title that I'm going to review ActiveGrid by WaveMaker. I wanted to review either Denodo, JackBe or AlchemyPoint by Orchestr8, but both Denodo and JackBe declined to allow me to evaluate their software, and Orchestr8 needs to publish some more documentation before I can do a good evaluation.

Why couldn't I test drive Denodo or JackBe?

Denodo declined because they sell enterprise middleware that requires some hand-holding to install and understand. They don't allow web downloads at all, let alone downloads by potential competitors. They did offer me the chance to participate in a directed demo, but having been in the software business for a long time, I just don't trust demos. I need to dig in and get my hands dirty. I want to thank them for making the offer, however.

JackBe was more straightforward. Serena is a competitor, and they aren't knowingly going to hand over their software to someone who could have malicious intent. That would be me, I suppose.

Fair enough. I'll just have to sign up under a different email address and company name and get my hands on it that way. Perhaps I'll use my Second Life avatar. Better watch out for a download request from Chuck Malibu, JackBe. (Note: This is a joke. I would never do such a thing, even to support my blogging habit.)

So while I can't talk to you about Donodo, JackBe or AlchemyPoint, I can talk to you about a surprisingly interesting web application building tool called ActiveGrid. WaveMaker is the company name, although as little as a week ago the company name was ActiveGrid, just like the product name. I'm not quite sure why they changed their name, but as long as they didn't change the tools, this review should still be accurate.

What is ActiveGrid?
ActiveGrid advertises itself as a Web 2.0 platform (who doesn't?) allowing non-technical or business-oriented people to build their own AJAX-based RIAs and mashups. Some of that is true, and some is a bit of a stretch. It's true that you don't need a lot of technical know-how to build data-based applications with highly interactive AJAX-driven interfaces, although some familiarity with the workings of a DBMS is necessary. When I started to build mashups, well that required a bit more technical know-how, which I'll talk about later.

ActiveGrid's application model is based on a database with a web front-end driven by a BPEL process engine. It reminds me strongly of the old Access-driven applications we built back in the 80s and 90s. Start with a database, add some user interfaces and some Visual Basic to manage the business logic, and there you have it, a home-built application. Of course, ActiveGrid is quite a bit more sophisticated.

I used ActiveGrid to build an application that used the various RSS feeds and REST interfaces I defined in OpenKapow. (See my earlier post.) It was easy to assemble the screens as long as I used the defaults. They do a fairly good job of putting the widgets, data fields, etc. on the display in a visually pleasing way. They have a small selection of style sheets which were adequate. Of course, just as with QEDWiki, you can use your own style sheets.

Building ActiveGrid Web Pages is a Mixed Bag
I liked a lot of the page-building features. Selecting service fields to use on a page was quite easy. I didn't need to map the fields to page elements, ActiveGrid did all that for me. One very nice feature allowed me to take individual pages I created and, using AJAX techniques, quickly aggregate them into a single page. I liked that a lot.

Some things I didn't like about the page builder. What I want, and what most web-based application page building tools provide, is a visual modeling tool with a drag and drop interface allowing me quickly to mock-up the page. That's not how GUI development works in ActiveGrid. Setting up page formats reminded me strongly of the way we built XWindows/Motif interfaces back in the day. Trust me, it was tedious and time consuming.

BPEL for Page Orchestration and an Intuitive Event Model
ActiveGrid use BPEL for page orchestration, which is an interesting although unorthodox choice for an interactive application. BPEL is usually associated with the WS-* stack for web service orchestrations, not with human workflow. They've done a good job of making it work by unifying their service model. Everything is a service. A call to a RSS feed is equivalent to a SOAP call is equivalent to script is equivalent to a database call. It's an interesting approach and one that works well. I found their event model, which they use to control page flow, to be easy to understand and easy to use. They only have synchronous events, but that's a small price to pay for simplicity. BPEL supports asynchronous events, so perhaps they will them in the future.

For Mashups, Better Get Some Expert Help
Now let's talk about building mashups. ActiveGrid absolutely qualifies as a tool for building business mashups. They aggregate data and visual elements into a unified end-user experience within a process-centric framework. However, just because it is a business mashup tool, it doesn't follow that ActiveGrid is a tool for business mashers. In fact, I'd say that ActiveGrid requires quite a bit of technical savvy to create mashups. More savvy than we can expect from your average business analyst-turned-masher.

As I've hinted earlier, you can call many types of services from within ActiveGrid. A call to an RSS feed is equivalent to a SOAP call is equivalent to a database query. From my perspective as a business masher, I agree with this approach. Why should I care whether my service is REST, SOAP, RSS, POX or just a database call. Admittedly, I need to know the type of service when I import it, but after that it's just inputs and outputs.

But, and this is a huge 'but,' when it comes time to aggregate these services so they can be used to present a common interface, we come to a real problem area: Field mapping and parameter passing. I can only say that field mapping and parameter passing within ActiveGrid, well, sucks. It's likely not their fault. I don't know any application builder, especially one based on BPEL, that does a good job with field mapping and parameter passing. When I had to pass parameters into a service call, it took me quite a while to figure out how to do it. In fact, simply following the steps in their 'getting started' application to pass a parameter from one page to another was confusing and took me several tries before I understood what they were doing.

Let's just say that ActiveGrid didn't seem to take a lot of trouble to hide the complexity of the BPEL field mapping monstrosity from their end-users. No graphical drag and drop, no suggested mappings. Not even pop-up help so you could read the help and look at the interface at the same time. Just a lot of steps to define input variables, output variables and a really bad interface to define the logic that associates the two together.

Yuck.

Botom Line

Business mashers can use ActiveGrid to build web-based applications quickly and easily. With some understanding of how databases work, it's quite simple to build highly interactive and appealing applications. When you want to turn them into mashups, however, you'd better get some serious help from your applications development organization because it's neither quick nor easy.

Friday, November 2, 2007

This just in: mashups may be on their way from The Peak of Inflated Expectations to the Trough of Disillusionment

Andy Dornan published an interesting article Monday in Information Week and replicated to internet evolution yesterday. His article was interesting because it went into some detail about whether Web 2.0 techniques (I won’t call them technologies) are ready for widespread use in the business, and he backs it up with some survey data. This has been a popular subject lately, as demonstrated by postings by Hinchcliffe, Baer, and even myself here and here.

I've already responded to Dornan about whether mashups will ever get 'business chops,' (his wording) so I won’t write about it today. Instead, I’d like to talk about some of the findings from the Information Week survey data that were the basis of Dornan's article.

Information Week surveyed 110 respondents about various Web 2.0 issues. First, respondents were asked, “What is Web 2.0”, which is a good question with which to start. 15% said Web 2.0 was a waste of time and bandwidth and 53% said it was an overhyped buzzword. (Multiple selections allowed.) I’m going to assume that the 15% overlapped the 53% and conclude that 38% of respondents believe that while Web 2.0 is an overhyped buzzword, which is true, it isn't a complete waste of time, which is also true.

This is is progress.

In the last year the value of AJAX, wikis, YouTube, blogging and RSS has been demonstrated to business. While more people may be sick of the term, they are also adopting the technologies. Definitely progress. Regarding mashups in particular, after all this is a mashup blog, 54% put browser-based applications under the Web 2.0 umbrella. Of course it would have been better if they had been called by their proper name: presentation mashups. But a year ago I doubt if 54% of respondents would have understood the significance of browser-based applications, mashing at the glass, and how that is different from the old web-based integrations we've been using for years. Again, progress.

The article also says that 40% of respondents thought mashups were a Web 2.0 attraction. At least they thought so in February. The survey results weren’t explained in the article, but I was interested to see that in September the rate dropped to less than 10%. I'm not quite sure I'm reading the correct meaning behind the numbers. I'm assuming they floated two identical surveys. One in February and one in September. This makes sense if you correlate it with Gartner’s Hype Cycle for Portal Ecosystems, 2006. If Gartner’s predictions were correct, we were due for a plummeting hype rating for mashups sometime in 2007. Personally, I hope it’s true because I’m all for the mashup hype calming down so we can roll up our sleeves and get some real work done. I’ve posted enough on this subject that I’ll let it rest for now. While marketing departments want the hype, the rest of us would rather move on to the Plateau of Productivity as soon as possible.

One set of numbers that is a bit more sobering is that over 60% of those asked to cite objections to Web 2.0 said that security was an issue. The high percentages demonstrates that the respondents weren’t stupid. In my mind security is the one problem with SOA, various Web 2.0 technologies in general and mashups in particular, that could stop progress in its tracks. At least for a while. The first time sensitive personal information gets leaked out in a mashup, and the masher is a member of a company with lots of money, we are going to see some serious lawsuits and a lot of backpedaling on the mashup adoption front. It won't stop us forever, but it could stop us for a while if we don't start to take security very seriously.

I’d like to close with some thoughts on the Thompson Financial mashup example cited in the piece. Thompson has been building true business mashups before mashups were cool. In Thompson's view, the business building their own mashups doesn't bring them into conflict with IT. Each does what they do best. IT provides the secure infrastructure, the backups, the servers. Thompson's subject matter experts build the applications. It's definitely a great model for turning the idea of business mashups into the reality of business mashups.

While I've been concentrating on the mashup data from this article, but mashups are by no means all Dornan covers. He discusses Wikis, social networking and other Web 2.0 'issues.' The article is well worth a read, even for those of us on the downward slope from The Peak of Inflated Expectations to the Trough of Disillusionment.