Who will be the business mashers?

One of the top two leading industry analyst firms has been talking for several months about how software applications are brittle, difficult to maintain and aren't architected to keep up with the fast-paced demands of today's business environment. That's not new news. It isn't even old news. What is news is that now we have analysts saying out loud what we've all known for quite a while: custom software development is broken.

I can't tell you which analyst firm has been talking about this issue because they will insist that I run this post through their vendor relations department. I don't really want to have my blog sanitized, so instead I'll just call them the Unknown Analyst. (UA)

UA has suggested some solutions to this problem. They predict more development teams will adopt agile software development practices as a way to make app dev more responsive. That will help, but agile methods will only take us so far. Even with agile development practices, multisourcing, offshoring and otherwise scaling up capacity, app dev resources are scarce and expensive. IT has been scaled to work on big-ticket and highly complex software projects, not the myriad interesting and potentially business transforming smaller applications that are backing up in app dev's queue.

Regardless of the development methodology employed by the app dev team and how many new developers they hire, IT simply isn't going to be able to satisfy the demands of the business for new and innovative applications. Especially when, as UA says, they need to change at the speed of business: constantly.

UA suggests that this gap will be filled by business analysts. The role of the business analyst will need to change to become as much someone who assembles applications as someone who simply defines them on behalf of the business. In our world this means the business analyst will turn into the business masher.

So where do business analysts come from, and will they have the necessary skills to become business mashers? What skills will the business masher need?

Good questions, both. UA says that business analysts are grown, usually from an IT staffer who is interested in business or more likely from a business expert or project manager of some sort who becomes interested in IT. Today, the most important skills business analyst posses have to do with effective communications. Can the BA communicate effectively both with the business and with IT? Does the BA know how to write, analyze a problem and mediate disputes between what can be two opposing forces?

Technical skills such as process modeling, usability engineering, and data modeling are less important. Likely because those skills can be learned, but knowing how to communicate effectively both to the business and IT isn't something that can be picked up out of an instruction manual.

But are the soft skills currently important to BAs going to be adequate when these BAs have to start turning out applications?

It's a bit of a rhetorical question because I think I have an answer already. No, these skills won't be enough. I've worked with a number of BPM tools in the past. I've also worked with a number of Web 2.0 application assembly platforms, as you will know if you've read my reviews of Intel's MashMaker, Kapow's RoboMaker, Alpha Works' QEDWiki, WaveMaker's ActiveGrid and even Serena Mashup Composer. These tools all either don't actually build applications, MashMaker, Mashup Editor, QEDWiki and RoboMaker fall into this category, or require some strong IT know-how when it comes to the deep-down nuts and bolts of web service calls.

Yes the tools will get better, but for now I can't agree with UA's assessment. Today's BAs aren't going to become tomorrow's business mashers. Most of them lack the necessary technical skills.

I want to digress a bit. A little over ten years ago there were a lot of complaints in IEEE circles because engineers would found companies only to have them run, owned and grown by MBAs. (Microsoft excepted, of course.) Many IT departments were run by MBAs working for the CFO rather than by technical people with knowledge of technical issues. We nerds put ourselves in technical silos, and then complained like crazy about the lack of technical know-how among our top executives.

IEEE put out a call and said, basically, "Quit complaining and go get MBAs." In other words, IEEE decided that if we nerds wanted leaders with strong technical knowledge, we'd have to grow these leaders from our own ranks. It was a great message and got a number of us to start worrying about things like entrepreneurialism, sales, marketing and finance in addition to whether Windows or Unix would end up ruling the world.

I think we have a similar chance to expand our horizons today. The BA role is changing. BAs today need soft skills, but the BAs of tomorrow will need more technical know-how. Most of these BAs come from the business side of the house and have learned about IT on the job. That means when these newly minted mashers start to create mashup-based business applications they will make mistakes. They won't necessarily think about version control. They won't know about consumer-side SOA Governance. They won't understand how to use XPath, call web services or in many cases how to set up a feedback system to manage changes to their applications.

Remember the last couple of times business built their own applications? Remember the Access databases from the 80s? Remember the web applications in the 90s? The business needed agility and they got it, but they also needed engineering discipline, and this they didn't get. In the end, when it became clear that these applications were business critical, they ended up back in IT's lap, and both IT and business lost. IT because they became villains. (Business: remember when we could make changes whenever we needed? Now everything takes forever.) Business because they lost their agility. (IT: those business guys don't understand that we have 400 things on our TODO list, and the capacity to do five of them.)

We don't want to repeat the mistakes of the past if we can help it. If we technical people want to keep control over our businesses, we have to learn about business. If we don't want to repeat the problems we had with business built applications in the 80s and 90s, we need to step into BA roles now so we can bring our hard-learned disciplines to bear on this new application construction paradigm.

Calling all IT application developers! Why not take the plunge and become a BA? Walk away from Eclipse and give up Java. (Although you will probably still need javascript.) Learn what makes your business tick. Read "Crossing the Chasm." Listen to a marketing podcast. Change those sneakers for loafers, throw a sports coat over that t-shirt and go apply for the BA job in sales your HR department just posted.

This time around, let's do it the right way.

Intel Mash Maker shows promise, but isn't there yet

I've spent the last couple of days playing around with Intel's Mash Maker, trying to see if it is a tool I would feel comfortable putting in front of a business masher. As with so many other mahsup tools, the answer is "No." Not only is it much too fragile at the moment (Their server has been down more than it has been up over the past couple of days.) but it still requires users to know HTML and XPath to scrape content.

However, unlike Google's Mashup Editor and Yahoo! Pipes, I can see the skeleton of a real business-user tool in the making.

For those of you who may not have read about Mash Maker, it is a semantic web-based HTML screen scraper developed at Intel's Berkley Research Lab. The idea behind the tool is to leverage the power of everyone in the world to classify, tag and otherwise bring under control the chaos of content that is now the web. Then when the content is tagged and understood, allow said content to be mashed with other content while providing hints about other available mashups.

Here is an example. There is nothing new about plotting all the good burrito restaurants near a specific location. We've been mashing up those sorts of sites for a couple of years now. However, what if you wanted to find the good burrito restaurants that were within walking distance of a tire repair shop? If you had to use Google or Yahoo! to develop this mashup, it wouldn't be worth your time. However, with Mash Maker you can create the mashup in minutes.

Run a search for burrito joints in your area code. Show the results as a table (A 'standard' mashup provided by Mash Maker) and copy them. Run a search for tire repair shops in your area. Paste in the burrito restaurant information and map the results. Now you have plotted on one map the burrito joints and the tire repair shops. It's faster to build the mashup than it would be to run two Google Map searches and tally the results on paper.

Why do I say the tool isn't yet ready for prime-time? Server crashing problems aside, getting to the point where Mash Maker understands the content in a web page is difficult. Any page that Mash Maker doesn't understand, and that includes nearly every website that I visited, needs to have a data extractor written to scrape out the content. Creating a data extractor requires the user to know how to write XPath expressions, and how to understand the relationship between the page's HTML and the rendered content. Sorry, but that's too much to ask of a business user. No, not because the business user is stupid, but because these users are experts in their subject area, not in HTML and XPath.

I have to conclude that "Mashups for the masses" is still only a pipe dream. However, I think Intel is on the right track. While Mash Maker isn't ready for my business mashers yet, it could be in the future.