Gartner says that initial SOA adoption rates are slowing. I think they are wrong.

I just read a Government Technology article stating that Gartner says there is a dramatic decrease of organizations planning first-time SOA projects. Here's the money quote.

Since the beginning of 2008, there has been a dramatic fall in the number of organizations that are planning to adopt SOA for the first time. In 2008, this was cut by more than one-half, down to 25 percent from 53 percent in 2007, while the number of organizations with no plans to adopt SOA more than doubled from 6 percent in 2007 to 16 percent in 2008.

Unfortunately, I no longer have direct access to the Gartner research (One of the things about leaving my former employer that I will miss the most.) so I have to comment only on the secondary source, the above mentioned article. That's too bad because I can't look at the methodology used to gather the research. Knowing Gartner from the past, however, I can make some guesses.

I'm guessing they talked to IT departments and asked, “So, what are your spending plans for SOA?” and probably, “Have you already adopted SOA within your organization?” Those aren't bad questions, but I think we could all have predicted the results. Given today's economic uncertainties, Big IT isn't going to spend lots of money on big IT projects that don't have an established track record for success. Unfortunately, the success of SOA in the enterprise, at least Big SOA, is not a given.

But these results don't even come close to telling the full story. I personally have worked on three projects over the past few months that were absolutely based on a services architecture. However, if you were to ask the IT departments of these organizations about the projects, they would not have tagged them as SOA.

Why not?

Because these weren't 'SOA projects,' they were business initiatives whose solutions happened to make use of SOA. In all three cases we didn't buy expensive middleware to run the software. We didn't embark on an orgy of service writing to SOA-enable myriad legacy systems. In two of the three cases it's probable that I was the only person who knew the underlying architecture of the solution was services oriented. These were true Guerrilla SOA projects.

So while I completely believe that Big IT departments are slowing down in their implementation of Big SOA projects, I don't believe for a minute that these same organizations aren't expanding their use of SOA. It's just that IT doesn't know. And what they don't know, they can't blab to Gartner.

So take what Gartner says with a grain of salt. Sure some of the Big IT SOA projects may be on hold, but don't assume that Big IT owns all the action. SOA is happening all over, without IT knowing anything about it.

Hello again, everyone. (Or all two of you)

Loyal readers may remember that I stopped writing this blog when my boss instituted a policy requiring all blogs be approved by PR before publication. Since I wouldn't submit my blog to PR, I stopped writing it, at least under my own name.

I'm no longer with my prior company, so it is time to start up this blog again. I've got a backlog of issues to write about. So many, in fact, that you can think of this as my Q4 2008 editorial calendar.

• Finally, after all my pleading, all my whining and all my cajolery, I get to test drive JackBe Presto. We've gone a round or two in the past, with JackBe refusing me access, and me swearing that I would give them a fair review. You can read my requests here, here and here. I'd like to say that my razor sharp reasoning finaly convinced JackBe that free and open access to their technology is the best marketing strategy, but it wouldn't be true. It's just that I now no longer work for a company JackBe perceives as a competitor.
  
  So far it has been a bit of a trial. Since I had to give back my corporate computer, I'm starting from scratch with a laptop having only, god help me, Vista, installed. Before I can install the Presto suite I have to install the JDK and Eclipse, which reminded me of all the myriad libraries and jar files I already had installed on my old machine. I suppose I'll figure out what's missing when I write code and don't have the appropriate jars...
  
  So I haven't quite gotten Presto installed yet, but hopefully some time today I'll be able to dig in.



• The financial services industry has been an early adopter of business-driven initiatives using guerrilla SOA and mashups. Needless to say, this market has other things on its mind just now. The question we are all asking is, "What's in store for us given the meltdown?" I've been building mashups for a handful of global banks for the past nine months, and I've got some opinions about what we can expect for the next nine months.
  


• Interneer looks like another promising application builder tool, along the same lines as Coghead and ActiveGrid (WaveMaker). Once I finish with Presto, I'll take Interneer through its paces and let you know how it stacks up against its rivals.



• Despite our best efforts, the hype surrounding SOA is dimming. This is actually a good thing, not a bad thing. SOA is no longer newsworthy because it is passe. Mainstream. Old school. Dull. As SOA moves to the mainstream, raging debates about SOAP versus REST, stateless web services, and whether Event-Driven Architecture (EDA) is an extension of SOA, seem to be naval gazing. I'll give my take on the state of SOA, where it is, and where I think it will be moving in the next year.



• The industry is lousy (Nasty word. Read the etymology here.) with WS* standards. Along with the product reviews, I'll start to explain some of these specifications and how they fit in to the evolving SOA 'standards.'

That should take me through Q4 2008, along with random rants from the field. Feel free to write with suggestions for topics, or just to say, "Hello."

It's good to be back.

Farewell Business Mashups Blog...I don't want a PR review

To all three of my readers out there, I'm sad to say that this will be my last post for this blog.

We now have a policy stating that all external communications have to be run through our PR department. Since I'm not comfortable having my blog reviewed, I'm going to stop this one.

No need to weep in despair, however. I will be blogging under another name and another account, so keep a lookout.

Sincerest Regards,

Kelly A. Shaw

IDC says HR to help Web 2.0 adoption. Um...what?

I just read an article by INQUIRER.net writer Lawrence Casiraya that had me raising my eyebrows. He interviewed IDC's Shalini Verma about the concept of "unified communications" and how Web 2.0 technologies such as social networking can help distributed organizations collaborate more effectively.

Here's the money quote.

So how can businesses begin to embrace Web 2.0? The task may fall into the hands of the human resource department.

What was Verma thinking? HR, together with IT, have been the main forces preventing adoption of Web 2.0 technologies in the workplace. According to Verma, HR should conduct a survey to find out what technologies are already being used and then make a plan for company-wide adoption of the most useful tools.

Well, I doubt that will work. Pretend we are listening to a conversation between two 'screenagers' when they get a survey from HR...

Nancy Networker: Did you see this survey from HR? They're trying to find out what sort of web apps we use. Why do they want to know?

Wally Widgetuser: (Smirking knowingly to Nancy) The survey says it's because they want to help us collaborate more effectively across the company.

Nancy and Wally have a good, cynical laugh.

Nancy: Well, I'm not telling them anything. The last thing I need is for them to shut down Facebook. It's the only way I keep in touch with the guys in Thailand.

Wally: You got that right. If I tell them I'm building mashups to keep track of my customer information, they'd freak.

Nancy: I'm just going to say I use Google Docs. I can live without them if HR and IT shut them down.

...

OK, the situation may not be that bad, but I'd wager that, given HR's history, there won't be a lot of trust between HR and the employees.

What's the answer?

I think HR could take the lead, but they'll have to do some groundwork first to establish some trust and credibility with the rest of the organization. Executive sponsorship wouldn't hurt either. In my own case, I spent well over a year blogging under a different name, knowing I'd be shut down if HR found out. I came out of the closet when Serena's executive leadership started promoting the use of social networking.

In addition to getting very visible executive support, HR could start publishing internal case studies about how people in the company use Web 2.0 to promote collaboration. Maybe someone in HR uses pbwiki to collaborate on ideas for the company picnic. Maybe someone in PR uses Second Life to conduct press conferences. (OK, that one's off the wall.) Perhaps someone in sales uses a mashup to get information before a customer call. If HR spends some time up front promoting the use of these collaborative tools, then maybe they will be trusted.

Maybe.

But we all remember when HR shut down MySpace, told us we couldn't blog, and forbade us from giving recommendations on LinkedIn. They've got a ways to go before we trust them with the dark secret that we use OpenKapow to scrape competitive information, or that we swap musical stations with clients on Pandora.

News from the mashup trenches

I’ve neglected this blog for a while, and now it’s time to get back to it. I didn’t think changing jobs would have much of an effect on my blogging output. Wrong again, Shaw. These days I have to do my reading and blogging mostly after hours, and that does change the dynamic. What takes priority, doing laundry, weeding the garden, helping with homework, balancing the checkbook, playing fetch with the dog, splitting wood, or...writing this blog? I'm afraid lately it's been those other things, and I'm sorry.

On the bright side, I’ve been in the trenches more than I have been in the past couple of years. It’s one thing to read analyst reports, discuss findings with customers and partners, write academic papers and pontificate from on high. It’s another to jump in and actually develop mashups. It is definitely harder to view the entire industry from a trench. Especially in the context of a fixed price contract. On the other hand, I’ve become reacquainted with the reason I went into technology in the first place: I enjoy it. I’ve written more applications (Don't be fooled. Mashups are applications.) in the past couple of months than in the past two years combined.

In a departure from my standard mashup posts, I’d like to share some things I’ve learned about mashups in the past few weeks rather than commenting on mashup news or reviewing some new mashup tool. Some of my experiences have reinforced what I thought before I embarked on this change in job title. Some have made me change my thinking, and in ways that might not make my current employer happy. Sorry in advance, Serena Software, if what I have to say doesn't agree with our marketing message.

First, I’d like to talk about the idea that mashups will allow the business to deploy their own applications without having to bother IT. I never thought this would work. After all, even applications in the tail will need access to back-end systems. Even applications in the tail will need version control of some sort. Even applications in the tail will need governance, even if that governance is light-handed compared to governance used to manage more strategic applications built by IT itself. After my time in the trenches, I’m even more convinced that mashup development has, at the very least, to be a loose partnership between the business and IT.

Let me give you an example.

I built a series of mashups as a prototype for a hardware technology company who wanted to get better control of their rebate program. It was a very intense three-day engagement where we spent the entire day working with the customer to understand their business problems, and then I got to spend the nights building mashup prototypes. (I got about four hours of sleep during those entire three days. I don’t recommend it steady-state, but it was nice to know I could get back to my developer roots if I had to do so. I even had my first fully caffeinated non-diet pop in years. I drew the line at pizza, however.)

The meetings with the client were sponsored by IT, and were run by the business. They were sponsored by IT because the business didn’t know mashups from shinola, but IT did. The business didn’t have a clue that mashups could help them, but IT did. The business was frustrated at IT’s repeated attempts to solve their growing rebate program problem, and IT responded by searching for alternatives to their same old development paradigm.

IT and business worked together to help us define the requirements. IT and business together helped identify what infrastructure was already in place to support the effort. IT and business worked together to help define what parts and pieces of the proposed new ‘system’ could be managed by end users and what parts and pieces would be governed by IT. In other words, there was deep collaboration between IT and the business. And while there were definitely self-preservation undertones in IT's motives, there was also realization on both sides that the old way of doing app dev didn’t work. Everyone was on-board to try something new.

If business had tried to run this meeting without IT, mashups wouldn't even have been on the table. It would have been strictly a shadow IT project that may or may not have taken advantage of recent Web 2.0 trends. And it would have been killed as soon as it became apparent that the project required access to CRM and financial systems. If IT had tried to run the meeting without the business, then we would likely have had a great architecture, and a deep understanding of how the pieces and parts were going to work together, but we never would have understood the deep and abiding frustration felt by the business, nor the financial imperatives that required something be done about the problem soon.

Both had to work together for success. Perhaps this isn't true for all mashup projects, but I suspect it will be true for many of them, especially any that require integration with the existing IT infrastructure.

Here’s something else I learned in the past few weeks. We’ve been saying for a while that business users would start writing their own applications because they can no longer wait for IT to do things for them. The idea is that account reps, shipping clerks, marketing program specialists, etc. would write their own mashups, and we will give them the tools to do so quickly and easily. Now that I’ve been out in the world a bit and have seen what is really going on, here’s what I think: Bullshit.

Even the young 'Net Gen' men and women on the business side don’t want to write their own apps. They may be forced to do so, but they aren’t doing it by choice. That’s because these younger workers want to make progress in their chosen career. Account reps of any age want to spend their time selling, not writing apps. Business analysts want to spend their time figuring out how to beat the competition, not writing apps. Marketing program specialists want to put programs in the field, not spend time writing apps.

So while these end users may be writing apps, they are doing so reluctantly because it takes time away from their chosen careers. Mashers are the exception rather than the rule.

What I do see is a rise in shadow IT. Certainly the business can’t wait for IT to get to their applications in the tail. But the business isn’t building these apps itself. they are hiring outside help. Unlike the wholesale shadow IT trend in the 80’s however, these new shadow IT projects have to operate below the radar. They have to be inexpensive enough to be funded within a departmental budget and not cause an IT governance blip. They also have to show fast ROI so that, when the project finally gets outed, the business can justify what it’s done in a ‘ask forgiveness not permission’ model.

(Note: web widgets are the exception. I'll write more about them in a future post.)

Here’s something that may surprise you. These shadow projects may be instigated by IT as well as by the business. IT itself is getting frustrated with its own inability to react quickly to requests for applications in the tail. In fact, I worked on a project a couple of weeks ago where I designed a guerrilla SOA infrastructure for an IT department that was trying to get an internal project into production before Big IT saw what was happening. The irony of IT setting up shadow IT to subvert Big IT made the job all the more enjoyable.

The take-away for this post is that IT and business have to work together for successful mashup projects, just as they do for any development effort. (Let's all hold hands and sing Kumbaya.) However, when business does take the lead, it is likely to be in the form of a low-cost shadow IT project rather than in the form of an internal masher. I know that isn't doctrine. Perhaps my opinion will change in the future. But for now, I calls it the way I sees it.

Mashups in the financial sector aren't just for the back office

I spent last week in NYC talking about mashups to a number of customers in the financial sector. I love going to NY, and last week the weather was beautiful and all the designer dogs were out in force in Central Park. The outlook from our financial institution clients wasn’t quite so perfect, however.

Here’s the message I heard over and over again: Mashups in the financial industry were only good for the back-office, not the front-line. So while we can help make their order-to-cash process mean and lean, we can’t help them bring innovative products to their customers.

I understand the reasons. Financial institutions have to be conservative. When bankers and investment institutions stray from the straight and narrow, somebody will likely be in front of Congress right before they go to jail. S&L bailout anyone? Would you like to invest in some junk bonds? Let's depend on Enron for our retirement portfolio. Oh yes, let’s not forget subprime mortgages.

So while I understand their reluctance to adopt mashups on the front-end of their business, I think it is a mistake. I wouldn’t expose the banking systems until we get better mashup security. But financial institutions have a lot of other offerings that aren’t tied directly to their transactional back-end systems.

Why should they bother?

Financial institutions have to walk a fine line. They are in a constant struggle to balance the need for governance, the heavy load of compliance, and a cutthroat competitive landscape. And the financial sector depends heavily on technology to be competitive. And not necessarily technology within a traditional IT organization.

According to a Booz Allen Hamilton study, for every dollar spent on ‘real’ IT, most industries also spend 78 cents on ‘shadow IT.’ That is, IT funded directly by, and implemented within the business. In the financial sector I’d be willing to bet the ratio is much higher. One bank employee I talked to said that embedding IT within the business is a necessary practice just to stay competitive. When one bank innovates, the others have to be right behind. That means tight coupling between the technologists and the business so new and innovative offerings can be out the door fast.

This sounds like a perfect job for mashups.

I’m not an expert on the institutional side, but I do have a number of personal and small business accounts with a couple of handfuls of banks and investment firms. As a consumer of financial services, I’ve got a number of ideas for how they could use mashups without compromising their core banking systems.

How about a money management mashup? Most banks have money management information, but wouldn’t it be a good idea to mash information from multiple sites, mashing book sales from Amazon? Then not only could the company provide good value to their customers, they might also be able to turn their website into a profit center.

Ditto for investment information. I have accounts with several investment firms, yet when I want to do any investment research, I have to search Yahoo! finance to get the financials and Google for any relevant news. I’d use a mashup that pulled that information together into a single page.

How about a mashup that pulls together many investment strategies? Again mashing up books from Amazon and information from some of the leading personal finance strategists. How about a mashup that lets me compare and contrast a company’s performance against some of its nearest competitors? Then mash in some Google Docs to let me save my analysis so I can retrieve it later.

And on the other side of the equation, banks and investment firms should turn some of their free web content into widgets. For example, Fidelity has a DJIA chart on their home page. If this was a widget, and they modified it to show provenance, Fidelity would get free advertising whenever someone added the widget to a mashup.

I’m not buying that mashups aren’t a good fit for the financial services industry. Most of the innovation, at least on the consumer side, isn’t in the back-end transactional systems. It’s out front, providing services, information and advice to customers.

Again, a perfect job for mashups.

Smoking pot and stealing music. Some things never change.

OK, I admit I wrote that title to see if I could trick some people into reading this post. But really, I will actually compare the two. My motivation is a recent article by Linda Tucci, a writer for SearchCIO.com. It made me smile because it was about how millennials don't respect organizational, hierarchical or other boundaries. These millennials are going to cause security headaches because they don't respect IT policies and procedures either.

This is a hot news flash?

In her defense, Tucci was simply reporting on the results of a Symantec survey, first blogged by Symantec employee Samir Kapuria. But those of us who have either been interacting with these younger workers, or have children of that age who are about to enter the workforce, already know we've got an IT compliance disaster waiting to happen. I know that my own daughters have absolutely no respect for IP rights. In their minds, anything on the public web is and ought to be theirs for the taking. Lectures about the morality of downloading music and video fall on deaf ears. As do discussions about network security and malware.

These conversations reminded me of discussions I had with my parents about pot smoking when I was a teenager. My parents lectured me on the evils of marijuana, but in my peer culture at the time, nearly everyone smoked it. In fact, the University of Michigan and Michigan State University had parties every spring, called the Hash Bash , to protest pot laws. While I never had the guts to light up on the steps of the capital and get carted away in nonviolent protest, I wasn't above cutting class (I was in High School at the time) and joining in the party.

Bear with me. This isn't just a stroll down memory lane. It really is about mashups.

In my view at the time, and the view of many in my generation, pot was not only a civil right, it was symbol. Sure, flaunting the anti-pot laws was fun. But it was also morally defensible to break the laws in protest of unnecessarily restrictive rules and regulations. I believed my parent's views were not only behind the times, not just old fashioned. They were wrong, and nothing they said changed my mind.

That's the attitude I see in my children. Talking to them about network security, IP rights, privacy, and even footnoting, is like talking to a brick wall. For them, free access and use of all information is not only a civil right. Breaking IP and security rules is a form of political protest against unnecessary and restrictive rules and regulations. Here's the money quote from the article.

When asked whether they feel entitled to use whatever application or device or technology they would like, regardless of source or corporate IT policies, 69% of millennials said yes, compared with 31% of other workers. Indeed, 75% of millennials have downloaded software on their work computer for personal use, vs. 25% of other workers -- even though 85% of the organizations surveyed indicate their policies restrict that practice. Millennials also regularly store their corporate data on personal devices: 39% on personal computers, 38% on personal USB devices, 20% on personal hard drives and 16% on personal smartphones.

CIOs should be very afraid of these survey results. Especially since the same survey showed that IT and other corporate leaders believe they have good rules in place, and that everyone understands and mostly obeys them. Those who don't comply get fired.

Most of the Millennials I know aren't afraid of losing their job. They aren't going to get intimidated by getting yelled at by the boss. Organizations who try to restrict the use of personal devices, who prohibit social networking and other Web 2.0 applications, who try to legislate the use of web content, are either going to be mired in lawsuits, or are going to find that they can't hire innovative and out-of-the-box thinkers.

What's the alternative? I'd like to fall back on the agreement I've now forged with my children. I've worked for companies that blocked sites, monitored email, recorded web access and filtered out 'bad' words in IM. I didn't care for it, and I wasn't going to turn around and do the same thing in my own home. Nor could I simply ignore the problem. While I know pirating is illegal, I also believe it is wrong.

We finally came to a compromise that we worked out together. They don't completely like it, still believing I'm backwards-thinking. I don't completely like it, believing they will have ample opportunity to break the law. But because it is a negotiated agreement rather than a dictated policy, I have some hope of success.

• They are now free to download anything that is really free, not pirated free. MySpace is full of 'really free' music and video, and a lot of it is quite good.
• They can keep their MySpace accounts, but they must allow me access to their profiles. (Neither of them like Facebook. Probably because I use it.)
• They have an iTunes budget. It isn't large, but it is enough to buy a few songs now and then.
• They won't download software without my approval. I can only deny the download if the software is harboring malware, if it's content is objectionable or if it will cost too much.
• They agree not to store any pirated content on their computer.
• I've asked them not to 'borrow' pirated content from their friends. I've told them I'll throw away any media that I believe has pirated content.

So far it's either working or they are very good at making it appear to work. I won't take bets.

I think IT has to do something similar. In old paternalistic, hierarchical organizations it might be considered a sign of weakness to negotiate policy with subordinates. Our millennials are going to change that mindset. Corporate leaders will need to work with their employees rather than dictate to them, or they will face not being able to recruit or retain the quality of worker they need. So instead of a restrictive IT policy based on sanctions and Big Brother thinking, we'll probably end up with something similar to the agreement I have with my kids.

With respect to mashups, I think we'll also end up with something similar.

• If you mash content from the web, note the source.
• If you mash content from behind the firewall, make sure the content isn't sensitive.
• If you are mashing services from the web, make sure they don't have viruses, understand the costs, and try to use reputable sources.
• If you are mashing services from behind the firewall, make sure the services don't expose sensitive information.

Are these guidelines bulletproof? Of course not. There isn't an IT policy today that's bulletproof. What these guidelines do is help the masher understand what the issues are and why he/she should be concerned. These guidelines treat the masher like an adult, not like a naughty child or convicted felon that must be monitored.

Some may think this is mere kowtowing to these new bad boys entering the workforce. Further proof that the world is going to Hell in a hand basket. Me? I can't wait until these younger workers roll in and shake everyone up. Will we have chaos? Will there be security problems? Are there going to be mistakes, upheavals and disasters?

Most certainly. But there will also be progress.

(Note to horrified readers: I stopped smoking pot in High School. I didn't then, and still don't, think there is anything wrong with it. I just needed to get my act together academically. After HS I always ended up in jobs that required a security clearance. And now it just doesn't interest me.)