In-Depth
A talk with the experts
If you ask five people what middleware is, you are likely to get five different answers. So we did just that to start our Middleware Special Report. We asked five voices of experience several key questions, and came away with some very interesting replies.
Is middleware on firmer ground than in past days, we asked some middleware mavens. The consensus? Yes ... but. Some assert that middleware is giving way to the concept of Enterprise Application Integration (EAI), which they describe as a new breed of software entirely. People we spoke with say the EAI industry will build its fortune upon the back of the middleware infrastructure created in the last decade. At the same time, middleware will be pushed down and commoditized, and ultimately find acceptance -- although as something of an afterthought ("Like TCP/IP," said Vitria's JoMei Chang.). Most of those ADT talked to, however, admit that this coming EAI golden age has been made possible by the serious efforts of message-oriented middleware makers, with most citing IBM as driving it to prominence in the '90s via MQSeries.
But there is still some controversy regarding messaging and object alternatives. The former is clearly a more loosely coupled asynchronous approach, while the latter is still usually associated with synchronous system design. Meanwhile, objects are said to offer more flexibility through abstraction. The experts generally concur: Your mileage may differ.
So, what is the relationship of EAI to middleware? "EAI needs middleware," said industry analyst Les Yeamans. "The way to get the benefits of EAI is with various types of middleware. Once the year 2000 comes and goes, EAI is going to be the burning issue in MIS."
For middleware, acceptance beckons -- IT shops are overburdened trying to connect diverse systems -- as well as obscurity, as EAI comes along to turn traditional middleware into mere plumbing. Looking at middleware is still looking through the glass darkly, but pictures are emerging. Taken together, we think the comments of our middleware crew prove useful, so we invite you to read our virtual middleware forum ...
How can one bring sense to the mystery of middleware? We thought simply asking insiders for their thoughts would be helpful. But definitions are still hard to come by. According to John Mann, senior consultant/analyst for Enabling Technologies at Boston's Patricia Seybold Group, middleware is a huge space and it is typically confusing.
"I generally think of middleware in terms of message-oriented middleware, but that's only a tiny piece," said Mann. "The most all-inclusive definition would describe the software that sits between the application and the operating system. I would exclude the databases. But it is a runtime software that the applications would otherwise have to have written inside the application. TP monitors, application servers, MOMs and request brokers -- those form a whole other class of middleware."
There is also another class, which Mann would call data access middleware. Commented Mann: "That would be the [Information Builders Inc.'s] EDA SQL, that kind of middleware that connects the applications to data so that applications don't have to know about 60 different kinds of data sources. They can use middleware as sort of a switch."
Tom Laffey is a middleware technical pioneer now deeply involved in the rapid commercialization of the technology. At Lockheed, an initial application development area for Laffey was satellite ground control. Now, as co-founder and CTO of Los Altos, Calif.-based Talarian Corp., he brings a new generation of technology to bear on the area he and others call zero-latency systems, basically rapid response enterprise systems that bring the information routing techniques of defense and Wall Street to commercial apps.
"From day one we had to do everything real-time," he said, looking back on the formation of Talarian. "At first we were about five years ahead of the market. Now the whole world is coming over to real-time. It is very natural. In our everyday lives we have One-Hour Photo -- and now you see that becoming 30-Minute Photo. In terms of middleware categories, publish-and-subscribe is really what we've been."
How does Laffey segment the middleware market? In terms of response, there is a continuum, he suggests, from 'batch' processing to real-time. EAI is an application built on such middleware technology. IBM's MQSeries, in Laffey's view, is just the next step up from batch.
Les Yeamans is an industry analyst, publisher of the Messageq.com Web site specializing in EAI trends and technologies, and a co-founder of the Message Oriented Middleware Association (MOMA). He focuses on middleware and EAI, and his credentials as a middleware expert are substantial. He was the general manager at System Strategies Inc. (SSI), which developed one of the first commercial messaging systems. SSI's work became part of the foundation of IBM's MQSeries software.
What is his historical view on the middleware market? "The market has really been growing in the last couple of years," he said. From 1988 through 1994 there was little demand -- there was more of a technology push underway by the vendors. In those days, said Yeamans, "You were really creating the market."
This changed, however, with the acceptance of MQSeries over the course of the '90s. "IBM invested a boat-load of money to create the market, and they did quite a good job," he said. "This was the first time IBM had a technology on day one available on 10 platforms."
"I think middleware probably started out in the earlier days more like a database middleware, and, when some of the distributed object technology came out, some people started calling that middleware," said Ron Zahavi, chief technical officer at Concept Five Technologies Inc., based in McLean, Va.
"Middleware, in my mind, is technology, or a set of technologies, that provides some level of abstraction for whatever it is people are trying to abstract," he said.
"People look at something like ORBs," said Zahavi, regardless of whether it is CORBA or DCOM, "as a certain type of middleware." He would suggest CORBA or DCOM be viewed more as technology. The real interest, according to Zahavi, is the applications that sit on top of that middleware. Thus, the new area of interest is the applications that help you process and analyze what the middleware pipes carry.
"The stuff that people usually relate to EAI, I think, is MOM really, and MOM-related," said Patricia Seybold's Mann. In fact, a lot of this resembles what was years ago known as 'operations.'
"Digital had something called DEC Message. Then came MQSeries. Tibco [also] had a special kind of approach. They had publish-subscribe, but they've moved past that now," noted Mann.
"The problem with MOM middleware, if there is a problem, is that it is low level. It is the old story about any kind of middleware: The more generality, the simpler it is, but the less it gives you. The original message-oriented middleware was like a pile of lumber -- you could use it to build an outhouse, a garage, a church or a house," he explained. "Once you start producing pretty fat modules, once you start formulating, then the range of possibilities gets restricted. Early MOM was versatile, simple and low level, and users had to add a lot to it."
We asked Mann about the role of MQ in all this. Said Mann: "I think the whole MOM industry has done a lot of education. But I would say that IBM getting into it kind of validated it and legitimized the market. Microsoft has a different view -- they work with developers, not enterprises. The developer, the programmer, has a different perspective on that than the enterprise [application manager]. The 'enterprise' is actually thinking in terms of looking over multiple apps and integrating them. The developer is still thinking about solving a specific problem."
In order to understand the technology and to figure out where it is going, we once again pick up our discussion of what middleware is and where it comes from. Middleware, notes Yeamans, "could be described as anything between the 'Net layer and app layer. Lots of things could be called middleware -- any API, even LU6.2. [Even] remote procedure calls as implemented in DCE could be described as middleware.
"There was a lot of buzz in the early '90s about DCE," continued Yeamans. "But people realized over the years -- except perhaps the died-in-the-wool synchronous 'guys' -- that a tightly coupled approach is going to be impractical in a distributed architecture with more than two machines. These tightly coupled architectures required all of these machines to be up and running at the same time. People have learned that these systems don't scale. When you get to large-scale systems, you need something loosely coupled. That's where message queuing has hit its pace."
But Yeamans does not dismiss the object-oriented synchronous approach associated with ORBs.
"All of these other technologies have their places -- there are various development paradigms out there. But I guess the issue is what technology you are going to use to tie it all together. That's where we get to EAI, and where it gets interesting. There are some new apps where you might want to use CORBA. But if you're not a new company, you have a legacy of apps you want to integrate with new and packaged apps. Then you need to come up with a technology that's going to tie that all together," he noted.
According to Concept Five's Zahavi, "We look at ORBs and OTMs; they add the capability in most cases of standardization like CORBA. They give you object-oriented capabilities and abstraction. There are also multilanguage, multiplatform capabilities they tend to provide. But you don't see one product-type today that necessarily gives you all the capabilities [you need]. So when we look at how we define EAI, we define it as features that cut across all types.
"A lot of vendors are trying to move from their area of strength into more of a middle ground," he explained. "You might see a message broker application trying to add more of the Web and component and Java capabilities [of an application server]. You might see an application server vendor trying to add transaction capability and security, to be more of an enterprise integration platform. We see EAI today as a term applied to integration that requires features across all of those. It's not necessarily middleware by itself that's a technology that competes, but it is a way to apply those technologies."
The application servers are very good at bringing together information from different sources, said Zahavi. "Developing new logic to go with that information, and presenting it to things like the Web, is an application server strength. But they do not necessarily do a lot of the legacy-to-legacy integration. They're good at developing applications and services, but are they the ultimate solution for the enterprise? Probably not, but they are a piece of that. The same can probably be said about all the other technologies.
"If we look at message brokers, we can say they are very good at integrating systems through other systems. They support a lot of the asynchronous capabilities; guarantee delivery, messaging, transformation and routing of messages; and they also guarantee the transformation of the information and formats," he said.
"Maybe the application servers lend themselves very well today to the Web and to components, but the messaging brokers don't necessarily do that well," Zahavi said. "They're concentrated more on the back-end messaging infrastructure. Again, you sort of need both capabilities, but there's really no single product out there today that wraps all of these enterprise application issues together."
"It seems the application servers are all using CORBA as their communications system," noted Talarian's Laffey. "They focus on small 'request-reply' applications that CORBA is good at. But true EAI is much more a many-to-many problem." It tends, in his view, to be more asynchronous and loosely coupled. Laffey would contend that the messaging approach is more scalable. That a closely coupled object broker approach may work on LANs but not on WANs.
We also could not resist asking about XML, which we hear may serve as an integration lingua franca [see "XML: The last silver bullet," p. 24, Application Develop- ment Trends, April 1999].
Said Laffey: "XML is a good thing, but it's not going to live up to its hype. You can put XML in a message and certainly use it [as meta data]. But if you care about performance, you're not going to want to use XML because it uses bandwidth. At some places on the middleware spectrum, XML has just too much overhead."
We wonder if the next generation of 900MHz Sparc chips might alter this equation, but we don't wonder out loud. We can't pretend to argue with Mr. Laffey's experience.
CORBA expert Zahavi somewhat echoes Laffey's (and Yeamans') comments on the merits of loosely coupled, messaging methods of middleware.
"You never can get a lot of internal departments to all agree on what a customer profile means and how to develop joint software that cuts across lines," he said. "In reality, when you get into the enterprise, the model where you look at integration across loosely coupled applications is going to work and scale very well.
"I think that what we saw in recent years is that the messaging software showed up at the proper place in the market," noted Zahavi. He sees merit in an emerging standard messaging enhancement to CORBA.
JoMei Chang's credentials as a middleware expert are complete. They include important work with Tibco (then known as Teknekron) in defining push-oriented middleware on Wall Street in the early '90s. More recently, she is president and CEO of Mountain View, Calif.-based Vitria Technology, founded as one of the key definers of EAI.
"After Tibco," said Chang, "we realized we had to provide a higher level of infrastructure.
"Ten years ago, when you talked about TCP/IP you had to explain it. Now people take it for granted. I see messaging becoming the 'plumbing piece,'" she said.
"The middleware component will gradually become a commodity in the next two to three years. MQSeries as messaging middleware is going to be more and more of a standard, and that's why you're not going to see messaging as the key selection criteria," said Chang.
Is EAI the same as middleware? Is middleware the same as EAI? "That's a major myth," she says. "EAI is a lot more than middleware."
Chang believes there is a need to think in terms of distinct layers: The plumbing or data layer, which may be supplemented by a messaging integration layer, and then two top layers -- the process automation layer and the real-time analysis layer. The key to business value is in the top two layers, she said
Where does she position application servers? While Chang thinks application servers will move toward EAI, she sees them as one lower layer component for EAI.
We note in our discussion that a Web search query on middleware returns less hits today than it did last year, while a query on "Enterprise Application Integration" sends the match-count through the roof. Is that a sign of overactive marketing?
Chang responds with a question: "Why is it every other week a new company calls itself an EAI vendor? EAI today is seen by analysts as the key pain area people are facing. EAI can be bigger than ERP and the database market combined."
Real-time is becoming another one of the buzzwords of the late decade. And the once-esoteric real-time-oriented middleware engine known as 'publish-and-subscribe' may move up as a mainstream offering as a result. "People needed to connect applications, and not only connect them, but connect them more or less in real-time," said Seybold's Mann. "People were once content to do EAI as batch data transfers -- it would be enough to be able to run two jobs overnight.
"But now people are talking about doing real-time. So the real-time requirement, and the fact that some middleware is so low level, made people aware that we needed to do something to make it easier," he said. This led to Enterprise Application Integration and a slew of vendors -- Mann cites "the Active Softwares and the Vitrias." He also noted that IBM has a "fairly comprehensive solution package.
"This is the market that people have recognized, defined and called 'Enterprise Application Integration,'" said Mann.
ADT's readers have often asked for information on Microsoft's messaging middleware entry, MSMQ. Is it making in-roads? It is perhaps still too early to tell. We asked our panel about MSMQ today.
"IBM has a strong position in the messaging layer," said Vitria's Chang. "I think Microsoft is going to have a stronghold. We have not seen so much with MSMQ. But I think it is a force to contend with."
Said Seybold's Mann: "I have attitude problems about Microsoft. I think they are culturally more interested in market forces than they are in identifying and solving business problems. But there may be a stage at which the typical Microsoft developer suddenly needs to be interested in building middleware-based products. In that case you'll see MSMQ take off."
Has analyst Yeamans seen MSMQ in the market yet? "It is starting to get a little more popular," he said. "The problem is that Microsoft is in a difficult position because they're not allowed to market MSMQ separately. It literally has to be marketed as part of NT server.
"It's not like IBM's MQSeries, which has a whole division that lives or dies on how many licenses are sold," he added. "MSMQ is getting better and more popular in development [circles]. It's not widely known, but NT is the leading distribution platform of MQSeries at IBM."
Our guests for this discussion were JoMei Chang, president and CEO of Mountain View, Calif.-based Vitria Technology; Tom Laffey, co-founder and CTO of Talarian Corp., Los Altos, Calif.; John Mann, senior consultant/analyst for Enabling Technologies at Boston's Patricia Seybold Group; Les Yeamans, an industry analyst and publisher of Messageq.com; and Ron Zahavi, chief technical officer at Concept Five Technologies Inc., McLean, Va.