In-Depth

XML tools: Who knows where or when?

• By Johanna Ambrosio
• April 1, 2004

One of XML’s core strengths can also become a source of confusion. People have begun using the language for so many different things -- structured and unstructured content, app integration and workflow -- that the ways in which developers work with it vary widely.

Some use XML-specific tools ranging from parsers to those that help with data transformation or integration, to content management software and workflow managers. Others opt for a fuller-figured XML development toolkit, analogous to an IDE. Some go the freeware route, or use text editors they already have, including emacs and vi.

But is new, XML-specific tooling necessary? Would it not be more efficient to use the development environment already in-house rather than spend time learning a new one?

The answer, industry watchers and customers say, depends on what you are doing with XML and the level of functionality built into your existing tools. There are good reasons to start with what you already have, and there are equally powerful motivations to adopt a separate, XML-focused suite.

The conundrum was summed up by Uttam Narsu, vice president at consultancy Forrester Research in Cambridge, Mass. While the XML-specific toolkits will often provide rich schema and design choices, they do not necessarily “connect up to everything else,” he said. “How do you interact with XML from an existing Java program? Many of the choices are quite painful.”

On the other hand, today’s existing IDEs often do not “give you enough XML design or management facilities,” he noted. This choice can leave users who are doing heavy-duty XML between a rock and a hard place.

Rikki Kirzner, research director for app development and deployment at Framingham, Mass.-based consultancy IDC, puts it this way: “There’s no general rule; it goes project by project. But the major question is: How specific do you need to become in your XML implementations or conversions?”

There are a lot of vertical takes on XML these days, for everyone from aviation to financial-services and high-tech manufacturing firms. If you are doing document conversion or sharing across multiple companies, you may well need an XML tool that allows you to implement or transform a specific vocabulary or schema.

But if it is something more general, like wrappering XML for a Web services app, “then pretty much all the big players have a very credible offering built in,” Kirzner said. Popular development tools from Borland, IBM, Microsoft and others already have various levels of XML support; whether you require more than that is a function of what you need to achieve and how extensively you need to manipulate, change or otherwise transform XML.

In general, “the need for standalone XML tooling was more acute in the early days when XML was identified as a separate thing,” said Ronald Schmelzer, a senior analyst at ZapThink LLC in Waltham, Mass. “These days, people expect tooling to be part of their existing” development environment, he said.

Indeed, Sandra Rogers, director of the Web services program at IDC, said her research confirms that “most people are using general tools, not Web services-specific types of tools” when it comes to XML. “They see it as an extension of their existing tool environment.” The exception: integration types of projects, when it is “more important to be specific” to ensure the technology covers the different databases or information types that need to be integrated.

Differences in approach

There may be other times when an XML-specific development environment is called for. Jonathan Robie, editor of several XQuery specifications, is XML program director at DataDirect Technologies, a Rockville, Md.-based vendor working on products that integrate XML and other data sources. He said he uses an XML-specific IDE “about half the time.” It helps to visualize relational schemas, XML schemas and the structure of XML files. Some XML IDEs allow you to generate code fragments simply by dragging structures into the code window, said Robie, which saves time and ensures that the paths are correct. And if the XML IDE also includes a good debugger, it is much easier to find bugs in the code.

And that is the crux of the difference between general application development tools and many of those that are specific to the XML world, he explained. “XML tools help you to visualize what you’re trying to do, because you can actually see XML. It’s both a machine- and human-readable language.” An XML IDE helps programmers to see both the data and the effects of each step in the transformation.

For their part, traditional programming environments are more about optimization and debugging than they are about visualization, Robie said.

For beginners particularly, one of the big benefits of an XML IDE is that everything is easy to install and “it all just works,” he noted. “Many XML gurus work with a programmer’s editor, Ant files or Makefiles, and a variety of separately installed tools, but that takes some time to set up and understand.”

Not an either/or proposition

Another factor to consider is that choosing an XML-specific toolkit does not necessarily mean it has to be entirely separate from your existing development environment. Altova’s XML Spy product, for example, is integrated with Microsoft’s Visual Studio.NET (VS.NET). So if a VS.NET user is working on an XML file, he can get XML Spy as the visualization and debugging environment for SOAP and XSLT -- within the familiar VS.NET interface.

“On the Java side we don’t have anything as tight as that, but we are cooking some things up,” said David Kershaw, manager of education and professional services at Altova in Beverly, Mass.

There are other motivations for specific XML tools, he said. It just does not pay a general tools vendor to focus as deeply as a vendor devoted to the XML and Web space, he noted. XML just “won’t ever be the highest priority” to a big development name like Microsoft or IBM, Kershaw said, as it is to a company that makes its living from XML.

Further, “there just aren’t that many tools to treat schemas the same way as we do for software code, components or classes,” he said.

At the same time, however, even Kershaw admits that Altova’s bread-and-butter customer is someone who spends virtually all of his or her time with XML. “We don’t pretend we’re a general-purpose IDE for Java or C++ code,” he said. “We’d rather plug into Visual Studio than replicate all that structure.”

Another point is that XML-specific tools are “what keeps people honest” after using the wizards and other features found in the general-purpose IDEs. “In the XML world, you may have a valid WSDL file, but not a completely correct WSDL file,” he said. “After I create something in VS.NET, I can go over to XML Spy to make sure it really does what I think it does.”

Finally, Kershaw suggested, “it’s difficult to make a really good XML schema editor -- that’s a serious modeling task,” and it is not something that all the tools do equally well.

The case for general tools

Be that as it may, IBM, Oracle and some other traditional tools vendors are making a case for integration. It is only with an environment that handles XML, SQL and other types of information -- all together -- that many pieces of a complex development problem can be provided, they say. And that is best handled with a general IDE that has XML features and functions built in.

“Integration is very important,” said Ted Farrell, architect and director of strategy for Oracle’s tools group. “We don’t have the luxury of only XML or content or software; systems are being built that are a combination of all of those things. So the problem is complicated.” This, in turn, requires tooling that “knows about” different types of information.

The new mantra from Oracle is “productivity with choice,” Farrell said. “Our tools are separate from the runtime environment” so customers can opt to go with a specialized XML tool to, say, define a schema that can later be brought into Oracle’s JDeveloper for transformation and other tasks. “If we’re all using the same file, you can check them in, then I can take them and extend them” via Oracle’s own toolset, he said.

Arthur Ryman, WebSphere Studio Development lead at IBM, explained that “what we’re pushing now is how XML is integrated into the programming model.”

This helps customers to do more complex builds like turning XML into HTML. “We created point XML tools early on,” he said, but nowadays the real value to customers is that XML is just one part of the overall WebSphere environment. “If you’re writing XML, it’s in relation to something else,” he said.

“You can’t look at XML as some isolated technology with its own specialized toolset,” Ryman said. “It’s just one of many technologies you need to get the job done.” Perhaps it is to help integrate Java and CICS applications, or to work with EJBs on the back end, or to integrate XML with stored database procedures. “It really is an integrated development effort, and you need an integrated development environment,” he explained.

That integration is getting more pervasive. Coming up in a future version of WebSphere Studio will be a debugger that can handle both Java and XSL, “so you can extend XSL to call out to Java functions,” Ryman explained. “So if I can’t express something easily in XSL, I can write a Java function. In our debugger framework, it’s all seamless.” With standalone XML tools, “you don’t necessarily get good integration with the underlying languages,” he said.

Microsoft’s view of the XML world

Like many of the general tools vendors, Microsoft is embracing XML “whole hog,” even though it is fairly early going. “There’s a lot of excitement around XML right now, but there hasn’t been widespread adoption yet,” said Tom Rizzo, director of product management for SQL Server. “You want XML to be pervasive yet cost-effective, because it’s an enabling technology” for so many other things, he said.

On the first rung is support with the Office suite, so business users can turn Word and other documents into XML, with Office handling the conversion behind the scenes so users do not need to touch the language directly. “End users don’t ever want to actually see XML itself,” Rizzo joked.

On the next rung is support within VS.NET, which includes an integrated XML editor as well as support for many of the XML object models, including XPath, XML DOM and others. Microsoft will support XQuery “when it becomes a finalized standard,” Rizzo promised. “We have a beta on our Web site, but it’s not final yet.”

On the storage side, SQL Server stores XML documents and can run queries against them. Rizzo promised even “tighter integration” between the SQL and XML worlds in the next release of SQL Server, due for commercial release by the end of this year. In that version, users will be able to do cross-domain queries and cross-domain insertions of both XML and SQL data.

“Let’s say you have a customer database -- with name, phone number and addresses. But maybe you’d want to store an XML column in that database with demographic information that’s stored in XML because it could change. So you could do a selection of the top 10 customers and at the same time filter it with demographic data, too,” he said.

For industry-specific versions of XML, there is Microsoft’s BizTalk Server, which can take proprietary or vertical schemas and then convert proprietary information to those schemas or industry standards. The next version of BizTalk, “due out pretty soon,” will make it easier to “build those transforms between XML and to orchestrate transforms,” Rizzo said.

Overall, Rizzo said, Microsoft’s XML support meets “80% to 90%” of customers’ requirements.

Different XML needs may require different tools

In general, there are two types of XML users, say Oracle’s Farrell and others: business analysts who like XML’s flexibility to deploy code once to many different kinds of platforms, and the programmers and serious XML coders who are using the language directly.

For the first set of users, visual and other “light” tools may be needed more than for the second batch of heavy-duty programmer types. From this second group of users arises a call for better schema manipulation and editing, as well as better transformation mapping. “No one feels this is done very well across the board; different products map these transformations, but all the tools have their limitations,” Farrell said.

It is precisely because of these different types of XML users that the world is moving in two directions. One is increased functionality and integration for the heavy-duty XML coder. And then there are the larger numbers of visual tools for business analysts and others who want to map vendor, product or part number and then parse or merge those elements into an application. They do not want or need to know how XML works, only that it does.

IDC’s Rogers explained that her research suggests that “many people want to have their XML or SOAP code automatically generated from a modeling tool, and then be able to get into that code both graphically and in the traditional coding way. And they want it bi-directional, so if there’s a change it shows up both graphically and in the code itself.”

But it is going to take a while for the technology to become that sophisticated. Until it does, XML developers already have many choices in their arsenal. As IDC’s Kirzner said, “if I need more than the standard eight crayons that come with the coloring book, I can always go out and buy the box with 64 crayons. And then maybe some magic markers, too.”

Please see the following related story:
"OneFamily, one connection layer" by Johanna Ambrosio