Web Services Testing: Beyond SOAP

Document ID: ZapFlash-08262002 | Document Type: ZapFlash
By: Jason Bloomberg
Posted: Aug. 26, 2002

Today's Web Services are often little more than software components wrapped in SOAP interfaces, and as such, today's software testing tool vendors only need to add simple XML support to their product lines in order to offer Web Services testing capabilities to their customers. However, over the next five years or so, Web Services herald a shift in distributed computing toward loosely coupled, standards-based, Service-oriented architectures. Testing these architectures and the Web Services that constitute them is another matter entirely. What's down the road for Web Services, and what functionality will testing tools require to support companies' testing needs in the future?

Phase One (2002 - 2003): Internally Focused Testing
The first phase of Web Services adoption (which describes the current situation) has an internal enterprise focus. Enterprises primarily use Web Services to simplify and reduce the cost of integration inside the firewall. Some companies are exploring the use of Web Services to communicate outside the firewall, but these instances are often pilot projects, and are typically with trusted business partners and known IT organizations.

In the first phase of Web Services adoption, three capabilities are the most important for Web Services testing tools:

 

• Testing SOAP messages -- moving beyond using SOAP as the interface to the Web Service to testing the format of the messages themselves.

   

• Testing WSDL files and using them for test plan generation -- WSDL files contain metadata about Web Services' interfaces. Testing tools can use these WSDL files to generate test plans automatically.

   

• Web Service consumer and producer emulation -- when a testing tool generates test messages that it sends to a Web Service, it is emulating the consumer for that Service. In addition to the Web Service producer, the consumer of the Service also sends and receives SOAP messages. A Web Services testing tool should therefore emulate the Web Service producer as well as the consumer.

Phase Two (2003 - 2005): Testing Service-Oriented Architectures
As new products and services on the market resolve the issues with Web Services security, management, and transactions, companies will be able to exchange Web Service messages with other companies (customers, suppliers, and partners) in a more uninhibited, loosely-coupled manner. Enterprises and established groups of business partners will find that UDDI-based Service registries will become a critical enabler of the dynamic discovery of Web Services within controlled environments.

During this second phase of Web Services adoption, the following capabilities for Web Services testing tools become important:

 

• Testing the publish, find, and bind capabilities of a SOA -- three fundamental characteristics of Service-oriented architectures are the publish, find, and bind capabilities of the constituent Web Services. Web Service testing tools should test each side of this triangle.

   

• Testing the asynchronous capabilities of Web Services -- today's early uses of Web Services are often as simplified remote procedure calls or requests for documents. In addition to these straightforward synchronous uses, SOAP also supports asynchronous messages, including notification and alert messages. Web Services testing tools should test each kind of SOAP message.

   

• Testing the SOAP intermediary capability -- the SOAP specification provides for message intermediaries. A particular SOAP message will typically have a designated recipient, but may also have one or more intermediaries along the message route that take actions based upon the instructions provided to them in the header of the SOAP message. Web Services testing tools must verify the proper functionality of these intermediaries.

   

• Quality of Service monitoring -- Traditionally, IT management is the province of operations, while software testing belongs to development. This distinction will gradually blur as Service-oriented environments become more prevalent. Therefore, Web Services testing tools should have runtime testing capabilities in addition to their design time capabilities.

Phase Three (2004 and beyond): Testing Dynamic Runtime Capabilities
As Service-oriented architectures mature and Web Service orchestration and business process automation tools become increasingly prevalent, Web Services will be dynamically described at runtime. In this phase, Web Service consumers dynamically discover Web Services as in phase two, but now those Services may change every time a consumer invokes them. Furthermore, the concept of Web Service "consumer" and "producer" become less meaningful in phase three, because complex orchestrations of Web Services will be the norm. Businesses will expose coarse-grained Web Services to their other companies (for example, a "product catalog" Service), and those Web Services will actually consist of a large, dynamic collection of finer grained Services (say, "product list," "current inventory," "preferred pricing") that may themselves consist of even more finely- grained Web Services. In addition, each of these component Web Services may contain Services provided by other companies (for example, "credit approval").

Web Services testing tools in phase three should have the following capabilities:

 

• Web Services orchestration testing -- One of the most powerful uses of Web Services is orchestration: combining many fine-grained Web Services into larger, coarse-grained Services. Such orchestrations of Web Services typically involve more than one company.

   

• Web Services versioning testing -- rolling out new versions of Web Services, especially when they are combined into complex orchestrations, will be especially complex and risky. For such rollouts to be successful, therefore, the enterprise must have testing tools that can test the new versions of individual Web Services either in production, or in an environment that parallels the production environment as closely as possible.

The ZapThink Take
As enterprises move toward the Service-oriented environment envisioned in phase three, the traditional "test after development" approach to software testing will no longer work at all. In fact, the ponderous "design, then develop, then test, then launch" testing processes familiar from the sequential waterfall software development methodology will be entirely inappropriate for the fully realized Service-oriented environment. Instead, software development groups will by necessity have to take what is now known as an agile approach to testing and development: work with the customer, tackle just what needs to be done, write a test, code as a team, pass the test, repeat until finished. Testing before coding will move from being "extreme" (as in the popular agile development methodology Extreme Programming) to being the only cost-effective method for dealing with the dynamic nature of distributed computing.

Taking the agile approach to testing, in fact, appears to be the only approach to software testing that makes sense in phase three. In the agile approach, testing begins by automating the collection and representation of new user requirements. The tests themselves, as well as the resulting code, develop iteratively, so that when the code is complete, the test is as well -- and the code always passes the test at each iteration. If each individual Web Service has its own automated test, then orchestrating Web Services also includes orchestrating the tests. As a development team works with users to define coarse-grained requirements and write the corresponding coarse-grained tests, the testing tools that the development team uses must be able to automate the coordination of the appropriate tests for all existing and new Web Services, on all levels of granularity.

Other changes to the traditional approaches to software development that Web Services will bring about are even more profound. In fact, Web Services promise to change the entire approach to software development. The whole concept of a software development lifecycle, where development teams design, build, launch, and eventually retire large software projects, will fall by the wayside as companies realize the advantages of orchestrating loosely coupled Services into coarse-grained business Services. The iterative methodologies that were such an improvement over the earlier waterfall methodology must now change as well, and become agile. Agile approaches to software development will by necessity become the only cost-effective, practical way of conducting the practice of creating software in the fully realized Service-oriented environment.