Spring WebSocket STOMP tips and tricks

Recently we successfully implemented secure Websockets in one of our projects and learned a lot of tricks to get things working together. Given below are some tips that would help teams embarking on implementing WebSockets in their programs.

1)  Spring uses the STOMP protocol for Web Sockets. The other popular protocol for Web Sockets is WAMP, but Spring does not support it. Hence if you are using Spring, make sure that your Android, iOS and JS  libraries support STOMP.

2)  Spring websocket library by default also supports SockJS as a fall-back for web JS clients. If your use-case only entails supporting Android and iOS clients, then disable SockJS in your Spring configuration. It might happen that a use-case might work on SockJS on a web-client, but fail in native mobile code.

3) The default SockJS implementation of Spring, sends a server heartbeat header (char 'h') every 25 seconds to the clients. Hence there is no timeout on the socket connection for JS clients. But on mobile apps (pure STOMP), there is no heartbeat configured by default. Hence we have to explicitly set the heartbeat on the server OR on the client to keep the connection alive when idle. Otherwise, idle connections get dropped after 1 minute. Reference Links: https://stackoverflow.com/questions/28841505/spring-4-stomp-websockets-heartbeat
Sample server side code below.

4) One Android websocket library that works very well with Spring websockets is https://github.com/NaikSoftware/StompProtocolAndroid. But unfortunately, this library does not support heartbeats. Hence you have to explicitly send heartbeats using sample code like this - https://github.com/NaikSoftware/StompProtocolAndroid/issues/18.

5) On iOS, the following socket library worked very well with Spring websockets - https://github.com/rguldener/WebsocketStompKit. We faced an issue, where this library was throwing Array Index Range exceptions on server side heartbeat messages, but we made small changes in this library to skip process for heart-beat header messages.

6)  It is recommended to give the URL scheme as wss://, although we found that https:// was also working fine. If you have SockJS enabled on Spring, then please append the URL with /websocket, as you would get a exception stating invalid protocol up-gradation. Hence clients should subscribe to/{endpoint}/websocket. https://github.com/rstoyanchev/spring-websocket-portfolio/issues/14

7)  Both libraries also support sending headers during the CONNECT step. Very often, teams send the authorization token during the CONNECT step in the header, that can be used to authenticate the client. To access these headers on the server we need to access the Nativeheaders hashmap. Sample code - https://github.com/rsparkyc/WebSocketServer/blob/master/src/main/java/hello/ConnectionListener.java

  

Using Spring MVC to create REST style services

We have been always using Apache CXF or Axis 2 to build web services, but recently I was pretty impressed with the simplicity with which we can create REST style services using Spring MVC. OOTB, Spring MVC integrates with Jackson (for JSON serialization) and JAXB (for XML serialization).

We just have to annotate our Spring Controller class with @RequestMapping and @ResponseBody attributes and let Spring handle the rest. The following links would provide more info on the same.

http://www.javacodegeeks.com/2013/04/spring-mvc-easy-rest-based-json-services-with-responsebody.html

http://xpadro.blogspot.in/2013/02/create-and-test-rest-services-with.html

Using Spring MVC with Ext-JS MVC
http://justinrodenbostel.wordpress.com/2013/03/15/restful-spring-mvc-and-extjs-episode-1-the-spring-stuff/
http://justinrodenbostel.wordpress.com/2013/04/02/restful-spring-mvc-and-extjs-episode-2-the-extjs-stuff/

Troubleshooting XML namespaces binding in SOAP request using JAXB

Recently helped a team resolve an issue regarding namespace handling in JAX-WS / CXF. Jotting down the solution as it might help other folks breaking their heads on this issue :)

Our Java application needed to consume a .NET webservice and were facing challenges in SOA interoperability. We created the client stubs using the WSDL2Java tool of CXF. The WSDL had - elementFormDefault="qualified"

The problem we ran across was that the complex types were all being returned as 'null'. Enabling the network sniffer, we checked the raw SOAP reponse reaching the client. The response was OK with all fields populated. Hence the real issue was in the data-binding that was happening to the Java objects.

A quick google search revealed that the default behavior of JAXB when it encounters marshelling/unmarshalling errors is to ingore the exception and put it as a warning messages !!. This was a shock, as there was no way to debug the issue on the server.

We then wrote a sample JAXB application and tried to un-marshall the XML/SOAP message. It is here that we started getting the following error stack:
org.apache.cxf.interceptor.Fault: Unmarshalling Error: unexpected element

Hence, it was proved that the real culprit was the JAXB unmarshaller that was not generating the Java classes with the correct namespace.

First we checked the generated Java classes and found a file called as "package-info.java" that had a XML namespace 'annotated' on the package name - essentially a package annotation. So this was supposed to work, but why is the unmarshaller throwing an exception?

We tried adding the following attributes to the annotation and then the unmarshaller started working !!!
@javax.xml.bind.annotation.XmlSchema (
    namespace = "http://com.mypackage",
    elementFormDefault = javax.xml.bind.annotation.XmlNsForm.QUALIFIED,
    attributeFormDefault = javax.xml.bind.annotation.XmlNsForm.UNQUALIFIED
  ) 

Not sure why the WSDL2Java tool did not add these automatically based on the elementFormDefault="qualified" present in the WSDL.  But this trick worked and we could consume the .NET webservice. We had to modify the build script to replace the "package-info.java" file everytime it was recreated.

Another option is to manually add the "namespace" attribute to all XMLTypes in the generated Java classes; but this is very tedious.

Understanding REST

In one of my old posts, I had elaborated on the differences between REST and simple POX/HTTP.

Recently came across an interesting post by Ryan Tomayko; where-in he trys to explain REST in a simple narrative style. A must read :)    http://tomayko.com/writings/rest-to-my-wife

Another interesting discussion thread on REST is available at StackOverFlow - regarding verbs and error codes.

Interoperability when using datasets in .NET webservices

Datasets are powerful data containers that are very popular in the .NET world. Quite often, a lot of .NET webservices return datasets in their web methods. This is ok, as long as we are sure that all the clients are also on the .NET platform. 

But if we want our services to be interoperable with other platforms (e.g. Java), then passing datasets back to webservice clients would not work. This is because a dataset is a generic container and is populated at runtime with data. Hence at design time, it is not possible to define the dataset datatype with a schema. Also when a dataset is serialized to XML, then the default .NET SOAP serializer adds .NET specific attributes to the XML schema. e.g. isDataset=true, Diffgram, etc.

Typed datasets do have a schema backing them, but here again there is a hack required to make typed dataset accessible to a Java program. The hack is to return a xml string or a XML Node instead of the typed dataset. We would also have to change the autogenerated WSDL and add the typed dataset schema to it. A lot of trouble for using datasets in a heterogenous environment :( 

For hassle free interop, the best option is to go for pure schema driven complex types that can be mapped to any language.

More information is available at the following links:

http://msdn.microsoft.com/en-us/magazine/cc188755.aspx

http://msdn.microsoft.com/en-us/library/ms998428.aspx

http://www.cnblogs.com/rickel/articles/157935.html

XML serialization tips in .NET

Today, my team ran across a strange issue. Some of the properties on a .NET class were not getting serialized to XML when used in webservices. There was no error message or any other warning. 

Closer examination revealed that the properties not getting serialized were 'read-only' properties; i.e. they had getters but no setters. The default XML Serializer in .NET calls the property getters/setters and hence if a read-only property is present, then it is not serialized.

Once we added setters to the properties, XML serialization worked fine.

There is another option. We can implement the IXmlSerializable interface and write custom serialization code. But this could be very tedious. 

Schema validation in webservices

Recently a friend of mine was working on a .NET SOA project and was facing a peculiar problem.The XML Schema he was using had a lot of restictions such as minOccurs, maxOccurs, int ranges, string regular expression patterns, etc.

But unfortunately in .NET 2.0 webservices, there was no easy way to validate input messages based on this schema. The reason for this behavior has to do with XmlSerializer, the underlying plumbing that takes care of object deserialization. XmlSerializer is very forgiving by design. It happily ignores XML nodes that it didn't expect and will use default CLR values for expected but missing XML nodes. It doesn't perform XML Schema validation and the consequence is that details like structure, ordering, occurrence constraints, or simple type restrictions are not enforced during deserialization.

Unfortunately there is no switch in .NET 2.0 that can turn on Schema validations. The only way to validate schemas is to use SOAP extensions and validate the message in the extension code block before deserialization occurs. The following links show us how to do it:
1. MSDN-1
2. MSDN-2

In the J2EE world, JAX-WS 2.0 has become the default standard for developing webservices. In most toolkits, we essentially have 2 options: Let the SOAP engine do the data-binding and validation or separate the data-binding and schema validation from the SOAP engine. For e.g. In Sun Metro project, validation can be enabled for the SOAP engine using the @SchemaValidation attribute on a webservice.(https://jax-ws.dev.java.net/guide/Schema_Validation.html)
In Axis 2, there is support to enable schema validation using the XMLBeans binding framework.

I found this whitepaper quite valuable in understanding the various options for schema validation in JEE webservices.

Loosely typed vs Strongly typed webservices

This article gives a good overview of the difference between a loosely coupled and strongly coupled webservice.

Jotting down some points from the article here:

A loosely typed webservice is one where the WSDL (interface definition) does not expose the XML schema of the message to be transferred. One example of a loosely typed description is Web service that encodes the actual content of a message as a single stringThe service interface describes one input parameter, of type xsd:string, and one output parameter.

Another way of indicating that no strict definition of a message format exists in a WSDL file is the use of the element. Its occurrence in a schema simply indicates that any kind of XML can appear in its place at runtime. In that respect, it behaves much like the case where single string parameters are defined. The difference is that here, real XML goes into the message, as opposed to an encoded string.

Pros and Cons:
The service consumer and service provider need to agree on a common format that they both understand, and they both need to develop code that builds and interprets that format properly. There is not much a tool can do to help here, since the exact definition of the message content is not included in the WSDL definition.

On the positive side, if the message structure changes, you do not have to update the WSDL definition. You just have to ensure that all the participants are aware of such a change and can handle the updated format.

Examples where Loosely coupled services make sense:
1. For example, assume you have a set of coarse-grained services that take potentially large XML documents as input. These documents might have different structures, depending on the context in which they are used. And this structure might change often throughout the lifetime of the service. A Web service can be implemented in a way that it can handle all these different types of messages, possibly parsing them and routing them to their eventual destination. Changes to message formats can be made so that they are backward compatible, that is, so that existing code does not have to be updated.

2. Another example is the use of intermediaries. They have a Web service interface and receive messages. But many times, they provide some generic processing for a message before routing it to its final destination. For example, an intermediary that provides logging of messages does not need a strongly typed interface, because it simply logs the content of any message that it receives.

3. Finally, a message format might exist that can not be described in an XML schema, or the resulting schema cannot be handled by the Web service engine of choice.

A strongly typed service contains a complete definition of its input and output messages in XML Schema, a schema that is either included in the WSDL definition or referred to by that WSDL definition.

Strongly typed Web services provide both consumers and providers of services with a complete definition of the data structure that is part of a service. Tools can easily generate code from such a formal contract that makes the use of SOAP and XML transparent to the client and server-side code.

Capabilities of an ESB

Just saw Mark Richards presentation on ESB. The presentation is simple and easy to understand without all the fancy jargon.
The streaming video is available at: http://www.infoq.com/presentations/Enterprise-Service-Bus

Capabilites of an ESB:

Routing: This is the ability to route a request to a particular service provider based on some criteria. Note that there are many different types of routing and not all ESB providers offer them i.e. static, content-based, policy-based. This is a core capability that an ESB has to offer (at least static routing). Without it a product cannot be considered an ESB.

Message transformation: Ability to convert the incoming business request to a format understood by the service provider (can include re-structuring the message as well).

Message enhancement: We don't want to change the client and in order to do that we often need to enhance (removing or adding information, rules based enhancement) the message sent to the service provider.

Protocol transformation: The ability to accept one type of protocol as input and communicate to the service provider through a different protocol (XML/HTTP->RMI, etc).

Message Processing: Ability to perform request management by accepting an input request and ensuring delivery back through message synchronization i.e queue management.

Service orchestration: Low-level coordination of different service implementations. Hopefully it has nothing to do with BPEL. It's usually implemented through aggregate services or interprocess communication.

Transaction management: The ability to provide a single unit of work for a service request for the coordination of multiple services. Usually very limited since it's difficult to propagate the transaction between different services. But at least the ESB should provide a framework for compensating transactions.

Security: Provides the A's of security. There are no "silos" in SOA so the ESB has to be able to provide this.

Adding trusted root certificates in Websphere 6.1

We had deployed an application to WAS 6.1 ND. This application contained a JAX-WS 2.0 webservices client that used to call a third-party service using SSL. The digital certificate used by the third-party was self-signed and hence we needed to accept it as a trusted party in our Trust Store.

We imported the certs into a trust-store (JKS format) using the keytool command of the JDK and wrote the following code:

System.setProperty
("javax.net.ssl.trustStore", trustFilename );
System.setProperty
("javax.net.ssl.trustStorePassword", "changeit") ;
System.setProperty
("javax.net.ssl.keyStore", trustFilename );
System.setProperty
("javax.net.ssl.keyStorePassword", "changeit");

But unfortunately this was not working on WAS6.1. (This works fine on Tomcat).
In earlier versions of WAS, the iKeyman tool provided an interface to manipulate digital certs, keys, Trust stores and Key stores. But in WAS 6.1, all these tasks can be done from the web-based admin console.

So to add the certs to the Trust store, I went to "Security (on the left panel) --> SSL certificate and key management > Key stores and certificates > NodeDefaultTrustStore > Signer certificates"
Add the new root digital certs that need to be trusted to this store. The JAX-WS client should now be able to connect to the HTTPS resource. Remove any system properties that have been set before.

A good article (for WAS 6.0) describing how SSL and Digital certs work in Websphere can be found here.

Hack to deploy JAX-WS 2.0 webservice on Websphere 6.1

Our team had developed a webservice using JAX-WS 2.0 and using the Reference implementation of Sun as the JAX-WS provider. The application was developed using Netbeans and the embedded Tomcat server.

But when the application was deployed on WAS 6.1, then it failed and resulted in ClassCast exceptions. The base installation of WAS 6.1 only supports JAX-RPC 1.1.
IBM did provide an option - installing the webservices feature pack for WAS 6.1
The installation was a bit of a pain, especially on a cluster. You essentially had to create new AppServer profiles, create a new cluster and move all applications to the new cluster. This would also required considerable downtime on a live production server.

I looked for an other option. I tried to understand what was happening under the hoods when Netbeans was creating a webservice. It was clear that the webservice created was using a Servlet provided by the RI of JAX-WS 2.0. Even all the annotations and JAXB libraries were included in the lib directory. Hence there was no dependancy on any server-specific library as such.

I realised that the ClassCast exceptions on WAS 6.1 were happening due to the ClassLoader heirarchy - which was set to 'Parent First'. I decided to try to change this configuration to 'Child First' - so that JAX-WS RI libraries are picked up from WEB-INF/lib first.

There were 2 changes that I made using the web admin console. Go to "Enterprise Applications --> {Application name} --> Class Loader" and changed the following items:
Select "Classes loaded with application class loader first" and "Single class loader for application". Save these changes to the master configuration and restart the AppServer.

This hack worked and we could deploy JAX-WS 2.0 webservices on WAS 6.1 :)

Tip: We ran into a few Linkage errors, which we could resolve by removing all duplicate jar files.
A linkage error would typically occur when a class tries to call another class loaded by a different class loader.

Is POX/HTTP same as REST?

There is a lot of confusion in the industry over REST webservices. A lot of applications offer POX/HTTP interfaces and call them REST webservices. But in theory, just having XML tunneled over HTTP does not satisfy all REST principles.

It's important to understand that REST is not a protocol. It is an architecture style that can be used to design systems. It brings in a noun-oriented approach to access resources (instead of a verbs).

The Apache CFX framework has new features (using the magic of annotations) that make it possible to have true REST style webservices.
More information on this can be found here and here.

Recently came across this article that states the REST principles that should be followed during design:

1. Give every resource an ID. Use URIs to identify everything that merits being identifiable, specifically, all of the "high-level" resources that your application provides, whether they represent individual items, collections of items, virtual and physical objects, or computation results. Examples:

http://example.com/customers/1234

http://example.com/orders/2007/10/776654

http://example.com/products/4554

2. Link things together. (Hypermedia as the engine of application state)

Example: Element in XML doc: product ref='http://example.com/products/4554'

We could use a simple 'id' attribute adhering to some application-specific naming scheme, too but only within the application’s context. The beauty of the link approach using URIs is that the links can point to resources that are provided by a different application, a different server, or even a different company on another continent because the naming scheme is a global standard, all of the resources that make up the Web can be linked to each other.

3. Use standard methods. For clients to be able to interact with your resources, they should implement the default application protocol (HTTP) correctly, i.e. make use of the standard methods GET, PUT, POST, DELETE

e.g. GET - get order details/list all orders

PUT - update order

POST - add order

DELETE - delete order

4. Communicate statelessly - This is for scalability.

Message style and encoding in WSDL documents

The WS-I specs state the following 2 types of encoding to be complaint to the Basic profile:
- RPC/literal
- Wrapped document/literal

Found this cool article on developerworks that explain all the styles and encoding in detail.

WSDL cheat sheet

With the plethora of tools available today for webservices development, we sometimes forget the basic fundamental semantics of a WSDL document. Here is a quick reference of what a WSDL contains:
1. Types: These are the data type definitions - expressed as a schema of complex/simple types.
2. Message: A message consists of a logical order of one or more types.
3. Operation: Defines a operation that consists of a input and output message.
4. Porttype: A collection of operations.
5. Binding: Specifies the concrete protocol data format specifications (e.g. document-literal) for a portType.
6. Port: Specifies a network addresss for a binding, thus defining a single communication endpoint.
7. Service: collection of endpoints or ports

Thus a service is defined by one or more endpoints/ports. A port is combination of a binding with a network address. Each binding is a combination of a porttype and concrete data format specifications. Each porttype is a collection of operations that define messages that are exchanged.

More information can be found here.