Generating KML for RESTful clients

We have already described how to integrate with JAX-RS using the JBoss RESTEasy implementation. In that last article we showed how straightforward it was with a good API to implement RESTful web services that serve XML representations of our data. We have always been fervent supporters of web services, so RESTful was just a natural evolution from our previous deployments of CORBA or SOAP.

We have recently made another jump forward towards the Web 2.0 throw-in-all-you-want buzzword concept: the integration of third party web applications (weblets?) in our VisibleLogistics service. I believe the new technical buzzword for this is mashup. As a service that tracks orders, catalogues and shipments over the world it seemed fairly logical to integrate some cool-looking interactive maps to display geographical information that would only look dull in plain HTML.

With the rise of Google Maps and Google Earth as the de-facto cool interactive map applications on the web, and their amazingly well-designed APIs, we were bound to create a mashup with them sooner or later. Now that Google Earth is available as a browser plugin it becomes very interesting to make those mashups.

We decided it was time to integrate Google Earth and Google Maps (because the Google Earth plugin is not available on Linux and Mac OS) in VisibleLogistics. In this article we describe how we implemented this integration.

Integrating Google Maps or Google Earth in our web application means we need to find a way to control these views — widgets — in order to instruct them how to display the data we want to display. In our case we want to highlight certain locations, attach some text to them, display locations that change over time and draw some lines on the map.

There is only one way to control these map widgets: through their JavaScript APIs. But there are two ways to tell them what to display: using API methods that create visual objects to place on the map, or using an XML file describing the visual objects in application-agnostic format: KML.

I have worked with many APIs over the years, not so many in JavaScript I admit, but still quite a few, and I have to say Google’s JavaScript APIs for Google Maps and Google Earth are good. They are not just good: they are very good. They are so well designed that even though I am not too fond of JavaScript I end up not complaining at a single thing about these APIs (with an exception I will mention later).

As much as I like their API, building the visuals using the API does not make sense because Google Maps and Google Earth have a different map object creation API. Moreover, they both work well with the language-agnostic KML format. Using KML allows us to not only work with Google Earth plugins, but the Google Earth autonomous application itself. KML documents can even be saved or viewed in other applications.

We thus decided to generate KML for the visual objects we want to display. The next questions are of course how we generate these KML documents, and how we serve them.

After a quick trial with stream.print("<kml>…​") and document.createElement("kml") (the DOM equivalent) it becomes clear that this is not how we want to generate KML. If only for the reason that the structure of the KML format is not reflected in our code and any change to the version of KML (now at 2.2 indicating several previous revisions, and strongly hinting at further ones) will not break our code, but will make our code generate invalid KML.

The ability to have the KML format reflected in our code is key to being able to spot KML syntactical errors as we write the code that generates it. Having our IDE of choice provide syntactical checks of the KML format, contextual help and completion are essential if we want to write KML correctly, quickly, and in a future-version-proof way (at least in the sense that we will know exactly where the new version has changed).

For all these reasons we decided to use the Java API for XML Binding (JAXB) and apply it to KML. This allows us to create KML as Java objects and have a syntactically correct object representation of KML in Java. A quick search reveals a few hits but mostly related to version 2.1 while the current version is 2.2.

Because we did not find any readily available KML 2.2 JAXB library, we set out to generate it ourselves since JAXB includes a tool to generate JAXB-annotated classes from an XML Schema. We started by getting our hands on the latest KML Schema (yes both the XML and Atom schemas) and tried our luck:

This error and a solution are already known but we were not too happy about modifying a schema. So We have looked for other solutions and found that it is possible to either insert some JAXB annotations in the schema or create an external binding customisation file, which would be used to guide xjc while it generates the JAXB classes.

Of course we prefer not to mess with the schema so we created an external binding customisation file in order to resolve the two conflicts in question by renaming the Java factory method for the scale and snippet elements.

After that we can run xjc again:

At that point it did generate all the JAXB classes, but with package names we were not very happy with so we changed the customisation file:

For the curious, yes, guessing that the URN urn:oasis:names:tc:ciq:xsdschema:xAL:2.0 made xjc look for a schema at http://docs.oasis-open.org/election/external/xAL.xsd was done using tcpdump ;)

We now have all of our JAXB files and once We have built the Javadoc and start using it, we notice a funny comment in net.opengis.kml.AbstractFeatureType.getRest():

So we told xjc to rename the field:

We now have only one remaining problematic getRest() method in net.opengis.kml.NetworkLinkType but this time the message in the Javadoc points us to the fact that xjc is confused about kml:Link and atom:Link having the same local part which means that since they are both present in the same class (via inheritance) one of them must be renamed:

As soon as we started using this generated code we noticed that there were many JAXBElement and no @XmlRootElement, which is definitely not how we were used to working with JAXB, and tends to get in the way of clean code. There turns out to be a reason for this and a solution (for some elements):

Although this solution does not work for all elements (so we still need to do with the ObjectFactory and JAXBElement in some cases), it is still better than nothing. If you know the solution to this, do tell us please.

Now we can use all the goodness KML provides. In our case we want to show a shipping container whose geographical location changes over time. Our data model maps containers as a Resource type, which can be placed in other resources over a period of time represented by a Placement. Let us start by creating the KML document, which is actually a <kml> element containing a <document> element:

Now let us add all the placement locations by creating a KML <placemark> for each location. The interesting thing is that if we associate <timespan> element elements to placemarks Google Earth allows you to animate the placements in time.

Now we have a KML document which shows all our placements over time, but we would like to add some graphical representation of the trip of our container, so we will create a line between each placement as well:

And we are done.

Now that we know how to generate all this KML goodness we need to find a way to provide this KML to the user. Because we are using Seam and Java EE 5, there are a number of options to do this. One is to make a Web bean (actually really a Seam component bean) or a Servlet (which is pretty lame and low-level these days). We would map this service to a URL such as http://visiblelogistics.com/view/resource/AWC/Package1/placements.

Hey, hold on, this looks suspiciously like a RESTful URL…​ and we already have a RESTful API which gives us the list of placements for a resource at http://visiblelogistics.com/rest/resource/AWC/Package1/placements except that this URL serves application/xml content in an XML format suitable for our RESTful API.

If you think about it, it is the same list of placements we want to access, but with a different representation: application/vnd.google-earth.kml+xml. This is exactly what RESTful APIs are meant to handle: the data is associated with a URL, and the type of representation returned to the client depends on what the client asks for using the Accept HTTP header. In JAX-RS/RESTEasy this is as simple as defining this EJB interface:

The implementation of the EJB bean is of course left to the reader, but We have already given the full code for the KML generation and in our previous article on RESTEasy We have given the rest, so it is not that hard.

We can now open links such as http://visiblelogistics.com/rest/resource/AWC/Package1/placements in Google Earth, and because it will request the proper content-type, our RESTful API will serve the KML format instead of the plain XML format. This works great for Google Earth.

But the next thing we would like to do is insert links to this KML in the web frontend on http://visiblelogistics.com so that the user could click those links and Google Earth would open and display the cool visuals. The problem here is that the web browser will not request KML, so when the user clicks the link, the browser will get a plain XML from the RESTful server, and will display it instead of asking Google Earth to open it.

Expecting the representation format to be determined solely by the HTTP headers is foolish: HTML links cannot specify such headers. We have to resort to a common trickery which is add meaning to file extensions. For example we can say that if our URL ends with /placements it is plain XML whereas if it ends with /placements.kml we will return KML. A bit of a return to the olden days where file extensions had meaning, but not so irrelevant, and it does allow us to create links within our web frontend that, when clicked by the browser, will load the data in Google Earth because the KML content-type will be returned by the server.

We wanted to do a mashup right? Now that we can generate KML let us just include a Google Earth plugin in our application which displays the same KML inline. This is fairly easy as you just need to insert this in your web page:

This gives us a nice Google Earth plugin view:

The unfortunate thing when using unpopular operating systems is that we are always last to get the good stuff. The Google Earth plugin is not available for Linux or Mac OS yet, so all the eye-candy We have just added is just useless for clients on those operating systems.

In order to still show something useful we decided to use Google Maps when the Google Earth plugin is not available. Testing for this is fairly easy given the good API Google Earth has, and Google Maps has the ability to load KML files:

Unfortunately, as simple as this sounds, it does not work for us for a very good (and not obvious at all) reason: while the Google Earth plugin (and Google Earth) are full-fledged applications that go and fetch the KML from the network, parse it and translates it into something visual, Google Maps is a light JavaScript library which offloads the KML parsing and translation to the Google servers. This means that when we add a GeoXml overlay on Google Maps, the Google servers are going to fetch and translate the KML into some JavaScript which will then be loaded by the client to display the KML data.

This sounds great right? Except when the KML in question is protected by authentication. We do not want people to access our API without signing in, and we certainly do not want to give Google servers the VisibleLogistics user’s credentials so it can obtain the KML file. What we do want is that the client’s computer fetches the KML and shows it in Google Maps. And we are not alone to want such a client-side JavaScript KML parsing library.

Enter the GeoXML library. It adds client-side KML parsing to Google Maps and does a great job. You only need to declare a global variable where it will store the KML representation, and load the geoxml.js script after your own JavaScript.

With all this, we finally manage to have our Google Maps fallback for Linux and Mac OS users: