Home > java > Custom URL protocols and multiple classloaders

Custom URL protocols and multiple classloaders

December 12th, 2009

In the introduction to Spring’s Resources, the following is mentioned in reference to the standard java.net.URL:

While it is possible to register new handlers for specialized URL prefixes (similar to existing handlers for prefixes such as http:), this is generally quite complicated, and the URL interface still lacks some desirable functionality, such as a method to check for the existence of the resource being pointed to.

Whilst I don’t disagree on the need for a simpler resource abstraction, I do think that the statement misses the key point that the URL protocol extension mechanism does not work well in multiple classloader environments (e.g. Webapps and OSGI bundles), hence I thought I would write this blog entry to explain:

  • The way custom URL protocol handlers are registered.
  • Why the extension mechanism does not work well with multiple classloaders.

Registering a custom URL protocol handler

To register a custom URL protocol handler the following needs to be done:

  1. Provide a concrete implementation for URLStreamHandler.
  2. Provide a concrete implementation for URLConnection.
  3. Name your URLStreamHandler class Handler and put it in a special package.

The following code illustrates a protocol called user, which will treat all file paths relative to the user’s home directory:

 
package sun.net.www.protocol.user;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.net.URLConnection;
import java.net.URLStreamHandler;
public class Handler extends URLStreamHandler {
  @Override
  protected URLConnection openConnection(URL url) throws IOException {
    return new UserURLConnection(url);
  }
  private static class UserURLConnection extends URLConnection {
    private String fileName;
    public UserURLConnection(URL url) {
      super(url);
      fileName = url.getPath();
    }
    @Override
    public void connect() throws IOException {
    }
    @Override
    public InputStream getInputStream() throws IOException {
      File absolutePath = new File(System.getProperty("user.home"), fileName);
      return new FileInputStream(absolutePath);
    }
  }
}

Note that the package is sun.net.www.protocol.user and the class is called Handler. The extension mechanism requires you to call your class Handler and put it in a subpackage that is your protocol name. The parent package can be arbitrary however by default sun.net.www.protocol will be searched. If you wish to use your own parent package you must also provide a system property called java.protocol.handler.pkgs which lists your package name (e.g. uk.co.cooljeff.protocol.user).

The following test case illustrates how the customer user protocol can be used:

 
public class URLTest {
  @Test
  public void testUserProtocol() throws Exception {
    URL url = new URL("user:.profile");
    InputStream ins = url.openStream();
    assertNotNull(ins);
    ins.close();
  }
}

This is all that needs to be done which I found pretty simple to do. The hardest bit was finding the documentation. Unfortunately the documentation for the extension mechanism is found on one of the java.net.URL constructors and not in the class level documentation for URL or the package documentation for java.net.

Multiple classloaders and custom URL protocol handlers

The custom user protocol described in the previous section works without any issue in an Eclipse project or in a vanilla standalone application. However if I were to supply my custom protocol in a web app deployed on tomcat, the following exception is likely to have been seen:

 
Caused by: java.net.MalformedURLException: unknown protocol: user
	at java.net.URL.(URL.java:574)
	at java.net.URL.(URL.java:464)
	at java.net.URL.(URL.java:413)
	at uk.co.cooljeff.protocol.user.URLTest.testUserProtocol(URLTest.java:12)
	. . . 5 more

To understand why, take a look at the logic used to locate custom protocols (found in URL.getURLStreamHandler(String)):

 
static URLStreamHandler getURLStreamHandler(String protocol) {
. . .
    String clsName = packagePrefix + "." + protocol + ".Handler";
    Class cls = null;
    try {
      cls = Class.forName(clsName);
    } catch (ClassNotFoundException e) {
      ClassLoader cl = ClassLoader.getSystemClassLoader();
      if (cl != null) {
        cls = cl.loadClass(clsName);
      }
    }
    if (cls != null) {
      handler  = (URLStreamHandler)cls.newInstance();
    }
. . .

You will notice that Class.forName() is used. The issue here is that the classloader that will be used is the classloader of the calling class. The calling class in this instance is URL. Hence if URL is loaded in a parent classloader of where the custom protocol is, it won’t be found. This is why custom URL protocols have to be put on the web containers classpath rather than being bundled in the web app that uses it.

Conclusion

The URL protocol extension mechanism is fundamentally flawed when it comes to multi classloader environments such as OSGI and Webapp containers. The registration mechanism does not work because in such environments, the URL class itself is likely to be in a parent classloader of the custom protocol, resulting in the custom protocol not being visible to the registration logic. In addition there is no standardization for the default package structure for where additional protocols should be found. This leads to either a non portable solution across JVMs or the need to use system properties which are not that friendly in enterprise web app containers.

Resources

java

  1. December 13th, 2009 at 02:13 | #1

    You might want to look at the URL Handlers Service (chapter 11 of the OSGi core specification http://www.osgi.org/download/r4v42/r4.core.pdf)

    It defines a service-based approach for registering URL handlers in a modular, dynamic application. The basic idea is that you can register an implementation of URLStreamHandlerService with the OSGi service registry – the various service implementations are then discovered by the real (singleton) URLStreamHandler that’s registered by the OSGi framework. The framework handler is responsible for sending incoming requests to the right handler service.

    You can find a concrete example over at http://wiki.ops4j.org/display/paxurl/Pax+URL which also works in a standard Java environment, except without the dynamic part.

    Of course, there are some wrinkles (mostly when embedding frameworks inside other containers) but most OSGi frameworks can also support this by using reflection to chain to existing handler factories.

  2. admin
    December 13th, 2009 at 13:02 | #2

    Stuart, thanks for this, I was not fully aware of how OSGI works around the various issues and Chapter 11 does a good job at explaining it.

  1. August 12th, 2014 at 11:48 | #1
Comments are closed.