Custom URL protocols and multiple classloaders
In the introduction to Spring’s Resources, the following is mentioned in reference to the standard java.net.URL:
While it is possible to register new handlers for specialized URL prefixes (similar to existing handlers for prefixes such as http:), this is generally quite complicated, and the URL interface still lacks some desirable functionality, such as a method to check for the existence of the resource being pointed to.
Whilst I don’t disagree on the need for a simpler resource abstraction, I do think that the statement misses the key point that the URL protocol extension mechanism does not work well in multiple classloader environments (e.g. Webapps and OSGI bundles), hence I thought I would write this blog entry to explain:
- The way custom URL protocol handlers are registered.
- Why the extension mechanism does not work well with multiple classloaders.
Registering a custom URL protocol handler
To register a custom URL protocol handler the following needs to be done:
- Provide a concrete implementation for URLStreamHandler.
- Provide a concrete implementation for URLConnection.
- Name your URLStreamHandler class Handler and put it in a special package.
The following code illustrates a protocol called user, which will treat all file paths relative to the user’s home directory:
package sun.net.www.protocol.user; import java.io.File; import java.io.FileInputStream; import java.io.IOException; import java.io.InputStream; import java.net.URL; import java.net.URLConnection; import java.net.URLStreamHandler; public class Handler extends URLStreamHandler { @Override protected URLConnection openConnection(URL url) throws IOException { return new UserURLConnection(url); } private static class UserURLConnection extends URLConnection { private String fileName; public UserURLConnection(URL url) { super(url); fileName = url.getPath(); } @Override public void connect() throws IOException { } @Override public InputStream getInputStream() throws IOException { File absolutePath = new File(System.getProperty("user.home"), fileName); return new FileInputStream(absolutePath); } } }
Note that the package is sun.net.www.protocol.user and the class is called Handler. The extension mechanism requires you to call your class Handler and put it in a subpackage that is your protocol name. The parent package can be arbitrary however by default sun.net.www.protocol will be searched. If you wish to use your own parent package you must also provide a system property called java.protocol.handler.pkgs which lists your package name (e.g. uk.co.cooljeff.protocol.user).
The following test case illustrates how the customer user protocol can be used:
public class URLTest { @Test public void testUserProtocol() throws Exception { URL url = new URL("user:.profile"); InputStream ins = url.openStream(); assertNotNull(ins); ins.close(); } }
This is all that needs to be done which I found pretty simple to do. The hardest bit was finding the documentation. Unfortunately the documentation for the extension mechanism is found on one of the java.net.URL constructors and not in the class level documentation for URL or the package documentation for java.net.
Multiple classloaders and custom URL protocol handlers
The custom user protocol described in the previous section works without any issue in an Eclipse project or in a vanilla standalone application. However if I were to supply my custom protocol in a web app deployed on tomcat, the following exception is likely to have been seen:
Caused by: java.net.MalformedURLException: unknown protocol: user at java.net.URL.(URL.java:574) at java.net.URL.(URL.java:464) at java.net.URL.(URL.java:413) at uk.co.cooljeff.protocol.user.URLTest.testUserProtocol(URLTest.java:12) . . . 5 more
To understand why, take a look at the logic used to locate custom protocols (found in URL.getURLStreamHandler(String)):
static URLStreamHandler getURLStreamHandler(String protocol) {
. . .
String clsName = packagePrefix + "." + protocol + ".Handler";
Class cls = null;
try {
cls = Class.forName(clsName);
} catch (ClassNotFoundException e) {
ClassLoader cl = ClassLoader.getSystemClassLoader();
if (cl != null) {
cls = cl.loadClass(clsName);
}
}
if (cls != null) {
handler = (URLStreamHandler)cls.newInstance();
}
. . .
You will notice that Class.forName() is used. The issue here is that the classloader that will be used is the classloader of the calling class. The calling class in this instance is URL. Hence if URL is loaded in a parent classloader of where the custom protocol is, it won’t be found. This is why custom URL protocols have to be put on the web containers classpath rather than being bundled in the web app that uses it.
Conclusion
The URL protocol extension mechanism is fundamentally flawed when it comes to multi classloader environments such as OSGI and Webapp containers. The registration mechanism does not work because in such environments, the URL class itself is likely to be in a parent classloader of the custom protocol, resulting in the custom protocol not being visible to the registration logic. In addition there is no standardization for the default package structure for where additional protocols should be found. This leads to either a non portable solution across JVMs or the need to use system properties which are not that friendly in enterprise web app containers.
