Tuesday, December 29, 2009

Dynamic class loader with URLClassLoader and howto unload class and delete it's jar file

Almost any java plugable solutions developer meet the problem "if you dynamically load jar with URLClassLoader and then unload it, there is no possibility to delete this jar under Windows until stopping/restarting JVM".

Here is a more detail problem description.
Some class loader is used to load class from externally plugable jar file. URLClassLoader is a most funtional thing here (provide a way to load classes not only from local place and jars). Short example:




ClassLoader clazzLoaderOld = Thread.currentThread().getContextClassLoader()
URL url0 = new File(filePath0).toURL();
URL url1 = new File(filePath1).toURL();
clazzLoader = new URLClassLoader(new URL[]{url0, url1}); // (1)
Thread.currentThread().setContextClassLoader(clazzLoader);
// here possible to use all classes from our class loader paths
// ...
Thread.currentThread().setContextClassLoader(clazzLoaderOld);




Remark: org.eclipse.jdt.apt.core.internal.JarClassLoader has no such problems, but this class loader has not all URLClassLoader functionality...

So, now our needs is unload useless classes. To do this just necessary nil all URLClassLoader (1) references. Here is only one:




clazzLoader = null;




Now should be possible to delete filePath jar or replace it with new one jar file,
but Windows tells what this is not possible cause file is opened/locked by some other process...

Here is example: "JBoss Tools -> Failed to delete project"
one more: "URLClassloader locks jars"
other: "Classloader won't let me delete jar file!"

It seems like Sun JDK developers do not use FILE_SHARE_DELETE flag when open jar with Windows native api...

So others in it's turn should looking for workarounds for this.
I wrote some big code snippet firstly



public void close() {
try {
Class clazz = java.net.URLClassLoader.class;
java.lang.reflect.Field ucp = clazz.getDeclaredField("ucp");
ucp.setAccessible(true);
Object sun_misc_URLClassPath = ucp.get(this);
java.lang.reflect.Field loaders =
sun_misc_URLClassPath.getClass().getDeclaredField("loaders");
loaders.setAccessible(true);
Object java_util_Collection = loaders.get(sun_misc_URLClassPath);
for (Object sun_misc_URLClassPath_JarLoader :
((java.util.Collection) java_util_Collection).toArray()) {
try {
java.lang.reflect.Field loader =
sun_misc_URLClassPath_JarLoader.getClass().getDeclaredField("jar");
loader.setAccessible(true);
Object java_util_jar_JarFile =
loader.get(sun_misc_URLClassPath_JarLoader);
((java.util.jar.JarFile) java_util_jar_JarFile).close();
} catch (Throwable t) {
// if we got this far, this is probably not a JAR loader so skip it
}
}
} catch (Throwable t) {
// probably not a SUN VM
}
return;
}





and then find exactly the same solution here:
"Classloaders Keeping Jar Files Open"

I must say "Thank you" to John Mazz for inspiration! Unfortantly, this doesn't work in my case... it was not possible to delete mysql-connector-java-5.0.7-bin.jar after all these...

As I guess not all references to JarFile were killed. One more "hack" was necessary.
Here it is:




/**
* cleanup jar file factory cache
*/
@SuppressWarnings({ "nls", "unchecked" })
public boolean cleanupJarFileFactory()
{
boolean res = false;
Class classJarURLConnection = null;
try {
classJarURLConnection = ReflectHelper.classForName("sun.net.www.protocol.jar.JarURLConnection");
} catch (ClassNotFoundException e) {
//ignore
}
if (classJarURLConnection == null) {
return res;
}
Field f = null;
try {
f = classJarURLConnection.getDeclaredField("factory");
} catch (NoSuchFieldException e) {
//ignore
}
if (f == null) {
return res;
}
f.setAccessible(true);
Object obj = null;
try {
obj = f.get(null);
} catch (IllegalAccessException e) {
//ignore
}
if (obj == null) {
return res;
}
Class classJarFileFactory = obj.getClass();
//
HashMap fileCache = null;
try {
f = classJarFileFactory.getDeclaredField("fileCache");
f.setAccessible(true);
obj = f.get(null);
if (obj instanceof HashMap) {
fileCache = (HashMap)obj;
}
} catch (NoSuchFieldException e) {
} catch (IllegalAccessException e) {
//ignore
}
HashMap urlCache = null;
try {
f = classJarFileFactory.getDeclaredField("urlCache");
f.setAccessible(true);
obj = f.get(null);
if (obj instanceof HashMap) {
urlCache = (HashMap)obj;
}
} catch (NoSuchFieldException e) {
} catch (IllegalAccessException e) {
//ignore
}
if (urlCache != null) {
HashMap urlCacheTmp = (HashMap)urlCache.clone();
Iterator it = urlCacheTmp.keySet().iterator();
while (it.hasNext()) {
obj = it.next();
if (!(obj instanceof JarFile)) {
continue;
}
JarFile jarFile = (JarFile)obj;
if (setJarFileNames2Close.contains(jarFile.getName())) {
try {
jarFile.close();
} catch (IOException e) {
//ignore
}
if (fileCache != null) {
fileCache.remove(urlCache.get(jarFile));
}
urlCache.remove(jarFile);
}
}
res = true;
} else if (fileCache != null) {
// urlCache := null
HashMap fileCacheTmp = (HashMap)fileCache.clone();
Iterator it = fileCacheTmp.keySet().iterator();
while (it.hasNext()) {
Object key = it.next();
obj = fileCache.get(key);
if (!(obj instanceof JarFile)) {
continue;
}
JarFile jarFile = (JarFile)obj;
if (setJarFileNames2Close.contains(jarFile.getName())) {
try {
jarFile.close();
} catch (IOException e) {
//ignore
}
fileCache.remove(key);
}
}
res = true;
}
setJarFileNames2Close.clear();
return res;
}





The whole redefined class loder code is here:

http://snipplr.com/view/24224/class-loader-which-close-opened-jar-files/

This code snippet is a part of Hibernate Tools - open source project, part of JBoss Studio for Java developer who uses Eclipse as IDE.

3 comments:

John Mazz said...

It is such a shame that the JDK forces us to do this kind of thing.

I do not know why the Java API does not have some kind of "close" or "destroy" method on the ClassLoader class to allow users of classloader instances to indicate when they should clean up. It seems to me it would be so easy to fix this if the Java implementations would provide a true "close()" method to help clean up classloaders.

Vitali Yemialyanchyk said...

look like Java 7 will has such method, but do not sure it will fix the problem with locked jars...

Rovo said...

I'm not sure if this problem is related only to URLClassLoader or to a Java 6 version prior u20 which I have currently installed, but I managed to delete jar files after unloading the ClassLoader (I extended ClassLoader rather then URLClassLoader and did the URL-stuff inside myself).

My tests succeeded even with replacing a "locked" jar-file with a new one while running my simple application. By locked I mean a jar-file which was loaded by my ClassLoader and therefore couldn't get deleted at runtime.

In Linux deleting loaded jars isn't any problem at all during runtime, on windows I'm allowed to delete jar-files only when the ClassLoader was removed by the GarbageCollector.

As I'm currently experimenting with my simple framework I might implement somekind of service-registry (think of something like osgi has) which only returns weak-references of services. that way the only strong reference to the class should be via the ClassLoader. And if this one gets removed, all weak references are considered not or only weak reachable and get garbagecollected too.

this way assuring that no reference to any class within a jar-file exists should be possible, which leads to unloadable jars even in heavy dependency situations.