What causes PermGen OutOfMemoryError on JBoss?
What is the underlying cause of a PermGen OutOfMemoryError in JBoss?
I am running JBoss AS 4.2.2 in my development environment, and this occurs after redeploying my web application a large number of times.
Christian Vest Hansen's blog gives JVM options that help a lot, but do not solve the problem completely:
-XX:+UseConcMarkSweepGC
-XX:+CMSPermGenSweepingEnabled
-XX:+CMSClassUnloadingEnabled
-XX:MaxPermSize=128m
As already mentioned, you're probably experiencing leaking class loaders. For some reason your classes are not being unloaded. This can happen for two reasons
- Objects of said classes still exist on the heap, objects always reference their class, or
- The class loader is referenced somewhere, for whatever reason, class loaders reference their classes in order to not load them twice
There's no catch-all solution to this problem. One helpful tool to help you find the root cause is the Eclipse Memory Analyzer Tool, which you can apply to a heap dump from your JVM (you can enable heap dumps on OOMs with the -XX:+HeapDumpOnOutOfMemoryError option). Perhaps start looking for java.lang.Class objects from your web app to see why they are being kept alive. Unfortunately, PermGen is not normally part of a JVM heap dump, so you can only try to find correlating artifacts in the rest of the heap (Class-objects are not stored in PermGen if im not mistaken, only the actual byte code is, please correct me if I'm wrong though).
HTH.
Edit:
Dave Cheney suggests in a comment that java.lang.Class-objects are indeed part of PermGen, and not included in a normal hotspot heap dump. Unless you have a JVM which writes this info in the heap dump, you'll need a different approach. You can still look for instances of your objects, but if you're leaking classes/class loaders (they both unfortunately imply each other) it seems you need to look for other signs (meta data objects from JBoss, etc).
The underlying cause is references to classes that have been discarded leaking outside their classloader, preventing the JVM from unloading those classes from the perm gen. Those flags you use may cause the JVM to aggressively purge classes that are unloadable, but it will not solve the underlying problem.
There is a good, abeit complex explanation here
The cause of the PermGen OutOfMemory error is the app redeploys. The underlying cause is leaked Class objects in PermGen from the redeploys.
Of course, the workaround is to restart the JVM after a certain number of redeploys.
This is a very difficult problem to totally solve, although with some sleuthing you can often make big improvements. Here's where you start: When your web app is stopped, make sure that:
- all Threads that you started are stopped
- all ThreadPools that you started are shut down
- all static references that you can release are released
These are some of the things that can cause a Class object to be trapped in PermGen.
Also, note that not all JVMs (or all versions of JVMs) will GC Class objects in PermGen. If you are running a JVM or a version of a JVM that will not GC Class objects in PermGen, then your only choice is to restart the JVM after a certain number of redeploys. This probably does not apply to you, given the JVM options you mention.