Is this very likely to create a memory leak in Tomcat?

I configured tomcat to work with a different external open source.

However, after the tomcat is running for a few minutes I get:

SEVERE: The web application [/MyProject] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1b3f02f]) and a value of type [org.apache.axis.MessageContext] (value [org.apache.axis.MessageContext@5dbd4e]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.

What could cause it?

Where do I have to look? Could it be datapooling on Tomcat?

And what does it mean Threads in Tomcat?

EDITED

Here is my full trace. The application seems to reloads its context while it's still running - and I don't know why!

Mar 13, 2011 10:56:12 PM org.apache.catalina.core.StandardContext reload
INFO: Reloading this Context has started
Mar 13, 2011 10:56:12 PM org.apache.catalina.core.StandardWrapper unload
INFO: Waiting for 1 instance(s) to be deallocated
Mar 13, 2011 10:56:13 PM org.apache.catalina.core.StandardWrapper unload
INFO: Waiting for 1 instance(s) to be deallocated
Mar 13, 2011 10:56:14 PM org.apache.catalina.core.StandardWrapper unload
INFO: Waiting for 1 instance(s) to be deallocated
Mar 13, 2011 10:56:14 PM org.apache.catalina.core.ApplicationContext log
INFO: Closing Spring root WebApplicationContext
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc
SEVERE: The web application [/MyProject] registered the JBDC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc
SEVERE: The web application [/MyProject] registered the JBDC driver [oracle.jdbc.driver.OracleDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/MyProject] appears to have started a thread named [NioSocketAcceptor-1] but has failed to stop it. This is very likely to create a memory leak.
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/MyProject] appears to have started a thread named [NioProcessor-1] but has failed to stop it. This is very likely to create a memory leak.
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/MyProject] appears to have started a thread named [NioProcessor-4] but has failed to stop it. This is very likely to create a memory leak.
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/MyProject] appears to have started a thread named [bitronix-disk-force-batcher] but has failed to stop it. This is very likely to create a memory leak.
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/MyProject] appears to have started a thread named [bitronix-scheduler] but has failed to stop it. This is very likely to create a memory leak.
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/MyProject] is still processing a request that has yet to finish. This is very likely to create a memory leak. You can control the time allowed for requests to finish by using the unloadDelay attribute of the standard Context implementation.
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/MyProject] appears to have started a thread named [NioProcessor-7] but has failed to stop it. This is very likely to create a memory leak.
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/MyProject] appears to have started a thread named [NioProcessor-2] but has failed to stop it. This is very likely to create a memory leak.
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/MyProject] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1b5a8e1]) and a value of type [org.mvel2.debug.DebuggerContext] (value [org.mvel2.debug.DebuggerContext@16259fd]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/MyProject] created a ThreadLocal with key of type [org.apache.axis.utils.XMLUtils.ThreadLocalDocumentBuilder] (value [org.apache.axis.utils.XMLUtils$ThreadLocalDocumentBuilder@84b0b4]) and a value of type [com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl] (value [com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl@16d2cfa]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/MyProject] created a ThreadLocal with key of type [null] (value [com.sun.faces.util.Util$1@16bbac9]) and a value of type [java.util.HashMap] (value [{com.sun.faces.patternCache={ = }}]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/MyProject] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1b3f02f]) and a value of type [org.apache.axis.MessageContext] (value [org.apache.axis.MessageContext@5dbd4e]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/MyProject] created a ThreadLocal with key of type [org.apache.axis.utils.XMLUtils.ThreadLocalDocumentBuilder] (value [org.apache.axis.utils.XMLUtils$ThreadLocalDocumentBuilder@84b0b4]) and a value of type [com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl] (value [com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl@378584]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/MyProject] created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Transactional resources]) and a value of type [java.util.HashMap] (value [{org.hibernate.impl.SessionFactoryImpl@ccc27b=org.springframework.orm.hibernate3.SessionHolder@4f6ada}]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: The web application [/MyProject] created a ThreadLocal with key of type [null] (value [com.sun.faces.application.ApplicationAssociate$1@1f01fcf]) and a value of type [com.sun.faces.application.ApplicationAssociate] (value [com.sun.faces.application.ApplicationAssociate@1b85528]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
2011-03-13 22:57:27,734 ERROR (            ContextLoader.java:220)     - Context initialization failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionManager' defined in class path resource [applicationContext-hibernate.xml]: Cannot resolve reference to bean 'sessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory' defined in class path resource [applicationContext-hibernate.xml]: Invocation of init method failed; nested exception is java.lang.OutOfMemoryError: Java heap space
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:106)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1325)

Solution 1:

The message is actually pretty clear: something creates a ThreadLocal with value of type org.apache.axis.MessageContext - this is a great hint. It most likely means that Apache Axis framework forgot/failed to cleanup after itself. The same problem occurred for instance in Logback. You shouldn't bother much, but reporting a bug to Axis team might be a good idea.

Tomcat reports this error because the ThreadLocals are created per HTTP worker threads. Your application is undeployed but HTTP threads remain - and these ThreadLocals as well. This may lead to memory leaks (org.apache.axis.MessageContext can't be unloaded) and some issues when these threads are reused in the future.

For details see: http://wiki.apache.org/tomcat/MemoryLeakProtection

Solution 2:

I added the following to @PreDestroy method in my CDI @ApplicationScoped bean, and when I shutdown TomEE 1.6.0 (tomcat7.0.39, as of today), it clears the thread locals.

/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */
package pf;

import java.lang.ref.WeakReference;
import java.lang.reflect.Array;
import java.lang.reflect.Field;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 *
 * @author Administrator
 * 
 * google-gson issue # 402: Memory Leak in web application; comment # 25
 * https://code.google.com/p/google-gson/issues/detail?id=402
 */
public class ThreadLocalImmolater {

    final Logger logger = LoggerFactory.getLogger(ThreadLocalImmolater.class);

    Boolean debug;

    public ThreadLocalImmolater() {
        debug = true;
    }

    public Integer immolate() {
        int count = 0;
        try {
            final Field threadLocalsField = Thread.class.getDeclaredField("threadLocals");
            threadLocalsField.setAccessible(true);
            final Field inheritableThreadLocalsField = Thread.class.getDeclaredField("inheritableThreadLocals");
            inheritableThreadLocalsField.setAccessible(true);
            for (final Thread thread : Thread.getAllStackTraces().keySet()) {
                    count += clear(threadLocalsField.get(thread));
                    count += clear(inheritableThreadLocalsField.get(thread));
            }
            logger.info("immolated " + count + " values in ThreadLocals");
        } catch (Exception e) {
            throw new Error("ThreadLocalImmolater.immolate()", e);
        }
        return count;
    }

    private int clear(final Object threadLocalMap) throws Exception {
        if (threadLocalMap == null)
                return 0;
        int count = 0;
        final Field tableField = threadLocalMap.getClass().getDeclaredField("table");
        tableField.setAccessible(true);
        final Object table = tableField.get(threadLocalMap);
        for (int i = 0, length = Array.getLength(table); i < length; ++i) {
            final Object entry = Array.get(table, i);
            if (entry != null) {
                final Object threadLocal = ((WeakReference)entry).get();
                if (threadLocal != null) {
                    log(i, threadLocal);
                    Array.set(table, i, null);
                    ++count;
                }
            }
        }
        return count;
    }

    private void log(int i, final Object threadLocal) {
        if (!debug) {
            return;
        }
        if (threadLocal.getClass() != null &&
            threadLocal.getClass().getEnclosingClass() != null &&
            threadLocal.getClass().getEnclosingClass().getName() != null) {

            logger.info("threadLocalMap(" + i + "): " +
                        threadLocal.getClass().getEnclosingClass().getName());
        }
        else if (threadLocal.getClass() != null &&
                 threadLocal.getClass().getName() != null) {
            logger.info("threadLocalMap(" + i + "): " + threadLocal.getClass().getName());
        }
        else {
            logger.info("threadLocalMap(" + i + "): cannot identify threadlocal class name");
        }
    }

}

Solution 3:

The key "Transactional Resources" looks like you are talking to the database without a proper transaction. Make sure transaction management is configured properly and no invocation path to the DAO exists that doesn't run under a @Transactional annotation. This can easily happen when you configured transaction management on the Controller level but are invoking DAOs in a timer or are using @PostConstruct annotations. I wrote it up here http://georgovassilis.blogspot.nl/2014/01/tomcat-spring-and-memory-leaks-when.html

Edit: It looks like this is (also?) a bug with spring-data-jpa which has been fixed with v1.4.3. I looked it up in the spring-data-jpa sources of LockModeRepositoryPostProcessor which sets the "Transactional Resources" key. In 1.4.3 it also clears the key again.

Solution 4:

Sometimes this has to do with configuration changes. When we upgraded from Tomncat 6.0.14 to 6.0.26, we had seen something similar. here is the solution http://www.skill-guru.com/blog/2010/08/22/tomcat-6-0-26-shutdown-reports-a-web-application-created-a-threadlocal-threadlocal-has-been-forcibly-removed/