Categories
Java Tomcat

Solution to “Tomcat can’t stop [Abandoned connection cleanup thread]”

I am preparing a web service for one of my larger projects, using Tomcat, Hibernate and MySQL. While developing I stumbled across several Tomcat error messages that suggest some threads could not be stopped. Most of the threads could simply be vanished by calling Hibernate’s SessionFactory.close() method. However, a few threads seem to remain, e.g.

Jul 09, 2014 10:55:40 AM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc
SEVERE: The web application [/examples] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to 
unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
Jul 09, 2014 10:55:40 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/examples] appears to have started a thread named [Abandoned connection cleanup 
thread] but has failed to stop it. This is very likely to create a memory leak.

and in fact, Tomcat stopped working after a few redeployments due to memory problems. Two main reasons:

  1. The MySQL AbandonedConnectionCleanupThread does not shutdown correctly.
  2. The MySQL J/Connector driver doesn’t unregister anymore from the DriverManager.

At least with J/Connector V5.1.31 the problem disappears by using this shutdown procedure (hooked in by a ServletContextListener):

1
2
3
4
5
6
7
8
9
10
11
12
13
   try {
      com.mysql.jdbc.AbandonedConnectionCleanupThread.shutdown();
   } catch (Throwable t) {}
   // This manually deregisters JDBC driver, which prevents Tomcat 7 from complaining about memory leaks
   Enumeration<java .sql.Driver> drivers = java.sql.DriverManager.getDrivers();
   while (drivers.hasMoreElements()) {
      java.sql.Driver driver = drivers.nextElement();
      try {
         java.sql.DriverManager.deregisterDriver(driver);
      } catch (Throwable t) {}
   }
   try { Thread.sleep(2000L); } catch (Exception e) {}
</java>

I still have to observe long-term effects and will keep you informed.

Categories
Java

Hibernate 4 with JTA Transactions

The following problem occurred when I was migrating from Hibernate 3 to Hibernate 4.2.3. Scrollable result sets were suddenly throwing exceptions:

org.hibernate.exception.GenericJDBCException: could not advance using next()
	at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:125)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:110)
	at org.hibernate.internal.ScrollableResultsImpl.next(ScrollableResultsImpl.java:121)
...
Caused by: java.sql.SQLException: Operation not allowed after ResultSet closed
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1078)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:975)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:920)
	at com.mysql.jdbc.ResultSetImpl.checkClosed(ResultSetImpl.java:804)
	at com.mysql.jdbc.ResultSetImpl.next(ResultSetImpl.java:6986)
	at org.hibernate.internal.ScrollableResultsImpl.next(ScrollableResultsImpl.java:116)

It turned out after 3 hours of debugging and research that Hibernate’s 4 JTA implementation relies on having a transaction started not via JTA User Transaction but through Hibernate’s own Session object. The correct configuration in such cases is simply to use CMTTransactionFactory as transaction factory class:

1
2
3
4
5
6
7
8
9
10
11
12
13
<hibernate-configuration>
   <session-factory>
      <!-- How to find the Transaction -->
      <property name="hibernate.transaction.factory_class">org.hibernate.engine.transaction.internal.jta.CMTTransactionFactory</property>
 
      <!-- How to produce transaction -->
      <property name="hibernate.transaction.jta.platform">org.hibernate.service.jta.platform.internal.JOTMJtaPlatform</property>
 
      <!-- Session context with JTA -->
      <property name="current_session_context_class">jta</property>
      ...
   </session-factory>
</hibernate-configuration>

As explanation: The error is primarily caused by a bug in JTA implementation within Hibernate (it does not synchronize Hibernate’s with JTA’s transaction object). It occurs mainly when you use the Entity Manager. In above’s exception, a second retrieval was executed within object iteration which caused Hibernate to think that there is no active transaction and hence, close any previous statements. That’s why the Scrollable Result loses its session.

The solution was published first by Koen Serneels. Many thanks!

Categories
Java RS Library

RS Library V1.1.0 released

I had to do some rework for the RS Library. The most important changes are:

  • RSLIB-19 – Improved DAO registration in abstract DaoFactory implementation
  • RSLIB-20 – Upgrade to Hibernate 4
  • RSLIB-22 – Provide modules as OSGI packages
  • RSLIB-23 – Cache Control for DAOs

A complete documentation can be found either through Javadocs or the appropriate module homepages:

The Maven coordinates are:

   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>baselib</artifactid>
      <version>1.1.0</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>data</artifactid>
      <version>1.1.0</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>data-hibernate</artifactid>
      <version>1.1.0</version>
   </dependency>