Safe Database Input/Output
Outside of using a decent framework, a lot of J2EE applications still handle their own DBIO. And just like memory resource leaks are a problem in C++, database resource leaks are a problem in virtually every J2EE application I’ve worked on.
But what do you do? You’ve got a legacy code base of several hundred thousand lines of code. Much of it written by (less than ideal) coders who have since left the project. There isn’t the money to fix every leak, even if you could find them all. And don’t even mention switching to a better framework. Just keep patching the worst offenders, right?
Well, I’m tired of that. So off the clock one weekend, I decided I needed a better solution. Something that would be low overhead, a complete solution, and most importantly could be implemented globally with nothing more than a simple sed script.
I call it SafeDBIO.
Abstract
Simply put, it’s garbage collection for database cursors. Keep track of the JDBC Statement and PreparedStatement objects, their ResultSets and other kin. Close them with worker threads when all references are lost. And create wrapper constructors so it’s easy to retrofit existing code. It’s all pretty basic stuff, but incredibly useful. To be honest, I don’t know why Oracle’s JDBC drivers don’t already do it for you.
The Code
There are two classes: SafeStatement and SafePreparedStatement. The code is long only because we have to wrap so many method calls from the JDBC drivers. But the guts are pretty simple. I’m going to deconstruct SafePreparedStatement here, for no other reason than to encourage people to use PreparedStatements wherever possible.
The Constructor
/**
* @param rps must not be null
*/
public SafePreparedStatement( PreparedStatement rps ) {
rps.getClass(); // check for null
// System.out.println("[SafePreparedStatement] creating "+ this +" for "+ rps);
this.rps = rps;
}
This is a wrapper class, so you pass it a PreparedStatement and it returns another PreparedStatement. Ideally you’ll perform a global replace on your code base (using sed or your IDE) to change:
java.sql.PreparedStatement ps = conn.prepareStatement(sql);into:
java.sql.PreparedStatement ps
= new org.kered.safeDBIO.SafePreparedStatement( conn.prepareStatement(sql) ); Note: I call rps.getClass() to throw a NullPointerException if we’re passed an invalid PreparedStatement. (standard fail-fast coding technique) Other than that, the constructor is uninteresting.
And to the point, wrapping your prepareStatement() calls should be the only code change necessary in your application.
Finalization
private boolean closedSafely = false;
/* (non-Javadoc)
* @see java.lang.Object#finalize()
*/
protected void finalize() throws Throwable {
if( !closedSafely ) {
try {
System.out.println("[SafePreparedStatement.finalize] closing "+ rps);
rps.close();
} catch( NullPointerException e ) {
System.out.println("[SafePreparedStatement.finalize] "+ e);
} catch( SQLException e ) {
System.out.println("[SafePreparedStatement.finalize] "+ e);
}
}
super.finalize();
}
When our SafePreparedStatement goes out of scope, eventually it will be picked up by the JVM’s garbage collection thread. In theory Java makes no guarantee that this method will be called. The VM can either never GC this object, or it can ignore destruction routines on application shutdown. But on exit, the Oracle drivers release open connections anyway. And in most VMs, the GC is run periodically. I don’t anticipate this being a problem, but please contact me if you find otherwise.
The finalize() method is simple. It just closes the root PreparedStatement if closedSafely is false. This variable is set to true if the calling code closes their object explicitly. Logging is optional.
Registered Return Objects
Closing a PreparedStatement when the object goes out of scope is dangerous. A developer can keep a ResultSet or Metadata object around a lot longer than the PreparedStatement that created it. You’re liable to introduce some very peculiar bugs into your system this way, especially if you’re like me and applying this to a large, weird code base. So every Object returned by the SafePreparedStatement is registered with the following method:
private final Map registeredReturnedObjects = Collections.synchronizedMap(new WeakHashMap());
private Object registerReturnedObject( Object o ) {
if( o!=null ) {
registeredReturnedObjects.put( o, null );
registeredSafePreparedStatements.add(this);
}
return o;
}
Note: Notice the call to registeredSafePreparedStatements.add(this). This is where SafePreparedStatements are registered to become ineligible for garbage collection. We’ll discuss this in a bit.
A WeakHashMap is used to allow the VM to clean these objects up as it normally would. As long as this Collection is not empty, we keep an internal reference to the SafePreparedStatement around to prevent it from being GCed before its time.
For example, the wrapped getResultSet() method:
/* (non-Javadoc)
* @see java.sql.Statement#getResultSet()
*/
public ResultSet getResultSet() throws SQLException {
return (ResultSet) registerReturnedObject( rps.getResultSet() );
}will return the same ResultSet object the root PreparedStatement would. But first it sticks a weak reference to it into our WeakHashMap. If you’re not familiar with weak references, they’re references that don’t prevent the Java garbage collector from cleaning up an object. (any strong reference will prevent the GC from cleaning an object) You never hold one directly; you always hold a strong reference to a WeakReference object which holds the weak reference for you. At any point before an object is GCed, you can request a strong reference back to the original object. WeakHashMaps have the unique feature of losing entries when their keys are GCed. If a SafePreparedStatement’s map is empty, it has no outstanding objects and is eligible for de-registration.
The De-registration Thread
Once we’ve registered a SafePreparedStatement (through the registration of one of its return objects), we need a way to unregister them. The following is our de-registration thread:
private static final Set registeredSafePreparedStatements
= Collections.synchronizedSet(new HashSet());
private static Thread deregistrationThread = new Thread() {
public void run() {
while( true ) {
for ( Iterator i=new HashSet(registeredSafePreparedStatements).iterator();
i.hasNext(); ) {
SafePreparedStatement sps = (SafePreparedStatement) i.next();
if( sps.registeredReturnedObjects.size()==0 ) {
registeredSafePreparedStatements.remove(sps);
}
}
try{ sleep(10000); }
catch( InterruptedException ie ) { /* ignore */ }
}
}
};
static {
deregistrationThread.setDaemon(true);
deregistrationThread.start();
} This code does three things. First, it provides a Set with which we can register SafePreparedStatements. (see the registerReturnedObject method above) This makes our object ineligible for garbage collection when it would be dangerous. Second, it creates a worker thread to periodically poll and unregister registered SafePrepareStatements (when they no longer have any registered return objects). Third, it sets a static init script to start the worker thread when the class is loaded. Got all that? 
If you’re not familiar with multi-threaded programming, you’re not alone. A disturbingly high number of programmers have no clue what they’re missing. If that’s you, this block is not going to make much sense. But explaining it is outside the scope of this article. I suggest this book.
Conclusion
Pretty simple, eh? Why is this not an optional part of Oracle’s JDBC drivers? Heck, if their source were available, I’d be able to embed this logic and not require any wrapper code at all. But that’s a whole other rant.
Future Work
There is still one gaping logic hole in this code. If red flags went off in your head before you hit the de-registration section, kudos to you. What happens if code using a SafePreparedStatement does the following?
- Gets a
ResultSet - Gets some object off that
ResultSet - Loses references to the
SarePreparedStatementand theResultSet - Tries to access the object they pulled out of the
ResultSet
If the object tries to do any DBIO, the cursor may all ready be closed causing it to fail. This can be rectified by simply extending the registration code sets to a theoretical SafeResultSet object, but there are so many methods to wrap it just didn’t seem worth it when I was finishing the SafePreparedStatement and SafeStatement objects. I’ve never come across code that does that, but it should be done anyway. But because we’ve already laid the groundwork, no project level changes will be necessary. All the wrapping can be done inside the SafeStatement and SafePreparedStatement objects. And this technique can be extended indefinitely for as many objects deep JDBC will go.
References
Download the finished code here: safeDBIO.tar.gz
All code is licensed under the GPL.
