IOC without an IOC-container

Spring is my framework of choice if I need to write enterprise applications. But in some occasions using Spring (or more dressed down IOC containers) is not an option, because:

1. it is too heavy/big. In some (rare) cases every byte counts
2. you can’t introduce new dependencies.
3. all kinds of political reasons

A practical example of such an strange application is a javaagent. I’m working on one for the ‘Concurrency Detector’ and javaagents typically are small. And introducing extra dependencies can introduce unforeseen side-effects, e.g. I had a problem with using AspectJ in my Javaagent in combination with AspectJ in a webapplication. The side effect was that my class transformations totally got bypassed. That is one of the main why I dropped AspectJ for the concurrency detector and decided to use ASM instead.

So what if you can’t use an IOC container, but do want to use IOC. Simple: just create one (perhaps more) class that glues all components together, just like in the Spring container. This class can know of all specific implementations, policies/strategies, initializations etc etc. All the stuff you normally do in a Spring container you can do in this class. The components of your system don’t need to be aware of the IOC-container they are running in, and that is how I like my OO design.

Letting the garbage collector do callbacks

Garbage collection in Java is great, but in some cases you want to listen when an object is garbage collected. In my case I had the following scenario:

1. I have a throw away key (a composition of objects) and a value (statistics) in some map.
2. When one of the components of the key is garbage collected, the key and value should be removed from the map.

The java.util.WeakHashMap was the first thing that came to mind, but there are some problems:

1. It is not thread safe. And wrapping it in a synchronized block is not acceptable for scalability reasons.
2. The key (a WeakReference containing the real key) is removed when the real key has been garbage collected. But in my case this isn’t going to work. I don’t want my throw-away key to be removed when it is garbage collected because it would be garbage collected immediately since nobody but the WeakReference is holding a reference to it.

So I needed to go a step deeper. How can I listen to the garbage collection on an object? This can be done using a java.lang.ref.Reference (java.lang.ref.WeakReference for example) and a java.lang.ref.ReferenceQueue. When the object inside the WeakReference is garbage collected, the WeakReference is put on a ReferenceQueue. This makes it possible to do some cleanup by taking items from this ReferenceQueue. This is the first big step to listen to garbage collection.

The other two constraints are now easy to solve:

1. thread safety: use a ConcurrentHashMap implementation
2. listen to an arbitrary object instead of the key: wrap the object inside a WeakReference and store the key & map inside. If you create a thread that consumes these references from the ReferenceQueue, this thread is now able to remove the key (and value) from the map

This weekend I created a proof of concept implementation and it can be found bellow.

public class GarbageCollectingConcurrentMap<K, V> {

    private final static ReferenceQueue referenceQueue = new ReferenceQueue();

    static {
        new CleanupThread().start();
    }

    private final ConcurrentMap<K, GarbageReference<K, V>> map = new ConcurrentHashMap<K, GarbageReference<K, V>>();

    public void clear() {
        map.clear();
    }

    public V get(K key) {
        GarbageReference<K, V> ref = map.get(key);
        return ref == null ? null : ref.value;
    }

    public Object getGarbageObject(K key){
        GarbageReference<K,V> ref=map.get(key);
        return ref == null ? null : ref.get();
    }

    public Collection<K> keySet() {
        return map.keySet();
    }

    public void put(K key, V value, Object garbageObject) {
        if (key == null || value == null || garbageObject == null) throw new NullPointerException();
        if (key == garbageObject)
            throw new IllegalArgumentException("key can't be equal to garbageObject for gc to work");
        if (value == garbageObject)
            throw new IllegalArgumentException("value can't be equal to garbageObject for gc to work");

        GarbageReference reference = new GarbageReference(garbageObject, key, value, map);
        map.put(key, reference);
    }

    static class GarbageReference<K, V> extends WeakReference {
        final K key;
        final V value;
        final ConcurrentMap<K, V> map;

        GarbageReference(Object referent, K key, V value, ConcurrentMap<K, V> map) {
            super(referent, referenceQueue);
            this.key = key;
            this.value = value;
            this.map = map;
        }
    }

    static class CleanupThread extends Thread {
        CleanupThread() {
            setPriority(Thread.MAX_PRIORITY);
            setName("GarbageCollectingConcurrentMap-cleanupthread");
            setDaemon(true);
        }

        public void run() {
            while (true) {
                try {
                    GarbageReference ref = (GarbageReference) referenceQueue.remove();
                    while (true) {
                        ref.map.remove(ref.key);
                        ref = (GarbageReference) referenceQueue.remove();
                    }
                } catch (InterruptedException e) {
                    //ignore
                }
            }
        }
    }
}

I am now able to run my concurrency detector on large projects like JBoss instead of running out of memory.

Now officially a Xitaan

Last 5 months I haven’t done a lot of software development for my work. I was given the chance to do more consultancy related tasks like performance/stability trouble shooting and audits. I could never have imagined it, but I lost my passion for ‘just’ writing software. I still do write software at home. At the moment I’m working on a tool that is able to detect which objects are accessed by multiple threads. One of the main challenges now is to replace AspectJ by a custom Java agent that gives me more flexibility and doesn’t interfere with other AspectJ configurations. But I’m not satisfied working on projects anymore; I was an unhappy developer and continuing on this road doesn’t provide a bright future.

A few weeks ago I was asked to make the switch officially from the development department (XSD) to the consultancy department (XITA) and I agreed:

1. XSD (Xebia Software Development) does projects
2. XITA (Xebia IT Architects): does more consultancy related tasks like audits, trouble shooting, advice etc.
3. XBR (Xebia Business Requirements): does a lot on the business side of IT like changing the development process or giving business advice.

Within XITA I want to continue specializing in everything related to concurrency and distributed computing. Also I’m thinking about speaking at JFall (a Dutch Java conference) about the Java Memory Model and I’m planning to give an internal? course on this subject as well.

Detecting Concurrency II: an Example

Last week I wrote a blogpost about detecting multithreaded access on objects. To make it a little bit less abstract, you can find a report here (sorry for the lame file sharing mechanism). The report is created by running the tool on Apache Tomcat (a project with some concurrency problems).

In the first section you can find which fields are touched by multiple threads, and a few other statistics like the maximum number of read/writes by a single thread, and the maximum total number of read/writes by all threads on some instance. In the second report section you can see which instances were touched by which threads, and you can also see the overlapping fields (so fields touched by 2 or more threads).

4 June my first public presentation in Haren (Groningen)

My employer Xebia has organized a knowledge sharing session in Haren (Groningen, The Netherlands) on the 4th of June (starting at 17:00u) to get some exposure in the north of The Netherlands. The evening starts with a presentation about the Agile process (one of the main drivers within Xebia). But to show that we are not just about process, but also about hardcore Java, I’m giving a presentation about the Java Memory Model (JMM). After the presentations we finish with a bbq.

The last few months I have been increasing the frequency of doing internal presentations (twice a month we have a knowledge sharing evening) and in most (probably all) cases I have been talking about concurrency control in some form. In the autumn I want to speak at JFall, a Dutch Java conference, and I expect the subject to be the JMM as well. Speaking at this knowledge sharing evening is a good training for speaking to a larger public.

So if you want hear something about the JMM, the Agile process or do some networking, send me an email and I’ll try to get you on the guest list.

The concurrency detector

One of the things I love to do is to check code for concurrency issues (for example in audits). If a system is aware of concurrency from day one, checking for concurrency problems is less hard (although it still can be very complex) because it is much clearer which objects are and are not thread safe. But in most cases the systems I check are systems that already have been build and thread safety didn’t always have the highest priority. To make it even more problematic, if the system contains hundreds or even thousands of classes, figuring out which object could have problems is almost impossible.

So I started thinking about a way to assist me to find problems; to see which objects are used by multiple threads. That is how the ‘concurrency detector’ was born. At the moment it used AOP (AspectJ) to add the ‘concurrency detection aspect’ to classes that I’m interested in. This aspect stores which thread has touched which field of which object. And based on this information it is quite easy to infer which objects are used by multiple threads. From that point on it is manual work to see if there are really problems, but it is a lot better than having no assistance at all. From time to time it writes all information to some HTML report and the system can be influenced by JMX (turned off/on, reset etc).

In the near future I want to add the following stuff:

1. more informative HTML reports: maximum numbers of writes/read by a thread, maximum total number of reads/writes etc
2. sortable tables in the HTML reports
3. problem ‘hotspots’: e.g. if an object has multiple reads and writes by multiple thread, it is more likely to have concurrency problems than a single write and a multiple reads. This helps getting ‘the biggest bang for the bug’
4. dynamic AOP (dynamic pointcuts): to add advice on classes on the fly. At the moment I’m using load time weaving, but I’m not very happy with it. It could interfere with existing Java Agents, it only works on Java 5 (and higher) and I need to create a new jar every time I want to look at different packages/classes. It is not quite clear to me if or how AspectJ is able to deal with dynamic AOP.

If you are interested in this tool, please stay in touch with my blog. I still have to decide what I’m going to do with the product (probably make it Open Source). Without in depth concurrency knowledge, the tool is useless anyway.

HashMap is not a thread-safe structure.

Last few months I have seen too much code where a HashMap (without any extra synchronization) is used instead of a thread-safe alternative like the ConcurrentHashMap or the less concurrent but still thread-safe HashTable. This is an example of a HashMap used in a home grown cache (used in a multi-threaded environment):

interface ValueProvider{
	V retrieve(K key);
}

public class SomeCache{

	private Map map = new HashMap();
	private ValueProvider valueProvider;

	public SomeCache(ValueProvider valueProvider){
		this.valueProvider = valueProvider;
	}

	public V getValue(K key){
		V value = map.get(key);
		if(value == null){
			value = valueProvider.get(key);
			if(value!=null)
				map.put(key,value);
		}
		return value;
	}
}

There is much wrong with this innocent looking piece of code. There is no happens before relation between the put of the value in the map, and the get of the value. This means that a thread that receives the value from the cache, doesn’t need to see all fields if the value has publication problems (most non thread-safe structures have publication problems). The same goes for the value and the internals (the buckets for example) of the HashMap. This means that updates to the internals of the HashMap while putting, don’t need to be visible to a thread that does the get.
So it could be that the state of the cache in main memory is not in an allowed state (some of the changes maybe are stuck in the cpu-cache), and the cache could start behaving erroneous and if you are lucky, it starts throwing exceptions. And last, but certainly not least, there also is a classic race problem: if 2 threads do a interleaved map.put, the internals of the HashMap can get in an inconsistent state. In most cases an application reboot/redploy would be the only way to fix this problem.

There are other problems with the cache behavior of this code as well. The items don’t have a timeout, so once a value gets in the cache, it stays in the cache. In practice this could lead to a webpage that keeps displaying some value, even though in the main repository the value has been updated. An application reboot/redeploy also is the only way to solve this problem. Using a Common Of The Shelf (COTS) cache would be a much saver solution, even though a new library needs to be added.

It is important to realize that a HashMap can be used perfectly in a multi-threaded environment if extra synchronization is added. But without extra synchronization, it is a time-bomb waiting to go off.

JMM: Thank God or the Devil for Strong Cache Coherence?

The Java Memory Model is something that most developers don’t understand, even though this knowledge is mandatory to create concurrent code that works without problems on any Java platform. A Memory Model describes under what conditions a write done by one thread, will be visible to another thread that does a read. Most developers think that the write of normal variables without any synchronization actions, is going to be visible to any reading thread (this behavior is called sequential consistency). They don’t realize that Java doesn’t provide this behavior out of the box, because it would prevent a lot of optimizations from being used (out of order execution of instructions, use of caches/registers etc).

The ‘nice’ thing is that code with JMM problems is not going to behave badly on a lot of modern hardware in a lot of cases. This is because most hardware provides a very strong cache coherence (far stronger than the JMM requires), so a value written in one cache is probably going to be visible in the other caches, even though synchronization actions are missing. The fact that the problems are not ‘that bad’ or ‘have not occurred’ is an argument I hear quite often.

The question remains of course if relying on the hardware to ‘fix’ a faulty application is a good approach. If the application is going to run on hardware that provides a much weaker cache coherence, very hard to track down problems could happen. This means that the Java application is not platform independent anymore. Another problem is that not just the cache could cause Memory Model problems, but the compiler can cause it as well. Currently it already is possible that a compiler optimization could break an application, no matter how strong the cache coherence of the underlying hardware is. And no guarantees can be made that a future JVM is not going to optimize much more aggressive, so even using a different JVM could cause problems.

Maybe it would have been better if the hardware didn’t provide such a strong cache coherence, so that writing correct concurrent code would have a much higher priority.

JMM: Constructors don’t have visiblity problems

One of the things that has been bugging me for some time is this: do I need to prevent visibility problems inside my constructor. With normal methods it is quite clear if there are visibility problems, eg:

class Foo{

	int a=0

	void init(){
		a=1;
	} 

	void print(){
		println("a="+a);
	}
}

If the init method is called by some thread, others threads don’t need to see the newest value of a (ever). This problem can easily be prevented by making a volatile or doing read/write of a inside a synchronized block(there are more ways). But constructors are a different story, because the reference to the newly created Foo instance, needs to be published safely before other threads are able to read fields of that instance.

class Foo{
	int a;

	Foo(){
		a=1;
	}

	void print(){
		println("a="+a);
	}
}

//safe publication
volatile Foo foo = new Foo();

This safe publication acts as the missing link to make ‘piggybacking on synchronization’ possible; so even though the newly created object(s) still has visibility problems, after the safe publication has been executed, all other threads will see a correctly constructed object when they read foo. It is very important to realize this will only work with a safe publication, otherwise there will be problems.

So from a technical point of view there are no issues. The question is if this is a correct way to deal with this situation. Personally I prefer to remove all visibility issues inside the constructor (in 99% of the cases by making the fields final) so it removes any doubt how this class is going to behave regarding visbility. When I inspect code being used by multiple threads, one of the first things I look at are the fields and the constructors. If there are visibility ‘problems’ inside the constructor, it is likely that others methods have problems as well.

ps:
Last year I wrote a blogpost about the Spring container providing a safe publication, you can check it here.

Why a lock should be a constant

Last week I read a post on the concurrency mailinglist about a piece of code from Apache Tomcat. The code was analyzed with Findbugs by Bill Pough (one of the inventors) and it is riddled with concurrency errors.

private InstanceListener[] listeners = new InstanceListener[0];

public void addInstanceListener(InstanceListener listener) {

        synchronized (listeners) {

            InstanceListener[] results = new InstanceListener[listeners.length + 1];

            for (int i = 0; i < listeners.length; i++) results[i] = listeners[i];

            results[listeners.length] = listener;

            listeners = results;

        }

}

Apart from a missing happens-before relation on the write and read of the listeners array causing visibility problems, and the same goes for the content of the array (piggy backing on synchronization won’t work), there are serious ‘old school’ concurrency problems as well.

Imagine what happens when the lock on the monitor of listeners array is obtained by one thread. When another thread wants to obtain the lock, it is blocked because the lock is not available. In the meantime the first thread continuous en replaced the listeners array by a new one (containing all previous InstanceListeners and the new one). When this thread completes, it releases the lock. The blocked thread obtains the lock on the monitor of the stale listeners array and starts working on the new listeners array without having a lock on it.

These problems could have been prevented by:

1. using a constant Object reference for a lock
2. using the CopyOnWriteArrayList (the CopyOnWriteArraySet maybe would be a better performing alternative if the order of the listeners doesn’t matter)