Managing Task Execution: Introduction

September 14, 2006

Abstract

In this post I want to discuss a few issues you are likely going to encounter when designing components that execute tasks/processes. In the next post I’m going to explain a construct I have used in Enterprise Java systems, that deals with these problems while localizing complexity.

Introduction

There are a lot of issues you have to deal with when you are executing tasks (running processes) in a multithreaded environment (especially in serverside systems):

  • synchronization of data structures
  • controlling the executing thread(s)
  • asynchronous calls
  • controlling the number of threads
  • Preventing processes from running parallel

Synchronization of datastructures

If multiple threads are allowed to access shared mutable data structures, usually synchronization is required to prevent isolation problems.

Synchronization can be complex and integrating it directly into components in a system can be a nightmare (deadlocks anyone?). Another problem is that the components get more responsibilities because of the added synchronization logic and in most cases this is something I try to prevent.

Controlling the executing thread(s)

In some cases you don’t want to run a task on the thread that ‘initiates’ the execution of a task, but want to run it on a thread you have complete control over instead.

A few reasons why you want to control the executing thread:

  • priorities: if the tasks is going to take some time to complete (a batch job for example) you don’t want it running on a thread with a high priority because it can lead to an unresponsive system. Threads that are used for user interaction usually have a normal to high priority. So if a user initiates such a task, you don’t want to have it running on that thread.
  • liveness problems: if the thread is completely under your control, you know exactly which resources it locks. This makes a system less likely to get into liveness problems like deadlocks.

Asynchronous calls

In some cases you don’t want that a caller blocks untill the task completes, but want to execute the task asynchronous instead. This way the caller can return, without waiting for the completion, and continue what he was doing.

But making a call asynchronous is not without issues either. There are a couple of things you have to deal with. The most important one is dealing with unprocessed jobs: if jobs are created faster than they are processed, it can lead to problems (like memory/resource problems). In some cases you want block untill the task can be placed. In other cases you want an exception to be thrown if the task can’t be placed or untill some timeout has been reached. Luckely there are also cases where the task can be dropped completely without any problems.

Controlling number of threads

In some cases you want to control the number of concurrent threads. If tasks are accepted without any control, it could lead to a lack of resources (like memory and processing power) and your system doesn’t degrade gracefully under heavy load (it could crash).

Preventing processes from running parallel

There are cases where you want to prevent some tasks from running concurrent, example:

  • optimizing a Lucene index.
  • adding documents to a Lucene index.

If these processes are allowed to run parallel, Lucene will start throwing exceptions.

You could add exclusion logic to these tasks, but this leads to an increased number of responsibilities and this is something I usually try to prevent.

Scattering of concurrency logic

Often the concurrency logic, that is added to components to make them useable in such an environment, is scattered all over the place. The core logic of the class gets mixed with the concurrency logic and trust me, this is something you want to prevent:

  • components are difficult to test
  • components are less reusable (this isn’t always an issue)
  • components are more difficult to reason about
  • components are more complex so more difficult to extend, maintain etc.
  • changing the multithreading behaviour is more difficult because it is integrated into the components.

That is why concurrency logic should not be scattered all over the place: make a few components aware of your multithreading design en let them worry about all these problems. The other components can focus on their core responsibilities. This makes it much easier to design and maintain a multithreaded system.

In the next post I’m going to introduce the design. So if you are interested, stay tuned.

Advertisements

Asynchronous calls are not an implementation detail

September 4, 2006

Abstract

In this post I want to make clear that the asynchronous behaviour of a method call, is not just an implementation or configuration detail, but should be made explicit in code and documentation.

What is the problem

One of the things I have seen in projects that use multithreading of some sort, is that some method calls are made asynchronous transparently. This means that the caller has no way of knowing if a method call is executed on a different thread or on the thread of the caller..

Origin of the problem

Making a method call asynchronous is quite easy with methods that return void: you could create some sort of asynchronous decorator that executes the real call on a different thread. The caller can return without waiting for the completion of the task, while the thread executes the task at some point in the future. example:

interface Watercooker{
..void activate();
}  

class AsynchronousWatercooker implements Watercooker{
..private final Watercooker cooker;
..public AsynchronousWatercooker(Watercooker cooker){
....this.cooker = cooker;
..}    

..public void activate(){
....Runnable task = new Runnable(){
......public void run(){
........cooker.activate();
......}
....};

....new Thread(task).start();
..}
}

Especially with a powerful framework like Spring it is quite easy to do:

  1. Spring allows unmanaged threads, unlike EJB. So your components are allowed to create threads (you are able to set the resources yourself).
  2. Spring promotes using interfaces rather than classes. So creating an asynchronous decorator is easy. And because classes don’t create instances of dependencies themselves (but rely on Dependency Injection) it is easy to inject an instance of the asynchronous decorator.
  3. Spring has powerful AOP functionality: it is quite easy to create an aspect that makes a call asynchronous.

It doesn’t mean Spring is a bad framework: it is powerful but you still need to design good software.

Why is it a problem

There are a few reasons why transparantly adding asynchronous behaviour could lead to problems:

  1. concurrency issues
  2. different interface requirements
  3. postcondition don’t hold

Why is it a problem: concurrency issues

If data is shared between threads, some sort of concurrency control is required. If the caller is not aware of this, it could lead all kinds of nasty problems like race problems and corrupted data structures.

Why is it a problem: different interface requirements

Interfaces of asynchronous calls are different than interfaces of synchronous calls. The most important one is the difference between exceptions: with a synchronous method call you get an exception that is a consequence of the operation, like CookerIsEmptyException. A asynchronous call typically doesn’t have this kind of exception because the job (in most cases) isn’t executed directly and the caller returns after the task is posted. Asynchronous calls typically have exceptions like TimeOutException (if the job isn’t accepted in a certain amount of time), AlreadyStartedException (if a different thread already has activated the cooker).

Why is it a problem: postconditions don’t hold

Another problem is that you don’t know if postconditions of a call will hold. In case of the Watercooker it could be that you return from the activate method, but the water is still cold: you don’t want to have cold coffee!

Conclusion

That is why the (a)synchronous behaviour of a call should be designed and documented explicitly. In some cases the asynchronous interface will look a lot like the synchronous interface, but there are big differences. Don’t make the mistake by treating them the same, because they are not.

ps:

I want to thank the guys on the concurrency mailinglist for some useful feedback.