Am I too stupid for @Autowired?

When I started with Spring, I finally had the feeling that I could write enterprise application in the same powerful way as I could write normal systems. And I have written quite complex Java stuff like expertsystems, prolog compilers and various other compilers and interpreters and currently working on Multiverse; a software transactional memory implementation. I also have written quite a lot of traditional enterprisy Spring applications (frontend, backend/batch processing). So I consider myself at least as smart as most developers.

The first applicationcontext files felt a littlebit strange, finding the balance between the objects that need to be created inside the application context and objects created in the Java objects themselves. But it didn’t took me long to find this balance and realise how powerful the application context is:

1. it acts like a executable piece of software documentation where I can see how a system works, just by looking at a few xml files. I don’t need to look in the source to see how stuff is wired.
   I compare it with looking at Lego instructions, by looking at the instructions I can see what is going to be build (not that I ever followed the design). To understand what is build, I don’t need to look at the individual parts.
2. separation of interfaces and implementation, so testing and using a lot of cool oo design stuff is a dream
3. having the freedom to inject different instances of the same interface on different locations. This makes it possible to do cool stuff like adding proxies, logging etc etc. I wasn’t forced to jump through hoops any more because of all kinds of container specific constraints

With the old versions of Spring (1.2 series) I had the feeling of total control and complete understanding.

The application context made me aware that I needed to work with 2 hats:

1. ‘Object’ designer; so actually writing the Java code and documentation
2. ‘Object’ integrator; where I assembly complete systems based on the objects

And I loved it.. I have more than 1.000 post on the Spring forum just because I believe in this approach.

But if I look at a lot of modern Spring applications, filled with @Component and @Autowired annotations, it feels like I have lost it all. It takes me a lot longer to realise how something works, even though I have a great ide (newest beta of IntelliJ) with perfect Spring integration that makes finding dependencies a lot easier. So I keep on jumping from file to file to understand the big picture. A lot of developers I meet/work with think that the auto-wiring functionality is great because it saves them a lot of time and prevents them from programming in XML (XML was a bad choice, but that is another discussion), but somehow my brain just doesn’t get it.

So my big questions are:

1. am I too stupid?
2. if I’m going to try it longer, am I going to see the light?
3. do I expect too much? So because less characters need to be typed, is it inevitable that some redundant/explicit information is lost
4. is this some hype and eventually people start to realise that it was a nice experiment, but doesn’t provide the value it promises (just like checked exceptions or EJB 2). And in 10 years we can all laugh about it.

Spring AOP: Too complex for day to day use.

I have done quite a lot with Spring and AOP in the past (have been a long time Spring supporter), but every time I need it in a project and look at the reference documentation, I find it more and more confusing and getting too complex for day to day use (especially if not every team member is a Spring expert). It is not because I don’t understand AOP:
– I have used it for quite some time
– I do quite a lot of bytecode manipulation on the Multiverse project.

What is wrong with Spring AOP?
– everything can be combined with everything in all kinds of different ways (so too many permutations)
– there are 10.000 ways to configure it and the configuration can be spread all over the place (including in the source).
– Spring documentation (which is very detailed) is not written in a way to get you up and running fast (so a complete working copy/paste hello world solution).
– bytecode weaving of AspectJ gives problems with load time weaving (administrators, influences other applications on the same jvm) but compile time weaving in combination with Maven also gives problems because the bytecode is modified before the unit tests are executed.

It doesn’t mean that Spring AOP is bad from a technology point of view, but it certainly is bad from a productivity point of view. I love to solve complex problems, but I don’t like to solve problems that are needlessly complex. And Spring AOP imho is a good example of Spring becoming the problem it tried to solve: needless complexity.

The Springforum has lost its pazzaz

I have been reading and posting on the Spring forum since before it became mainstream, to be specific I joined on 22 November 2004. And under the name ‘Alarmnummer’ I have placed more than 1000 posts (in most cases answers to questions).

But the quality of the forum has dropped dramatically. Perhaps it is caused by:

1. Spring becoming yet another bloated environment
2. closed design by Spring-team and not the community
3. Spring getting too commercial

Thereby loosing touch with the community and especially with smart early adaptors.

Most questions I see on the forum are not about design any more, but in a lot of cases RTFM (Read the Fucking Manual) or UTFS (Use the Fucking Search) and not fun to answer at all.

IOC without an IOC-container

Spring is my framework of choice if I need to write enterprise applications. But in some occasions using Spring (or more dressed down IOC containers) is not an option, because:

1. it is too heavy/big. In some (rare) cases every byte counts
2. you can’t introduce new dependencies.
3. all kinds of political reasons

A practical example of such an strange application is a javaagent. I’m working on one for the ‘Concurrency Detector’ and javaagents typically are small. And introducing extra dependencies can introduce unforeseen side-effects, e.g. I had a problem with using AspectJ in my Javaagent in combination with AspectJ in a webapplication. The side effect was that my class transformations totally got bypassed. That is one of the main why I dropped AspectJ for the concurrency detector and decided to use ASM instead.

So what if you can’t use an IOC container, but do want to use IOC. Simple: just create one (perhaps more) class that glues all components together, just like in the Spring container. This class can know of all specific implementations, policies/strategies, initializations etc etc. All the stuff you normally do in a Spring container you can do in this class. The components of your system don’t need to be aware of the IOC-container they are running in, and that is how I like my OO design.

Quickstart project: Terracotta/Spring/Ant

Terracotta is a great way to created a distributed heap, but setting up a complete working example using Spring/ANT from scratch takes some time. So one of the things I do in such a situation is to create some kind of template project I can reuse so I’m up and running faster the next time. I decided to share this template with the Java community. The current template project contains a producer of messages (strings) and a consumer of messages. The messages are distrubuted by using a BlockingQueue that is distributed by Terracotta.

The only thing that needs to be changed are the settings in the tc.properties file.

You can download the project here:

Executing long running task from UI

A colleague asked me how one could prevent the execution a long running task on some UI thread (e.g. the thread of a Servlet container or the Swing event dispatching thread) and also how one could prevent the concurrent execution of that same task. So I send him an email containing a small example and decided to place it on my blog to help others struggling with the same issue.

The long running task

The FooService is the service with the long running method ‘foo’.

interface FooService{
        void foo();
}

There is no need to add threading logic to the FooService implementation. It can focus purely on the realization of the business process by implementing the business logic. The code should not be infected with concurrency control logic, because it makes testing very hard, and makes code also hard to understand, reuse or change. So this is one of the first potential refactoring I often see in code. I’ll post more about this in ‘Java Concurrency Top 10’.

The execution-service

The FooExecutionService is responsible for executing the FooService and preventing concurrent execution (if the correctly configured executor instance is injected). Personally I prefer to inject the executor instead creating one inside the FooExecutionService, because it makes it hard to test and change/configure.

class FooExecutionService{

        private final FooService fooService;
        private final Executor executor;

	public FooExecutorService(FooService fooService, Executor executor){
		this.fooService = fooService;
		this.executor = executor;
	}

        /**
	 * Starts executing the FooService. This call is asynchronous, so
         * won't block.
	 *
	 * @throws RejectedExecutionException if the execution 
	 *         of foo is not accepted (concurrent/shutdown).
	 */
	public void start(){
                executor.execute(new Task());
        }

        class Task implements Runnable{
                void run(){
                        try{
                                fooService.foo();
                        catch(Exception ex){
                                log.error("failed to ...", ex);
                        }
                }
        }
}

The FooExecutionService could be improved in different ways: it could provide information when a task already is executing. This could be realized by placing a dummy task in the executor and check if the task is rejected. A different solution would be to let the Task publish some information about the status of the current execution. If the task is very long running, and you want to be able to stop the task, you could shutdown the executor by calling the shutdownNow method. This interrupts the worker-threads and if you periodically check the interrupt status of the executing thread while doing to long running call, you can end the execution.

Some Spring configuration

The Executor is injected from the outside by some Spring configuration, i.e.:

<bean id=""fooService" class="FooServiceImpl"/>

<bean id="fooExecutionService" class="FooExecutionService">
        <constructor-arg 	index="0" 
				ref="fooService"/>
        <constructor-arg 	index="1">
            	<bean 	class="java.util.concurrent.ThreadPoolExecutor"
			destroy-method="shutdownNow">
			<!-- minimal poolsize (only 1 thread) -->
                	<constructor-arg 	index="0"
                                 		value="1"/>
                	<!-- maximum poolsize (only 1 thread)-->
                	<constructor-arg 	index="1"
                                 		value="1"/>
                	<!-- the timeout (we don't need it) -->
                	<constructor-arg 	index="2"
                                 		value="0"/>
                	<!-- the timeunit that belongs to the timeout argument (we don't need it) -->
                	<constructor-arg index="3">
                    		<bean 	id="java.util.concurrent.TimeUnit.SECONDS"
                          		class="org.springframework.beans.factory.config.FieldRetrievingFactoryBean"/>
                	</constructor-arg>
                	<!-- the workqueue where unprocessed tasks get stored -->
                	<constructor-arg index="4">
                    		<!-- we don't want any unprocessed work: a worker needs to be available,
                 		or the task gets rejected. -->
                    		<bean class="java.util.concurrent.SynchronousQueue"/>
                	</constructor-arg>
        	</bean>
	</constructor-arg>    
</bean>

If there are multiple long running methods, it would be an idea to extract the creational logic of the executor to a factory method.

The UI-controller

And the FooExecutionService can be hooked up to some controller like this:

class StartFooController extends SomeController{
        final FooExecutionService fooExecutionService;
      	
	StartFooController(FooExecutionService fooExecutionService){
		this.fooExecutionService = fooExecuctionService;
	} 
	
        String handleRequest(Request request, Response response){
                try{
                        fooExecutionService.start();
                        return "success";
                }catch(RejectedExecutionException ex){
                        return "alreadyrunningorshuttingdownview";
                }
        }
}

Prevention of concurrent execution of different tasks

If you want to prevent concurrent execution of different long running methods, you could create a single execution-service for all methods, and share the same executor between the execution of the different tasks:

Versioned id’s

If you are using multiple system transactions that span a single business transaction, there is chance of isolation problems: a business transaction could see changes made by other transactions and this could lead to erroneous behavior.

Example:
there is a web page that can be accessed by management, that contains a list of all employees. Each row contains the id, the name, the date of the last bonus and a button to give the bonus. When the button is pressed, the id of the employee (eventually) is send to the following service method:

@Transactional
void giveBonus(long id){
	//load throws a special exception if the employee is not found
	//Spring: ObjectRetrievalFailureException if employee isn't found
	Employee e = employeeDao.load(id);
	e.giveBonus();
}

The problem with this approach is that 2 managers could access the page at roughly the same moment and both see that some employee didn’t get a bonus for some time and decide to give the bonus. The consequence is that the employee has received the bonus twice. Although this example is a little bit hypothetical, in some cases this behavior could lead to serious problems.

Optimistic locking is one of the best known techniques to deal with these issues: it doesn’t lock (unlike pessimistic locking) but hopes (is optimistic) about the fact that no other transaction is going to conflict. If a different transaction does make a change, the version of the data is changed, and this is something that can be checked. As soon as a difference between versions is found, an optimistic locking failure exception can be thrown.

Just carrying around the version and id, doesn’t feel that good. It makes method signatures less pretty and you have to carry it around. Especially when you need to deal with multiple entities, the solution gets more ugly. The solution is simple: integrate id and version into a single structure: VersionedId. VersionedId’s contain the id of an entity and also contain the version of that entity; so a VersionedId can uniquely identify an entity in time and could look like this:

class VersionedId{
	long version, id;
}

the previous example could be rewritten by using a VersionedId:

@Transactional
void giveBonus(VersionedId id){
	//load throws a special exception if the employee is not found or version doesn't match
	//Spring: ObjectRetrievalFailureException if employee not found
	//Spring: OptimisticLockingFailureException if version doesn't match
	Employee e = employeeDao.load(id);
	e.giveBonus();
}

As you can see there is not much difference. But you have to make sure that the load method of the EmployeeDao is able to handle versioned id’s and starts throwing some kind of optimistic locking failure if the Employee with the given id and version can’t be found. And instead of just placing the id of the employee on the html-page, the version needs to be placed as well.

Using detached objects (and reattaching them) is also a solution. But there are some issues:

1. The detached objects need to be stored and the most likely place is the HttpSession. This could lead to replication issues if you are in a clustered environment
2. Vague transactional behavior
3. I have seen some weird issues with detaching and reattaching entities with Hibernate (especially not detaching completely)

Service layer woes

I have posted an entry on the blog of my employer:

http://blog.xebia.com/2007/07/18/service-layer-woes/

DSL’s: a higher abstraction to the rescue

One of the things that bother me is that it is very easy to loose overview in large wired structures and I think DSL’s can limit the distance between specification and implementation. Example of something set up with a very low abstraction in Spring:

    <bean id="fileWritingProcess"
          class="com.xebia.clustering.processes.FileWritingProcess"/>

    <bean id="resequenceProcess"
          class="org.codehaus.prometheus.processors.ResequenceProcess"/>

    <bean id="fileWritingProcessor"
          class="org.codehaus.prometheus.processors.standardprocessor.StandardProcessor">
        <constructor-arg index="0"
                         type="org.codehaus.prometheus.channels.InputChannel"
                         ref="fileWritingProcessor-input"/>
        <constructor-arg index="1">
            <list>
                <ref bean="resequenceProcess"/>
                <ref bean="fileWritingProcess"/>
            </list>
        </constructor-arg>
    </bean>

    <bean id="fileWritingProcessorRepeater"
          class="org.codehaus.prometheus.repeater.ThreadPoolRepeater"
          init-method="start"
          destroy-method="shutdown">
        <constructor-arg index="0">
            <bean class="org.codehaus.prometheus.processors.ProcessorRepeatable">
                <constructor-arg ref="fileWritingProcessor"/>
            </bean>
        </constructor-arg>

        <property name="exceptionHandler">
            <bean class="org.codehaus.prometheus.exceptionhandler.Log4JExceptionHandler"/>
        </property>

        <property name="shutdownAfterFalse"
                  value="true"/>
    </bean>

And this is only 1/4 of the XML configuration.

Or now written in a DSL in Groovy (I’m still playing with the concepts and Groovy, so it could be that the syntax isn’t correct).

def piped = environment{
    queues{
        queue(name:'queue1'),
        queue(name:'queue2'),
        queue(name:'queue3')
    }

    pipeline{
        [parserprocess,
         queue1,
         parallel(threadcount:10){fibonacciprocess},
         queue2,
         parallel(threadcount:10){piprocess},
         queue3,
         [resequenceprocess,filewritingprocess]]
    }
}

I think most can imagine what the last script is doing without knowing anything of the application. That is the power of a DSL.

Spring and visibility problems

I posted a blogentry on Spring and visibility problems on the company website. If you want to read the post, you can find it here:

http://blog.xebia.com/2007/03/01/spring-and-visibility-problems