Minimal Agile documentation

One of the things that wonder me when looking at source code of Agile projects, is the lack of documentation. The minimal amount of documentation on classes/interfaces is the description of its core responsibility: what is the purpose of this class? It doesn’t need be be very long; a few sentences would be enough. And if the class/interface is part of some design pattern, this should be added to the documentation as well. If the the class is completely self contained, and the pattern is well known (like a Strategy or Decorator), I could live without the documentation but not everyone is a pattern freak, I would rather see it in the documentation.

Another important document is some sort of architectural overview to get a quick understanding of the system without needing to browse through all sources. This document should at least cover the main components of the system, what the responsibilities are of these components and how they interact and why this approach is being used. Without this information it is very hard to say anything about performance, security, scalability, re-usability.

I know that not writing documentation shouldn’t be blamed on the Agile methodology, but on its misinterpretation.

ps:
If you want to remove redundant parts in the documentation, why is the @author tag not removed from the javadoc? If I want to figure out who wrote it (or changed it) I could look at the blame functionality most version control systems provide.

Flexible quality

One of the things I have observed is that some projects get into problems caused by quality issues. The following 3 properties are part of every project:

1. functionality: the stuff that needs to be implemented
2. time: the total amount in man hours (budget is also expressed in time)
3. quality: structural/code quality, maintainability, number of bugs

If functionality and time are fixed, the only thing that is flexible is quality. So especially in the end of an iteration, when the pressure is felt, the quality goes down: code doesn`t get refactored and the structure degrades, test are not complete or totally missing. The problem gets even worse because the next iteration is going to contain debt from the previous one. If this debt isn`t added as in issue for the next iteration to solve, you will get a lot of hidden work, and hidden work is a recipe for project failure.

The Agile methodology promotes fixed time and fixed quality, so the only thing that is flexible is functionality. I’m a very pragmatic guy, so flexible time (so increasing the length of an iteration) would be negotiable (although you could loose your rhythm) but when quality is negotiable I think you are on the road to project hell: unhappy customers, developers and managers.

Ad hoc design vs reference architectures

I had a few discussions about references architectures vs ad hoc design in agile projects. Personally I think both have value, but when you have a standard problem, and a lot of them are standard, I prefer using a standard solution.

The main example in the conversation was if a service layer always is required. A lot of simple web-applications are crud oriented, and especially in such a verbose language as Java, creating a service layer feels like an exercise in futility. I have made my fair share of services that only delegate to doa’s without adding any logic. This means that code needs to be written/tested/maintained, that doesn`t add much value directly. And especially within agile teams, this code just screams ‘remove me’.

But even in crud oriented solutions, having a service layer has value. To give a few arguments:

1. you know where security is added. If you want to protect some data, you really don’t want the dao’s to be exposed in the presentation layer. Unluckily this is something I see too often, although they are often tucked away in some sort of request binding logic.
2. you know what the transactional setting are. I see a lot of code that is (partially) missing transactional settings. Luckily most databases provide one implicitly, but I don’t find this a professional way of writing software, because you can’t say anything about the consistency of the data: there could be isolation problems like inconsistent views (data that is read from multiple transactions) or lost updates.

So really good arguments have to be provided to convince me from not using a service layer.

If I look on a more higher level, my opinion is that applying reference solutions provides value:

1. you know where logic can be found. I guess we all know how irritating it is if you can’t understand a system not because the domain is complex, but the implementation is complex: logic is difficult to find, scattered all over the place and even worse: repeated.
2. you know how a system can be changed without needing to know all details of the system. This makes an application, in my opinion, more agile than one with a lot of ad hoc solutions.
3. it is easier for new team members to start with the project because they see familiar solutions.
4. chance is big that other people have experience with problems of the solution, so there often is knowledge you can fall back on.

So if I see a standard problem, I prefer using a standard solution. Enough books have been written about all kinds of problem domains, for example ‘Patterns of Enterprise Application Architecture’ from Martin Fowler about enterprise applications. But just as important as knowing reference solutions, is knowing when they are not a good match. Using a bad solution is a lot worse than using an ad hoc solution, because the former one always leads to unnecessary complexity.

A deeper concern is that I have the the impression that developers see the Agile methodology as a free card to question all established knowledge. Although I think that questioning knowledge is a good thing, because it helps to remove clutter, ignoring knowledge isn’t. Agile development requires more knowledge about reference solutions because you are designing all the time, and not just in the beginning. If a system isn’t well designed, you are not able to be agile because a system isn’t able to change gracefully.

Agile and upfront design

One of the things I get annoyed by is that people consider Agile being a methodology without any form of design. Big upfront design, like the waterfall method has, is not a good thing. In most cases you don’t know exactly what needs to be build and what all the constraints are. And that is why big upfront design leads to frozen ignorance: bad technical choices, caused by a lack of understanding, make a system hard to change. Another consequence is that the system is hard to work with because you have to deal with unnecessary complexity.

But my feeling is that some Agile developers are out of balance, because they reject any form of upfront design. Doing some light up front design (maybe a few hours in front of a white board), especially for more complex systems, can be a big help. Often there are some architectural patterns you can use like MVC, Layers, Pipes and Pilters, etc and it helps you to get a better understanding of the system. This makes it possible to do a calculated guess. My personal experience is that a good guess often pays of. If it doesn’t pay off, you should not stick to the choice, and refactor it so becomes the best solution for the problem (if this is possible).

Make deployment part of continuous integration

Last year I started working with Cruise Control as continuous integration server. Continuous integration is very important for Agile projects, because it is a source of continuous feedback: without continuous feedback you can’t be agile. But I have been focusing on the unit testing aspect too much: I thought of Cruise Control as a mechanism to run all unit tests when it detects changes in the source repository. But a continuous integration server can do much more than just running your unit tests.`

It can also build the war/ear and deploy it. This provides feedback on a lot of stuff that is hard to test otherwise:

1. is the application able to deploy on the target application server? Often there are differences between the development and production environment: different operating systems, different versions of the application server, different virtual machines, different configurations, etc. Personally I don’t mind much if there are small differences, but the closer you get to the production environment, the smaller the differences should be.
2. I use Spring for most projects, so are there errors in the applicationcontext? The best way to make sure the applicationcontext is valid, is to deploy. This could fail for a lot of reasons: an error in the jndi-name of the datasource for example.

If you make the deployment part of the continuous integration process, and make sure the build breaks when the deployment fails, you have a very good source of feedback.

At the moment I’m using Oracle OC4J as application server. I added a deploy target in my ANT script and this task is added to the list of targets that Cruise Control executes when it detects a change. Letting the deploy target drop the war/ear in a directory where it is picked up eventually is not enough btw, the build won’t break when the deploy fails.