Ad hoc design vs reference architectures

May 28, 2007

I had a few discussions about references architectures vs ad hoc design in agile projects. Personally I think both have value, but when you have a standard problem, and a lot of them are standard, I prefer using a standard solution.

The main example in the conversation was if a service layer always is required. A lot of simple web-applications are crud oriented, and especially in such a verbose language as Java, creating a service layer feels like an exercise in futility. I have made my fair share of services that only delegate to doa’s without adding any logic. This means that code needs to be written/tested/maintained, that doesn`t add much value directly. And especially within agile teams, this code just screams ‘remove me’.

But even in crud oriented solutions, having a service layer has value. To give a few arguments:

  1. you know where security is added. If you want to protect some data, you really don’t want the dao’s to be exposed in the presentation layer. Unluckily this is something I see too often, although they are often tucked away in some sort of request binding logic.
  2. you know what the transactional setting are. I see a lot of code that is (partially) missing transactional settings. Luckily most databases provide one implicitly, but I don’t find this a professional way of writing software, because you can’t say anything about the consistency of the data: there could be isolation problems like inconsistent views (data that is read from multiple transactions) or lost updates.

So really good arguments have to be provided to convince me from not using a service layer.

If I look on a more higher level, my opinion is that applying reference solutions provides value:

  1. you know where logic can be found. I guess we all know how irritating it is if you can’t understand a system not because the domain is complex, but the implementation is complex: logic is difficult to find, scattered all over the place and even worse: repeated.
  2. you know how a system can be changed without needing to know all details of the system. This makes an application, in my opinion, more agile than one with a lot of ad hoc solutions.
  3. it is easier for new team members to start with the project because they see familiar solutions.
  4. chance is big that other people have experience with problems of the solution, so there often is knowledge you can fall back on.

So if I see a standard problem, I prefer using a standard solution. Enough books have been written about all kinds of problem domains, for example ‘Patterns of Enterprise Application Architecture’ from Martin Fowler about enterprise applications. But just as important as knowing reference solutions, is knowing when they are not a good match. Using a bad solution is a lot worse than using an ad hoc solution, because the former one always leads to unnecessary complexity.

A deeper concern is that I have the the impression that developers see the Agile methodology as a free card to question all established knowledge. Although I think that questioning knowledge is a good thing, because it helps to remove clutter, ignoring knowledge isn’t. Agile development requires more knowledge about reference solutions because you are designing all the time, and not just in the beginning. If a system isn’t well designed, you are not able to be agile because a system isn’t able to change gracefully.


The joy of open source software development

May 22, 2007

At the moment I’m almost finished with the first release of a concurrency library I’m working on: Prometheus. The library is released under the MIT license and I hope to make a release very soon. The code almost is finished, but a lot of time goes in the details (one button release, site, documentation, setting it up with Bamboo etc). The cool thing is that I’m able to play with all kinds of very cool technologies that were granted because Prometheus is open source.

  1. JIRA: great issue and feature tracking system. I have used others (Rational ClearQuest, Trac) but JIRA is a still my preferred choice.
  2. Confluence: great wiki environment
  3. JProfiler: great profiler. I haven’t used it on the Prometheus library (yet)
  4. Yourkit: also a profiler I heard great stories about. I’m certainly going to give it a try.
  5. Clover: a test coverage tool. It really is amazing, it have found some untested code and unused testcode after checking the Clover report. Another cool thing is that it is safe to use in a multi threaded environment (this is quite handy for a multi threaded library like Prometheus). I also was granted a license for Clover 2 beta: it looks great and provides a lot more information. So I’m certainly going to give it a try.
  6. IntelliJ IDEA: what is there to say, it is my preferred editor. Would you spend $500 for Eclipse? But personally I don’t care much what other developers are using: if you can program as fast in vi as I in IDEA, I have no problem with vi.
  7. Bamboo for continuous integration. I normally use Cruise Control, but it appears that Bamboo is taking it to the next level. I’m going to integrate Prometheus on Bamboo as soon as I have time. There are some issues I have to work out, for example: repeated execution of tests to increase the chance of finding concurrency problems.
  8. Structure101: a tool for structure analysis. I have asked the guys from Headway Software if they have plans for an open source license, and luckily they do. So I’m going to play with it on Prometheus. If I’m able to extract relevant information from it, I plan to use it within Xebia the company I work for to assist with quality assurance.
  9. Fisheye: great for extracting information from Subversion. I haven’t played with it on Prometheus much, but I can imagine it is very useful if there is a large code base.

It almost feels like Christmas 🙂

There are also a bunch of open source products I use that can’t be left unmentioned:

  1. JUnit: what is there to say?
  2. EasyMock: easy when I just need to mock something for testing purposes. For the Prometheus library I decided not to use a mocking approach in most cases because I want to know if a component is functioning correctly and not if I have recorded correctly. For most standard objects mocking is good enough, but for something as complex as concurrency control, I didn’t want to trust on my recording abilities.
  3. ANT: After 2 years I still don’t care much about Maven 1 and 2. I think they are straitjackets and as long as you can obey the Maven rules, you are a happy guy. But as soon as you need something special, you are on your own. So ANT still is my preferred choice even though it takes more time to set up a script.
  4. Groovy & Gant. Ant provides a great infrastructure but it was a very bad choice to use XML as syntax (it always was a bad choice, but XML isn’t as hot as it used to be and now you are allowed say that it should not be used for anything else than data exchange. I have switched from ANT to Groovy ANT (Gant): the convenience of a DSL, but the power of a full blown language. I also use Groovy to generate the Prometheus site. It is great to have a scripting environment that integrates perfectly with Java.
  5. FindBugs: it is great for finding bugs. I have run it on Prometheus and immediately it discovered some bugs (not calling the super in TestCase.setUp in quite a few cases for example). Checkstyle also is something I’m familiar with but spending more time to make it shut up than I get value from it.
  6. Subversion: next to CVS it is ‘the’ Code Version Control system.

And last but certainly not least: I have to thank Codehaus. A lot of licenses were provided by Codehaus and I’m also happy I can make use of their servers (although I have backed up my code not only on their machines 😉 )