Friday, 15 January 2010

CruiseControl.NET Campfire plugin

On my current project, we’re using 37Signals’ Campfire for team chat and collaboration. We also use CruiseControl.NET for continuous integration. I thought it would be nice to have CruiseControl.NET automatically post build results into the Campfire chat room, to raise visibility of the build status out of the system tray and into the browser.

Implementing CruiseControl.NET plugins is quite straightforward, and Campfire has a very simple XML-over-HTTP API for posting messages into chatrooms. The code for the plugin is available on my github account. There are instructions there for using the plugin in your environment.

For the CruiseControl.NET integration, I found this blog post from Kris Selden very useful. The one thing I wasn’t sure about was how to handle errors. I didn’t want to risk exceptions being thrown from my plugin causing the build to fail. If something’s going to go wrong, I’d rather the plugin failed silently – we’ll quickly notice the lack of messages appearing in Campfire and look into it – so I made sure that I catch all exceptions and use CruiseControl.NET’s log4net facility to write to the server log.

I did have one problem with the Campfire API. I forgot to set the Content-Type HTTP header. Without this I was getting the HTTP response code 422 Unprocessable Entity, with a message “Body can't be blank”. This confused me for a while as I was definitely sending a well formed XML message with a body element. Setting the Content-Type header to application/xml fixed the problem.

Wednesday, 7 October 2009

Red-Green-Refactor at story level

A quick thought that came up this morning. When working on a story, we have two sets of responsibilities:
  • to the acceptance criteria and business value
  • to the technical quality of the codebase
We often forget to attend to the codebase once we've finished attending to the business value. We may end up with an unfinished refactoring. We are often in such a rush to claim the points. It's important that we keep the codebase clean. Red-Green-Refactor and "Make it work, make it right" apply at the story level as well as the code level. Satisfy the criteria, then make sure you're happy with the code quality.

Friday, 4 September 2009

NHibernate Bidirectional Cascade – when is it appropriate?

I recently found a situation where I needed to use NHibernate’s cascade facility from both ends of a relationship. Normally, I’d use it only in one direction, but in this scenario the only practical solution was to use it in both directions. I’m unsure whether this is a good thing to do, so in this post I’ll:

  • talk about why I normally use it in one direction
  • talk about the situation that led to the bidirectional cascade
  • ask whether this was the right solution.

One-directional cascades

Normally, the cascade is tied to the concept of ownership in the model. We cascade changes from the parent (e.g. an Order) to its logical children (e.g. OrderLines). This fits in with the Domain-Driven Design concept of an aggregate root. If we have the appropriate cascades set up from the aggregate root to the contained entities, then we can follow a simple workflow:

  1. load up an aggregate root from the repository
  2. modify, add and remove entities within the aggregate
  3. let NHibernate automatically persist the changes to the database when the session is flushed

This makes for very clean domain layer code which remains persistent ignorant. The domain code that is performing these modifications doesn’t have to explicitly save or delete any of the entities within the aggregate.

I suppose this model does not say that we can’t have cascade running from children to parents. But I always felt that the natural order of things was to have cascades running in one direction.

Why we needed bidirectional cascades

transient

In this scenario, the Customer is an aggregate root containing the ShoppingOrder and OrderLine objects. I’ve set up cascades from the Customer down to the OrderLine class, which allows the following code:

using (new SessionScope())
{
    var customer = ActiveRecordMediator<Customer>.FindByPrimaryKey(customerId);
    var shoppingOrder = customer.AddOrder(some, parameters);
    var newOrderLine = shoppingOrder.AddLine(some, parameters);
}

which executes SQL like this:

SELECT ... FROM Customer WHERE Id = ...
INSERT INTO ShoppingOrder ...
INSERT INTO OrderLine ...

Perfect. Note that I’m using Castle.ActiveRecord to configure and access NHibernate, with auto-flushing sessions. As far as I know, this scenario would translate directly to an equivalent plain NHibernate set up.

The problem comes with the OrderLineState. In our application, we have a separate part of the system which tracks the state of orders. There are two implications of this being a separate concern:

  • The OrderLineState is not considered part of the Customer aggregate. There’s no relation from the OrderLine to the OrderLineState.
  • The piece of code that creates the initial state object for an OrderLine is logically a long way from the piece of code that creates the OrderLine.

This means that when we create the OrderLineState, we need to explicitly tell NHibernate about the new entity:

using (new SessionScope())
{
    var customer = ActiveRecordMediator<Customer>.FindByPrimaryKey(customerId);
    var shoppingOrder = customer.AddOrder(some, parameters);
    var newOrderLine = shoppingOrder.AddLine(some, parameters);

    // In some other part of the system, perhaps responding to a domain event
    var state = new OrderLineState(newOrderLine);
    ActiveRecordMediator<OrderLineState>.Save(state);
}

But this code throws an exception from the call to Save(state):

NHibernate.PropertyValueException: not-null property references a null or transient valueNHOddities.OrderLine.Parent

This makes sense. We’ve asked NHibernate to save the OrderLineState object. This has a non-null, cascading reference to the OrderLine, so it tries to save the OrderLine. Because the relationship OrderLine.Parent (the Order) does not have cascading configured, NHibernate won’t try to save the Order yet. So the Order referenced by OrderLine.Parent is a transient object, and we get the corresponding exception.

There are a variety of ways to fix this. All of them work, in that they allow the code above to run and make the correct INSERT statements in the database.

  1. Remove the NHibernate NotNull constraint on the OrderLine.Parent property. This lets NHibernate save the OrderLine before it’s saved the order. I don’t like this for two reasons. Firstly, the NotNull constraint reflects a genuine constraint: an OrderLine must always belong to an order. Secondly, that NotNull constraint is probably reflected on the database column definition, so we’d have to change the database schema, removing a meaningful and useful constraint. In general, I don’t like having to change what is a genuine reflection of the domain just to make things convenient for the framework I’m using.
  2. Explicitly flush the session after adding the Order and OrderLine to the Customer aggregate, and before saving the new OrderLineState. I don’t like this one either. It requires my domain code to explicitly deal with session management. We usually consider this to be an infrastructural concern. We have a web app, so we handle session and transaction management at the request level.
  3. Add a cascade on the BelongsTo relation OrderLine.Parent.

Introducing this bidirectional cascade seems like the best (or at least, least-worst) solution. But it doesn’t quite seem right.

Is there anything wrong with a bidirectional cascade?

As I discussed earlier, it feels like cascades should run from parents to children, not the other way, and not in both directions. This isn’t written down as a rule or guidance anywhere that I can find in the NHibernate documentation, but it seems to be the standard approach. Indeed when I tweeted about this, I got a reply from none other than one of the authors of NHibernate In Action himself:

@ascordellis Cascade on both sides feels heavy, are you sure that both sides need to be the big daddy of the relationship?

For now, I’m leaving it in. For our problem, it’s the least intrusive solution. But I’d love to hear your opinion. Is there a better way to solve our problem? Should NHibernate just deal with it for us?

Tuesday, 21 July 2009

How to recover lost commits in a git repository

Today we had a problem: we lost several git commits when a rebase went wrong. We’re still not sure how this happened, but we were able to recover the commits. Here’s a quick summary of how we managed that, which I hope will help others get out of similar situations.

We tried to abort a rebase that had a lot of conflicts, and it took us from a situation like this:

pre-rebase

to one like this:

post-rebase

Disaster! Our “last big commit on my story” has vanished! We were pretty sure it was still in the repository somewhere, it just wan't reachable from any named branch. We hunted around in git log, git fsck and other commands with no luck. It turns out that the answer lies in git reflog show

$ git reflog show
3330b4e HEAD@{0}: checkout: moving from ms2 to my-story
3330b4e HEAD@{1}: checkout: moving from 3330b4e0b9af502908ce2c59958ec1e7f83a1a4b to ms2
3330b4e HEAD@{2}: checkout: moving from my-story to 3330b4e
5b50352 HEAD@{3}: rebase: updating HEAD
8f4a119 HEAD@{4}: rebase: more work on my story
c715f09 HEAD@{5}: rebase: some work on my story
6e1441d HEAD@{6}: checkout: moving from my-story to 6e1441d03b200d62f9ffc28d902c60a289d2ea3b^0
5b50352 HEAD@{7}: checkout: moving from mainline to my-story
6e1441d HEAD@{8}: commit: more work by the rest of the team
e72f33f HEAD@{9}: checkout: moving from my-story to mainline
5b50352 HEAD@{10}: commit: last big commit on my story
3330b4e HEAD@{11}: commit: more work on my story
a7d49b0 HEAD@{12}: commit: some work on my story
f3d00fb HEAD@{13}: checkout: moving from master to my-story
f3d00fb HEAD@{14}: checkout: moving from mainline to master
e72f33f HEAD@{15}: commit: some work done by other developers
f3d00fb HEAD@{16}: checkout: moving from master to mainline
f3d00fb HEAD@{17}: commit: commit b
2fcece2 HEAD@{18}: commit (initial): commit a

There it is! We now have the SHA for the “last big commit on my story”. We can check that this really is the right one:

$ git show 5b50352

commit 5b503520f132e1a64c510816fcd61cc97626791e
Author: Alex Scordellis 
Date:   Tue Jul 21 20:26:21 2009 +0100

    last big commit on my story

diff --git a/anotherfile.c b/anotherfile.c

  ...diff contents elided...

Now all we had to do was reattach a branch name to that SHA, and we were back where we started:

ascordel@ALEXSCORDEL /c/alex/code/git-recovery
$ git checkout 5b50352
Note: moving to '5b50352' which isn't a local branch
If you want to create a new branch from this checkout, you may do so
(now or later) by using -b with the checkout command again. Example:
  git checkout -b 
HEAD is now at 5b50352... last big commit on my story

ascordel@ALEXSCORDEL /c/alex/code/git-recovery (5b50352...)
$ git checkout -b recovered-my-story
Switched to a new branch 'recovered-my-story'

post-fix

Now we’re back where we were at the beginning. We can start again and try to get the rebase through this time. We still don’t know where it went wrong in the first place, but we were pretty relieved to get our work back after a short period of panic!

Wednesday, 25 March 2009

Problems mixing functional and object-oriented paradigms: NHibernate

Imagine a typical operation being performed in a regular CRUD business app. In most cases, this will be written in an object-oriented language and will involve mutable objects.

  1. grab some objects from the DB
  2. modify their state according to some business rules
  3. save the new state back to the DB

This pattern, in which an object changes state over time to represent changes in the ‘real world’ that we are modelling, is probably the most prevalent model found in business software systems which use an object-oriented language. Let’s call it the mutable-object pattern.

On the other hand, if we were to write this operation in a functional programming language, we are likely to do things differently:

  1. take some data structures as input to the function
  2. compute the values which result from applying some operation to that data
  3. return a new set of data structures representing those results

Crucially, this code written in the functional style does not alter, or mutate, the data values in the input data structures. These structures are typically immutable.

There’s been some debate recently about how, when and if we can, or even should, mix these two styles. The addition of LINQ to C# has brought a functional paradigm (delayed execution of queries over collections) to an OO language in which most developers are used to the mutable-object pattern, and really brought this issue into the mainstream.

I think that it’s important to represent each part of your code in the way which most clearly expresses your intention. If your intention is to modify some existing data in a relatively simple way according to some rules, the mutable-object pattern works well. If you need to operate on collections of data structures, performing operations which are more algorithmic in nature, a functional style works well. For example, a SQL query expresses very clearly, using a functional paradigm, what you are trying to achieve. What would you rather read? SELECT MAX(Age) FROM People; or:

max = 0;
foreach (var person in people)
    if(person.Age > max)
        max = person.Age;

So there is benefit from mixing the mutable-object and functional styles, but we need to be careful at the boundary between areas of code written in these different styles. I came across a significant example of this recently. NHibernate assumes that you will use the mutable-object style I described at the top of the post. In particular, it expects that there is only one copy in memory of each entity represented by a database row. If it returns you a CLR object in response to a query, it doesn’t let you save a different CLR object back to the same database row. If you want to modify the data in the database, you need to modify the CLR object that NHibernate gave you.

We shouldn’t really be surprised by this. The boundary between object-oriented programming and relational data storage has long provided thorny problems for software development, because of the natural impedance mismatch between OO programming and relational data (ironically, this is exactly the problem that NHibernate does a pretty good job of solving). There’s a similar impedance mismatch between the mutable-object pattern and functional styles involving immutable data structures.

Footnote

I originally wrote this post using the phrase “the OO paradigm” to describe the mutable-object pattern, until Chris Leishman pointed out that using an OO language does not necessarily imply mutability. Mutability is just the typical way that we work with objects in an imperative language like C#. Similarly, I am not asserting that all functional programming involves algorithms consuming immutable objects and churning out new immutable objects, just highlighting one pattern typically found in functional programming.

Saturday, 21 March 2009

ASP.NET doesn’t fire PostRequestHandlerExecute when you Response.Redirect

The problem

We recently discovered that our application was leaking database transactions. Checking the logs revealed that we were not closing transactions whenever we redirect within the application.

The cause

We had been closing transactions in the event handler for the PostRequestHandlerExecute event exposed by HttpApplication. It turns out that this event is not fired when you call Response.Redirect(newUrl).

The solution

We moved our transaction closing code to the EndRequest handler. This event does get called, even when you redirect. You can also change your call to Response.Redirect(newUrl, false);, which causes ASP.NET to execute all the usual events in the HTTP pipeline.

The post-mortem

The MSDN documentation for HttpApplication gives you an ordered list of the events that are fired during a request, but does not tell you that you can disrupt the events with a Redirect. The MSDN documentation for Response.Redirect does tell you that you can control “whether execution of the current page should terminate”, but doesn’t tell you what that means. It’s not clear which events will and will not still fire.

We are using the Castle.Windsor/Castle.ActiveRecord/NHibernate stack, so we based our application class on the UnitOfWorkApplication in Ayende’s Rhino.Commons library. It uses the PostRequestHandlerExecute event to close the unit of work after handling a request. A unit of work isn’t quite the same thing as a transaction, so it may be deliberate that the unit of work is not closed on redirect. I’ve posted a message to the Rhino Dev Google group to find out.

Tuesday, 17 March 2009

My day at #QCon

I spent last Wednesday at QCon. It was definitely an interesting day, but I felt like I missed a lot of stuff on other tracks. I also bumped into a few ex-colleagues from my pre-TW days and an old mate from uni who I hadn’t seen in years – I guess enterprise software is a small world…

First up was Sir Tony Hoare. Not having a background in computer science, I didn’t have quite the same feeling of awe as half the rest of the room. He made a few comparisons between software engineering (pragmatic, seeking dependability, short term view) vs computer science (perfectionist, seeking correctness, long term view), which were largely uncontroversial. Things got a bit interesting towards the end, where he stated his vision: “One day, software will be the most reliable component of every product which contains it … because of the successful interplay of research into the science of programming and the engineering of software”, and then fielded some interesting questions. It emerged that he sees complete specification as the limit of testing. I came away thinking that it’s important to have the computer scientists researching new ways for us to be certain about the behaviour of computer software, and to have the software engineers deciding pragmatically which of those methods are useful, and in what doses. Whether this will ever get us to Sir Tony’s vision, I’m not sure.

I popped into Ola Bini’s introduction to the emerging languages track, to see him explain why languages are important. My highlight here was Ola’s point that it’s all about communication, and we need to consider the best medium (i.e. language) for communicating each part of our systems to the machine, and to other humans. Since the target of our communication changes, so should the medium, and this is why we need different programming languages.

Next up, ThoughtWorks’ Ian Robinson on how to start an SOA initiative at enterprise scale. This was my pick of the day. It was one of those talks that makes you consider a whole new approach to an area of your work. I’ve picked out some key messages.

  • We should focus on capabilities which realise outcomes which deliver benefits (the what), rather than on services for services’ sake (the how). This allows us to focus on delivering those capabilities which differentiate our business from our competitors.
  • There’s a key difference between who is responsible for a business capability, and who is responsible for keeping a web service up. Engaging those in the first group will help engender a sense of ownership and urgency and ultimately lead us to more value-driven systems. For example, Ian suggests Responsibility Focused Stories (as <role>, I am responsible for <outcome>, which delivers <benefit>) to help identify these people and their responsibilities. From my own recent experience, I can attest that getting these business capability owners involved is often a significant challenge, particularly inside large companies, and I hope to use Ian’s ideas to help me do this.
  • Different service implementations are needed for different business capabilities, even if they operate on the same data. Don’t always assume that your business entities will take the same form and live in the same services. In particular the way we use and represent an entity may change significantly over the lifetime of that entity (consider the different capabilities involved in signing up a new customer, and performing daily financial trades on behalf of that customer). Update: see comment from Ian below for a clarification.
  • Use consumer-driven contracts to reduce coupling between service providers and consumers.
  • Have long-lived capability delivery teams responsible for business capabilities, but augment them with additional resources and knowledge from new projects that are taking advantage of the capabilities.

After lunch I was back in Ola’s track for Jonas Bonér on real-world Scala. He showed a simple chat application using the Lift framework, and whistled through a few of Scala’s more unusual constructs. Overall, I wasn’t convinced by what would make me choose Scala over any other functional programming language – it largely resembled Java with a few extra constructs thrown in. I think Jonas missed an opportunity to demonstrate the practical benefits that these constructs could bring and so get people excited it.

I then decided to go to Michael Feathers and Steve Freeman talk about the history of TDD. Having written and co-written Working Effectively with Legacy Code and Mock Roles, Not Objects respectively, I have a lot of respect for these guys.  Those texts played key roles in my understanding of good OO design. The session was an interesting history lesson, a useful reminder of the core principles, and contained a few nice anecdotes. I was particularly tickled by the fact that early practitioners of testing (Gerry Weinberg, Kent Beck) just assumed that everyone else was doing testing too, as they considered it to be part of doing a professional job. Plus ça change.

Martin Fowler then described our experiences with Ruby projects at ThoughtWorks over the last few years. It was good to see some statistics backing up the idea that Ruby works in the enterprise. 36 of the 41 projects ThoughtWorks has undertaken in Ruby would use it again, and of those 5 that wouldn’t, 4 said that the reason was due to the problems getting people to adapt to Ruby, rather than Ruby’s ability to perform the job required. An amusing anecdote midway through: developers on a Ruby project estimated their productivity boost (over, say, Java) as 2-5x in earlier parts of the project, reducing to 1.5x in later parts, but jumping back up to 2-3x when they had to go back to Java and remembered just how bad it was! Martin’s a great speaker, so as always it was an entertaining session.

In conclusion, I enjoyed all the sessions and heard some interesting things. But the biggest lesson I’ll take away is to go to talks about things I don’t know about, not things I do. I’m sure I’d have learnt a lot more new ideas in Rich Hickey’s Clojure presentation than in Michael and Steve’s TDD history. Not through any fault of Michael and Steve (and I certainly don’t want to pick on them), but because I’ve read a lot of that stuff before. There were also a number of talks on web architecture and RESTful services that I’d like to have caught. I also wish I’d taken notes in more of the sessions; as well as helping me take more of the information on board it would have helped me writing this post!

For those who couldn’t make it (and others like me who want to catch up on sessions they missed), all of the slides are on the QCon site, although some may need an attendee-only password.