In the past couple of years two of the key software development programmes within the Agency have switched from a traditional outsourced, waterfall-led arrangement, to a much more agile, internally controlled framework. These programmes seemed like walking two huge remote-control elephants through a massive risk-filled minefield to a distant glade some miles away. Whilst tied together. Through treacle.
If you’ve got to navigate a minefield you tend to look down a lot. So even though you know you’re tethered together, you only look up occasionally to take a look at your partner. And usually then just to see how tight the rope is!
So things have changed now and we’ve swapped the elephants for some little go-karts. There’re 25 go-karts for each elephant, the minefield is still there and there’s no remote control. We do the driving. But you’ve still got to manage that piece of rope….
The Funding and Contracting Transformation (FCT) programme was the second to change the elephant for the go-karts. The related Data Collections and Funding Transformation (DCFT) programme had done this some time ago. When we kicked off FCT, we knew this would be the first major instance where we would have to manage some complex integration, not just between the software components, but between FCT and DCFT - two agile programmes at very different stages of maturity. Now we’re getting into the testing phases of these programmes, we thought it would be useful to reflect.
Let’s talk about the good stuff. Having the dev teams on site and accessible has been a refreshing experience. No longer are we talking about contracts, requests for work, terms of reference, etc: instead we are able to get the key architects and business analysts in a room with (several) whiteboards and thrash out how the solution needs to work, test some concepts, fail fast and deliver product faster. Most importantly for me – and as discussed in Chris's last blog – there was a clear shared context of what we needed to do and the ‘Chinese Whispers’ were quickly resolved. It took a little while and we have to concentrate hard on sharing and testing our assumptions but what we ended up with is a far simpler and mutually beneficial solution than we would have had before.
More importantly, the respective interfaces have been specified, built and tested in a four week period, an unheard of achievement prior to this way of working. Even better, they’re simpler and we’ve identified lots of small improvements we can make for the future that will have a big impact on the later efficiency of the business.
It’s also important to talk about the stuff that we need to learn from. Firstly, agile doesn’t like you to look too far forward when you're building software. This can be tricky when you have big programmes with lots of things going on. For example, the FCT teams "sprung" their interface requirements on the DCFT teams with a fraction of the notice they would normally give. There was a significant chance the two programmes could have ended up derailing each other by placing a big stack of stories in the backlog that would need to trump everything else. It was fortune, rather than some creative rescheduling, that enabled us to get the stuff done in the end. This also extended to the architecture teams who were having to build and design corporate schemas and interface patterns on the fly to ensure that the rest of the Agency’s integration requirements would still be satisfied (eg security, recoverability). We know there’s stuff in there that we could do better. And we could have achieved that improvement with more notice.
Finally, we’ve got to get the overall Agency programme management right. Trying to manage dependencies through the project management process is fine, but ultimately we identified that we need to improve the communications between Product Owners and Senior Responsible Owners (SROs) both up-and-down and across. A few too many times we hit the problem of trying to identify which SRO had jurisdiction – who should make the call as to which programme was more important for the Agency. Our software development is now agile but not the entirety of the Agency operations. We need to find the right balance here in the future so that these types of issues can be resolved with the same cadence as the sprint teams want to work.
But we don’t want the elephants back.
7 comments
Comment by Paul Moffat posted on
I'm not sure why you are only just entering the testing phase of two agile programmes. testing should be performed throughout surely?
Comment by Mark Winspear posted on
Hi Paul
Testing has been ongoing throughout each project. I think the article is referring to being at the point where the two systems are actually talking to each other.
Comment by Chris Thompson posted on
Hi, Paul, We've done the respective testing in sprints as you would expect. The difference here is that we have two agile project teams now having to manage dependencies outside of their normal "epic" level - these are not just separate sprint teams but completely separate programmes with separate governance and stakeholders and have meant cross functional testing has needed to be managed separately. There have also been security barriers that have prevented us from properly testing the integrations as early as we would like.
However, you are right and in the future we would like to remove these blockers and test across the piste in line with the agile development cycle.
Comment by Col Sweetman posted on
I'm aware of the journey you have all taken to get to where you are now. It's a terrific achievement, and I take my metaphorical hat off to you all. Onwards and upwards, Inspect and Adapt.
Comment by Brian Crowley posted on
Hi Chris,
I'd come across this blog post after reading the previous one regarding service-based design. Service-oriented architecture would appear to be a potential candidate to alleviate some of your issues here, so I'm wondering whether you came to the same conclusion?
Comment by Chris Thompson posted on
Hi, Brian. You are right, we came to the same conclusions and are following a service-oriented architecture strategy. With our need to be much more agile in terms of both business response and software development it was the most appropriate option. The trick is to avoid falling into the bad practices that some SOA-architecture driven initiatives fall into and ensure we continue to build the foundations and culture across the Agency.
Comment by Scott Miller posted on
12 months ago it was "living the dream" and now you are making the dream !!!! Great article and very pleased to hear about the progress which has been made. Bring on R01 🙂