Agile development for government IT systems – beware of the hype

Agile development gets lots of hype. Governments around the world are saying that we must be agile and use agile practices for IT systems development. Whatever the question, ‘agile’ is the answer.

I am sympathetic to the agile manifesto and I think some agile practices such as time-bounded increments and test-driven development are universally useful. I would guess that agile approaches, in some form, are almost universal in companies developing software products.

Agile approaches place a great deal of focus on the ‘user’ – they may include users as part of the development team, they develop requirements in parallel with implementation with extensive user involvement, they rely on users to help develop ‘system tests’. This works really well for product development where the products are clearly aimed at users. Of course, real users are rarely involved but some form of user proxy can be involved – someone from a sales team, other developers playing the role of users, support staff who have real user feedback, etc.

In those cases, if the ‘users’ ask for something that’s too hard to build or too expensive, the organisation itself can decide what to do. It owns both the specification (what’s to be done) and its implementation. It can adapt by changing either or both.

However, when it comes to enterprise systems and, especially, government systems, then things are different. The owner of the specification and the system developer are not the same – some requirements can’t simply be dropped because they are too complex or expensive. Furthermore, the notion of what is a ‘user’ becomes much more complex. Typically, these are large systems focused on complex problems – such as medical record keeping – and there are many different types of user. These systems may have complex governance arrangements, may have to conform to national and international laws and regulations, may have stringent security requirements and their success or failure may affect the careers of politicians. In short, they are very complex systems.

Their are various problems with a user-driven approach to development in such circumstances:

1.     Users tend to put their convenience first and other requirements later. They don’t want the overhead of security and don’t always understand the restrictions that are imposed by those involved in system governance.

2.     Users are not lawyers. They don’t know what the rules and regulations that apply to the system are.

3.     As those involved in the system governance are often not actually users of the system, it is difficult to know how to include them in an agile requirements process. Often they don’t have functional requirements but they simply place constraints on the system.

4.     Users are very busy people. They often simply don’t have time or the inclination to stop what they are doing and discuss requirements for a system which may or may not affect them sometime in the future. When users get involved, they are sometimes the wrong people – those who have a personal interest in technology and are not typical of real users.

Agile methods don’t really, as far as I can see, have good ways of coping with these issues. They present an idealised world where users are engaged and interested and where user interests rather than enterprise constraints are what matter most. This is not the kind of world that I see when looking at national IT systems.

It makes lots of sense to adopt some agile practices for government systems and to try to engage end-users during the development process. However, I am convinced that there is still a need for old-fashioned requirements engineering where all stakeholders are considered, rather than simply diving into agile development.


Filed under agile methods, complexity, requirements, software engineering

8 responses to “Agile development for government IT systems – beware of the hype

  1. Very much agree. Nice post! Living in Washington, DC as a Software Engineering Consultant, I see this on a daily basis. Government Agencies are saying, “We want to go Agile!”. Then the question becomes, “What is Agile?”. And then the kicker, “How does that comply with government requirement 802.3?” Not easy questions to answer. There is no silver bullet!!!!!

  2. Great article. Agile practices should be encouraged but not to the detriment of security.

  3. I’m confused by your assertion that agile “works really well for product development where the products are clearly aimed at users” – what products are *not* aimed at users? After all, what is a user other than a consumer of a ‘product’?

    If government systems are complex and have many security and legal constraints, plus regulatory requirements, perhaps these are just part of the mandatory requirements, and non-negotiable. Engaging users in the development of product, even ‘medical record keeping’, is still a fruitful activity, isn’t it?

    Perhaps the issue here is less to do with agile versus traditional requirements engineering, and more with regard to successful incremental delivery. If more government IT projects were engaged with user needs, and delivered a series of incremental solutions, perhaps there would be fewer monumental disasters for the media to decry.

    Just my 2p 🙂

  4. Badly worded – I should have said ‘only aimed at users’. Of course, users of government systems are important but my experience is that many of them don’t understand the complex constraints on these systems. Why should they – it’s not their job but it’s the reason why they often don’t get the system that they would like.

    The problem with incremental delivery (as distinct from incremental development) is that for many systems, you can’t deliver incomplete functionality – think of a medical system that only could maintain records of cholesterol tests but not blood pressure. Or a tax system that only taxed people who were employed by companies but not who were self-employed.

    • Thanks for the clarification Ian.

      I may be misunderstanding what you mean by ‘incremental development’ but if this means that a working system is not delivered for a significant period of time, then I would take the partial solution from ‘incremental delivery’ any day.

      A medical system that can only record results of cholesterol tests would be better than no system at all – the vapourware promise of an all-singing solution ‘sometime in the future’ is of no use to anyone, is it?

      And without digressing into political views, some may consider that we *already* have a tax system that only taxes some of the people….

  5. Kiran,

    If there is no system in place then of course a partial system may be better. But this is hardly ever the case – there is an existing system which is to be replaced. People will always continue to use the existing system rather than an incomplete new system (a) because the new one may not allow them to so what they want to do and (b) because they will not wish to devote effort to understanding the new system.

    It is often the case that new systems are not introduced to make the life of users easier but because they save money, allow for more management control, etc. So there is little for users to gain by experimenting with a new system.

    • Perhaps this is not the clearest forum to discuss this, as I fear I am misunderstanding the point of your blog post and your latest comment.

      Having said that, if the discussion is now about requirements engineering for the replacement to an existing system, then it sounds like two core requirements are outlined in your comments: (a) the replacement system needs to allow users to do what they need to do (assuming what they want to do is permitted in the regulatory framework); and (b) there should be enough benefits, eg making their lives easier, to entice users to spend time learning the new system.

      If the system is being introduced to save money or to increase management control, and not driven by the desire to make users’ lives easier, or more productive (do more with less) then of course engaging the user base would be fruitless.

      Perhaps the problem in your hypothetical scenario is not agile versus traditional approaches in complex government systems; rather that the underlying requirements (save money, more management control) are incompatible with Amy methodology that involves users in the requirements specification.

      Happy to continue this debate here or on another channel.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s