What is failure?

The terms fault and failure are sometimes used loosely to mean the same thing but they are actually quite different. A fault is something inherent in the software – a failure is something that happens in the real world. Faults do not necessarily lead to failures and failures often occur in software that is not ‘faulty’.

The reason for this is that whether some behaviour is a failure or not, depends on the judgement of the observer and their expectations of the software. For example, I recently tried to buy 2 day passes on the Lisbon metro for myself and my wife. They use reusable cards so you buy 2 cards then credit them with the appropriate pass. The dialogue with the machine went as follows:

How many cards (0.5€ each): 2
How many passes (3.7€ each): 2
Total to pay: 15.8€

To put it mildly, I was surprised. I tried twice, the same thing happened. I then bought the passes one at a time and all was fine – I paid the correct fee of 8.4€.

From my perspective, this was a software failure. It meant that I had to spend longer than I should have buying these passes. On the train, I tried to think about what might have happened. What I guess is the situation is that it is possible to have buy more than 1 day pass at a time and have it credited to the card. So, the 2nd question should have been:

How many passes on each card?

From a testing perspective, the software was probably fine and free of defects and, if you understood the system, then you would have entered 1 pass per card.

So, failures are not some absolute thing that can be tested for. They will always happen because different people will have different expectations of systems. That’s the theme of my keynote talk at SEPGEurope 2010 conference in Porto. We need to design software to help people understand what its doing and help them recover from failures.

Every cut has a silver lining

The UK Government has announced cuts of up to 25% which will be imposed on universities over the next 4 years.

I’ve been reading a recent document from IBM called Capitalising on Complexity, which emphasises the importance of innovation and creativity and this has triggered a reflection on the contribution that the computer science research community in universities can make to this. Sadly, the conclusion I’ve come to is “if we can do anything it is in spite of rather than because of existing research structures and management”.

The problems that we suffer from are primarily imposed by the need for research quality to be ‘measured’ – either at the individual level (career progression, tenure, etc.) or at the institutional level. We are all encouraged to publish regularly in ‘high-quality journals or conferences’ and to write research proposals for external research support. More and more people are now competing for very limited funding.

The end-result of this is conservatism and incrementalism. It is dangerous to your career to go into a new area or to think differently as there are no ‘high quality’ journals and conferences to publish in. If you make proposals where you suggest interesting questions to explore with no clear idea of the results you will achieve (what I think of as real research), you have zero chance of funding because your proposal will inevitably have lots of holes in it that reviewers can challenge.

Research funding bodies, to their credit, are aware of this problem and sometimes support special initiatives (like the LSCITS project) to try and be more innovative. By and large, however, these rarely work as the pressures for incrementalism that are imposed by the current university system are just too great. Researchers have to think of their future – if they take 3 or 5 years out to ‘think differently’, then they will probably never get another research job.

All of this means that CS research in universities is not the lever for innovation that it should be, it does not encourage creativity, nor is it addressing the grand societal challenges that we face.

Paradoxically, perhaps, the inevitable cuts in university and research funding may offer us a way out of this situation. If there are no research jobs, then the notion of a research career is less important and smart people don’t have to be so concerned about publications. Cuts in travel budgets mean that less time is spent travelling to conferences to present papers to people who are mostly reading their email anyway. The hateful research assessment may disappear and we can start thinking long term rather than writing about another incremental advance.  Maybe some of us oldsters will be kicked into early retirement before senility sets in and we will have time to think  differently.

But we must try and maintain support for our PhD students. PhD’s themselves are mostly incremental – students have to write and defend a thesis and innovation is inherently risky. But PhD students have time to think, to be innovative and to come up with new and exciting ideas for the future. With fewer research jobs, they may focus on startups who are, it seems to me, to be the true source of innovation nowadays.

Email and the tragedy of the commons

The tragedy of the commons is a situation where multiple individuals behave rationally in their own self-interest and ultimately destroy a useful shared resource, even when it is perfectly clear that it is not in the best interests of the group as a whole for this to happen. So, if a ‘common’ can support grazing for 50 sheep and 5 people put 10 sheep each on it then all is well. However, if 1 person puts 11 sheep on the common, this starts an arms race where each individual then feels compelled to match this. Ultimately, the common is over-grazed and destroyed.

To handle this problem, we have invented regulators who are supposed to stop individual behaviour that threatens the good of the whole (although it didn’t work too well with the banks).

The analogy with email is striking. Email in the 1990s was a fantastically useful tool for supporting distributed working. It appeared to be a cheap and effective way of distributing information to lots of people. However, as it became universal, more and more information was distributed by email to the extent that, in many jobs, it has become impossible to manage the amount of electronic information received. Email is easy – adding one more recipient takes very little effort – and it stimulates responses – even if these are automatically generated responses saying that the receiver is out of the office. We have also placed the burden on the receiver to read their mail – all too often we are told when something hasn’t been done that we have received an email about this.

We have created a situation where a shared, incredibly useful resource, has been degraded to the extent that its value is now questionable in many situations.  Charging does not help – this might reduce spam but wouldn’t reduce the volume of email generated within an organisation. Social media (blogs, wikis, social networks) have the capability to improve things but the lack of standards, the image of these systems as ‘unprofessional’ and sometimes clunky interfaces don’t help.  Organisations really need to rethink their policies and approach to information dissemination and, instead of just relying on email, provide a range of mechanisms through which people can communicate and share information.

And maybe we also need organisational e-mail regulators to make sure that everyone complies with these policies.