Tag Archives: complexity

Abstraction and complexity

I gave a talk recently about complex systems engineering at Stirling University where I discussed my notions that software engineering is essentially reductionist and we need to rethink software engineering approaches to cope with the complex systems that we are now building. I was challenged by a questioner who claimed that abstraction was an effective way to deal with complexity and I’m afraid that I dismissed this rather glibly without any real rationale of why it was inappropriate.

I have now thought about this and I now think that I can present a better rationale of why abstraction is ineffective for complexity management. In a nutshell, complexity arises because of the interactions between the elements of a system (see my blog post on complexity). Systems are inherently complex when these interactions are dynamic and where they change their nature over time and in response to environmental stimuli. Complicated systems are ones where there are many elements, perhaps of different types and where elements may have many distinct characteristics but where the relationships between these elements are static.  For example, a topographic map is complicated but it is not complex.

Abstraction, however, is a mechanism for dealing with diversity in the system elements where abstractions represent the essential (for that system) characteristics of a collection of elements. Therefore, if we are building a transport model, we may have an abstraction ‘car’ which has characteristics of size and speed – we don’t care about marque, colour, etc. This is an absolutely essential mechanism for understanding and reasoning about systems and for helping us create software – but it helps us deal with complicated not complex systems.


Filed under LSCITS

What is complexity?

I’m part of the LSCITS project where LSCITS stands for ‘Large Scale Complex IT Systems’ and we have been having discussions about what is meant by ‘complexity’. Some argue that the term complexity should be reserved for complex adaptive systems, systems which are dominated by emergent behaviour. Others argue that ‘conventionally’ engineered systems can also be complex in that their non-functional characteristics and (sometimes) their functional behaviour cannot be predicted. This is particularly likely when we create systems by integrating different parts (often other systems) which are independently developed and managed. In such cases, it is practically impossible to predict how the characteristics of one system will interfere with the characteristics of others.

We suggested that ‘complex’ and ‘complicated’ were not the same. A complicated system is one that is understandable in principle although, in practice, the effort involved may be so great that it is not understandable in practice. This was my own view at one time, but I’ve now changed my mind and think that there is no practical difference between a complex and a very complicated system.  This position has emerged from musings on the roots of complexity.

Some systems are inherently complex – we cannot deduce their properties by a study of their components; we cannot predict the consequences to changes to these systems. System behaviour and properties are emergent and non-deterministic. I believe that such inherent complexity stems from the fact the there are dynamic, dependent relationships between the parts of the system. These relationships evolve in time and according to stimuli from the system’s environment. New relationships may be created and existing relationships may change. As a consequence, deterministic modelling techniques cannot be used to make predictions about such systems although statistical approaches may be used in some cases.

As soon as you consider the people who use a system to be part of the system, you have dynamic, dependent relationships between components in the system so I argue that all large, socio-technical systems can be considered to be complex systems.

There is also another aspect to complexity – what might be called epistemic complexity. This relates to the predictability of system properties when changes are proposed. If you don’t have enough knowledge about a system’s components and their relationships, you cannot make predictions about it, even if, in principle, that system does not have dynamic dependent relationships between its components. Therefore, I argue that large complicated systems are also complex systems when it is practically impossible to acquire the necessary knowledge to understand the system.

This means, of course, that complexity is not just a property of the system but also of the system observer. We have all encountered system experts who know about some system and can make changes to it in a reliable way. Their knowledge is hard to articulate and when they are no longer available, someone taking over the system may find it impossible to develop the same level of understanding. Therefore, what was a complicated system has become a complex system.

Where is all this leading – who cares?

Well, I think it is important to emphasise that complexity isn’t simple. Striving for a simple, universal definition of complexity isn’t really going to get us anywhere.

If we are to try and manage complexity we need a toolbox of theories and methods to do so. To give an example, if you think about dynamic dependencies between components, formal methods of computer science don’t really help. However, if you think about epistemic complexity, they may be very useful indeed as they allow us to state ‘truths’ about a system – filling in our knowledge about that system.

The notion of dynamic, dependent relationships may also be useful in helping us manage complexity. By developing a better understanding of such relationships (e.g. through socio-technical analysis of organisations), we may be able to change the type of these relationships from dynamic to static and hence reduce the complexity of the system.

As I said, complexity isn’t simple so there’s lots of scope for disagreement here.


Filed under LSCITS