Cloud Society: Learning from research on IT complexity

The problem with IT systems is that they develop a complexity of their own. Can cloud cut through this - or add another layer?

As humans, we’re very good at seeing things from certain perspectives (notably our own), but the broader picture can be elusive.

This factor is particularly relevant for IT, in which conversations so often start from the technological point of view, and frequently end in discussions about people, process and politics. We’ve seen this in cloud computing as much as anywhere else. But what if we could start with the latter, rather than the former?

One UK initiative which has been tussling with this challenge is the Large Scale Complex IT Systems (LSCITS) group. Overseen by Professor Dave Cliff of the University of Bristol, LCSITS was set up five years ago to consider what happens when computer systems – or systems of computer systems – grow beyond a certain scale. For ‘large’ think ‘impossibly large’: for example, the NHS computing environment (in its entirety) is one such ‘large’ system. 

Cloud computing gets a direct mention in the models defined by LCSITS. The term was a mere twinkle in the eye of the IT industry at the time the initiative was started and as such, came under the banner of ‘novel computational approaches’. Cloud-based systems can form one part of the systems-of-systems covered in the LSCITS stack.

As we have seen, the term ‘cloud’ has grown to become a standard part of industry parlance. At the same time it has hit a complexity glass ceiling of its own, manifested in the way that people have talked about hybrid clouds. "The future is public," says one set of evangelists; "the future is private," says another. "The future is hybrid," say the pragmatists, which has the consequence of leaving organisations to work out what ‘hybrid’ means, in terms of architecture, process, and how it enables business requirements to be met.

As per usual, we have approached the debate around cloud from the bottom up, thinking about technology first rather than starting from the top of the layered model captured by LCSITS. Notably, it defines a new field of systems engineering to sit above the technical layers of the stack– that of socio-technical systems engineering. This covers, from the website, “approaches and findings from sociology, psychology, and management theory.”

We have approached the debate around cloud from the bottom up

While enterprise architects work in the socio-technical domain, they can tend (once again) to focus on process, structure and control, rather then such ‘wooly’ areas as the above, which fall more into the camp of change management. In other words, current approaches are fragmented – and the starting point “this is a socio-technical issue” – remains a topic for bars and workshop sessions, rather than being front of mind during technology projects. 

While we might all agree that socio-technical approaches are best in principle, good practice remains elusive. The good news is that LSCITS has been defining structures, principles and methodologies that include requirements capture, interface design, deployment and continuity planning. University syllabuses have been defined, handbooks produced. Indeed, the only real question which arises is – outside of the circles in which such work has taken place, why haven’t the lessons been given wider visibility?

Still, with the LCSITS programme coming to a close, the insights it has amassed are invaluable to those responsible for complex systems in general, and those involving large-scale distribution of resources – aka cloud computing – in particular. The future is indeed hybrid, but not just technologically. Only by understanding of both the cloud’s place in the overall, complex architecture, and also the social aspects involved in any large-scale deployment, can we really hope to create large-scale, hybrid technology environments in the longer-term. With initiatives such as LSCITS, at least, we have made a start.