Seeing the Woods and the Trees

Andy O’Kelly

Innovation Network Solutions News

Our ISO 20000 network management service creates a lot of data. How can networks be visualised in a useful way for business?

This week I visited our Managed Services Centre to review how we manage our largest customer network, which has over 3,500 active devices connected nationally.

Visualising the network in a customer-friendly way

The sheer volume of polling for a network of this size is something readily resolved in the architecture we use – as the volume of devices under management increases, we can scale out horizontally with additional Virtual Machine servers. The real challenge was how to visualise the network in a way that represented the customer’s business, as well as the underlying structure of the eir network. Part of the problem was that the graphical capability of the otherwise excellent network management tools we use restricted what could be delivered. While these tools are great at gathering and interpreting the most nuanced and precise network device stats and metrics, there has been a perpetual lag between what those tools offered in terms of a dashboard and the customer expectation. This expectation was informed by the wider development of graphical interfaces available on the web. The best consumer service sets the pace. The solution here was an obvious one in retrospect – simply to use the graphical interfaces available on the web, while continuing to use the excellent network management functions of the tools we selected previously.

Combining the best of the web with in-house network management tools

So we are now using mash-ups of traditional open-source and Google APIs, combined with in-house development to represent information within our management system’s database. The design of the database includes the data-sets used in instrumenting the network via polling, tables populated with customer information specific to how they want to view their network, and also longitude and latitude information relating to eir network structure used to show where retail outlets are located.

The interface is simple, and capable of representing a very large network in an intuitive and useful way that shows condition, geography, and business impact.

If that’s how we resolve the service visualisation challenge for our largest customer, how do we handle the problem of getting all our customers’ conditions visualised in real-time? One way is to have a view that includes all correlated customer incidents and events, but as the volume and network scale increases, this becomes less and less useful.

Gource provides a key breakthrough in effective network management

The breakthrough here has come from the open source community, and a utility called Gource. Gource displays directory structures as an animated tree, with the root at the centre, directories as branches, and files as leaves. There are excellent examples of how it has been used on YouTube – for instance to visualise software version control

We use it in an unmodified form to look at our structured data, to provide a consolidated view of all managed customer network incidents. This resembles a biological structure, with what looks like the head of a dandelion per customer that grows organically in real-time to reflect incidents detected and resolved, including those that have not had an impact on customer service but point to underlying areas of concern. This represents patterns of incidents and problems very well. There is a running event total for each customer too, which becomes a comparator identifying outlier networks that have a higher than normal level of incidents. This focuses attention, and acts as a sort of league table for the expert engineers who are responsible for customer network stability. Looking at a time-lapse playback of a customer network using Gource, the reduction in the number of incidents can be seen as the expert optimises the network – the resulting improvement is clear to see in the animation.

I learn something new every time I meet our Managed Services Centre team. It was great to see how they have combined Google, open source, and enterprise network management in such a clever and impactful way.
Andy O’Kelly is Chief Architect at eir and a speaker on IT solutions and innovation. Follow Andy on Twitter, connect on Linked In.