Many sites. How can service quality be compared? (Observability Part 3)

A manufacturing site, a sales office and a retail branch all use the same cloud application. Yet each accesses it through a different mix of connections, providers and security components. How can the central IT department tell where service quality meets requirements and where action is needed?

Observability therefore needs to account for local conditions in a way that allows teams to assess sites and services on a clear, consistent basis. This requires common criteria, with the depth of analysis tailored to each site’s importance to the business.

Establish common criteria

Any meaningful comparison starts with a clear definition of what is being assessed. Connection availability, application response times and the stability of a business process measure different things. If a report conflates them, a site may appear to be functioning normally even though employees struggle to carry out their work.

For services that matter to the business, organizations should define which aspects of quality they need to measure and how. Response times can only be meaningfully compared if it is clear where the measurements were taken, how the service was accessed during testing and which period the results cover. A test run from a local site, for example, is not directly comparable to a measurement taken from a central data center.

The reporting period also needs to reflect how the business operates. A daily average that looks normal can conceal delays during a brief but critical period. Across international operations, reports must account for local operating hours. A service report should make clear whether an issue occurred during production, during trading hours or outside the hours when the service is used.

Using unique names for sites and services, alongside clearly defined measurement methods, helps teams determine whether differences in results reflect service quality issues or variations in measurement conditions.

Match the depth of analysis to business impact

Using common criteria does not mean applying the same thresholds or collecting the same amount of data at every site. What matters is the impact a service issue would have on the business process concerned.

At a manufacturing plant, a time-sensitive process may call for a detailed examination of the local network and its dependencies. At a retail branch, the priority may be point-of-sale systems and their availability. A sales office may place greater emphasis on the quality of communication and collaboration. The services actually in use, and how critical they are to the business, should determine the scope of observability.

A practical approach is to define a common minimum set of information covering the site, its connection, its provider, key services and relevant incidents. Additional measurements can then be introduced for particularly critical services. These might provide a closer view of the local Wi-Fi network, add further access tests or bring security events into the analysis.

This gives teams a clear basis for assessing sites while keeping the effort proportionate to local needs. A small office can still run a critical service, so site size alone should not determine the depth of analysis.

Assess provider performance in the context of the service

Site assessments also need to take external providers into account. Their contractual performance targets cover specific sections of the overall network path. A provider may meet those targets even while an application remains slow at the site.

An assessment must therefore distinguish between what a provider is contracted to deliver and the service quality observed in practice. Which measurement points fall within the provider’s scope? Which sections of the network are the responsibility of other parties? Is there evidence of problems at the site or with access to the cloud? Observability can put this information in context and help teams focus their investigation on the sections affected.

Site reports should therefore group comparable types of sites and services. Recurring issues associated with a provider warrant investigation. Establishing whether those issues stem from the provider’s performance requires an examination of the network paths involved, the measurement conditions and other dependencies.

A useful starting point for observability is to compare one important service across a small number of sites with different characteristics. Where are shared definitions missing? Which local conditions affect the assessment? The findings can inform a standard that can be extended gradually to other sites.

Would you like a clearer basis for assessing service quality across your international sites? Our experts can help you establish common criteria and determine the appropriate depth of analysis for your sites and services.


Image: created with AI