Cloud or Edge? That’s the Wrong Question.

Why enterprise AI infrastructure needs to follow the workload

As artificial intelligence moves from experimentation into everyday business processes, organizations are having to make increasingly practical infrastructure decisions. Where should AI applications run? Which workloads belong in the cloud? When does processing closer to users, machines or data make more sense?

It can be tempting to frame this as a choice between cloud and edge. But enterprise AI rarely fits neatly into one environment.

An AI assistant supporting employees has very different requirements from a computer vision system inspecting products on a manufacturing line. A predictive maintenance application processing equipment data has different needs again.

The more useful question is therefore not “Cloud or edge?” but “Where should this particular AI workload run?” The answer depends on what the workload needs to do, where its data originates and how quickly, securely and reliably it needs to operate.

AI does not have one infrastructure requirement

The term “enterprise AI” covers an increasingly broad range of applications.

Some AI workloads process information centrally and can take advantage of the scalability and computing resources available in the cloud. Others interact directly with physical environments, generating and analysing data continuously at factories, warehouses, stores or other locations.

Consider a computer vision application used for quality control on a production line. Cameras may generate a continuous stream of images that need to be analysed quickly enough to identify a defect while the product is still moving through the manufacturing process.

Sending every image to a central cloud environment for analysis may not always be the most appropriate approach. Processing closer to the production line can reduce the amount of data that needs to travel across the network and allow decisions to be made locally.

An enterprise AI assistant presents a different situation. It may need access to applications and information distributed across the organization and benefit from centralized cloud-based AI services.

Both are AI workloads, but their infrastructure requirements are very different.

Food for thought: Are you deciding where AI should run based on an overall technology strategy, or on the actual requirements of each workload?

Sometimes AI should move closer to the data

Much of the discussion around AI infrastructure focuses on compute: where organizations can access the processing capacity required to run increasingly sophisticated models. But there is another side to the equation: the data itself. AI depends on data, and moving large volumes of data between locations, clouds and applications can introduce network requirements, latency and cost. In operational environments, some of that data may also be highly time-sensitive or may no longer need to be retained or transferred once it has been analysed. This is where edge computing becomes particularly relevant.

Instead of continuously moving data to a central location for processing, organizations can place computing resources closer to where that data is generated. AI inference can then happen locally, while selected results or relevant information are sent to central systems.

A manufacturing site, for example, could analyse video or sensor data locally and send alerts, anomalies or aggregated information to a central platform rather than transferring every piece of raw data.

This changes the infrastructure question. Instead of always asking “How do we get the data to the AI?”, organizations can also ask: “Should we bring the AI closer to the data?”

When milliseconds – and connectivity – matter

For many enterprise applications, a short delay in receiving an AI-generated response is perfectly acceptable. For others, it can affect the business process itself.

AI used in industrial automation, computer vision, logistics, connected mobility or other operational environments may need to analyse information and respond very quickly, making processing closer to where data is generated or decisions are required particularly important.

Processing closer to where an event occurs can help reduce the distance that data needs to travel before a decision is made.

Resilience is another consideration. If an AI-supported process depends entirely on connectivity to a distant cloud environment, what happens when that connection is interrupted? Not every application needs to continue operating independently during a network disruption, but for business-critical processes, the ability to perform at least some processing locally can become an important architectural consideration. This does not remove the need for connectivity. In fact, distributed AI environments can make reliable connectivity even more important because models, applications, data and management platforms still need to communicate across locations. The difference is that not every decision necessarily has to travel the entire distance before it can be made.

Cloud still has an important role

Moving AI closer to where data is generated does not mean moving away from cloud.

Cloud environments provide access to scalable computing resources, AI platforms and services that would be difficult for many organizations to reproduce independently across every location. They can also provide a central environment for managing applications, models and data across the enterprise.

The relationship between cloud and edge can therefore be complementary.

An organization might develop and train an AI model centrally, using scalable cloud resources to provide the computing capacity needed during training or retraining. Once trained, the model can be deployed at multiple edge locations for inference, where it processes time-sensitive data closer to where that data is generated. Cloud resources can then be scaled down until they are needed again, while central platforms continue to support model updates, broader analysis and management.

Data is not the only consideration

Latency and data movement are important, but they are not the only factors that determine where an AI workload should run.

Organizations may also need to consider the sensitivity of the information being processed, regulatory requirements, resilience, available computing capacity and the cost of operating infrastructure across different environments.

A workload involving sensitive operational or customer information may have different requirements from one analysing public information. An AI service used occasionally by employees may have a different cost profile from an application continuously processing video from hundreds of locations.

Sovereignty can influence the decision too. As discussed in our previous article, organizations need to understand which infrastructure and providers their AI depends on and where retaining greater control matters.

The result is rarely a universal rule such as “cloud first” or “edge first”; the appropriate location depends on the workload.

From cloud strategy to workload placement

For many organizations, cloud strategy has traditionally focused on deciding which applications should move to the cloud and which should remain on-premises. AI adds another dimension. As AI becomes embedded in physical operations as well as digital processes, infrastructure may need to extend across central cloud environments, enterprise data centres and locations closer to where data is created.

This makes workload placement an increasingly important architectural decision. The objective is not simply to find the technically fastest environment. Organizations need to balance performance, data movement, resilience, security, control and cost.

Those requirements can also change over time. An AI application may begin as a limited pilot in the cloud and later become part of a critical operational process. Data volumes may grow, new locations may be added and performance requirements may change.

Infrastructure therefore needs enough flexibility to allow workloads to evolve rather than locking every AI application into the environment where it happened to begin.

Food for thought: If an AI workload needed to move closer to your operations tomorrow, could your current architecture support it?

Questions technology leaders should ask

As enterprise AI expands across applications and locations, technology leaders may want to consider:

• What latency does this AI workload actually require?
• Where is the data it needs generated and how much of that data needs to move?
• Could processing some information locally reduce unnecessary data movement?
• Does the workload need to continue operating if connectivity to a central environment is interrupted?
• What security, regulatory or sovereignty requirements influence where the workload can run?
• Which parts of the AI lifecycle benefit from centralized cloud resources and which could run closer to the business?
• How will the economics change as the workload, data volumes or number of locations grow?
• Can we move or redistribute the workload if its requirements change?

There will rarely be one answer that applies to every AI application. The purpose of these questions is to make workload placement a deliberate decision rather than simply an extension of an existing cloud strategy.

Building infrastructure that follows the workload

Enterprise AI is unlikely to have a single destination. Cloud will remain important for scalable computing, centralized platforms and access to AI services. Edge environments can bring processing closer to data, users and physical operations, while on-premises infrastructure will continue to make sense for workloads with particular performance, security, integration or control requirements. For many organizations, the result will be a combination of all three.

The challenge is therefore not choosing between cloud and edge, but building an infrastructure that allows AI workloads to run where they make the most sense – and to move as their requirements evolve.

As AI becomes more deeply embedded in enterprise operations, the infrastructure should follow the workload, rather than forcing every workload to follow the infrastructure.

Continue the conversation

As AI workloads spread across cloud, edge and on-premises environments, choosing where they should run becomes an important part of enterprise AI architecture.

If you would like to discuss how cloud, edge and connectivity can support the performance, resilience and flexibility your AI workloads require, our specialists are available to help.


Image: created with AI