What is Edge AI?

Edge AI is the use of machine learning (AI) algorithms that run at or near the source of data they analyze (i.e., the edge). Edge AI is a new term that describes what is fundamentally an old way of doing things — using on-device processing to crunch data and predict an outcome, as opposed to sending that data up to the cloud for processing. The reason edge AI has emerged as a term, though, has a lot to do with the specific kind of AI we’re usually talking about, in the form of on-device machine learning models.

How Does Edge AI Work?

Machine learning models are core to the concept of edge AI. In the very simplest terms, a machine learning model uses vector math to analyze data inputs and generate a predictive outcome. A given ML model is often trained on a particular use case, such as speech recognition, object detection, or otherwise determining the likelihood of a particular condition (e.g., the number of defects in a stainless steel bolt). It’s important to remember that ML models are predictive, not conclusive — they are trained on relevant data that is intended to “teach” them how to predict an output for a given use case. This is what is meant by “ML model training.” The more relevant data a model is trained on, ideally, the more accurate the predictions of that model will be in the real world. (Ideally is a very big word here, model training is a lot harder than it sounds! ML model training is extremely computationally expensive, and more data doesn’t always make a better model.)

These ML models often run on extremely powerful server clusters in the cloud, as some ML models are incredibly large — hundreds of gigabytes in size or more. And the amount of processing power needed to run models of such sizes is immense; not even a powerful desktop workstation would provide usable performance. But other ML models are quite specialized and designed only to determine very specific conditions, and these models tend to be much smaller than the “general intelligence” large language models (LLMs) of companies like OpenAI. So small, in fact, that they can be deployed on devices like smartphones, individual blade servers, desktop or laptop computers, or even IoT devices. When a machine learning model is deployed on one of these devices, we call it “edge AI.” The edge refers not to the cutting edge (though that’d be cooler), but the fact that these devices exist far away from the centralized cloud computing infrastructure where most AI processing happens.

In many ways, on-device AI is edge AI, as you’ll see when we define edge AI use cases below.

What Are Edge AI Use Cases?

One of the most widely deployed forms of edge AI today is assisted (or automated) driving in passenger cars. Using a suite of cameras and sensors, many modern vehicles “see” the road you drive on, recognizing speed limit signs, lane markers, pedestrians, and other vehicles. Machine learning (in this case, computer vision) algorithms running locally on processors inside the vehicle analyze the data produced by these sensors to determine road conditions. Those conditions then are processed by behavioral models on the car into an outcome, such as adjusting vehicle speed automatically when the speed limit of a road changes. In this scenario, the car’s machine learning algorithms likely only receive major updates infrequently (if ever, in some cases), and data about the behavior of the vehicle and performance of the model may not even be transmitted back up to the cloud — such systems can operate in an entirely “closed loop” fashion. 

In truth, there is no meaningful difference between “on-device AI” and “edge AI” in most cases; the two are one and the same. Further, “on-device processing” functionally describes what edge AI is in many circumstances; it is, in some ways, a buzzword label.

Edge AI could even be reduced in application to the algorithm that detects faces in the viewfinder of a standalone digital camera. After all, the camera is running a computer vision algorithm (i.e., doing vector math on image data), which is a form of machine learning, and that algorithm is running on-device at the “edge.” Therefore, edge AI. And the term edge AI has been retroactively applied to other use cases ranging from medical pacemakers, glucose monitors, industrial sensors, and smart thermostats, to video surveillance and retail automation. In truth, edge AI is an extremely slippery concept that can be applied to a huge number of systems and applications given the correct framing.

Some of the most pertinent examples of edge AI to understand the broader concept are:

• Autonomous vehicles: Automated driving requires a lot of local computing power and demands near-real-time responsiveness. The data processed is complex, heterogeneous, and highly specialized. This is a clear-cut example of edge AI.
• Retail robotics: Robots designed to spot hazards, validate placement of goods, and identify security risks in retail stores must process very large amounts of visual data from many cameras and other sensors. This means on-device processing is crucial, and requires a sophisticated machine learning model. Another strong case for the “edge AI” label.
• Retail automation: Retailers are increasingly exploring the idea of using computer vision to identify the goods a customer has decided to buy and provide a seamless checkout experience, as opposed to traditional barcode scanner-based checkout systems. Such implementations require byzantine camera and sensor networks that must process data rapidly on edge AI servers inside the retail location.
• Manufacturing automation: Manufacturing goods requires validating the quality of product coming off the line, and computer vision and various sensors can be indispensable to the quality assurance process. Similarly, ensuring the performance of manufacturing systems through constant monitoring with AI advances this goal. Real-time or near-real-time performance is crucial in such environments, where a production fault could easily cascade into substantial lost productivity or damage to equipment.

How is Edge AI Different from Cloud AI?

Edge AI definitionally runs at or very near the source of the data that the AI processes to produce a prediction or result. Cloud AI runs on a centralized resource (e.g., a server farm) and requires sending data from the source to the cloud. Depending on the type of data, cloud AI could be extremely bandwidth-intensive and produce highly undesirable latency, making it unworkable for certain use cases like vehicle automation or medical device monitoring. 

What are the Benefits of Edge AI?

The benefits of edge AI (as compared to cloud AI) tend to be very similar to the benefits of local processing versus cloud processing — as they are fundamentally similar concepts. Some of those benefits include:

• Bandwidth: Processing data for use by AI directly on-device (or very, very close to the data source) means no to little bandwidth cost, as there is no data transfer occurring or, if there is, it is only occurring inside local resources. This means the only real limitation on data size and complexity comes in the form of the local processing power you have at the edge.
• Compute power: With cloud computing of any kind, you “rent” the processing power you’re using, which is very costly. If you already own the edge devices your AI model is running on, you’re using compute power you already own. 
• Security: The data you don’t send is the data someone else can’t steal (well, usually). By keeping AI processing on-device, you’re inherently operating in a more secure, high-trust environment.
• Extensibility and scalability: Assuming your use case doesn’t require a constant internet connection, edge AI allows you to deploy models wherever a device physically can be accommodated. This is absolutely crucial for use cases like vehicles or medical devices. Scalability also benefits, as the necessary AI capability is built directly into the hardware.
• Real-time performance: By far the greatest benefit of edge AI is in enabling real-time use cases. Vehicles, medical devices, and high-sensitivity uses like manufacturing and industrial safety require near real-time responsiveness that cloud AI simply can’t guarantee.

What is the Future of Edge AI?

Edge AI’s growth is dependent on two inverse functions: the growth of on-device AI computing power, and the efficiency of on-device machine learning models. As devices grow more powerful, they can use more complex ML models for on-device AI to generate more accurate predictions, eventually opening up new use cases. And as ML models become more efficient, the more successfully they can leverage on-device processing.

Real-time medical monitoring seems like one area where edge AI will grow in the coming decades. Vehicular autonomy will certainly be another. Retail, manufacturing, and industry will all benefit, too. Unlike the more generic large language model (LLM) AI space, these use cases rely on highly performant ML models trained for very specific data, and they are already successfully proving themselves out in the real world. While it’s unclear where a breakthrough moment is most likely, edge AI is going to continue its growth in the enterprise and B2C segments for the foreseeable future.