Deep neural networks (DNN) have produced groundbreaking results in many complex applications of AI, such as natural language processing, facial recognition, sentiment analytics and object recognition. For instance, the accuracy of Google’s machine translation system improved 60% using a DNN approach.

Finding the right network architecture – that is, the components of the network and how they are instantiated and connected – is essential to this process. If the architecture is chosen based on history and convenience, the network will not reach its full potential.

Much of the recent research in DNNs has focused on designing specialized architectures that excel with specific tasks. With AutoML, DNN design is done automatically. Essentially, the machine optimizes itself, providing effective DNNs to many business problems. Examples of works in progress include image classification for detection of diabetic retinopathy and facial attribute recognition.

Evolving AI Capabilities

Evolutionary AutoML is an advanced form of AutoML in which architectural optimization happens via evolutionary computation. It leverages and extends existing evolutionary algorithms to evolve the network structure, creating and testing many generations of designs to arrive at unique and sometimes unexpected architectures that we will discuss more below. In this way, evolutionary AutoML forms a foundation for improving AI, making it practical, democratizing it and extending it to a wider range of applications in the following four ways:

1) Discovering Better DNN Configurations

Adding evolutionary computation to AutoML makes it possible to customize the network architecture for specific tasks, such as language interfaces, sentiment analysis and self-driving cars. Evolutionary AutoML discovers the principles of the domain and encodes them into the architecture. By doing so, it eliminates human, historical or simplicity biases; instead, it explores the entire space of possible solutions. With enough computing capacity, evolutionary AutoML can find solutions that are surprising, complex and often better-performing than the best human designs.

For instance, evolutionary AutoML may home in on solutions via multiple paths, allowing several hypotheses to be evaluated in tandem. The resulting architectures make innovative applications possible in situations where a high degree of accuracy is important. Such applications include life-saving identification of lung diseases in radiographic images and the recognition and flagging of toxic content on social media.

2) Making AI Practical

Many commercial applications of deep learning need to be run on smartphones or similar-sized processors in wearables, vehicles, appliances and potentially even toys. Unfortunately, the hundreds of millions of parameters of modern DNNs can’t fit into the few gigabytes of RAM in most smartphones.

An important goal here is to minimize the complexity or size of the network while maximizing its performance. The speciation and complexification heuristics of evolutionary AutoML allow it to be easily adapted to multi-objective optimizations to find such minimal architectures.

This functionality could be applied to object recognition systems in smartphones to take advantage of DNNs even with limited memory. Possible applications would be identifying products in a store, automatically recognizing people in pictures or identifying text for translating traffic signs.

3) Taking Advantage of AI Without AI Expertise  

One of the main goals of AutoML is to make it possible for anyone to use AI, including in-house talent without expertise in the field. However, current AutoML systems, for example those from Google and Yelp, are limited in terms of the applications that they support and the amount of optimization they do. They focus on optimizing superficial features of the architecture and do not provide user insights.

In contrast, evolutionary AutoML optimizes the entire architecture, leading to further domain insights, including which variables matter and how they interact. It democratizes AI and makes modern architectures available to solve business problems even when employees do not have specific DNN expertise. For example, using evolutionary AutoML, insurance companies could construct a DNN that analyzes the complex array of risk variables in underwriting.  

4) Expanding the Range of AI Applications

Evolutionary AutoML makes it possible to apply deep learning to a broader range of business problems, including those that lack the millions of training examples and high data volumes usually required for deep learning. Evolutionary AutoML can evolve an architecture that combines a small amount of data, which is all that may be available for a new task, with datasets of other related tasks, such as those in other vision or language datasets.

By learning concurrently, rather than in isolation, each task can be acquired more effectively. A customized architecture discovered through evolutionary AI is crucial for taking advantage of this process. This concurrent learning capability would be highly useful in an environment where multiple related tasks exist, as in recognizing handwritten characters in multiple alphabets or spoken words in multiple different languages.

In these four ways, evolutionary AutoML is transforming how AI can be applied to real-world problems. As this technology evolves, we will likely see much of the manual creation of algorithms and DNNs replaced by more sophisticated, general-purpose automated machines that will aid scientists in their research or engineers in the design of AI-enabled products.

Click here to learn more about evolutionary AI.

Risto Miikkulainen

Risto Miikkulainen

Risto Miikkulainen is Associate VP of Evolutionary AI at Cognizant and a Professor of Computer Science at the University of Texas at... Read more