In some implementation, a system for identifying malicious attacks on a convolutional neural network (CNN) model includes a target computing system that performs classification of objects using a CNN model, and an attack identification computing system that identifies an injected neural attack. The attack identification computing system can be configured to generate, based on the CNN model and associated parameters, an ecosystem of CNN models by modifying original weights of the parameters associated with the CNN model; update the original weights of the parameters with the modified weights; store, in a secure data store, the updated weights of the parameters; generate, based on the updated weights, n update file for the CNN model; update, using the update file, the CNN model; and transmit the updated CNN model to a targeting computing system configured to detect neural attacks by an attacker computing system based on the updated CNN model.Systems and programs that rely on neural networks are susceptible to cyber-attacks, including the injection of malicious training data during the training process. This can enable attackers to extract valuable training data and model parameters and can lead to targeted modification of the network behavior, all while being stealthy and difficult to detect. Our researchers have developed a system for securely deploying convolutional neural networks (CNNs) to critical edge applications – such as self-driving cars. The system consists of methods for the detection of and defense against Trojan attacks in the CNN. The technology leverages a process called stochastic parameter mutation to mutate the original CNN into an ecosystem of unique CNNs. The ecosystem is then deployed to the hardware devices, where each device has its own unique variant of the CNN model. This, coupled with a special update mechanism, enables a one-to-one relationship between a model (device) and any updates made to it. If a device receives an update meant for another device, it will not work properly, and the attack will be easily detected. This makes it difficult for an attacker to insert a stealthy Trojan and reduces the likelihood of a Trojan update from spreading to other devices in the ecosystem.
Brochure
