Damon Woodard

Applies Object Localization Algorithm to Identify and Locate Objects in Large, Complex Images

This object localization algorithm and window search technique identifies and locates target objects from scanning electron microscope images of integrated circuits to assist detection of defects and malicious circuits. Integrated circuits play key roles in nearly all modern electronics and are valued at $391 billion globally. The integrated circuit supply chain is vulnerable to security breaches including the sale of substandard, counterfeited circuits that can lead to poor quality or entirely non-functional electronics. In the past, object localization algorithms identified target objects in other fields using optical images, but these did not assess the quality of integrated circuits due to limitations of the optical images themselves.

Researchers at the University of Florida have developed an object localization algorithm that identifies target objects in integrated circuits from scanning electron microscope images. This algorithm quickly identifies and locates problems such as defects in integrated circuits.

Application

Object localization algorithm that identifies and locates target objects in integrated circuits from scanning electron microscope images

Advantages

• Quickly identifies and locates target objects in integrated circuits, allowing for rapid, high-quality hardware assurance of integrated circuits
• Enables the localization of various structures with high reliability, allowing for the search of multiple target objects simultaneously
• Reduces search time in comparison to traditional window search techniques, speeding up hardware assurance of integrated circuits

Technology

This object localization algorithm uses a 2D search-string algorithm in combination with the window search technique and a knowledge-based mask window to search for multiple target objects from scanning electron microscope images of integrated circuits.

Reduces Scanning Electron Microscope Imaging Time and Creates Standard Cell Libraries Directly from Circuit Images

These algorithms apply artificial intelligence to reverse engineering and hardware assurance, decreasing the time required to image integrated circuits and enabling successful logic extraction when no standard cell library is available. The semiconductor industry commonly uses scanning electron microscopy (SEM) to image chips for failure analysis, hardware assurance, and reverse engineering. To ensure image accuracy, SEM typically uses a high magnification setting for all chip structures regardless of shape or size, which inflates the imaging time requirement, especially for increasingly complex circuits. Available algorithms for extracting the overall logic of an integrated circuit require its template standard cell library, which inhibits the reverse engineering of commercial off-the-shelf components with no design information available.

Researchers at the University of Florida have developed a set of algorithms that improve SEM imaging and reverse engineering of integrated circuits. One algorithm intelligently and automatically adjusts magnification to improve overall SEM imaging time while maintaining accuracy, and the other creates standard cell libraries directly from SEM images to facilitate reverse engineering of integrated circuits with no known design details.

Application

Intelligent SEM imaging of integrated circuits that is much faster for more efficient hardware assurance; automated algorithm that allows reverse engineering with no knowledge of an integrated circuit’s design rules

Advantages

• Automatically updates SEM magnification, greatly reducing imaging time requirements for IC hardware assurance or reverse engineering
• Accounts for shape and size of IC structures, tuning magnification to make features in image clearly visible
• Imaging algorithm is simple and parallelizable, improving efficiency of any existing imaging platform
• Extracts the standard cell library from an SEM image, enabling reverse engineering of ICs without knowing any of its design rules

Technology

The first algorithm incorporates the idea of shapes and sizes to integrated circuit structures in scanning electron microscope (SEM) imaging and dynamically adjusts parameters so that larger structures get imaged with lower magnification and smaller structures with higher magnification. This introduces the flexibility of automation to a previously hand-tuned imaging platform, saving considerable time in the imaging step of hardware assurance and reverse engineering. The second algorithm determines design details of an integrated circuit from raw SEM images of the contact layer. It extracts a circuit’s standard cell library itself, allowing successful logic extraction from chips with completely unknown design rules, such as certain commercial off-the-shelf devices.

Heightens Contrast in Lower Quality Scanning Electron Microscopy Images for More Efficient Reverse Engineering or Failure Analysis

This image-processing algorithm segments the structural elements of integrated circuits in scanning electron microscopy (SEM) images to enable reverse engineering or failure analysis. The increasing complexity of advanced integrated circuit (IC) chips has rendered optical imaging obsolete for efficient reverse engineering. Computer vision algorithms now expedite reverse engineering of present-day integrated chips by processing higher-resolution scanning electron microscopy images of the densely packed chips in order to automate structural component segmentation. However, available segmentation algorithms only work on higher-quality scanning electron microscopy images, which take a very long time to process and require human interaction to optimize image parameters.

Researchers at the University of Florida have developed an algorithm that analyzes a low-quality scanning electron microscopy image of an integrated circuit and automatically segments the image into structural elements for reverse engineering. The algorithm enables faster integrated circuit imaging and eliminates any need for human interaction, increasing efficiency in the reverse engineering process.

Application

Computer vision algorithm that automatically segments the structural elements of integrated circuits in scanning electron microscopy images to aid reverse engineering, failure analysis, debugging, hardware security, and intellectual property protection

Advantages

• Segments integrated circuit structure in low quality scanning electron microscopy images, reducing time requirements for image processing in reverse engineering applications
• Does not require parameter optimization, enabling automatic image segmentation to cut processing time and costs
• Avoids modeling noise sources, producing high contrast images of the internal structure of an integrated circuit
• Simplifies otherwise lengthy and expensive integrated circuit reverse engineering, benefitting applications including device reconstruction, optimization, debugging, hardware security, and intellectual property protection

Technology

This algorithm applies histogram-based auto segmentation to the integrated circuit (IC) structures present in scanning electron microscopy (SEM) images acquired under low magnification and/or having poor qualities. The algorithm sets the scanning electron microscopy image through a series of stages in which it extracts the histogram of the image, corrects it, and segments the histogram based on its number of peaks. The algorithm does not try to model noise sources and does not require parameter fine-tuning. The segmentation algorithm relies on the working principles of scanning electron microscopy imaging to produce a high-contrast integrated circuit image suitable for reverse engineering or failure analysis. This greatly simplifies the traditionally lengthy and expensive integrated circuit reverse engineering workflow.

‘BLOcKeR’ Uses Hardware Obfuscation to Restrict Access and Operation of Critical Device Hardware Without Biometric Verification

This biometric recognition system secures Internet of Things (IoT) devices, safeguarding user’s sensitive biometric data against common attacks. Shortly, billions of devices will be electronically connected to the Internet of Things (IoT). The IoT enables users to connect to the Internet across many devices, such as smart homes, cars, IoT wearables, and mobile devices, allowing them to take full advantage of their services and the data they generate. However, with the growing use of these connected devices, users must maintain and secure the IoT devices.

Internet of Things-enabled devices are especially vulnerable to supplying fraudulent data unless stringent protocols are employed to verify the user's identity. With billions of IoT endpoints, traditional forms of access control, such as passwords, are feasible. However, a strong password is difficult to remember and employ for various devices. Dongles and smart cards have become popular alternatives for strong passwords, but theft and misuse threaten their use. Biometric recognition, a scheme for identifying people based on fingerprints or other specific traits about their physical appearance, is a compelling solution for verifying the users of IoT devices. However, a usable biometric recognition system must incorporate protection from attempts to steal the biometric information previously stored and authorized by the system, as well as attempts to use biometric information stolen from other systems to gain access. There is a need for low-cost access control schemes that allow humans to activate and maintain IoT services and systems.

Researchers at the University of Florida have developed a biometric recognition system, BLOcKeR, safeguarded by two hardware security mechanisms: physically unclonable functions (PUFs) and hardware obfuscation. This system fingerprints each IoT device using PUFs, so authorized biometric information only works to unlock the device it was collected. It also employs hardware obfuscation to turn off the components that process raw biometric data into a mathematical identifier – a template -- and then match it to those in the database of authorized templates, ensuring no access to the sensitive database without the proper biometric information.

Application

Biometric identification for internet-of-things devices, secured from common attacks such as template theft by hardware obfuscation and physically unclonable functions

Advantages

• The user’s biometric information is not collected in its raw form, preventing attacks aimed at stealing and misusing biometric information
• Deploys physically unclonable functions to fingerprint the device or access point that the user is authorized for, preventing template hijacking attacks to gain entry to another database
• Secures the system biometrically at every juncture, avoiding reliance on passwords that are forgotten or easily guessed

Technology

This biometric identification system incorporates security measures to safeguard its usage. In the context of biometric recognition, the circuits store, retrieve, and process biometric data in its raw form and protect the template of the authorized user. Hardware obfuscation achieves this by restricting the operation of these circuits unless a hardware key is present to unlock them.

In this biometric system, the key is derived from the user's biometric data, eliminating the need for passwords. This results in the system not retrieving or processing authorized templates from the database unless presented with the correct biometric data. Since the retrieval and processing steps are sensitive to attacks, locking these steps with hardware obfuscation enhances the system’s security. It uses physically unclonable functions (PUFs): characteristic defects of an electronic circuit created by subtleties of the manufacturing and impossible to reproduce. These defects allow PUFs to distinguish one device from another, and this system harnesses that distinguishability to irrevocably link authorized biometric data to the device it was collected on, preventing misuse of biometric data across devices.

Classifies low-resolution images of integrated circuits to achieve compatibility with faster, cheaper imaging

This image-processing algorithm produces robust segmented images from low-resolution scanning electron microscope images of integrated circuits without a reference ground-truth image. Arriving at images of integrated circuits that are segmented, meaning that each pixel is labeled as part of the integrated circuit or not, is a vital step when checking that an integrated circuit was built as intended (hardware assurance) or determining the logical function of an integrated circuit from its physical structure (reverse engineering). Scanning electron microscopes are powerful tools for imaging integrated circuits, but achieving images of today’s intricate integrated circuits high-quality enough for segmentation requires the microscope to dwell for a long time over each cell of the integrated circuit, quickly adding up to weeks of imaging times. Therefore, a procedure for segmenting low-quality integrated circuit images produced with short dwell times is necessary to avoid the time costs of integrated circuit reverse engineering and hardware assurance spiraling out of control.

Researchers at the University of Florida have developed an image-processing algorithm that is more effective in segmenting low dwell-time images, resulting in impressive time savings during scanning electron microscope imaging without sacrificing segmentation reliability. This parallelizable algorithm also leverages machine learning so that it can segment features of various length scales without fine-tuning, making it widely applicable.

Application

Produce reliable segmented images of integrated circuits using only easily available, low-quality scanning electron microscope data as input

Advantages

• Effectively analyzes low dwell-time scanning electron microscope images, enabling faster imaging time
• Segments multiple regions of the scanning electron microscope image in parallel, reducing the required computational time
• Processes images using an unsupervised algorithm, achieving broad applicability without the need for fine-tuning or ground-truth reference images

Technology

Scanning electron microscopes work by firing a beam of electrons at a small region of the sample. These electrons subsequently scatter off and are collected to form an image of the region. Dwelling over the same region in a firing/collecting state for a longer time correlates with higher-quality images and shorter dwell times can significantly increase image noise as well as reduce image quality. For reverse-engineering purposes, each pixel of the final scanning electron microscope image must be classified according to whether it is part of the integrated circuit or not. The many different length scales of integrated circuit structures render this task challenging. However, machine learning algorithms are well suited to learn how to identify whatever features may be present. This algorithm deploys Gaussian mixture models in combination with different low-pass filters to train itself to identify the various frequencies present in the image. Rather than being programmed to pick out certain structures, this algorithm benefits from an unsupervised workflow that allows it to choose the best filter to remove noise but recover sharp edges. It achieves all this with significant time savings due to accepting low dwell time scanning electron microscope images and being parallelizable.