AI Inferencing Solutions

Deep Learning

Once an AI model has been trained, the resulting streamlined neural network can then be presented with new data for it to identify - whether it be to recognise and diagnose medical images, identify spoken words or predict habits – these results, dependant on the model’s training, is called inference.

The first approach when inferring looks at parts of the neural network that don’t get activated after it’s trained. These sections just aren’t needed and can be “pruned” away. The second approach looks for ways to fuse multiple layers of the neural network into a single computational step. It’s akin to the compression that happens to a digital image. Designers might work on these huge, beautiful, million pixel-wide and tall images, but when they go to put it online, they’ll turn into a jpeg. It’ll be almost exactly the same, indistinguishable to the human eye, but at a smaller resolution. Similarly, with inference you’ll get almost the same accuracy of the prediction, but simplified, compressed and optimized for runtime performance.