CANON

The company showed an impressive “AI” to improve photos, removing noise and aberrations

Artificial intelligence has been a big topic these past few weeks, but that's also true of the fundamentals it's based on, like deep learning. That's what the company Canon did in the development of new algorithms for editing photos and removing many problems that can arise during shooting, be it noise, Bayer interpolation or optical defects of the lens. An article appeared on the company's Japanese website that describes several new algorithms using deep learning to significantly improve photo quality.

One of the traditional problems is noise. This creates a grainy image and its removal is possible (and fairly easy), but too much effort to suppress it also leads to a loss of detail. This is often seen, for example, in “fuzzy” hair, grass, trees. Developing a new noise reduction system was not easy. Early attempts led to many strange artifacts, and in some cases the results were even worse than conventional methods. However, by changing the structure of the neural network, the learning process and the training data, it was possible to develop a system that works much better.

Another persistent problem is the moiré effect, which arises as a result of Bayer interpolation. Only one color is captured at each sensor pixel (except for Sigma's Foveon), so the remaining two colors must be estimated based on surrounding pixels. This leads to color artifacts in some types of scenes, which can be noticed, for example, in striped shirts. Efforts to suppress this effect have so far resulted in suppressed colors and reduced detail. Now the interpolation is done by a neural network, which gives more priority to the brightness change than to the color, but also manages to suppress the false colors of the moiré effect.

This was a problem with the sensor chip. However, the lens can also introduce disease into the image. The third area that Canon has focused on is precisely these optical aberrations, such as reduced sharpness in the corners. Using simulations, he created images on which he trained this system. This should help, for example, landscape photography, where it's always a battle between low and high apertures. Low apertures will bring out a sharp center and (usually) unclear corners, high apertures will sharpen the corners but bring out a slight overall blur due to diffraction.

The system also dares to remove chromatic aberration, which mainly occurs in the contrasting edges closer to the corners of the image. Here, too, computer graphics and training data generation came into play, a lot of trial and error, until a really impressive result was achieved.

But even in this case it was not won. While each of these functions works well individually, the big problem has been connecting them together so that making one correction doesn't negatively affect the other. This was finally done and now the question remains when we will see this system in a software product that we will be able to use.

by: Milan Šurkala