DIGITAL LIFE

How Google is fighting camera racism

With Real Tone technology, Google trains its algorithms to properly represent racialized people. Numerama met Florian Koenigsberger, the project manager, and Natacha Ikoli, a colorist specializing in lighting and presentation.

Are the cameras of our smartphones racist?...If they were not designed for this specific purpose, the majority of them continue to neglect people of color today. Trained for decades on models that resemble their creators, often white men, image algorithms are full of biases. Without doing it on purpose, they usually choose to highlight a white subject at the expense of a black person. People of color know that they are victims of a form of digital discrimination, since their skin often appears underexposed, overexposed or simply different from reality.

Beyond the question of the choice of prioritization between a black and a white, the algorithms of the cameras are simply not trained for black skins (the tools dedicated to color grading are based on light skins). A problem highlighted by Google with its Pixel 6 in 2021, the first smartphones officially equipped with Real Tone technology (the Pixel 5, released a year earlier, was secretly experimenting with it). Numerama met the teams behind this important technological progress.

Real Tone, awareness of the problem...The Real Tone project starts from Florian Koenigsberger, who joined Google as an intern in 2017. This son of a black Jamaican mother and a white German father, himself a photographer, began to take an interest in the question algorithmic biases in 2018. After talking to his manager about his idea to improve this aspect on Pixel smartphones, which are known to be excellent in photos, Florian took the helm of a project that was only officially announced three years ago. later. Research, meetings… The hard part with Real Tone was knowing how to touch on such a sensitive subject in the right way. “We know how to do computational photography, but we can’t speak for the community. We needed a lot of people” explains the project manager.

To achieve this, Google has created a kind of “task force” made up of racialized people. Several experts from the world of photography came to help the company make the right choices, to finally teach its smartphones to properly represent colored skins. “It was very important not to involve experts only at the end of the project”, comments Florian. There has obviously been a lot of machine learning, to teach the Pixel photo algorithm to treat people differently based on their color, instead of doing everything on the model of white skin. “No one asks black photographers if they know how to photograph white people”remarks the project manager.

In the Google team, there are several artists and representation specialists. Among them, Natacha Ikoli, born in Congo and raised in Paris. Having become a colorist and a specialist in character presentation issues, Natacha is an expert in video colorimetry “When you’re a black person, you see when certain people come across as overexposed or too dark”, she explains. A problem that white people don’t always see.

In 2020, Google started rolling out Real Tone. In 2021, he officially unveiled it. Thanks to this technology, currently reserved for Pixel smartphones, skin tones are more faithfully represented on Pixel smartphones than on other mobiles. The textures are also better, since the algorithms anticipate what to do when they recognize a black person.

Can this technology become the standard at Apple, Samsung and Xiaomi?...Today, Real Tone is limited to the Pixel camera. There are filters in Google Photos, but Florian Koenigsberger suggests that only photos actually taken with Google smartphones use his new algorithm really well: “There are ways to improve old photos, but if your starting work is bad, it won’t be good”.

What about other Google products, such as Google Meet (the video call service with which we conducted the interview with Florian Koenigsberger)? Real Tone isn’t used in other Google services today, which means webcams still don’t do black skin justice.

“It’s complicated, but we’re working on it” explains Florian Koenigsberger, who highlights compression issues.

Finally, how to explain that no other brand of telephones has looked into the subject? Google says it is discussing with manufacturers of Android devices, but the reality is that it is the only one to communicate on this aspect in 2023. Should we see a lack of interest from other brands? Florian Koenigsberger does not think so: “It’s not for nothing that we took so long. It’s not a quick job”. According to him, all the other major brands (Apple, Samsung, Xiaomi, etc.) are working on this subject, but none is yet ready to highlight their progress. “They know that when we talk about this, people will ask you questions”. “It’s good that others take their time”.

In a few years, our smartphone cameras might be able to capture dark skin with ease, so much so that the question won’t even arise. In the meantime, Pixel smartphones are the only ones to really claim this quality.

by Nicolas Lellouche