Google launches way to “hear” Kandinsky painting using AI to interpret colors as sounds

In collaboration with the Centre Pompidou in Paris, the Google Arts & Culture used IA to try to figure out how Wassily Kandinsky heard his paintings. The Russian artist had synesthesia, a neurological condition that allowed him to hear music in colors and see colors in music. “The sound of colors is so definite that it would be difficult to find someone who could express bright yellow as bass notes and dark lake as treble,” he wrote.

Google researchers tried to replicate this kind of experience using machine learning. Google’s Transformer neural network was fed music from Kandinsky’s era, then they used AI to generate new sound compositions inspired by those samples. The result is Play a Kandinsky, where you can “hear” the work Yellow, Red, Blue, which the artist painted in 1925.

In the experience, users press play on the different colors to hear the rhythms and instruments that Kandinsky associated with each tone. Short texts for each button explain the emotions that the painter associated with each color, such as red and agitated, and yellow and bold. On the interactive website, users can also create a composition based on their feelings and share it on social media.

The project's sound artist, Antoine Bertine, explains in the behind-the-scenes video: “By inviting the machine to express itself, the music kind of emerged on its own, and we felt that as artists we were more like intermediaries from Kandinsky's time to today, rather than interpreters of theory.”

Listening to Kandinsky's painting through this Google project may not be the same as actually having synesthesia, but it is an almost psychedelic experience involving colors and sounds.

Through which channels you reach those people, classic and out of the box. The Next Web