Googles musical AI experiments include a rapping smartphone and an infinite drum machine

Google is in the middle of a big machine learning push right now, announcing yesterday a pair of big-name hires as well as new product roadmap that will offer more AI services and support via Google Cloud. But that’s just the business side of the equation, and the company also seems to be having a hell of a lot of fun with its AI research, publishing a bunch of fun and creative experiments powered by machine learning.

You may have already seen Quick, Draw!, a web app which uses neural networks to identify your terrible doodles, but here are our three favorite experiments:

A.I. Duet — a computer that respond to your melodies with its own improvisations

Google says it’s working on getting this experiment onto the web, but right now it only exists as some freely available code. It’s basically a computer program that acts as a musical partner. Play it a melody, and it will respond in kind. The software works by using neural networks to analyze a catalog of melodies. From this, it derives vague patterns (like which notes follow other notes and how to mirror things like rhythm) and uses this to come up with its own tunes. This technique of generative AI music is becoming pretty common, and has been used to create everything from pop songs to jazz numbers.

Infinite drum machine — using machine learning to categorize (and play) a catalogue of percussive sounds

This one uses a technique that pops up in a bunch of experiments — something called t-SNE. This is a machine learning algorithm that’s particularly good at organizing high-dimensional data, or data with lots of different aspects or characteristics. (Here's a good video explainer.) This could be data from image recognition software (because there are lots of different ways to analyze any particular image) or, in this case, sound files. Researchers used t-SNE to sort a massive catalog of everyday sounds, mostly percussive in nature, grouping them into similar families. You can explore the results here, and the web app also includes a rudimentary drum machine.

Sadly, this experiment isn’t available to download, but I’m pretty sure someone will be incorporating it into an app soon. This basically combines a few different tools, including a machine learning-powered image recognizer (powered by Google’s Cloud Vision API), a speech synthesizer, and an electro soundtrack courtesy of legendary Italian DJ and musician Giorgio Moroder. This isn’t the first app we’ve seen that carries out this say-what-you-see trick, but it is the most musical.

ncG1vNJzZmivp6x7tbTEr5yrn5VjsLC5jmtnam5fZn5wfZVoaGxuZG6EdXyOoKaon5yaeqK1jJ6vqZ2inrqmutOsZKato56w