Google Built An AI That Can Produce Music From Word Descriptions, But Won't Make It Publicly Available.

Google's remarkable new AI system can make music in any genre based on a written description. However, because to the concerns, Google has no imminent plans to release it.

Image Source: www.nytimes.com

Google's MusicLM isn't the first generative AI system for songwriting. Other attempts include included Riffusion, an AI that composes music by picturing it, Dance Diffusion, Google's own AudioML, and OpenAI's Jukebox. However, because to technological constraints and inadequate training data, none of them have been able to generate songs that are exceptionally sophisticated in composition or high-fidelity.

MusicLM might be the first to do so.

MusicLM: Generating Music From Text

Presents MusicLM, a model for generating high-fidelity music from text. MusicLM generates music at 24 kHz that remains consistent over several minutes.

proj: https://t.co/8vzBONkPe3
abs: https://t.co/vzW01q7VpH
data: https://t.co/LERn2mZMtO pic.twitter.com/u4L4ui0RwU

— Aran Komatsuzaki (@arankomatsuzaki) January 27, 2023

For example, "enchanting jazz tune with a memorable saxophone solo and a solo singer" or "Berlin '90s techno with a deep bass and forceful kick," MusicLM was developed on a dataset of 280,000 hours of music to learn to produce cohesive songs for descriptions of, in the words of the researchers, "substantial complexity." 

Its songs, surprisingly, seem like something a human artist would write, but not as innovative or musically harmonious. 

Given that there are no musicians or instrumentalists in the loop, it's difficult to stress how fantastic the samples sound. MusicLM captures details like as musical riffs, melodies, and emotions even when fed rather extensive and meandering descriptions.

As an example, the sample below, title stated that it "induces the sense of being lost in space," and to my ears, it certainly does just the job.

Another example, this time created from a description that begins with the words "The primary soundtrack of an arcade game." Isn't it possible?

MusicLM's powers go beyond only creating song clips. The researchers at Google demonstrate that the algorithm can build on existing tunes, whether hummed, sung, whistled, or performed on an instrument. Furthermore, MusicLM may take a series of descriptors (for example, "time to meditate," "time to wake up," "time to run," "time to give 100%") and construct a melodic "story" or narrative lasting up to several minutes - ideal for a movie soundtrack.

But, to be honest, MusicLM is far from perfect. The altered quality of some of the samples is an inevitable side effect of the training process. And, while MusicLM can synthesize vocals, including choir harmonies, it falls short of expectations. The majority of the "lyrics" span from barely English to outright gibberish, and are performed by synthetic voices that sound like mash-ups of other singers.

🎵 Generate music from text🎵

From Google Research: MusicLM, a model generating high-fidelity music from text descriptions.

The craziest part? MusicLM takes a painting and generates audio based on the description of the art: pic.twitter.com/X06uGDdrB5

— AI Breakfast (@AiBreakfast) January 27, 2023

Nonetheless, the Google researchers point out the numerous ethical issues that a system like MusicLM presents, such as the inclination to include copyrighted content from training data into the created songs. During an experiment, they discovered that around 1% of the music created by the system was directly copied from the songs on which it was trained – a figure that appears to be high enough to discourage them from publishing MusicLM in its current condition.

"We realize the danger of potential misuse of creative work associated with the use case," stated the paper's co-authors. "We clearly underline the importance of more future effort in addressing these dangers related with music generating."

 According to a whitepaper written by Eric Sunray, now a legal intern at the Music Publishers Association, AI music generators like MusicLM violate music copyright by creating "tapestries of coherent audio from the works they ingest in training, thereby infringing the reproduction right under the United States Copyright Act." 

Reviewers have also questioned if teaching AI models to play music that is protected by copyright since the debut of Jukebox. Concerns have also been raised about the training data used in image-, code-, and text-generating AI systems, which is frequently collected from the internet without the developers' permission.

There may soon be some understanding of the situation. Several litigation now pending in court will almost certainly have an impact on music-generating AI, including one involving the rights of musicians whose material is used to train AI systems despite their knowledge or agreement. Time will tell, though.