Are music deepfakes considered fair use?
The past few years have seen “music deepfakes” — or the unlicensed replication of a celebrity’s music and voice — emerge as a distinct, potentially disruptive category in creative A.I.
Scrappy developers have published deepfakes of rappers like Jay-Z, Eminem, Travis Scott on the Internet for free, using complex text-to-speech models and hundreds of hours of training data (i.e. artists’ back catalog). A growing laundry list of big-tech corporations and research groups like Google, Facebook, Sony and OpenAI are also investing in their own A.I.-driven music projects — with the benefit of many times more capital, training data and machine-learning talent than what indie artists and developers can access. Some of these companies are taking an explicitly deepfake-oriented approach, creating entirely new songs in the style of deceased performers like Nirvana and Frank Sinatra. And this doesn’t even take into account visual rather than auditory deepfakes, which abound for celebrities like Paul McCartney, Tom Cruise and Maisie Williams.
In all of the above cases, the underlying goal is to scale the believable replication of celebrity brands, regardless of the celebrity’s direct involvement. But all of these artists and/or their estates also have deep relationships with labels, publishers and other organizations who help manage and protect their sprawling portfolio of copyrights. So the natural question that follows is: Are the companies that make music deepfakes getting licenses to use copyrighted songs as training data?
In most cases, the answer is no. Instead, companies take the classic Silicon Valley approach of asking for forgiveness rather than permission — meaning they could potentially be committing copyright infringement on the scale of millions of tracks. For instance, to build their Jukebox neural net, OpenAI scraped the web to gather 1.2 million songs with corresponding lyrics and metadata from the now-defunct site LyricWiki, to build music and voice models in the style of artists like Elvis Presley, Frank Sinatra and Katy Perry whose music and likeness are clearly copyrighted.
To be fair, not all generative or adaptive music companies infringe on music rights holders. Some, like now-defunct Jukedeck (acqui-hired by TikTok in 2019), have drawn from music in the public domain as their training data; others, like Endel and Weav, have specific licensing agreements with labels and publishers to build generative music experiences around existing catalogs.
But many of the most talked-about, most-consumed A.I. music projects on the Internet today are unlicensed celebrity deepfakes, with the music industry largely moving reactively rather than proactively to quell them (if they move at all). As these use cases continue to expand, rights holders and tech developers would do well to develop a licensing and remuneration framework for A.I. that supports all stakeholders involved.
A common defense that tech companies lean on to overstep this kind of licensing process is via the fair-use doctrine in the U.S. For music deepfakes, this is a flimsy gamble at best, because the wholesale appropriation of copyrighted music to create soundalikes that commercially promote a given company does not fall into traditional categories of fair use, like news reporting or criticism.
Yet, with some recent developments, tech companies might think that unlicensed A.I. music projects actually now have sufficient legal backing.
Back in December 2020, I wrote for Water & Music about the ongoing Supreme Court case Google v. Oracle, consisting of a feud between Google and Sun Microprocessors (later acquired by Oracle) as to whether Google copy-and-pasting 11,500 lines of declaring code from Sun’s Java SE API into Google’s Android operating system was considered fair use. Google argued that its use of the declaring code — which Sun previously only used in desktop applications — was fair use because it occurred in a wholly different, innovative medium, and that its use of this code was necessary because it enabled Java developers to create applications for Android without learning a new programming language. On the flip side, Oracle argued that Google generated billions in profit by unfairly commandeering its declaring code for a slightly different medium after past licensing negotiations between Google and Sun fell through.
After over a decade in the U.S. courts system, the Supreme Court held in April 2021 that Google’s unauthorized reimplementation of Sun’s (now Oracle’s) Java SE API declaring code was in fact fair use. While many in the tech industry celebrated this outcome, the verdict also shocked some onlookers in I.P. law.
At worst, as I suggested in my previous piece, a verdict in favor of Google could mean the end of music rights holders’ participation in A.I.-driven music creation technology, giving license to startups to include an exact copy of a copyrighted song in their training data, technologies and products for free.
Unfortunately, even with this April verdict, music rights holders won’t have any definitive closure as to whether this doomsday stance will ultimately play out, given the flexible interpretation of fair use. However, we can have a good idea about what Google v. Oracle will ultimatelymean for the music industry — and how to start more productive conversations about licensing and compensation for A.I. music companies — by analyzing the logic behind the Supreme Court’s decision.
Breaking down the Google v. Oracle verdict
To understand the somewhat bizarre nature of the Google v. Oracle verdict, let’s revisit the elements of fair use, which enables creators — in limited circumstances — to borrow a portion of another creator’s work without a license.
While Congress codified this doctrine into copyright law in 1976, judges continue to adjust its interpretation to meet the creative needs of the modern technological landscape. Unfortunately, the only certainty fair use provides is that it is inherently uncertain; no one knows whether they have a successful fair-use defense until it has been litigated in court.
Today, someone asserting a fair-use defense is required to satisfy a four-part balancing test:
- The purpose of the use and whether the use is “transformative” (i.e. whether it adds “something new, with a further purpose or different character, and do[es] not substitute for the original use of the work,” in the words of the U.S. Copyright Office);
- The nature of the work being copied (e.g. functional vs. expressive);
- The amount and substantiality of the proportion of the original work being used; and,
- The effect of the new work on the market for the original copyrighted work, which can include the amount of the loss and the sources of the loss.
In securing Google’s victory, the Supreme Court found each of the four prongs weighed in favor of fair use. Even though Google used the declaring code from the Java SE API for the exact same purpose as in Sun’s API (namely, calling upon Java API packages), the Supreme Court deemed the use “transformative” because Google arguably intended its reimplementation of the declaring code to help others create Android applications. Without this declaring code, programmers could not have used their Java skills to build these apps. The Court also effectively disregarded the commercial purpose of Google’s reimplementation of Sun’s declaring code, and questioned whether bad faith had any bearing on a fair use analysis — quoting from the Campbell v. Acuff-Rose Musiccase that “[c]opyright is not a privilege reserved for the well-behaved.”
As for the fourth or “market effects” prong of fair use, a majority of the Justices found that Oracle was not subject to quantifiable monetary harm from Google’s actions, and that Sun was in a poor position to succeed in or monetize the mobile phone market anyway, despite initially trying to license the API to Google for Android. Moreover, the Justices argued, the source of any theoretical loss felt by Oracle was not tethered to the copyright of the Java SE API alone. Instead, any profits Google derived from the declaring code was a result of the adjacent investment that programmers had made in developing their skills using Java. Had Oracle prevented programmers from utilizing their Java expertise in Android — a right not contained in U.S. copyright law, in that copyright owners don’t have exclusive rights over the knowledge or skills people develop from engaging with their I.P. — the public would have been injured by a stifling of innovation.
Finally, the Court noted that because the declaring code was almost purely functional (as compared to expressive like a painting or song), and that the declaring code represented only 0.4% of the entire Java SE API code, that the second and third prongs also weighed in favor of Google’s fair use.
Fair-use calculus: Music is not the same as an API
While the extent to which this landmark case will influence industries beyond software is uncertain, it is already impacting how copyright infringement cases are being argued in the art world. For instance, the Second Circuit had ruled in March 2021 that Andy Warhol’s use of Lynn Goldsmith’s photograph of Prince in the visual artwork collection “Prince Series” was not considered fair use. But after the Google v. Oracle verdict, the Andy Warhol Foundation for the Visual Arts is urging the court to rehear the case, arguing that the Prince Series satisfies the “transformative” criterion for fair use that played a key role in the Supreme Court siding with Google.
Companies training their algorithms on unlicensed music might similarly gamble that the Google v. Oracle verdict will liberally impact the transformative and market-effects prongs of fair use in the companies’ favor; these two are the most important for music fair-use defenses, as the goal of copyright law in these cases is to protect the financial incentives of artists while not stifling new works of art.
“Music deepfake” developers and other unlicensed A.I. music companies might try to argue that their use of music copyrights is transformative because their intention is to enable others to create — that their technology allows anyone to create a new composition with a click of a button. They could go on to say that being required to license all of the sound recordings and compositions on the internet would be impossible, which would result in them training their algorithms on a smaller number of songs; this smaller dataset would be less useful and rich, which would hurt the public by depriving them of the full potential of the underlying technology. Last but not least, these fair-use proponents might argue that, like Sun, music rights holders alone are not equipped to hone A.I. technology and therefore are not in a good position to succeed commercially in the world of A.I. music creation.
Although these arguments follow the logical structure of Google v. Oracle, none of them will likely win the day for unlicensed A.I. music developers. The underlying flaw in these arguments is that they assume the declaring code used in the Google v. Oracle case is equal to music in the eyes of copyright law, and that the software industry shares common and necessary practices — including the reimplementation of unlicensed copyrighted work — with the music industry.
This is simply not true. The breadth of a creator’s rights in a copyright is proportional to the work’s originality and creativity, and the creators of inherently expressive content (like music) are afforded more robust copyright protections compared to those that make more functional creations (like computer code). Consequently, courts tend to take a more conservative approach in their fair-use analysis when music is the subject matter.
As emphasized in the iconic Betamax case (Sony Corp. of America v. Universal City Studios, Inc.), fair use is also intended to help rebalance and offset the monopolization of copyright ownership as technology evolves — which varies from market to market, medium to medium and industry to industry. Again, there is no one-size-fits-all for the doctrine. In Google v. Oracle, the Supreme Court interpreted the transformative prong of fair use to fit the contours and necessities of the software industry. Without this standard, the Court recognized that innovation and the public would be burdened; each software company could be required to create their own programming languages, which would lessen interoperability and make developing applications for multiple operating systems more cumbersome. (A large swath of the tech industry today would argue that APIs should not be copyrightable at all.)
Any new transformative standard for fair use that arises from Google v. Oracle will likely be reserved for thin, functional copyrights only. Otherwise, extending such a standard to creative copyrights like songs would essentially eliminate the copyright holder’s exclusive right to prepare derivative works, such as the right for authors to convert novels into films.
In regards to the fourth prong about market effects, fair-use proponents will have a difficult time relying on Google v. Oracle to justify unlicensed music in creative A.I. apps, because wholesale copying is arguably not necessary for these apps to operate, or for the music industry to utilize A.I. technologies in general. As discussed earlier in this piece, companies like Jukedeck have been able to promote innovation in creative A.I. without using unlicensed catalogs. Applying a broad reading of this case to music and other creative industries would arguably disincentivize artists from creating, decrease the over $919.7 billion arts and culture contributes to the U.S. economy, and disturb fair use’s purpose of promoting the arts and sciences. As a result, any Google v. Oracle alteration of the market-effects prong of fair use will also be limited to instances of fair use among more functional copyrights.
Conclusion: Is music deepfake compensation possible?
Notwithstanding how Google v. Oracle will impact the reimplementation of code in the software industry, the stark differences between functional APIs and expressive music copyrights means that this case will likely not win the day for unlicensed A.I. music companies. For compositional algorithms to utilize protected music copyrights created by humans — and for the music industry to proactively rather than reactively benefit from the proliferation of music deepfakes — there needs to be a clear conversation about compensation before it becomes too late.
Similar to how sampling agreements are negotiated, this conversation needs to cover how much (if not all) of the resulting music from a given A.I. technology will be owned and controlled by the owner of the original copyright, versus by the software developer and/or the end user of an A.I. music creation tool. As Water & Music has covered previously, there is no standard right now for which of these stakeholders ultimately owns an A.I.-created song. Some tools, like those developed under Google Magenta, do not treat themselves as coauthors and immediately grant all rights to the end user. Others, like, Boomy and Splash (f.k.a. Popgun), maintain 100% ownership themselves and give users only a perpetual royalty-free license — without revealing anything in their user agreements about the role of the underlying training data. In the context of an official licensing agreement, a situation may arise in the future where the original music rights holders get more ownership and control over deepfakes around their I.P., with developers then relinquishing ownership over these deepfakes and instead gaining the secondary benefit of refining their composition algorithms — the ultimate source of value for their own users.
However, unlike with compulsory licenses for compositions, owners of sound recordings are not required by law to license their music to A.I. developers. Consequently, before these above conversations can even occur, deepfake developers will need to convince artist estates, contemporary creators and record labels to come to the negotiation table. They will need to explain that their ultimate purpose is not to create celebrity lookalikes forever, but to advance music creation technologies to enrich the future of the industry as a whole. Moreover, they will have to convince rights holders that these technologies and deepfakes will not dilute their catalogs or scar their legacies, but rather will contribute to their legacies by paying it forward to future creators in the form of A.I.-driven music creation tools.
This amounts to a somewhat more radical, bottom-up view of how copyrights accrue value over time, which the music industry needs — but perhaps is not ready — to embrace in order for music deepfake compensation to become a reality.
Special thanks to Cliff Fluet for his feedback and expertise.