Copyright Protection in LLM AI Training Part 1

27th January, 2025 - 4:34 am Categories: Copyright 0 Comments

Note: In this two-part blog series, the authors examine the recent controversy surrounding GenAI’s Learning Models and Copyright infringement arising through it. In the first part the authors delve into the methods behind GenAI’s training and the arguments made in the American jurisdiction.  The second part delves into the Indian landscape in light of the ANI v. Open AI litigation

ChatGPT owned by Open AI has recently been in controversy over its training practices. Media houses all over the world have filed suits against it claiming violation of their protected works in the outputs provided by ChatGPT. Their primary allegation is that ChatGPT has used their content without permission and hence violated their copyright over said content.

This blog seeks to examine whether the practices of ChatGPT fall within the ambit of fair use as defined under §52 of the Indian Copyrights Act. The blog does this by examining the concurrent cases going on in the United States. The blog further examines the arguments from a more local stand point by reviewing the order of the Delhi High Court in ANI v. Open AI.

1. The Allegations: Legal Position in the US

In the U.S this litigation is consolidated into two major actions. First is the New York Times and Daily News v. Open AI and Microsoft. The second, is a group of independent Pulitzer winning authors against Open AI and Microsoft, consolidated as In Re Chat GPT. They both allege that their copyrighted works have been illegally used by Open AI’s ChatGPT and Microsoft’s Bing to provide outputs to their users. They allege that this violates their intellectual property rights.

The New York Times (‘The Times’) alleges that Defendants’ generative artificial intelligence tools, including Microsoft’s Bing Chat and Open AI’s ChatGPT, unlawfully use The Times’s copyrighted content to develop large-language models (LLMs) without permission or payment, threatening its ability to provide independent journalism. These tools rely on large-scale copying of millions of The Times’s news articles, investigations, opinion pieces, reviews, and how-to guides, emphasizing its uniquely valuable content. The Times claims Defendants’ AI outputs mimic its expressive style, generate verbatim or closely summarized content, and attribute false information, undermining its credibility and damaging reader relationships. Microsoft’s Bing allegedly uses Times content to produce detailed summaries, depriving The Times of subscription, licensing, advertising, and affiliate revenue.

Defendants have claimed the defense of the “transformative use”. The doctrine states that if any work is thoroughly transformed then such transformation is protected by fair use doctrine. It is the defendant’s case that the LLM does not enjoy the copyrighted work in its expressive (i.e., protected) sense. It merely reads the meta data of such works to train its model to generate appropriate responses. It then transforms this Meta data to give its users a unique and transformed output.

The publishers assert that this unlicensed use is not transformative or fair use, as the tools compete directly, stealing audiences and eroding its journalism’s value. They invoked the 4-Factor test. The test requires that any claim for fair use must overcome four factors, these are:

1. Purpose and Character of the use.
2. Nature of the work.
3. Amount and substantiality of the portion used.
4. Effect on the market or the potential market for the original work.

The judgement in these cases is still pending with the publishers having been directed to file evidence in favor of negative effect on the market of their works due to Open AI’s use of their content. However, to understand the application of transformative use it is imperative to understand the extent of the transformation itself. The next section deals with how ChatGPT generates its responses.

• How does ChatGPT Function?

ChatGPT is at its core a software. It does not “learn” anything in the human sense. It also has no “perception” or “expression”. Therefore, first and foremost, ChatGPT does not have the capability to express another person’s expression. The model merely reads the meta data embedded within such “expressive works” and collects the information, data and patterns. These attributes help ChatGPT to determine the best sequence of language and words for every query made to it by a user. ChatGPT is therefore called a “Large Language Model”. The imperative takeaway is that ChatGPT does not egress into the expression of any work. It is trite law that Copyrights do not protect ideas, but only the expression of such ideas.

On a more in depth note, ChatGPT operates using a transformer-based neural network trained on large text corpora through two phases: pre-training and fine-tuning. During pre-training, it predicts the next word in a sequence by analyzing extensive datasets, learning grammar, facts, and reasoning patterns. In the fine-tuning phase, the model is refined with curated datasets and human feedback to align responses with user expectations. When processing queries, it tokenizes input text into smaller units (tokens) and interprets them contextually, understanding relationships between words and phrases. Using its training, ChatGPT generates responses by predicting the next sequence of tokens, dynamically synthesizing answers based on patterns learned during training rather than retrieving information from a database.

1. What are the data sets used for training ChatGPT?

The data (referenced from the suit filed by the Times) which was and is still being used to train the model making up ChatGPT comes from various sources like Common Crawl, WebText2, Books 1, Books 2, and Wikipedia. These sources are machine readable and contain millions of links to different sources that the model accessed for creating its logical and factual datasets. The information in these data sets is then contextual interpreted through tokens to provide the user with a relevant output.

These datasets lie at the heart of this issue. New York Times alleges that “WebText 2” is created by Open AI itself. It further alleges that WebText 2 features unique links to Articles by the Times more than 200 thousand times. Despite relying on other methods like Open Crawling (which generates 410 billion tokens) WebText 2 (which generates only 19 billion tokens) has a 22% weight in the training data. This brings home the publisher’s allegation that ChatGPT in increasingly trained on data generated from WebText 2 which includes a large quantity of the publisher’s copyrighted content. The above table clarifies this further.

Therefore, it can be concluded that ChatGPT has used the content of the publishers to train its LLM model. As highlighted above, a large quantity of this training data comes from copyrighted articles, often behind paywalls. The outputs provided by ChatGPT are many a times exact reproductions which contradicts the key argument on transformative use.

Understanding the Effect on Market under the Fourth Factor

The doctrine of fair use has no application if the alleged fair use upends the market of the original product by depriving the original owner of their income. This is true even in situations where there is no direct competition. Therefore, commercial use cannot be made of any copyrighted work without making sure that the new use of the work does not affect the market and expression of the original work.

In the Times case, Open AI claims that it has not affected the market of the original product as it is not a new agency. Its flagship LLM model, against which the suit is primarily filed, makes open declarations that the outputs made by it do not constitute facts and that the user must cross-check the facts before using them.

However, in a practical sense, the operation of ChatGPT makes this safeguard meaningless. Latest models of ChatGPT are able to surf the internet to provide real time answers to the user’s queries. In doing so, the model also lists a directory of links which reflect the sources of this information. This, if seen in light of the fact that ChatGPT is capable of reproducing paywalled content makes it clear that ChatGPT is a substitute to the publishers.

The publishers have already set out in their complaint that they earn their revenue through monetizing on their reported articles. This is why their articles are behind a paywall. If ChatGPT can read these articles, process them to cite them as a source to user queries, and then also reproduce the content behind the paywall it does (in operation) substitute the carefully monetized market of the publishers.

Open AI and its Non-Profit Charter:

Open AI was founded as a non-profit entity in 2015 with its sole fiduciary duty to the good of humanity. Open AI since 2019 has been capped for profit entity called Open AI Global. Microsoft, a long-time investor has promised to invest 13 billion dollars into Global OpenAI and has agreed that at the end of the 13 billion dollars it will own a 49% stake in Global OpenAI. Other investors will own the another 49% with a mere 2% stake left with the non-profit entity.

Open AI since 2019 has used its vast amounts of trained data to train its LLM models and also better its other AI models. It has sold these models at high prices to prominent firms for a hefty charge. Open AI has confirmed that 92% of the Fortune 500 companies depend of software created by Open AI. Moreover, ChatGPT is increasingly becoming a competitor to Google Search Engine. Although their mode of revenue differs, their user base is the same.

Interestingly, the publishers have already negotiated with Google and allowed it to crawl its content in return for better results for their search engine and summarizing of information at the top of each search. Google does not crawl and use the data free of cost, rather it pays a compensation to the publishers (100 Million Dollars in 2023). In the complaint the publishers have pointed out that similar negotiations with Open AI have failed. Therefore, identical acts have had different outcomes and there can be no denying the publishers argument that ChatGPT and the publishers are in the same market for some portion of the activities and that stealing of the publishers’ content amounts to fulfilment of the fourth factor. Needless to say, the four-factor test and doctrine of transformative use are not legislatively recognized in India. The next section of this blog deals with how this issue is potentially going to be tackled by Indian Courts.

The controversy over ChatGPT’s training practices raises critical questions about copyright and generative AI. While OpenAI claims transformative use, evidence like reliance on WebText2 and reproduction of paywalled content suggests possible copyright violations impacting market value.  As U.S. cases shape global interpretations, India faces unique challenges due to the absence of doctrines like the four-factor test in its laws. Balancing innovation with creators’ rights will be key to addressing these complexities. In the next part of the blog the authors discuss the effect of the global interpretation in India.

Author: Sarthak K., in case of any queries please contact/write back to us via email to chhavi@khuranaandkhurana.com or at Khurana & Khurana, Advocates and IP Attorney.