Beyond the Prompt: The Law's Race to Catch Up with Generative AI

Two things are simultaneously reshaping the world of AI. The first is AI-generated content: text, images, music, and art created not by a human but by a machine that has studied millions of human creations, all from a short typed command. The second is deepfake and synthetic media: AI-generated videos, audio clips, and images that replicate real faces, voices, and likenesses to the point where an average viewer cannot tell what is real and what is fabricated. Both belong to a new dimension of AI called generative AI, a facet of AI that learns patterns from data and produces new content resembling what it is trained on.

Hundreds of millions of photos, documents, and artworks collected from the internet are used to train AI content generators. The model studies patterns and learns to produce output resembling what it has seen. One such example is the Ghibli frames, which took animators months to produce, and the AI studied thousands of them. And this is where the problem began. The AI did not learn to create; It learned to imitate the people who create. The model consumed their work as raw material and learned from it, replicating the original work, which was now termed as ‘AI-generated ’.

This isn't just a philosophical argument. Polish digital artist Greg Rutkowski became one of the most replicated artists on AI platforms, with millions of images generated in his style. As a result, he reported losing commissions while others profited from imitations of his work. It's not only about collecting but also about stealing data without permission.

Further, deepfakes have moved into active harm. Celebrities have found fabricated explicit videos of themselves online. Politicians have been shown in fake speeches. Beyond celebrities, deepfakes have been used in financial scams, misinformation campaigns, and identity fraud, revealing that the threat extends far beyond entertainment.

The conversation around accountability is already live. Getty Images has taken Stability AI to court for training its model on millions of licensed photographs without permission. Meanwhile, researchers warn of “model collapse” as AI content floods the internet, next-generation models increasingly train on AI outputs rather than human reality, quietly drifting from truth. 

AI-generated content and deepfakes are not a distant future concern. They are actively reshaping how art is valued, how trust across platforms functions, and how vulnerable individuals are to having their identity used without consent. 

​

Settled, Contested, and Still Unknown 

The involvement of AI in today’s time revolves around the aspects of originality, copyright, and the privacy of work and individuals. Many concerns arise related to infringement of photos, documents, artworks, as well as deepfakes.

​However, the questions that demand an answer are accountability and the value of originality.

​In India, various concerns arising out of synthetically generated data are primarily dealt with under the Informational Technology Act,2000, its rules of 2011 and 2021 and the Digital Personal Data Protection Act, 2023, and its accompanying rules.

​The 2021 rules were updated in February 2026, keeping in mind the proliferation of deepfakes and cyber fraud, safeguarding personal privacy and dignity, and protecting democratic processes and public order, and structuring intermediary compliance and Safe Harbour protection to address Artificial Intelligence and synthetic manipulation.​

Rule 3 outlined the baseline mandate that all internet intermediaries must implement to remain eligible for immunity, with Rule 3 (1)(b) mandating platforms to actively inform users not to host, upload, or share prohibited content and Rule 3 (3) implementing a dedicated Synthetically Generated Information (SGI) framework. Further, rule 4 specified additional due diligence to be observed by the significant social media intermediaries.

Rule 7 remains the operational hammer of the framework, wherein it dictates that if an intermediary fails to observe the notified rules, the provisions of sub-section (1) of section 79 (the Safe Harbour) of the Act shall not apply to such intermediary, and he may be liable for punishment.

The update significantly compressed response timelines, where the intermediaries were now required to comply within 2 hours in case of non-consensual nudity and deepfakes upon receiving a user complaint, within 3 hours in case of unlawful content notice by a court order or an authorized government agency officer, and within 7 days for standard grievance redressal.

Further, section 3(c)(ii) of the DPDP Act, 2023, states that the Act does not apply if a data principal (individual to whom the personal data relates)  voluntarily makes their own personal data publicly available. However, this is most fiercely debated, as companies may rely on this to scrape the data for training datasets. The legal consensus warns that if a user sets their social media to private after a model has scraped it, continuing to use that data for AI may enter into a legal gray zone.

In the recent case of Getty images brought proceedings against Stability AI alleging, inter alia, that millions of its copyrighted photographs had been used without permission in the training and development of Stable Diffusion. However, Getty abandoned its training and development claim after accepting that there was no evidence that the relevant training had occurred in the UK. The court therefore did not determine whether the alleged use of copyrighted photographs for AI training constituted copyright infringement. It did, however, reject Getty's separate argument that the resulting stable diffusion model was itself an ‘infringing copy’ under section 27(3) of CDPA (Copyright, Designs and Patents Act 1988).

​The principle established by the judgement was that the use of copyrighted works during the training of an AI model does not, by itself, make the resulting AI model an ‘infringing copy’; the resulting article must itself satisfy the statutory concept of a ‘copy’ under CDPA. Further, the judgement recognized that an intangible object can potentially constitute an article for the CDPA, and an AI developer may be liable for trade-mark infringement arising from marks appearing in AI-generated outputs where the developer exercises sufficient control over their generation and the statutory requirements for infringement are satisfied.​

A similar case emerged in India, where ANI (Asian News International), an Indian news agency, alleged OpenAI trained ChatGPT on its copyrighted articles without a licence, and that ChatGPT reproduced content in outputs to Indian users. It reflected the same territoriality problem as Getty (training on foreign servers), but was decided the opposite way. The Hon’ble Delhi High Court held that it did have jurisdiction. However, found no output-side infringement and held that storing copyrighted works for training is, prima facie, fair dealing under Section 52 (private use/research, no proven market harm). Henceforth, no interim injunction was granted as demanded by ANI.

​In a recent order dated 31st August,2026, the copyrights office dismissed U.S. researcher Stephen Thaler’s application the name DABUS (device for the autonomous bootstrapping of unified sentience) as the author of an artwork ‘A Recent Entrance to Paradise’, holding that although it did meet the originality requirement under Indian Copyright Law, DABUS could not be recognized as its author because it is neither a natural person nor a juristic person under law.

However, in 2020, Ankit Sahni created an artwork titled ‘Suryast” by taking a digital photograph he shot of a sunset and using a custom-built AI app called RAGHAV to apply the artistic style of Vincent van Gogh’s The Starry Night. Sahni applied for copyright registration in India as ‘human co-author and AI co-author’. In November 2020, the Indian copyright office granted the registration, effectively recognizing an AI system as co-author of an artwork.

As this was widely considered a historic, global first, it also triggered immediate scrutiny over giving an app legal authorship without deeply assessing the statutory implications of it. The office later issued a withdrawal notice, asking about RAGHAV’s legal status. However, Sahni contested this notice, arguing that the office does not have the statutory authority to revoke/review its own granted registrations. At present, suryast technically remains listed.

To infer, various existing statutes deal with the concerns arising from the growing field of Artificial Intelligence in terms of accountability, originality, privacy, authorship, and are constantly amended as the area demands but yet a clear and concise admission on the above terms remains unanswered which leaves us with the question to our curiosity, that to what extent will the law evolve itself when the limits of AI itself are unknown.