AI Content regulation in India: When labelling norms are ineffective

The Union finance minister’s images are manipulated through Artificial Intelligence (AI) deepfakes and misused to perpetuate financial fraud — this is, unfortunately, not a hypothetical case. Indeed, many such cases have surfaced lately. Celebrities and non-celebrities alike have fallen victim to AI-enabled deepfakes, often involving sexual imagery and harming their privacy and dignity. Now, there are reports of attempts to manipulate voter choice in the upcoming elections using AI-generated fake images of actors, wherein their likeness endorses or criticises a party. Digital arrest scams are also increasingly using AI deepfake imagery and voice to perpetuate fraud.