ChatGPT is generating fake Aadhaar and PAN cards: Alarming AI misuse surfaces online | Mint-OxBig News Network

ChatGPT, or more precisely GPT-4o’s native image generation capabilities, were unlocked by OpenAI last week. Since then, users have generated over 700 million images, finding various use cases for the new AI tool—most notably, creating Studio Ghibli-style portraits. However, as awareness of ChatGPT’s new capabilities spreads, so does the risk of misuse.

In one such scenario, some social media users have started sharing images of fake Aadhaar cards with their images using ChatGPT’s new image generator. While there has long been a debate about AI companies introducing features that could be misused in the wrong hands, with ChatGPT’s new ability to create photorealistic images, this grim reality may be coming to a head.

Seeing the flurry of images on social media, we tried to generate an Aadhaar card-like image via ChatGPT. The results were very close to the real Aadhaar card image, with only the facial details being inconsistent in this scenario.

Other users on X also shared images of OpenAI CEO Sam Altman and Tesla CEO Elon Musk‘s pictures superimposed onto the Indian identification card along with a proper QR code and Aadhaar number.

Not just Aadhaar cards! ChatGPT is also generating fake PAN cards:

The misuse of ChatGPT is not just limited to Aadhaar cards, social media users have pointed out instances of the chatbot also being able to generate PAN cards with remarkable accuracy. Aadhaar card is issued by the Unique Identification Authority of India (UIDAI) while PAN card is issued by the Income Tax department.

Restrictions on ChatGPT’s new image generator:

ChatGPT’s native image generation capabilities essentially mean that the underlying foundation model running the chatbot can generate images directly, rather than relying on external models like DALL-E 3. This allows the chatbot to follow detailed natural language instructions and generate more nuanced and accurate images.

In its GPT-4o Native Image Generation system card, OpenAI admitted that the new model has the potential to create more risks compared to previous DALL-E models.

“Unlike DALL·E, which operates as a diffusion model, 4o image generation is an autoregressive model natively embedded within ChatGPT. This fundamental difference introduces several new capabilities that are distinct from previous generative models, and that pose new risks… These capabilities, alone and in new combinations, have the potential to create risks across a number of areas, in ways that previous models could not.” the company stated.

ChatGPT currently has strict restrictions on creating photorealistic images of children (including minor Public figures), Erotic content and violent, abusive, and hateful imagery.

What experts are saying?

Wriju Ray, chief business officer at IDfy – a leading integrated identity platform – told Mint that Aadhaar verification is still easier to track given that card details and facial data can be matched with the back-end system. However, verifying the authenticity of other government-issued IDs, such as PAN or driving licences, may be more challenging.

“Aadhaar is a relatively easier case to solve as both text and face information can be checked against a back end system. Other ID cards like PAN and DL (Driving License) don’t have faces in the government databases. So one needs to look for signs of tampering in these cards too, to ensure that the fraudsters haven’t tampered the face on the ID card” Ray said.

Meanwhile, Rohit Kumar, founding partner at Delhi-based public policy firm The Quantum Hub (TQH), outlined the need for robust measures to mitigate the risk of misuse, which could lead to identity theft and fraud.

“While there are concerns that personal data may have been ingested during model training, the more urgent need is to establish robust safeguards at the output stage. This includes denying potentially harmful requests and embedding features such as digital watermarks or provenance metadata to prevent the misuse of generated content and the unintentional disclosure of sensitive information,” Kumar told Mint.

“A failure to act decisively could lead to further erosion of trust, with wide-reaching societal and economic consequences.” he cautioned.