Article is written to illuminate and prompt contemplation on the imminent and potential hazards that may arise in the near future.
Intent
It is not a secret, nor is it historically unproven, that cyber-criminals have adeptly embraced newer AI-based technologies for malicious purposes, be it for financial gain or the conveyance of ideological messages, purely for disruptive purposes.
Numerous articles have delved into AI-based Large Language Models (LLMs), exploring their application for illicit purposes through the circumvention of implemented safeguards, jail breaking, and/or prompt engineering.
The primary objective of this article is to underscore how AI-generated images from text can instigate disruption and lead to changes in policies, regulations, and operations, particularly in sectors such as Banking and Finance. Industries dependent on online customer verification for service provision are also susceptible.
Both the FBI and Europol have articulated their stance and understanding on this matter. If interested, the referenced links are provided below:
Science and technology revolutionize our lives, but memory, tradition and myth frame our response. — Arthur Schlesinger
Background
According to Forbes, In 2022, the global AI market size was estimated to be at $87 billion and is projected to reach $407 billion by 2027. Read More
The Mckinsey State of AI in 2022 report states, AI adoption has more than doubled from 2017 to 2022. In 2017, 20% of respondents reported adopting AI in at least one business area, whereas in 2022, that figure stands at 50%. Read more
With 64% of business owners believing AI will boost productivity, all we can expect is larger part of internet utilizing AI-based technologies in scaling and improving their businesses for better profit with lesser cost. Read more.
List of Potential Threats and Interesting Projects
Attack Vector : Impersonation and Identity Theft
With the combination of basic opensource research on targeted victim and stalking on social media platforms, a cyber criminal can achieve identity impersonation given on every day and historically basis databases full of personally identifiable information and high resolution image dumps of passport and other identity-verification documents gets shared/published on cybercrime discussion places.
Example: Using Stable Diffusion and LoRA
Attack Vector : Disinformation campaigns
- India’s Tryst with Deepfakes: Viral deepfakes of celebrities Rashmika Mandanna and Kajol, and political uses like during BJP’s 2020 election campaign, where Manoj Tiwari’s speech was altered for linguistic versatility, exemplify this trend.
- Pro-China Influence Campaign: A pro-China influence campaign used computer-generated avatars to promote the interests of the Chinese Communist Party. The campaign was first observed by intelligence company Graphika in late 2022.
- Bangladesh Government’s Policies: In September 2023, Agence France-Press’ fact-checking team in Bangladesh unearthed a coordinated campaign of hundreds of op-eds by fake experts praising the Bangladeshi government’s policies. Read More
Attack Vector : Social engineering campaigns
- Generative AI in Phishing: Generative AI tools like ChatGPT and Midjourney were used to make social engineering campaigns more believable. Read More
- Large-Scale Scam Campaigns: Sophos reported on large-scale scam campaigns made possible by generative AI. These campaigns combined code, text, images, and audio to build hundreds of unique websites and their corresponding social media advertisements. Read More
- Voice Phishing Attacks: The Federal Trade Commission warned about scammers using AI voice cloning technology to impersonate family members and con victims into transferring money on the pretext of a family emergency. Read More
Resources & References
- The article by Timur Yunusov exposes the exploitation of leaked databases on the dark web, where cybercriminals buy and sell stolen card information. It delves into the creation of fake IDs, powered by AI-generated images, for money laundering and sanctions evasion. The author’s personal experiment demonstrates the ease of bypassing KYC checks, underscoring vulnerabilities in the financial system and the pressing need for heightened security measures. Read More
- dot (aka Deepfake Offensive Toolkit) makes real-time, controllable deepfakes ready for virtual cameras injection. dot is created for performing penetration testing against e.g. identity verification and video conferencing systems, for the use by security analysts, Red Team members, and biometrics researchers. Refer
- 5 methods to generate consistent face with Stable Diffusion.
- Next generation face swapper and enhancer
- An arbitrary face-swapping framework on images and videos with one single trained model!
- Stable Diffusion Prompt Guides
In the upcoming article, we shall deep dive into nitty gritty about Selfie identity verification works. Stay tuned. If you wish to connect, quick OSINT should help in connect with me. Thanks! :)
