Possible adversarial threats that may arise from the use of AI language-based applications and minimization of its impact are described below:

People are using the AI language-based applications to understand, interpret, and enumerate cyber security contexts, to review security events and logs, to interpret malicious codes and malware samples etc. The applications have the potential to be used in vulnerability scanning, translation of security code from one language to another or transfer of code into natural languages, performing security audit of the codes, VAPT, or integration of application with SOC and SIEMs for monitoring reviewing, and generating alerts.

• A threat actor could use the application to write malicious codes for, exploit a vulnerability, conduct scanning, perform privilege escalation & lateral movement, to construct a malware or a ransomware for a targeted system.
• AI based applications can generate output in the form of text as written by human. This can be used to disseminate fake news, scams, generate misinformation, create phishing messages, or produce deep fake texts.
• A threat actor can ask for a promotional email, a shopping notification, or a software update in their native language and get a well-crafted response in English, which can be used for phishing campaigns.
• Creation of fake websites and web pages to host and distribute malwares to users through malicious links or attachments using the domain similar to AI based applications.
• Creation of fake applications impersonating AI based applications.
• Cybercriminals could use AI language models to scrape information from the internet such as articles, websites, news and posts, and potentially taking Personal Identifiable Information (PII) without explicit consent from the owners to build corpus of text data.

• Educate developers and users about the risks and threats associated with interacting with AI language models
• Verify domains and URLs impersonating AI language-based applications, avoid clicking on suspicious links.
• AI language-based applications are based on learning on large set of internet data. The model can collect all accessible data including sensitive information also. Hence, implement appropriate controls to preserve the security and privacy of data. Do not submit any sensitive information, such as login credentials, financial information or copyright data to such applications.
• Ensure that the text generated is not being used for illegal, unethical activities or for dissemination of misinformation.
• Use content filtering and moderation techniques within organization to prevent the dissemination of malicious links, inappropriate content, or harmful information through such applications.
• As the malicious codes written by threat actors may bypass the existing detection mechanism, making detection more challenging. Enhance and implement the relevant monitoring and security measures to detect these threat activities.
• Secure and conduct regular security audits and assessments of the systems and infrastructure to identify potential vulnerabilities and information disclosures.
• Enable multi-factor authentication (MFA) to prevent unauthorized access of accounts directly to AI based applications and protect user accounts from being compromised.
• Organizations may continuously monitor user interactions with AI language-based applications for any suspicious or malicious activity within their infrastructure.
• Organizations may prepare an incident response plan and establish the set of activities that may be followed, in case of an incident happens.
• Stay up-to-date on the latest security threats and vulnerabilities and take appropriate action to protect yourself and your data.

• Preventing Data Breaches / Data Leaks (CIAD-2021-0004))
  https://www.cert-in.org.in/s2cMainServlet?pageid=PUBVLNOTES02&VLCODE=CIAD-2021-0004



• Mobile based Malware: Methods and Countermeasures (CIAD-2022-0014)
  https://www.cert-in.org.in/s2cMainServlet?pageid=PUBVLNOTES02&VLCODE=CIAD-2022-0014