Beware of Hallucinations and Misleading Data

• LLMs will hallucinate and confidently assert that their answers are correct. You must always validate their output.
• Do not allow your LLM to execute dangerous functions without prior validation (Human-in-the-Loop).
• The challenge with hallucinations and misleading data is not only in their detection but also in their unpredictability. Evaluate the implementation of adaptive learning mechanisms to continuously improve the accuracy and reliability of the model.

Do Not Provide IP/Private Information to Public GPT Engines

• Do not share anything you would not want to be publicly known.
• Assume that anything you input into a public AI engine will be used for future training.
• Assess whether enterprise-grade tools are suitable for the intended purpose; they usually have special security measures.
• Define policies for the use of chat tools and train your teams.
• Consider additional measures such as data obfuscation or the use of private, self-hosted AI models.

Familiarize Yourself with the OWASP Top 10 for LLMs

• OWASP has created a Top-10 list for LLMs, describing the 10 most significant risks for LLMs, with examples and suggestions for appropriate countermeasures.
• Check whether there are additional risks beyond these from your specific use case.

Keep a Human in the Loop

• Be cautious with AI-generated code: validate it during code review and integrate code security tools into the IDE to test both first-party and third-party code or libraries.
• This is particularly important when using autonomous agents like Langchain, etc., that give AI direct execution access.
• Be careful about allowing your LLM to execute internal/external functions/system calls. Always apply the principle of least privilege.
• Be cautious about allowing your LLM to manipulate/change important/sensitive data. Explicit approvals/tests may be sensible (Human-in-the-loop).
• Human supervision is crucial, but it is also essential to train the human reviewers to understand the nuances of AI-generated code and to be able to detect errors.

Monitor Your AI Supply Chain

• Document the dependencies of data sources you use for training/tuning your LLM.
• Validate the data wherever possible.
• If AI recommends the use of tools or SDKs, carefully evaluate and validate these tools before first use. Attackers will spoof SDKs and tools that AIs are likely to recommend.
• The AI supply chain can be more opaque and complex than traditional software supply chains, necessitating specialized tools and approaches for effective management and security.

Prompt Injection

With Prompt Injection (similar to SQL Injection), attempts are made to manipulate the LLM by providing inputs intended to coerce the LLM into performing tasks it should not perform.

• Train your teams about Prompt Injection; the same principles used to prevent SQL Injection can be considered.
• For interactions between the LLM and your data, the principle of least privilege should always apply throughout the deployment chain.
• Restrict the access of your LLM to sensitive data. Only provide the data necessary for the respective task. The data must be carefully curated (mask personal/sensitive data appropriately).
• Use function calls where possible to avoid unstructured data that might change the context for the LLM or the desired behavior.
• Also familiarize yourself with indirect Prompt Injection, where data sources are the target of the attack, which then forces the LLM into unwanted actions or responses.
• Ideally, avoid prompts provided directly by users unless absolutely necessary. Mask the interface to the LLM with a suitable GUI.
• Additional security layers, such as real-time monitoring and anomaly detection, can further improve protection against such attacks.

Restrict Data Access for Your LLM

• Treat your LLM with the same sensitivity/criticality as your user data and consider direct access between your LLM and your data critically.
• Adhere to the rules of least privilege. Do not give your LLM more data than it needs for its work.
• Add audit routines around LLM input and output interactions to check input from users and output from the LLM.
• Note the dynamic nature of AI systems. Security measures must be adaptable to evolving data patterns and AI behaviors to ensure they remain effective over time.

Secure Your Vulnerabilities

• Treat code generated by LLM like code from inexperienced developers: validate, test, and correct it primarily during code reviews in the IDE.
• Evaluate tools that can automate the testing of AI-generated code.
• Test and fix first-party code generated by LLM in the IDE if possible (where the code is generated).
• Always manually verify open-source libraries recommended by AI before the first use.
• Establish a combination of automated tools and expert review.

Use Good Training Data

• Fine-tune your LLM model (expensive, slow).
• Use different in-context learning methods (fast, fairly accurate, easy, cheap).
• Validate/verify data sources.
• Protect data sources from external access, so external data cannot be added unintentionally and without review.
• Continuous monitoring and updating of data sources are essential to maintain the security integrity of the LLM.

Use Hybrid AI Models Wherever Possible

• Use the right tool for the specific job. LLMs are best suited for creating broad answers to general problems where absolute accuracy is less important.
• While hybrid models offer advantages, they also introduce complexity in terms of integration and security. Ensuring seamless and secure interoperability between different AI models can be challenging.



Conclusion

AI security requires a very robust strategy, especially due to the fact that Generative AI is still an technology in its early stage, and both attackers and defenders are in a steep learning curve.

Whether it's through refining access controls, maintaining a human oversight, or rigorously validating data, securing AI powerd applications requires great care.

By incorporating the above described practices, developers can aspire to not only enhance the security of their AI systems but also elevate the trust and reliability of the solution.