The AI/ML Security Working Group is a collaborative research initiative within the OpenSSF that explores security risks associated with large language models, generative AI, and machine learning systems.
The working group addresses the growing intersection of AI/ML and security by investigating how artificial intelligence and machine learning technologies pose risks to open source projects, maintainers, and their adopters. It examines both security challenges in AI development itself—such as supply chain security, data poisoning, and model training risks—and the application of AI as a tool for improving security. The scope encompasses prompt injection attacks, adversarial attacks, privacy and secret leakage, licensing concerns, and other emerging threats specific to AI/ML systems. The group pursues this work through collaborative research and peer engagement, drawing on expertise from both security and AI/ML specialists while coordinating with related communities including CNCF's AI Working Group, LFAI & Data, the AI Alliance, and MLCommons.
This initiative is most relevant for open source maintainers and organizations integrating AI technologies into their projects or products. It serves projects seeking guidance on secure AI development practices and those wanting to understand how to detect and defend against AI-related security threats. The working group functions as a central repository for recommendations spanning the entire model lifecycle, from data collection through production deployment, covering both "security for AI"—building AI systems securely—and "AI for security"—leveraging AI to enhance security of other systems.
The project operates as a sandbox-level working group within the OpenSSF structure, indicating it is in an exploratory phase focused on establishing foundational research and community engagement. The working group emphasizes collaboration across multiple communities and organizations rather than developing proprietary tooling, positioning itself as a knowledge aggregation and guidance resource for the open source ecosystem navigating AI/ML security challenges.
