Today, we are launching MLSEC.IO, an educational Machine Learning Security Evasion Competition (MLSEC) for the AI and security communities to exercise their muscle to attack critical AI systems in a realistic setting. Hosted and sponsored by Microsoft, alongside NVIDIA, CUJO AI, VM-Ray, and MRG Effitas, the competition rewards participants who efficiently evade AI-based malware detectors and AI-based phishing detectors.
Machine learning powers critical applications in virtually every industry: finance, healthcare, infrastructure, and cybersecurity. Microsoft is seeing an uptick of attacks on commercial AI systems that could compromise the confidentiality, integrity, and availability guarantees of these systems. Publicly known cases documented by MITRE’s ATLAS framework, show how with the proliferation of AI systems comes the increased risk that the machine learning powering these systems can be manipulated to achieve an adversary’s goals. While the risks are inherent in all deployed machine learning models, the threat is especially explicit in cybersecurity, where machine learning models are increasingly relied on to detect threat actors’ tools and behaviors. Market surveys have consistently indicated that the security and privacy of AI systems are top concerns for executives. According to CCS Insight’s survey of 700 senior IT leaders in 2020, security is now the biggest hurdle companies face with AI, cited by over 30 percent of respondents1.
However, security practitioners are unaware of how to clear this new hurdle. A recent Microsoft survey found that 25 out of 28 organizations did not have the right tools in place to secure their AI systems. While academic researchers have been studying how to attack AI systems for close to two decades, awareness among practitioners is low. That is why one recommendation for business leaders from the 2021 Gartner report Top 5 Priorities for Managing AI Risk Within Gartner’s MOST Framework published2 is that organizations “Drive staff awareness across the organization by leading a formal AI risk education campaign.”
It is critical to democratize the knowledge to secure AI systems. That is why Microsoft recently released Counterfit, a tool born out of our own need to assess Microsoft’s AI systems for vulnerabilities with the goal of proactively securing AI services. For those new to adversarial machine learning, NVIDIA released MINTNV, a hack-the-box style environment to explore and build their skills.
With the launch today of MLSEC.IO, we aim to highlight how security models can be evaded by motivated attackers and allow practitioners to exercise their muscles attacking critical machine learning systems used in cybersecurity.
“There is a lack of practical knowledge about securing or attacking AI systems in the security community. Competitions like Microsoft’s MSLEC democratizes adversarial machine learning knowledge for the offensive and defensive security communities, as well as the machine learning community. MLSEC’s hands-on approach is an exciting entry point into AML.”—Christopher Cottrell, AI Red Team Lead, NVIDIA
The competition involves two challenges beginning on August 6 and ending on September 17, 2021: an Anti-Malware Evasion track and an Anti-Phishing Evasion track.
In addition, for each of the Attacker Challenge tracks, the highest-scoring submission that extends and leverages Counterfit—Microsoft’s open-source tool for investigating the security of machine learning models—will be awarded a bonus prize.
“The security evasion challenge creates new pathways into cybersecurity and opens up access for a broader base of talent. This year, to lower barriers to entry, we are introducing the phishing challenge, while still strongly encouraging people without significant experience in malware to participate.”—Zoltan Balazs, Head of Vulnerability Research Lab at CUJO AI and cofounder of the competition.
To learn more about the 2021 Machine Learning Security Evasion Competition:
This competition is part of broader efforts at Microsoft to empower engineers to securely develop and deploy AI systems. We recommend using it alongside the following resources:
Register now to participate in the Machine Learning Security Evasion Competition that begins on August 6 and ends on September 17, 2021. Winners will be announced on October 27, 2021.
To learn more about Microsoft Security solutions, visit our website. Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us at @MSFTSecurity for the latest news and updates on cybersecurity.
1CCS Insight, Senior Leadership IT Investment Survey, Nick McQuire et. al, 18 August 2020.
2Gartner, Top 5 Priorities for Managing AI Risk Within Gartner’s MOST Framework, Avivah Litan, et al., 15 January 2021.
Microsoft, in collaboration with OpenAI, is publishing research on emerging threats in the age of AI, focusing on identified activity associated with known threat actors Forest Blizzard, Emerald Sleet, Crimson Sandstorm, and others. The observed activity includes prompt-injections, attempted misuse of large language models (LLM), and fraud.
The Microsoft Incident Response team takes swift action to help contain a ransomware attack and regain positive administrative control of the customer environment.
With the coming wave of AI, this is precisely the time for organizations to prepare for the future. To be properly ready for AI, Zero Trust principles take on new meaning and scope. The right endpoint management strategy can help provide the broadest signal possible and make your organization more secure and productive for years to come.
Business email operators seek to exploit the daily sea of email traffic to lure victims into providing financial and other sensitive business information.
