GitHub News and Insights | Microsoft Security Blog http://approjects.co.za/?big=en-us/security/blog/tag/github/ Expert coverage of cybersecurity topics Mon, 08 Jun 2026 17:30:14 +0000 en-US hourly 1 https://wordpress.org/?v=6.9.4 Microsoft Build 2026: Securing code, agents, and models across the development lifecycle http://approjects.co.za/?big=en-us/security/blog/2026/06/02/microsoft-build-2026-securing-code-agents-and-models-across-the-development-lifecycle/ Tue, 02 Jun 2026 17:15:18 +0000 http://approjects.co.za/?big=en-us/security/blog/?p=147521 Discover how Microsoft enables fast, secure AI development with MDASH and new security capabilities.

The post Microsoft Build 2026: Securing code, agents, and models across the development lifecycle appeared first on Microsoft Security Blog.

]]>

Today, developers and security teams are caught in growing tension. AI is accelerating development and introducing new issues around insecure code, opaque models, data exposure, and compliance. Add the challenges of shadow AI and tool sprawl and the result is a widening gap between innovation and control. As developers move faster, security teams struggle to keep up with visibility, governance, and oversight. The resulting friction across the development lifecycle is forcing a tradeoff between speed and safety that doesn’t need to exist. Security needs to move upstream to become part of how developers actually work: built into their day-to-day tools and connected to the tools security teams use.

At Microsoft Build 2026, we are announcing new security tools and capabilities to give developers clear guidance in real time, scale with the complexity of tasks, and provide security teams with a consistent view across the full lifecycle so innovation can move fast and securely without the business losing control. Learn more about our solutions to help secure your code, secure your agents, and secure your models.

Secure your code

Today’s headlines reflect the tension around the power of AI models and the potential threat they pose when used to find and exploit vulnerabilities. It is forcing a shift as security teams look for solutions to help them safely harness the power of these models. At the same time, developers want to use these same models to efficiently identify real, exploitable risk and remediate it within their flow of work. That’s why we developed the Microsoft Security multi-model agentic scanning harness (codename MDASH) and added native integration between Microsoft Defender and GitHub Code Security (part of the former GitHub Advanced Security suite) to help both security and developer teams identify and close gaps early.

Discover and validate exploitable vulnerabilities with codename MDASH

The new Microsoft Security multi-model agentic scanning harness (codename MDASH) is available in an expanded preview for eligible organizations and now includes integration with Microsoft Defender. This new agentic security system orchestrates a pipeline of more than 100 specialized AI agents using an ensemble of models to discover, validate, and prove exploitability across codebases written in popular programming languages.

This approach is unique in the industry. Our multi-model agentic scanning harness uses a configurable panel of models, ranging from state-of-the-art (SOTA) models as the heavy reasoners, to more cost-effective models for high-volume operations. This allows us to trade speed, recall, and cost, and minimize dependency on any specific model.

The combination of multiple models, hundreds of agents, and over 100 trillion signals a day helps identify real risk over theoretical noise, to help teams focus on what can be exploited. The strategic implication is clear: AI vulnerability discovery has crossed from research curiosity into production-grade defense at enterprise scale, and the durable advantage lies in the agentic system around the model rather than any single model itself. MDASH recently jumped roughly 10% in less than three weeks to a new CyberGym industry benchmark score of 96.55%.

“At Accenture, we’re always looking toward the next frontier in protecting our clients and our enterprise. What Microsoft is building with MDASH reflects a meaningful shift from reactive, rule-based scanning to agentic systems that can reason across complex codebases like a skilled security researcher,” says Kris Burkhardt, Chief Information Security Officer at Accenture. Accenture is one of a select group of Security partners and Microsoft Intelligent Security Association (MISA) members that are engaged in the preview to shape MDASH and accelerate agentic AI vulnerability discovery.

Our partner engagements reflect a shared focus on moving from reactive detection to proactive identification of exploitable risk. “We’re seeing cyber threats evolve rapidly, with AI accelerating both the scale and sophistication of attacks. Microsoft’s investment in MDASH reflects a strong commitment to helping organizations stay ahead of this curve. Based on our early discussions and exposure to the innovation, we see strong potential for MDASH to simplify and strengthen SecOps, helping organizations operate with greater resilience and confidence,” says Morgan Adamski, Principal and Deputy Platform Leader of Cyber, Data, and Tech Risk at PwC US.

Together, we are partnering across the industry to use leading models paired with our platforms and expertise to deliver protection at scale. “We’re excited to work with Microsoft on MDASH because it addresses one of the most pressing challenges our customers face: reducing the time between discovering a vulnerability and taking meaningful action. Microsoft’s role as a trusted security vendor matters here—customers need innovation, but they also need confidence, governance, and a partner they can rely on. Our early experience with MDASH has been encouraging, and we see real opportunity for it to help organizations modernize how they approach vulnerability discovery and remediation,” says Jason Rader, Insight CISO.  

Reach out to your Microsoft account representative for more information on the expanded preview of codename MDASH.

Prioritize and remediate code vulnerabilities with Microsoft Defender and GitHub Code Security

While codename MDASH identifies and validates what’s truly exploitable, the integration between Microsoft Defender and GitHub Code Security (part of the former GitHub Advanced Security suite), now generally available, brings runtime context into development and security workflows so that teams can prioritize and address risks early minimizing the impact to human resources. Vulnerabilities discovered in code are automatically enriched with real production signals, such as internet exposure and data sensitivity to inform prioritization. Developers can then remediate issues using AI-assisted fixes that are generated, assigned, and validated through GitHub Copilot Autofix and the GitHub Copilot cloud agent.

To support responsible, coordinated disclosure of findings that represent both real and potential vulnerabilities, role-based access controls ensure that only authorized individuals can view and act on them. Together, the production signal enrichment, AI-assisted remediation, and secure handling of findings within a single workflow help security and developer teams focus on real risk and enable teams to act quickly.

Secure your agents

Agents are quickly becoming a new layer of the application stack. As developers build agents and move them into production, they need the tools to ship fast without sacrificing security, including built-in identity, governance, and safety testing. Security teams have overlapping needs: visibility into what’s running, control over what agents can access, and consistent governance across clouds and endpoints. Microsoft is delivering new solutions to help.

Build secure agents from day one

At Build 2026, Microsoft is introducing new capabilities to help developers build secure, enterprise-ready agents by default. With the general availability of the Agent 365 SDK, developers can integrate controls directly into their development workflows, bringing observability, access controls, and compliance enforcement into how agents are designed and deployed. This enables teams to build custom agents for any AI platform that are compliant, and enterprise-ready, and compose well with Agent 365.

Security extends beyond development and into how agents run. On Windows, the Microsoft Execution Container (MXC) SDK provides OS-level control over agent execution, giving developers and IT teams the ability to define containment and policy, applied by the OS through isolation technologies such as process and session isolation. Windows 365 for Agents, now generally available, enables you to run any agent in a fully isolated, policy-governed Cloud PC. Native Windows integration with Agent 365 provides a common foundation for observability, security, and governance, including built-in Intune capabilities to set policies that govern agent runtime execution and control how agents operate.

These new capabilities are now in early preview.

Observe, govern, and secure agents at scale with Agent 365—now including local agents

As agents proliferate across environments, gaining visibility and control over them becomes critical. Agent 365 introduces new capabilities to manage agent sprawl and risk, including an Agent 365 Agent Registry that surfaces unmanaged local agents discovered by Microsoft Defender, Microsoft Entra, and Microsoft Intune—all working together. The registry supports more than 20 types of local agents, including coding agents, AI desktop applications, and both local and remote Model Context Protocol (MCP) servers. From there, Intune policies can be used to block common execution methods for OpenClaw agents.

Security teams also need the ability to defend against emerging threats without slowing developer productivity. Microsoft Defender, Entra, and Intune work together to provide the visibility, runtime protections, and context needed to manage agent risk without slowing developer productivity. Defender enables analysts to investigate agent activity using advanced hunting and provides an exposure graph that helps teams understand how agents are connected across the network. Preview of these capabilities coming soon.

Protecting data is foundational to securing agents at scale. Microsoft Purview controls to prevent data exfiltration, Data Security Posture Management risk discovery, and agentic risk detection for coding agents Claude Code, GitHub Copilot, OpenAI Codex, and OpenClaw. This enables visibility on how local agents access sensitive data, runtime protections for risky prompts, and insights into unsafe agent behaviors. Microsoft Purview Audit also logs all agent activity for full traceability. Preview of these capabilities coming soon.

Trust agents with your data

Developers also need direct, real-time insight into data security posture and risk signals associated with the agents they build. With Purview data risk signals embedded in the Foundry Control Plane, generally available, these signals provide guidance to developers on where to enforce protections before sensitive data is exposed. For example, Purview flags in real time when an agent surfaces sensitive financial data during testing and guides developers to mask or restrict access before deployment.

To further reduce risk, Purview introduces runtime data loss prevention (DLP) for agent prompts in Foundry, in preview with Agent 365. This capability detects, blocks, and audits sensitive data before it is processed by the agent, ensuring that sensitive information never reaches AI models.

Secure your models

Before AI reaches production, teams need to verify that the models they depend on are safe. Now developers can inspect model artifacts, whether platform-native or bring-your-own, with Defender AI model scanning, in preview. To help close gaps early model Defender AI model scanning detects and blocks potentially vulnerable or compromised models across registries, workspaces, and CI/CD pipelines to verify model integrity before deployment.

Trust starts with security

There should never be a choice between innovation and safety.

The capabilities announced today span the full development lifecycle: discovering what’s exploitable, governing what’s running, protecting the data AI depends on, and verifying that agents behave as intended before they reach production. Microsoft security is embedded directly into the platforms and workflows developers already use, supporting innovation across Microsoft Foundry, Copilot Studio, GitHub, and open-source frameworks, and bringing discovery and governance to shadow AI.

But real progress in AI depends on more than breakthrough capabilities—it depends on whether organizations can trust the systems they are building and deploying. That is the common thread across the innovations announced at Build 2026 and the principle guiding our approach. Because the future of AI will belong not just to those who move fastest, but to those who can innovate with trust.

To learn more about Microsoft Security solutions, visit our website. Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us on LinkedIn (Microsoft Security) and X (@MSFTSecurity) for the latest news and updates on cybersecurity. To learn more about how security is built into the Windows platform, explore the Windows Security book and Windows Server Security book.

The post Microsoft Build 2026: Securing code, agents, and models across the development lifecycle appeared first on Microsoft Security Blog.

]]>
New Microsoft guidance for the CISA Zero Trust Maturity Model http://approjects.co.za/?big=en-us/security/blog/2024/12/19/new-microsoft-guidance-for-the-cisa-zero-trust-maturity-model/ Thu, 19 Dec 2024 17:00:00 +0000 New Microsoft guidance is now available for United States government agencies and their industry partners to help implement Zero Trust strategies and meet CISA Zero Trust requirements.

The post New Microsoft guidance for the CISA Zero Trust Maturity Model appeared first on Microsoft Security Blog.

]]>
The Cybersecurity Infrastructure Security Agency (CISA) Zero Trust Maturity Model (ZTMM) assists agencies in development of their Zero Trust strategies and continued evolution of their implementation plans. In April of 2024, we released Microsoft guidance for the Department of Defense Zero Trust Strategy. And now, we are excited to share new Microsoft Guidance for CISA Zero Trust Maturity Model. Our guidance is designed to help United States government agencies and their industry partners configure Microsoft cloud services as they transition to Zero Trust, on their journey to achieve advanced and optimal security.

Microsoft has embraced Zero Trust principles—both in the way we secure our own enterprise environment and for our customers. We’ve been helping thousands of organizations worldwide transition to a Zero Trust security model, including many United States government agencies. In this blog, we’ll preview the new guidance and share how it helps United States government agencies and their partners implement their Zero Trust strategies. We’ll also share the Microsoft Zero Trust platform and relevant solutions that help meet CISA’s Zero Trust requirements, and close with two examples of real-world deployments.

CISA Zero Trust Maturity Model

Use this guidance to help meet the goals for ZTMM functions and make progress through maturity stages.

CLO25-Security-Lifestyle-Getty-1312953595

Microsoft supports CISA’s Zero Trust Maturity Model

CISA’s Zero Trust Maturity Model provides detailed guidance for organizations to evaluate their current security posture and identify necessary changes for transitioning to more modernized federal cybersecurity.

The five CISA Zero Trust Pillars: Identity, Devices, Networks, Applications & Workloads, and Data, as well as capabilities uniform across all pillars – including Visibility & analytics, Automation & orchestration, and Governance.
Figure 1. CISA Zero Trust Maturity Model.

The CISA Zero Trust Maturity Model includes five pillars that represent protection areas for Zero Trust:

  1. Identity: An identity refers to an attribute or set of attributes that uniquely describes an agency user or entity, including non-person entities.
  2. Devices: A device refers to any asset (including its hardware, software, and firmware) that can connect to a network, including servers, desktop and laptop machines, printers, mobile phones, Internet of Things (IoT) devices, networking equipment, and more.
  3. Networks: A network refers to an open communications medium including typical channels such as agency internal networks, wireless networks, and the internet as well as other potential channels such as cellular and application-level channels used to transport messages.
  4. Applications and workloads: Applications and workloads include agency systems, computer programs, and services that execute on-premises, on mobile devices, and in cloud environments.
  5. Data: Data includes all structured and unstructured files and fragments that reside or have resided in federal systems, devices, networks, applications, databases, infrastructure, and backups (including on-premises and virtual environments) as well as the associated metadata.

The model also integrates capabilities that span across all pillars, to enhance cross-function interoperability—including visibility and analytics, automation and orchestration, and governance. The model further includes the four maturity stages of the Zero Trust Maturity Model:

  • Traditional: The starting point for many government organizations, where assessment and identification of gaps helps determine security priorities.
  • Initial: Organizations will have begun implementing automation in areas such as attribute assignment, lifecycle management, and initial cross-pillar solutions including integration of external systems, least privilege strategies, and aggregated visibility.
  • Advanced: Organizations have progressed further along the maturity journey including centralized identity management and integrated policy enforcement across all pillars. Organizations build towards enterprise-wide visibility including near real time risk and posture assessments.
  • Optimal: Organizations have fully automated lifecycle management implementing dynamic just-enough access (JEA) with just-in-time (JIT) controls for access to organization resources. Organizations implement continuous monitoring with centralized visibility. 

Microsoft’s Zero Trust Maturity Model guidance serves as a reference for how government organizations should address key aspects of pillar-specific functions for each pillar, across each stage of implementation maturity, using Microsoft cloud services. Microsoft product teams and security architects supporting government organizations worked in close partnership to provide succinct, actionable guidance that aligns with the CISA Zero Trust Maturity Model and is organized by pillar, function, and maturity stage, with product guidance including linked references.

The guidance focuses on features available now (including public preview) in Microsoft commercial clouds. As cybersecurity threats continue to evolve, Microsoft will continue to innovate to meet the needs of our government customers. We’ve already launched more features aligned to the principles of Zero Trust—including Microsoft Security Exposure Management (MSEM) and more. Look for updates and announcements in the Microsoft Security Blog and check Microsoft Learn for Zero Trust guidance for Government customers to stay up to date with the latest information.

Microsoft’s Zero Trust platform

Microsoft is proud to be recognized as a Leader in the Forrester Wave™: Zero Trust Platform Providers, Q3 2023 report.1 The Microsoft Zero Trust platform is a modern security architecture that emphasizes proactive, integrated, and automated security measures. Microsoft 365 E5 combines best-in-class productivity apps with advanced security capabilities and innovations for government customers that include certificate-based authentication in the cloud, Conditional Access authentication strength, cross-tenant access settings, FIDO2 provisioning APIs, Azure Virtual Desktop support for passwordless authentication, and device-bound passkeys. Microsoft 365 is a comprehensive and extensible Zero Trust platform that spans hybrid cloud, multicloud, and multiplatform environments, delivering a rapid modernization path for organizations.

Diagram displaying Microsoft’s Zero Trust Architecture across six pillars: Identities, Devices, Data, Apps, Infrastructure, and Network.
Figure 2. Microsoft Zero Trust Architecture.

Microsoft cloud services that support the five pillars of the CISA Zero Trust Maturity Model include:

Microsoft Entra ID is an integrated multicloud identity and access management solution and identity provider that helps achieve capabilities in the identity pillar. It is tightly integrated with Microsoft 365 and Microsoft Defender XDR services to provide a comprehensive suite of Zero Trust capabilities including strict identity verification, enforcing least privilege, and adaptive risk-based access control. Built for cloud-scale, Microsoft Entra ID handles billions of authentications every day. Establishing it as your organization’s Zero Trust identity provider lets you configure, enforce, and monitor adaptive Zero Trust access policies in a single location. Conditional Access is the Zero Trust authorization engine for Microsoft Entra ID, enabling dynamic, adaptive, fine-grained, risk-based, access policies for any workload.

Microsoft Intune is a multiplatform endpoint and application management suite for Windows, MacOS, Linux, iOS, iPadOS, and Android devices. Its configuration policies manage devices and applications. Microsoft Defender for Endpoint helps organizations prevent, detect, investigate, and respond to advanced cyberthreats on devices. Microsoft Intune and Defender for Endpoint work together to enforce security policies, assess device health, vulnerability exposure, risk level, and configuration compliance status. Microsoft Intune and Microsoft Defender for Endpoint help achieve capabilities in the device pillar.

GitHub is a cloud-based platform where you can store, share, and work together with others to write code. GitHub Advanced Security includes features that help organizations improve and maintain code by providing code scanning, secret scanning, security checks, and dependency review throughout the deployment pipeline. Microsoft Entra Workload ID helps organizations use continuous integration and continuous delivery (CI/CD) with GitHub Actions. GitHub and Azure DevOps are essential to the applications and workloads pillar.

Microsoft Purview aligns to the data pillar activities, with a range of solutions for unified data security, data governance, and risk and compliance management. Microsoft Purview Information Protection lets you define and label sensitive information types. Auto-labeling within Microsoft 365 clients ensures data is appropriately labeled and protected. Microsoft Purview Data Loss Prevention integrates with Microsoft 365 services and apps, and Microsoft Defender XDR components to detect and prevent data loss.

Azure networking services include a range of software-defined network resources that can be used to provide networking capabilities for connectivity, application protection, application delivery, and network monitoring. Azure networking resources like Microsoft Azure Firewall Premium, Azure DDoS Protection, Microsoft Azure Application Gateway, Azure API Management, Azure Virtual Network, and network security groups, all work together to provide routing, segmentation, and visibility into your network. Azure networking services and network segmentation architectures are essential to the network pillar.

Microsoft Defender XDR plays key roles across multiple pillars, critical to both the automation and orchestration and visibility and analytics cross-cutting capabilities. It is a unified pre-breach and post-breach enterprise defense suite that natively coordinates detection, prevention, investigation, and response actions. It correlates millions of signals across endpoints, identities, email, and applications to automatically disrupt cyberattacks. Microsoft Defender XDR’s automated investigation and response and Microsoft Sentinel playbooks are used to complete security orchestration, automation, and response (SOAR) activities.

Microsoft Sentinel is essential to both automation and orchestration and visibility and analytics cross-cutting capabilities, along with any activities requiring SIEM integration. It is a cloud-based security information and event management (SIEM) you deploy in Azure. Microsoft Sentinel operates at cloud scale to accelerate security response and save time by automating common tasks and streamlining investigations with incident insights. Built-in data connectors make it easy to ingest security logs from Microsoft 365, Microsoft Defender XDR, Microsoft Entra ID, Azure, non-Microsoft clouds, and on-premises infrastructure.

Real-world pilots and implementations utilizing Microsoft guidance

The United States Department of Agriculture (USDA) implements multifaceted solution for phishing-resistance initiative—In this customer story, the USDA implements phishing-resistant multifactor authentication (MFA)—which is important aspect of the identity pillar of the CISA Zero Trust Maturity Model. By selecting Microsoft Entra ID, the USDA was able to scale these capabilities to enforce phishing-resistant authentication with Microsoft Entra Conditional Access for their four main enterprise services—Windows desktop logon, Microsoft M365, VPN, single sign-on (SSO). By integrating their centralized WebSSO platform with Microsoft Entra ID and piloting more than 600 internal applications, the USDA incrementally and rapidly deployed the capability to support the applications and services relevant to most users. Read more about their experience making incremental improvements towards stronger phishing resistance with Microsoft Entra ID.

The United States Navy collaborates with Microsoft on CISA Zero Trust implementation—In this customer story, the United States Navy was able to utilize Zero Trust activity-level guidance to meet or exceed the Department of Defense (DoD) Zero Trust requirements with Microsoft Cloud services. And now with Microsoft guidance tailored for the United States government agencies, the aim is to help civilian agencies and their industry partners to do the same—meeting the CISA ZTMM recommendations at each maturity stage with Microsoft Cloud services. Together with Microsoft, the Navy developed an integrated model of security to help meet their ZT implementation goals. Read more about their collaboration with Microsoft.

Access Microsoft guidance for the United States Government customers and their partners. Embrace proactive and proven security with Zero Trust.

Learn more

To learn more about Microsoft Security solutions, visit our website. Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us on LinkedIn (Microsoft Security) and X (@MSFTSecurity) for the latest news and updates on cybersecurity.


1Forrester Wave™: Zero Trust Platform Providers, Q3 2023, Carlos Rivera and Heath Mullins, September 19th, 2023.

The post New Microsoft guidance for the CISA Zero Trust Maturity Model appeared first on Microsoft Security Blog.

]]>
AI security risk assessment using Counterfit http://approjects.co.za/?big=en-us/security/blog/2021/05/03/ai-security-risk-assessment-using-counterfit/ Mon, 03 May 2021 16:00:52 +0000 Counterfit is a command-line tool for security professionals to red team AI systems and systematically scans for vulnerabilities as part of AI risk assessment.

The post AI security risk assessment using Counterfit appeared first on Microsoft Security Blog.

]]>
Today, we are releasing Counterfit, an automation tool for security testing AI systems as an open-source project. Counterfit helps organizations conduct AI security risk assessments to ensure that the algorithms used in their businesses are robust, reliable, and trustworthy.

AI systems are increasingly used in critical areas such as healthcare, finance, and defense. Consumers must have confidence that the AI systems powering these important domains are secure from adversarial manipulation. For instance, one of the recommendations from Gartner’s Top 5 Priorities for Managing AI Risk Within Gartner’s MOST Framework published in Jan 20211 is that organizations “Adopt specific AI security measures against adversarial attacks to ensure resistance and resilience,” noting that “By 2024, organizations that implement dedicated AI risk management controls will successfully avoid negative AI outcomes twice as often as those that do not.”

However, performing security assessments of production AI systems is nontrivial. Microsoft surveyed 28 organizations, spanning Fortune 500 companies, governments, non-profits, and small and medium sized businesses (SMBs), to understand the current processes in place to secure AI systems. We found that 25 out of 28 businesses indicated they don’t have the right tools in place to secure their AI systems and that security professionals are looking for specific guidance in this space.

This tool was born out of our own need to assess Microsoft’s AI systems for vulnerabilities with the goal of proactively securing AI services, in accordance with Microsoft’s responsible AI principles and Responsible AI Strategy in Engineering (RAISE) initiative. Counterfit started as a corpus of attack scripts written specifically to target individual AI models, and then morphed into a generic automation tool to attack multiple AI systems at scale.

Today, we routinely use Counterfit as part of our AI red team operations. We have found it helpful to automate techniques in MITRE’s Adversarial ML Threat Matrix and replay them against Microsoft’s own production AI services to proactively scan for AI-specific vulnerabilities. Counterfit is also being piloted in the AI development phase to catch vulnerabilities in AI systems before they hit production.

To ensure that Counterfit addresses a broader set of security professionals’ needs, we engaged with a diverse profile of partners spanning large organizations, SMBs, and governmental organizations to test the tool against their ML models in their environments.

“AI is increasingly used in industry; it is vital to look ahead to securing this technology particularly to understand where feature space attacks can be realized in the problem space. The release of open-source tools from an organization such as Microsoft for security practitioners to evaluate the security of AI systems is both welcome and a clear indication that the industry is taking this problem seriously.”

Matilda Rhode, Senior Cybersecurity Researcher, Airbus

Three key ways Counterfit is flexible

As a result of internal and external engagements, Counterfit is flexible in three key ways:

  1. Counterfit is environment agnostic—it can help assess AI models hosted in any cloud environment, on-premises, or on the edge.
  2. Counterfit is model agnostic—the tool abstracts the internal workings of their AI models so that security professionals can focus on security assessment.
  3. Counterfit strives to be data agnostic—it works on AI models using text, images, or generic input.

Under the hood, Counterfit is a command-line tool that provides a generic automation layer for adversarial AI frameworks such as Adversarial Robustness Toolbox and TextAttack. Our tool makes published attack algorithms accessible to the security community and helps to provide an extensible interface from which to build, manage, and launch attacks on AI models.

Designed for security professionals

Counterfit uses workflows and terminology similar to popular offensive tools that security professionals are already familiar with, such as Metasploit or PowerShell Empyre. Security professionals can benefit from the tool in the following ways:

  • Penetration testing and red teaming AI systems: The tool comes preloaded with published attack algorithms that can be used to bootstrap red team operations to evade and steal AI models. Since attacking AI systems also involves elements of traditional exploitation, security professionals can use the target interface and built-in cmd2 scripting engine to hook into Counterfit from existing offensive tools. Additionally, the target interface can allow for granular control over network traffic. We recommend using Counterfit alongside Adversarial ML Threat Matrix, which is an ATT&CK style framework released by MITRE and Microsoft for security analysts to orient to threats against AI systems.
Demo of Microsoft Counterfit used to scan an AI model for model evasion vulnerability.
  • Vulnerability scanning for AI systems: The tool can help scan AI models using published attack algorithms. Security professionals can use the defaults, set random parameters, or customize them for broad vulnerability coverage of an AI model. Organizations with multiple models in their AI system can use Counterfit’s built-in automation to scan at scale. Optionally, Counterfit enables organizations to scan AI systems with relevant attacks any number of times to create baselines. Running this system regularly, as vulnerabilities are addressed, also helps to measure ongoing progress toward securing AI systems.
  • Logging for AI systems: Counterfit also provides logging to record the attacks against a target model. Telemetry may help data science and engineering teams improve their understanding of failure modes in their AI systems.

This tool is part of broader efforts at Microsoft to empower engineers to securely develop and deploy AI systems. We recommend using it alongside the following resources:

  • For security analysts to orient to threats against AI systems, Microsoft, in collaboration with MITRE, released an ATT&CK style Adversarial ML Threat Matrix complete with case studies of attacks on production ML systems.
  • For security incident responders, we released our own bug bar to systematically triage attacks on ML systems.
  • For industry practitioners and security professionals to develop muscle in defending and attacking ML systems, we hosted a realistic Machine Learning Evasion Competition.
  • For developers, we released threat modeling guidance specifically for ML systems.
  • For engineers and policymakers, Microsoft, in collaboration with Berkman Klein Center at Harvard University, released a taxonomy documenting various ML failure modes.

Learn more

To learn more about this effort:

To learn more about Microsoft Security solutions visit our website. Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us at @MSFTSecurity for the latest news and updates on cybersecurity.


1Gartner, Top 5 Priorities for Managing AI Risk Within Gartner’s MOST Framework, Avivah Litan, et al., 15 January 2021.

The post AI security risk assessment using Counterfit appeared first on Microsoft Security Blog.

]]>