At Microsoft, we remain steadfast in our commitment to security, which continues to be our top priority. Through our Secure Future Initiative (SFI), we’ve dedicated the equivalent of 34,000 full-time engineers to the effort, making it the largest cybersecurity engineering project in history—driving continuous improvement in our cyber resilience. In our latest update, we share insights into the work we are doing in culture, governance, and cybernorms to promote transparency and better support our customers in this new era of security. For each engineering pillar, we provide details on steps taken to reduce risk and provide guidance so customers can do the same.

Insights gained from SFI help us continue to harden our security posture and product development. At Microsoft Ignite 2024, we are pleased to unveil new security solutions, an industry-leading bug bounty program, and innovations in our AI platform. 

Transforming security with graph-based posture management 

Microsoft’s Security Fellow and Deputy Chief Information Security Office (CISO) John Lambert says, “Defenders think in lists, cyberattackers think in graphs. As long as this is true, attackers win,” referring to cyberattackers’ relentless focus on the relationships between things like identities, files, and devices. Exploiting these relationships helps criminals and spies do more extensive damage beyond the point of intrusion. Poor visibility and understanding of relationships and pathways between entities can limit traditional security solutions to defending in siloes, unable to detect or disrupt advanced persistent threats (APTs).

We are excited to announce the general availability of Microsoft Security Exposure Management. This innovative solution dynamically maps changing relationships between critical assets such as devices, data, identities, and other connections. Powered by our security graph, and now with third-party connectors for Rapid 7, ServiceNow, Qualys, and Tenable in preview, Exposure Management provides customers with a comprehensive, dynamic view of their IT assets and potential cyberattack paths. This empowers security teams to be more proactive with an end-to-end exposure management solution. In the constantly evolving cyberthreat landscape, defenders need tools that can quickly identify signal from noise and help prioritize critical tasks.  

Beyond seeing potential cyberattack paths, Exposure Management also helps security and IT teams measure the effectiveness of their cyber hygiene and security initiatives such as zero trust, cloud security, and more. Currently, customers are using Exposure Management in more than 70,000 cloud tenants to proactively protect critical entities and measure their cybersecurity effectiveness.

Announcing $4 million AI and cloud security bug bounty “Zero Day Quest” 

Born out of our Secure Future Initiative commitments and our belief that security is a team sport, we also announced Zero Day Quest, the industry’s largest public security research event. We have a long history of partnering across the industry to mitigate potential issues before they impact our customers, which also helps us build more secure products by default and by design.  

Every year our bug bounty program pays millions for high-quality security research with over $16 million awarded last year. Zero Day Quest will build on this work with an additional $4 million in potential rewards focused on cloud and AI—— which are areas of highest impact to our customers. We are also committed to collaborating with the security community by providing access to our engineers and AI red teams. The quest starts now and will culminate in an in-person hacking event in 2025.

As part of our ongoing commitment to transparency, we will share the details of the critical bugs once they are fixed so the whole industry can learn from them—after all, security is a team sport. 

New advances for securing AI and new skills for Security Copilot 

AI adoption is rapidly outpacing many other technologies in the digital era. Our generative AI solution, Microsoft Security Copilot, continues to be adopted by security teams to boost productivity and effectiveness. Organizations in every industry, including National Australia Bank, Intesa Sanpaolo, Oregon State University, and Eastman are able to perform security tasks faster and more accurately.² A recent study found that three months after adopting Security Copilot, organizations saw a 30% reduction in their mean time to resolve security incidents. More than 100 partners have integrated with Security Copilot to enrich the insights with ecosystem data. New Copilot skills are now available for IT admins in Microsoft Entra and Microsoft Intune, data security and compliance teams in Microsoft Purview, and security operations teams in the Microsoft Defender product family.   

According to our Security for AI team’s new “Accelerate AI transformation with strong security” white paper, we found that over 95% of organizations surveyed are either already using or developing generative AI, or they plan to do so in the future, with two thirds (66%) choosing to develop multiple AI apps of their own. This fast-paced adoption has led to 37 new AI-related bills passed into law worldwide in 2023, reflecting a growing international effort to address the security, safety, compliance, and transparency challenges posed by AI technologies.³ This underscores the criticality of securing and governing the data that fuels AI. Through Microsoft Defender, our customers have discovered and secured more than 750,000 generative AI app instances and Microsoft Purview has audited more than a billion Copilot interactions.⁴  

Microsoft Purview is already helping thousands of organizations, such as Cummins, KPMG, and Auburn University, with their AI transformation by providing data security and compliance capabilities across Microsoft and third-party applications. Now, we’re announcing new capabilities in Microsoft Purview to discover, protect, and govern data in generative AI applications. Available for preview, new capabilities in Purview include Data Loss Prevention (DLP) for Microsoft 365 Copilot, prevention of data oversharing in AI apps, and detection of risky AI use such as malicious intent, prompt injections, and misuse of protected materials. Additionally, Microsoft Purview now includes Data Security Posture Management (DSPM) that gives customers a single pane of glass to proactively discover data risks, such as sensitive data in user prompts, and receive recommended actions and insights for quick responses during incidents. For more details, read the blog on Tech Community. 

Microsoft continues to innovate on our end-to-end security platform to help defenders make the complex simpler, while staying ahead of cyberthreats and enabling their AI transformation. At the same time, we are continuously improving the safety and security of our cloud services and other technologies, including these recent steps to make Windows 11 more secure. 

Next steps with Microsoft Security

From the advances announced to our daily defense of customers, and the steadfast dedication of Chief Executive Officer (CEO) Satya Nadella and every employee, security remains our top priority at Microsoft as we deliver on our principles of secure by design, secure by default, and secure operations. To learn more about our vision for the future of security, tune in to the Microsoft Ignite keynote. 

Microsoft Ignite 2024

Gain insights to keep your organizations safer with an AI-first, end-to-end cybersecurity approach.

Register now

Are you a regular user of Microsoft Security products? Review your experience on Gartner Peer Insights™ and get a $25 gift card. To learn more about Microsoft Security solutions, visit our website. Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us on LinkedIn (Microsoft Security) and X (@MSFTSecurity) for the latest news and updates on cybersecurity. 

¹ Microsoft Digital Defense Report 2024.

² Microsoft customer stories:

• National Australia Bank invests in an efficient, cloud-managed future with Windows 11 Enterprise
• Intesa Sanpaolo accrues big cybersecurity dividends with Microsoft Sentinel, Copilot for Security
• Oregon State University protects vital research and sensitive data with Microsoft Sentinel and Microsoft Defender
• Eastman catalyzes cybersecurity defenses with Copilot for Security

³ How countries around the world are trying to regulate artificial intelligence, Theara Coleman, The Week US. July 4, 2023.

⁴ Earnings Release FY25 Q1, Microsoft. October 30, 2024.

The post AI innovations for a more secure future unveiled at Microsoft Ignite appeared first on Microsoft Security Blog.

84% of surveyed organizations want to feel more confident about managing and discovering data input into AI apps and tools. This report includes research to provide you with the actionable industry-agnostic insights and guidance to better secure your data used by your generative AI applications. 

Microsoft Data Security Index

Gain deeper insights about generative AI and its influence on data security.

Discover more

In 2023, we commissioned our first independent research that surveyed more than 800 data security professionals to help business leaders develop their data security strategies. This year, we expanded the survey to 1,300 security professionals to uncover new learnings on data security and AI practices.   

Some of the top-level insights from our expanded research are:  

1. The data security landscape remains fractured across traditional and new risks due to AI.
2. User adoption of generative AI increases the risk and exposure of sensitive data.
3. Decision-makers are optimistic about AI’s potential to boost their data security effectiveness.

The data security landscape remains fractured across traditional and new risks

On average, organizations are juggling 12 different data security solutions, creating complexity that increases their vulnerability. This is especially true for the largest organizations: On average, medium enterprises use nine tools, large enterprises use 11, and extra-large enterprises use 14. In addition, 21% of decision-makers cite the lack of consolidated and comprehensive visibility caused by disparate tools as their biggest challenge and risk.

Fragmented solutions make it difficult to understand data security posture since data is isolated and disparate workflows could limit comprehensive visibility into potential risks. When tools don’t integrate, data security teams have to build processes to correlate data and establish a cohesive view of risks, which can lead to blind spots and make it challenging to detect and mitigate risks effectively.

As a result, the data also shows a strong correlation between the number of data security tools used and the frequency of data security incidents. In 2024, organizations using more data security tools (11 or more) experienced an average of 202 data security incidents, compared to 139 incidents for those with 10 or fewer tools.

In addition, a growing area of concern is the rise in data security incidents from the use of AI applications, which nearly doubled from 27% in 2023 to 40% in 2024. Attacks from the use of AI apps not only expose sensitive data but also compromise the functionality of the AI systems themselves, further complicating an already fractured data security landscape.

In short, there’s an increasingly urgent need for more integrated and cohesive data security strategies that can address both traditional and emerging risks linked to the use of AI tools.

Adoption of generative AI increases the risk and exposure of sensitive data

User adoption of generative AI increases the risk and exposure of sensitive data. As AI becomes more embedded in daily operations, organizations recognize the need for stronger protection. 96% of companies surveyed admitted that they harbored some level of reservation about employee use of generative AI. However, 93% of companies also reported that they had taken proactive action and were at some stage of either developing or implementing new controls around employee use of generative AI.  

Unauthorized AI applications can access and misuse data, leading to potential breaches. The use of these unauthorized AI applications often occurs with employees logging in with personal credentials or using personal devices for work-related tasks. On average, 65% of organizations admit that their employees are using unsanctioned AI apps.

Given these concerns, it is important for organizations to implement the right data security controls and to mitigate these risks and ensure that AI tools are used responsibly. Currently, 43% of companies are focused on preventing sensitive data from being uploaded into AI apps, while another 42% are logging all activities and content within these apps for potential investigations or incident response. Similarly, 42% are blocking user access to unauthorized tools, and an equal percentage are investing in employee training on secure AI use.

To implement the right data security controls, customers need to increase their visibility of their AI application usage as well as the data that is flowing through those applications. In addition, they need a way to assess the risk levels of emerging generative AI applications and be able to apply conditional access policies to those applications based on a user’s risk levels.

Finally, they need to be able to access audit logs and generate reports to help them assess their overall risk levels as well as provide transparency and reporting for regulatory compliance.

AI’s potential to boost data security effectiveness

Traditional data security measures often struggle to keep up with the sheer volume of data generated in today’s digital landscape. AI, however, can sift through this data, identifying patterns and anomalies that might indicate a security threat. Regardless of where they are in their generative AI adoption journeys, organizations that have implemented AI-enabled data security solutions often gain both increased visibility across their digital estates and increased capacity to process and analyze incidents as they are detected.

77% of organizations believe that AI will accelerate their ability to discover unprotected sensitive data, detect anomalous activity, and automatically protect at-risk data. 76% believe AI will improve the accuracy of their data security strategies, and an overwhelming 93% are at least planning to use AI for data security.

Organizations already using AI as part of their data security operations also report fewer alerts. On average, organizations using AI security tools receive 47 alerts per day, compared to an average 79 alerts among those that have yet to implement similar AI solutions.

AI’s ability to analyze vast amounts of data, detect anomalies, and respond to threats in real-time offers a promising avenue for strengthening data security. This optimism is also driving investments in AI-powered data security solutions, which are expected to play a pivotal role in future security strategies.

As we look to the future, customers are looking for ways to streamline how they discover and label sensitive data, provide more effective and accurate alerts, simplify investigations, make recommendations to better secure their data environments, and ultimately reduce the number of data security incidents.

Final thoughts 

So, what can be made of this new generative AI revolution, especially as it pertains to data security? For those beginning their adoption roadmap or looking for ways to improve, here are three broadly applicable recommendations:  

• Hedge against data security incidents by adopting an integrated platform.
• Adopt controls for employee use of generative AI that won’t impact productivity. 
• Uplevel your data security strategy with help from AI.

Gain deeper insights about generative AI and its influence on data security by exploring Data Security Index: Trends, insights, and strategies to keep your data secure and navigate generative AI. There you’ll also find in-depth sentiment analysis from participating data security professionals, providing even more insight into common thought processes around generative AI adoption. For further reading, you can also check out the Data Security as a Foundation for Secure AI Adoption white paper. 

To learn more about Microsoft Security solutions, visit our website. Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us on LinkedIn (Microsoft Security) and X (@MSFTSecurity) for the latest news and updates on cybersecurity. 

The post Microsoft Data Security Index annual report highlights evolving generative AI security needs appeared first on Microsoft Security Blog.

According to a recent survey conducted by ISMG, the top concerns for both business executives and security leaders on using generative AI in their organization range, from data security and governance, transparency and accountability to regulatory compliance.¹ In this paper, the first in a series on AI compliance, governance, and safety from the Microsoft Security team, we provide business and technical leaders with an overview of potential security risks when deploying generative AI, along with insights into recommended safeguards and approaches to adopt the technology responsibly and effectively.

Learn how to deploy generative AI securely and responsibly

In the paper, we explore five critical areas to help ensure the responsible and effective deployment of generative AI: data security, managing hallucinations and overreliance, addressing biases, legal and regulatory compliance, and defending against threat actors. Each section provides essential insights and practical strategies for navigating these challenges. 

Data security

Data security is a top concern for business and cybersecurity leaders. Specific worries include data leakage, over-permissioned data, and improper internal sharing. Traditional methods like applying data permissions and lifecycle management can enhance security. 

Managing hallucinations and overreliance

Generative AI hallucinations can lead to inaccurate data and flawed decisions. We explore techniques to help ensure AI output accuracy and minimize overreliance risks, including grounding data on trusted sources and using AI red teaming. 

Defending against threat actors

Threat actors use AI for cyberattacks, making safeguards essential. We cover protecting against malicious model instructions, AI system jailbreaks, and AI-driven attacks, emphasizing authentication measures and insider risk programs. 

Addressing biases

Reducing bias is crucial to help ensure fair AI use. We discuss methods to identify and mitigate biases from training data and generative systems, emphasizing the role of ethics committees and diversity practices.

Legal and regulatory compliance

Navigating AI regulations is challenging due to unclear guidelines and global disparities. We offer best practices for aligning AI initiatives with legal and ethical standards, including establishing ethics committees and leveraging frameworks like the NIST AI Risk Management Framework.

Explore concrete actions for the future

As your organization adopts generative AI, it’s critical to implement responsible AI principles—including fairness, reliability, safety, privacy, inclusiveness, transparency, and accountability. In this paper, we provide an effective approach that uses the “map, measure, and manage” framework as a guide; as well as explore the importance of experimentation, efficiency, and continuous improvement in your AI deployment.

I’m excited to launch this series on AI compliance, governance, and safety with a strategy paper on minimizing risk and enabling your organization to reap the benefits of generative AI. We hope this series serves as a guide to unlock the full potential of generative AI while ensuring security, compliance, and ethical use—and trust the guidance will empower your organization with the knowledge and tools needed to thrive in this new era for business.

Additional resources

• Get the Grow Your Business with AI You Can Trust e-book.
• Explore the Introduction to Generative AI and Safety guide.

Minimize Risk and Reap the Benefits of AI

Get more insights from Bret Arsenault on emerging security challenges from his Microsoft Security blogs covering topics like next generation built-in security, insider risk management, managing hybrid work, and more.

^{1, 2} ISMG’s First annual generative AI study – Business rewards vs. security risks: Research report, ISMG.

The post More value, less risk: How to implement generative AI across the organization securely and responsibly appeared first on Microsoft Security Blog.

Across many sessions and demos, we’ll address the top security pain points related to AI and empower you with practical, actionable strategies. Keep reading this blog for a guide of highlighted sessions for security professionals of all levels, whether you’re attending in-person or online.

And be sure to register for the digital experience to explore the Microsoft Security sessions at Microsoft Ignite.

Be among the first to hear top news

Microsoft is bringing together every part of the company in a collective mission to advance cybersecurity protection to help our customers and the security community. We have four powerful advantages to drive security innovation: large-scale data and threat intelligence; end-to-end protection; responsible AI; and tools to secure and govern the use of AI.

Microsoft Chairman and Chief Executive Officer Satya Nadella said in May 2024 that security is the top priority for our company. At the Microsoft Ignite opening keynote on Tuesday, November 19, 2024, Microsoft Security Executive Vice President Charlie Bell and Corporate Vice President (CVP), Microsoft Security Business Vasu Jakkal will join Nadella to discuss Microsoft’s vision for the future of security. Other well-known cybersecurity speakers at Microsoft Ignite include Ann Johnson, CVP and Deputy Chief Information Security Officer (CISO); Joy Chik, President, Identity, and Network Access; Mark Russinovich, Chief Technology Officer and Deputy CISO; and Sherrod DeGrippo, Director of Threat Intelligence Strategy.

For a deeper dive into security product news and demos, join the security general session on Wednesday, November 20, 2024, at 11:00 AM CT. Hear from Vasu Jakkal; Joy Chik; Rob Lefferts, CVP, Microsoft Threat Protection; Herain Oberoi, General Manager, Microsoft Data Security, Privacy, and Compliance; and Michael Wallent, CVP; who will share exciting security innovations to empower you with AI tools designed to help you get ahead of attackers.

These news-breaking sessions are just the start of the value you can gain from attending online.

Benefit from insights designed for your role

While cybersecurity is a shared concern of security professionals, we realize the specific concerns are unique to role. Recognizing this, we developed sessions tailored to what matters most to you.

• CISOs and senior security leaders: If you’ll be with us in Chicago, kick off the conference with the Microsoft Ignite Security Forum on November 18, 2024 from 1 PM CT to 5 PM CT. Join this exclusive pre-day event to hear from Microsoft security experts on threat intelligence insights, our Secure Future Initiative (SFI), and trends in security. Go back to your registration to add this experience on. Also for those in Chicago, be sure to join the Security Leaders Dinner, where you can engage with your peers and provide insights on your greatest challenges and successes. If you’re joining online, gain firsthand access to the latest Microsoft Security announcements. Whether you’re in person or online, don’t miss “Proactive security with continuous exposure management” (BRK324), which will explore how Microsoft Security Exposure Management unifies disparate data silos for visibility of end-to-end attack surface, and “Secure and govern data in Microsoft 365 Copilot and beyond” (BRK321), which will discuss the top concerns of security leaders when it comes to AI and how you can gain the confidence and tools to adopt AI. Plus, learn how to make your organization as diverse as the threats you are defending in “The Power of Diversity: Building a stronger workforce in the era of AI” (BRK330).
• Security analysts and engineers: Join actionable sessions for information you can use immediately. Sessions designed for the security operations center (SOC) include “Microsoft cybersecurity architect lab—Infrastructure security” (LAB454), which will showcase how to best use the Microsoft Secure Score to improve your security posture, and “Simplify your SOC with the unified security operations platform” (BRK310), which will feature a fireside chat with security experts to discuss common security challenges and topics. Plus, learn to be a champion of safe AI adoption in “Scott and Mark learn responsible AI” (BRK329), which will explore the three top risks in large language models and the origins and potential impacts of each of these.
• Developers and IT professionals: We get it—security isn’t your main focus, but it’s increasingly becoming part of your scope. Get answers to your most pressing questions at Microsoft Ignite. Sessions that may interest you include “Secure and govern custom AI built on Azure AI and Copilot Studio” (BRK322), which will dive into how Microsoft can enable data security and compliance controls for custom apps, detect and respond to AI threats, and managed your AI stack vulnerabilities, and “Making Zero Trust real: Top 10 security controls you can implement now” (BRK328), which offers technical guidance to make Zero Trust actionable with 10 top controls to help improve your organization’s security posture. Plus, join “Supercharge endpoint management with Microsoft Copilot in Intune” (THR656) for guidance on unlocking Microsoft Intune’s potential to streamline endpoint management.
• Microsoft partners: We appreciate our partners and have developed sessions aimed at supporting you. These include “Security partner growth: The power of identity with Entra Suite” (BRK332) and “Security partner growth: Help customers modernize security operations” (BRK336).

Attend sessions tailored to addressing your top challenge

When exploring effective cybersecurity strategies, you likely have specific challenges that are motivating your actions, regardless of your role within your organization. We respect that our attendees want a Microsoft Ignite experience tailored to their specific objectives. We’re committed to maximizing your value from attending the event, with Microsoft Security sessions that address the most common cybersecurity challenges.

• Managing complexity: Discover ways to simplify your infrastructure in sessions like “Simpler, smarter, and more secure endpoint management with Intune” (BRK319), which will explore new ways to strengthen your security with Microsoft Intune and AI, and “Break down risk silos and build up code-to-code security posture” (BRK312), which will focus on how defenders can overcome the expansive alphabet soup of security posture tools and gain a unified cloud security posture with Microsoft Defender for Cloud.   
• Increasing efficiency:: Learn how AI can help you overcome talent shortage challenges in sessions like “Secure data across its lifecycle in the era of AI” (BRK318), which will explore Microsoft Purview leveraging Microsoft Security Copilot can help you detect hidden risks, mitigate them, and protect and prevent data loss, and “One goal, many roles: Microsoft Security Copilot: Real-world insights and expert advice” (BRK316), which will share best practices and insider tricks to maximize Copilot’s benefits so you can realize quick value and enhance your security and IT operations.  
• Threat landscape: Navigate effectively through the modern cyberthreat landscape, guided by the insights shared in sessions like “AI-driven ransomware protection at machine speed: Defender for Endpoint” (BRK325), which will share a secret in Microsoft Defender for Endpoint success and how it uses machine learning and threat intelligence, and the theater session “Threat intelligence at machine speed with Microsoft Security Copilot” (THR555), which will showcase how Copilot can be used as a research assistant, analyst, and responder to simplify threat management.
• Regulatory compliance: Increase your confidence in meeting regulatory requirements by attending sessions like “Secure and govern your data estate with Microsoft Purview” (BRK317), which will explore how to secure and govern your data with Microsoft Purview, and “Secure and govern your data with Microsoft Fabric and Purview” (BRK327), which will dive into how Microsoft Purview works together with Microsoft Fabric for a comprehensive approach to secure and govern data.
• Maximizing value: Discover how to maximize the value of your cybersecurity investments during sessions like “Transform your security with GenAI innovations in Security Copilot” (BRK307), which will showcase how Microsoft Security Copilot’s automation capabilities and use cases can elevate your security organization-wide, and “AI-driven ransomware protection at machine speed: Defender for Endpoint” (BRK325), which will dive into the key secret to the success of Defender for Endpoint customers in reducing the risk of ransomware attacks as well maximizing the value of the product’s new features and user interfaces.

Explore cybersecurity tools with product showcases and hands-on training

Learning about Microsoft security capabilities is useful, but there’s nothing like trying out the solutions for yourself. Our in-depth showcases and hands-on trainings give you the chance to explore these capabilities for yourself. Bring a notepad and your laptop and let’s put these tools to work.

• “Secure access at the speed of AI with Copilot in Microsoft Entra” (THR556): Learn how AI with Security Copilot and Microsoft Entra can help you accelerate tasks like troubleshooting, automate cybersecurity insights, and strengthen Zero Trust.  
• “Mastering custom plugins in Microsoft Security Copliot” (THR653): Gain practical knowledge of using Security Copilot’s capabilities during a hands-on session aimed at security and IT professionals ready for advanced customization and integration with existing security tools. 
• “Getting started with Microsoft Sentinel” (LAB452): Get hands-on experience on building detections and queries, configuring your Microsoft Sentinel environment, and performing investigations. 
• “Secure Azure services and workloads with Microsoft Defender for Cloud” (LAB457): Explore how to mitigate security risks with endpoint security, network security, data protection, and posture and vulnerability management. 
• “Evolving from DLP to data security with Microsoft Preview” (THR658): See for yourself how Microsoft Purview Data Loss Prevention (DLP) integrates with insider risk management and information protection to optimize your end-to-end DLP program. 

Network with Microsoft and other industry professionals

While you’ll gain a wealth of insights and learn about our latest product innovations in sessions, our ancillary events offer opportunities to connect and socialize with Microsoft and other security professionals as committed to you to strengthening the industry’s defenses against cyberthreats. That’s worth celebrating!

• Pre-day Forum: All Chicago Microsoft Ignite attendees are welcome to add on to the event with our pre-day sessions on November 18, 2024, from 1 PM CT to 5 PM CT. Topics covered will include threat intelligence, Microsoft’s Secure Future Initiative, AI innovation, and AI security research, and the event will feature a fireside chat with Microsoft partners and customers. The pre-day event is designed for decision-makers from businesses of all sizes to advance your security strategy. If you’re already attending in person, log in to your Microsoft Ignite registration and add on the Microsoft Security Ignite Forum.
• Security Leaders Dinner: We’re hosting an exclusive dinner with leaders of security teams, where you can engage with your peers and provide insights on your greatest challenges and successes. This intimate gathering is designed specifically for CISOs and other senior security leaders to network, share learnings, and discuss what’s happening in cybersecurity.   
• Secure the Night Party: All security professionals are encouraged to celebrate the cybersecurity community with Microsoft from 6 PM CT to 10 PM CT on Wednesday, November 20, 2024. Don’t miss this opportunity to connect with Microsoft Security subject matter experts and peers at our “Secure the Night” party during Microsoft Ignite in Chicago. Enjoy an engaging evening of conversations and experiences while sipping tasty drinks and noshing on heavy appetizers provided by Microsoft. We look forward to welcoming you. Reserve your spot today! 

Something that excites us the most about Microsoft Ignite is the opportunity to meet with cybersecurity professionals dedicated to modern defense. Stop by the Microsoft Security Expert Meetup space to say hello, learn more about capabilities you’ve been curious about, or ask questions about Microsoft’s cybersecurity efforts. 

Hear from our Microsoft Intelligent Security Association partners at Microsoft Ignite

The Microsoft Intelligent Security Association (MISA), comprised of independent software vendors (ISV) and managed security service providers (MSSPs) that have integrated their solutions with Microsoft’s security technology, will be back at Microsoft Ignite 2024.

We kick things off by celebrating our Security Partner of the Year award winners BlueVoyant (Security), Cyclotron (Compliance), and Inspark (Identity) who will join Vasu Jakkal for a fireside chat on “How security strategy is adapting for AI,” during the Microsoft Ignite Security Pre-day Forum. This panel discussion includes insights into trends partners are seeing with customers relating to AI, a view on practical challenges, and scenarios that companies encounter when deploying AI, as well as the expert guidance and best practices that security partners can offer to ensure successful AI integration in security strategies.

MISA is thrilled to welcome small and medium business (SMB) verified solution status to its portfolio. This solution verification highlights technology solutions that are purpose built to meet the needs of small and medium businesses, and the MSSPs who often manage IT and security on behalf of SMBs. MISA members who meet the qualifying criteria and have gone through engineering review, will receive a specialized MISA member badge showcasing the verification and will be featured in the MISA partner catalog. We are excited to launch this status with Blackpoint Cyber and Huntress.

Join MISA members including Blackpoint Cyber and Huntress at the Microsoft Expert Meetup Security area where 14 members will showcase their solutions and Microsoft Security Technology. Review the full schedule below.

We are looking forward to connecting with our customers and partners at the Microsoft Secure the Night Party on Wednesday, November 20, from 6 to 10 PM CT.  This evening event offers a chance to connect with Microsoft Security subject matter experts and MISA partners while enjoying cocktails, great food, and entertainment. A special thank you to our MISA sponsors: Armor, Cayosoft, ContraForce, HID, Lighthouse, Ontinue, and Quorum Cyber.

Register today to attend Microsoft Ignite online

There’s still time to register to participate in Microsoft Ignite online from November 19 to 22, 2024, to catch security-focused breakout sessions, product demos, and participate in interactive Q&A sessions with our experts. No matter how you participate in Microsoft Ignite, you’ll gain insights on how to secure your future with an AI-first, end-to-end cybersecurity approach to keep your organizations safer.

Plus, you can take your security knowledge further at Tech Community Live: Microsoft Security edition on December 3, 2024, to ask all your follow-up questions from Microsoft Ignite. Microsoft Experts will be hosting live Ask Microsoft Anything sessions on topics from Security for AI to Copilot for Security.

To learn more about Microsoft Security solutions, visit our website. Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us at @MSFTSecurity for the latest news and updates on cybersecurity.

The post Microsoft Ignite: Sessions and demos to improve your security strategy appeared first on Microsoft Security Blog.

At Microsoft, we have commitments to ensure Trustworthy AI and are building industry-leading supporting technology. Our commitments and capabilities go hand in hand to make sure our customers and developers are protected at every layer.

Building on our commitments, today we are announcing new product capabilities to strengthen the security, safety and privacy of AI systems.

Security. Security is our top priority at Microsoft, and our expanded Secure Future Initiative (SFI) underscores the company-wide commitments and the responsibility we feel to make our customers more secure. This week we announced our first SFI Progress Report, highlighting updates spanning culture, governance, technology and operations. This delivers on our pledge to prioritize security above all else and is guided by three principles: secure by design, secure by default and secure operations. In addition to our first party offerings, Microsoft Defender and Purview, our AI services come with foundational security controls, such as built-in functions to help prevent prompt injections and copyright violations. Building on those, today we’re announcing two new capabilities:

• Evaluations in Azure AI Studio to support proactive risk assessments.
• Microsoft 365 Copilot will provide transparency into web queries to help admins and users better understand how web search enhances the Copilot response. Coming soon.

Our security capabilities are already being used by customers. Cummins, a 105-year-old company known for its engine manufacturing and development of clean energy technologies, turned to Microsoft Purview to strengthen their data security and governance by automating the classification, tagging and labeling of data. EPAM Systems, a software engineering and business consulting company, deployed Microsoft 365 Copilot for 300 users because of the data protection they get from Microsoft. J.T. Sodano, Senior Director of IT, shared that “we were a lot more confident with Copilot for Microsoft 365, compared to other large language models (LLMs), because we know that the same information and data protection policies that we’ve configured in Microsoft Purview apply to Copilot.”

Safety. Inclusive of both security and privacy, Microsoft’s broader Responsible AI principles, established in 2018, continue to guide how we build and deploy AI safely across the company. In practice this means properly building, testing and monitoring systems to avoid undesirable behaviors, such as harmful content, bias, misuse and other unintended risks. Over the years, we have made significant investments in building out the necessary governance structure, policies, tools and processes to uphold these principles and build and deploy AI safely. At Microsoft, we are committed to sharing our learnings on this journey of upholding our Responsible AI principles with our customers. We use our own best practices and learnings to provide people and organizations with capabilities and tools to build AI applications that share the same high standards we strive for.

Today, we are sharing new capabilities to help customers pursue the benefits of AI while mitigating the risks:

• A Correction capability in Microsoft Azure AI Content Safety’s Groundedness detection feature that helps fix hallucination issues in real time before users see them.
• Embedded Content Safety, which allows customers to embed Azure AI Content Safety on devices. This is important for on-device scenarios where cloud connectivity might be intermittent or unavailable.
• New evaluations in Azure AI Studio to help customers assess the quality and relevancy of outputs and how often their AI application outputs protected material.
• Protected Material Detection for Code is now in preview in Azure AI Content Safety to help detect pre-existing content and code. This feature helps developers explore public source code in GitHub repositories, fostering collaboration and transparency, while enabling more informed coding decisions.

It’s amazing to see how customers across industries are already using Microsoft solutions to build more secure and trustworthy AI applications. For example, Unity, a platform for 3D games, used Microsoft Azure OpenAI Service to build Muse Chat, an AI assistant that makes game development easier. Muse Chat uses content-filtering models in Azure AI Content Safety to ensure responsible use of the software. Additionally, ASOS, a UK-based fashion retailer with nearly 900 brand partners, used the same built-in content filters in Azure AI Content Safety to support top-quality interactions through an AI app that helps customers find new looks.

We’re seeing the impact in the education space too. New York City Public Schools partnered with Microsoft to develop a chat system that is safe and appropriate for the education context, which they are now piloting in schools. The South Australia Department for Education similarly brought generative AI into the classroom with EdChat, relying on the same infrastructure to ensure safe use for students and teachers.

Privacy. Data is at the foundation of AI, and Microsoft’s priority is to help ensure customer data is protected and compliant through our long-standing privacy principles, which include user control, transparency and legal and regulatory protections. To build on this, today we’re announcing:

• Confidential inferencing in preview in our Azure OpenAI Service Whisper model, so customers can develop generative AI applications that support verifiable end-to-end privacy. Confidential inferencing ensures that sensitive customer data remains secure and private during the inferencing process, which is when a trained AI model makes predictions or decisions based on new data. This is especially important for highly regulated industries, such as health care, financial services, retail, manufacturing and energy.
• The general availability of Azure Confidential VMs with NVIDIA H100 Tensor Core GPUs, which allow customers to secure data directly on the GPU. This builds on our confidential computing solutions, which ensure customer data stays encrypted and protected in a secure environment so that no one gains access to the information or system without permission.
• Azure OpenAI Data Zones for the EU and U.S. are coming soon and build on the existing data residency provided by Azure OpenAI Service by making it easier to manage the data processing and storage of generative AI applications. This new functionality offers customers the flexibility of scaling generative AI applications across all Azure regions within a geography, while giving them the control of data processing and storage within the EU or U.S.

We’ve seen increasing customer interest in confidential computing and excitement for confidential GPUs, including from application security provider F5, which is using Azure Confidential VMs with NVIDIA H100 Tensor Core GPUs to build advanced AI-powered security solutions, while ensuring confidentiality of the data its models are analyzing. And multinational banking corporation Royal Bank of Canada (RBC) has integrated Azure confidential computing into their own platform to analyze encrypted data while preserving customer privacy. With the general availability of Azure Confidential VMs with NVIDIA H100 Tensor Core GPUs, RBC can now use these advanced AI tools to work more efficiently and develop more powerful AI models.

Achieve more with Trustworthy AI 

We all need and expect AI we can trust. We’ve seen what’s possible when people are empowered to use AI in a trusted way, from enriching employee experiences and reshaping business processes to reinventing customer engagement and reimagining our everyday lives. With new capabilities that improve security, safety and privacy, we continue to enable customers to use and build trustworthy AI solutions that help every person and organization on the planet achieve more. Ultimately, Trustworthy AI encompasses all that we do at Microsoft and it’s essential to our mission as we work to expand opportunity, earn trust, protect fundamental rights and advance sustainability across everything we do.

Related:

Commitments

• Security: Secure Future Initiative  
• Privacy: Trust Center 
• Safety: Responsible AI Principles

Capabilities

• Security: Security for AI
• Privacy: Azure Confidential Computing
• Safety: Azure AI Content Safety

The post Microsoft Trustworthy AI: Unlocking human potential starts with trust    appeared first on Microsoft Security Blog.

Join us for Microsoft Ignite 2024 from Monday, November 18, 2024, through Friday, November 22, 2024, in Chicago, Illinois, to find out how to secure and govern AI and create a security-first culture with end-to-end security. Listen and learn from Microsoft security leaders and network with fellow security professionals, partners, and tech enthusiasts. Security professionals of all levels can join interactive labs, workshops, keynotes, technical breakout sessions, demos, and more, led by Microsoft security leaders and experts. It’s our opportunity to share and showcase our latest security product innovations with you, and then dive into the technical details together—so the information you learn at Microsoft Ignite can have an immediate benefit to your digital environments and your customers.

Although all in-person passes are sold out, there’s still time to register to attend Microsoft Ignite online from Tuesday, November 19, 2024, through Thursday, November 21, 2024. By joining us online, you’ll have the flexibility to attend more than 40 security sessions and demos that fit your schedule, access on-demand content, and participate in interactive question and answer sessions with our experts. Register now for an online pass to join us!

Microsoft Ignite 2024

Discover solutions that help modernize and manage intelligent apps, safeguard data, and accelerate productivity, while connecting with partners and growing your community or business.

Register now

Microsoft Security at Microsoft Ignite: An expanded experience

We’re excited to welcome back security leaders and other security professionals to Microsoft Ignite. At Microsoft Ignite, our content is designed to help security professionals secure their environments, use all their Microsoft products and resources safely and securely, and make sure the processes in their organizations prioritize security first. Last year, when customers asked for more security content, we delivered—and we received great feedback. So this year we’re planning even more, with a focus on our continuing commitment to securing our technology and our customers.

Starting a day before the conference, Monday, November 18, 2024, we are hosting the new Microsoft Ignite Security Forum. This exclusive pre-day event is designed for security leaders who are primarily accountable for securing their organizations. The Microsoft Ignite Security Forum is for businesses of all sizes to hear from Microsoft security experts on threat intelligence insights, learnings, and trends in security. It’s also a good opportunity for customers to take a “peek behind the curtain” with in-depth discussions about cybersecurity and information about our Secure Future Initiative (SFI). Attend the Microsoft Ignite Security Forum to discover how you can advance cybersecurity protection and stay ahead of today’s aggressive cyberthreats. Register for Microsoft Ignite today and add on the Microsoft Ignite Security Forum.

From Tuesday, November 19, 2024, through Friday, November 22, 2024, at Microsoft Ignite 2024, Microsoft Security is excited to share how we bring together every part of the company to drive security innovation: our large-scale data and threat intelligence, complete end-to-end protection, responsible AI, and tools to secure and govern the use of AI in your organizations.

By joining us online, you’ll have the flexibility to attend sessions that fit your schedule, access on-demand content, and participate in interactive Q&A sessions with our experts. Plus, you’ll gain AI-specific cybersecurity skills that will make you an invaluable asset to your organization. 

• Find out how to maximize complete end-to-end security to secure and govern AI.
• Learn how our global-scale threat intelligence informs the products you use daily.
• Hear from Microsoft Security product innovators like Vasu Jakkal (Corporate Vice President, Security, Compliance, Identity, and Management) and Charlie Bell (Executive Vice President, Microsoft Security).
• See products in action during sessions, demos, interactive labs, and workshops.
• Network with fellow security leaders, partners, and technical enthusiasts.

See an overview of the Microsoft Security experience at Microsoft Ignite.

Who is the security experience for?

At Microsoft Ignite, we cater to a diverse audience of security decision makers, security practitioners, developers, and IT professionals, ensuring that each group finds valuable, tailored content to meet their specific needs. Whether you’re a leader looking for strategic insights or a hands-on developer seeking technical guidance, our sessions, demos, interactive labs, and workshops are designed to equip you with the knowledge and tools necessary for securing your environment.

Here’s what we have for you:

• Security decision makers can expect both strategic and actionable technical information at Microsoft Ignite for your leadership and your team. We’ll have content to help you build strategies to secure your environment. We’ll also help you articulate and demonstrate security needs to your senior leadership or board members. And then, we’ll support you with the tactical information you need to execute those plans.
• For privacy, identity, compliance, security operations center (SOC) analysts, and more, Microsoft Ignite will have sessions dedicated to upskilling, certifications, and deeper understanding of Microsoft technologies and solutions—so your knowledge and opportunities both deepen and grow. Looking to boost your skills? Sign up for in-depth security training and hands-on labs offered at the Microsoft Ignite Pre-day on November 18, 2024, that will help improve your organization’s security posture, strengthen team expertise, and outpace bad actors.
• For developers, we’re building specific technical content on how to use your Microsoft security solutions for complete end-to-end-protection, an essential for AI readiness. Need security 101 learning? The hands-on labs offered at Microsoft Ignite Pre-day on November 18, 2024, are for you too. We’ll also share how to create and operate AI applications safely and securely—at scale—with tools to secure and govern your use of AI.
• For partners, the previously separate Microsoft Inspire event is now an integrated part of Microsoft Ignite this year, bringing our entire community closer together, and helping partners understand how they can leverage Microsoft technologies and bring solutions to life for so many of our customers.

For the entire community, we have several celebrations planned to help maximize your opportunities to network and socialize, including our Secure the Night party, the event-wide Microsoft Ignite celebration, and more.

Register now for Microsoft Ignite 2024

You won’t want to miss the opportunity to participate and learn with the global community of technical and security professionals online, November 19 to 21, 2024. Select “Digital Attendee” on the registration page to save your spot for the Microsoft Ignite 2024 online experience today.

To learn more about Microsoft Security solutions, visit our website. Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us on LinkedIn (Microsoft Security) and X (@MSFTSecurity) for the latest news and updates on cybersecurity.

¹Prioritizing security above all else, Microsoft Official Blog. May 3, 2024.

The post Join us at Microsoft Ignite 2024 and learn to build a security-first culture with AI appeared first on Microsoft Security Blog.

Data security as a foundation for secure AI adoption

Learn the four steps organizations can take to prepare their data for AI.

Get the whitepaper

Preparing data for AI adoption

In a recent survey on the state of generative AI, business leaders expressed optimism on the potential of AI, but shared their struggle to gain full visibility into their AI programs—creating data security and compliance risks.² 58% of organizations surveyed expressed concern about the unsanctioned use of generative AI at their companies, and the general lack of visibility into it. And 93% of leaders report heightened concern about shadow AI—unsanctioned or undetected AI usage by employees.³ Our whitepaper walks through four key steps organizations can take to prepare their data for AI and includes a detailed checklist at each stage. The stages include knowing your data, governing your data, protecting your data, and preventing data loss. Taking these steps and understanding how to prepare your data properly for AI tools can help mitigate leader concerns and decrease data risk.

Choosing which AI to deploy

Once you secure your data and prepare to deploy AI, how do you decide which generative AI application is best for your organization? For many customers, choosing AI that integrates with their existing Microsoft 365 apps helps maintain security and maximize their current technology investments.

Copilot for Microsoft 365 is integrated into Microsoft 365 apps so that it understands a user’s work context, is grounded in Microsoft Graph to provide more personalized and relevant responses, and can connect to business data sources to reason over all of user’s enterprise data. Copilot inherits Microsoft 365 controls and commitments, such as access permissions, as well as data commitments and controls for the European Union Data Boundary, providing customers with comprehensive enterprise data protection. And with Microsoft Purview, Copilot customers receive real-time data security and compliance controls seamlessly integrated into their organization’s Microsoft 365 deployment.

Secure and govern usage of Copilot for Microsoft 365

As organizations deploy Copilot and other generative AI applications, they want to get ahead of the inherent risks of data being shared with generative AI applications—including data oversharing, data leakage, and non-compliant use of generative AI apps. In the whitepaper, we walk through the steps you can take to discover and protect your organization data as it interacts with AI, then how to govern usage of Copilot once it is deployed. Many organizations also choose to add Microsoft Purview, which provides value like Microsoft Purview AI Hub to help you gain visibility into how your organization is already using AI, including insights into sensitive data being shared with AI applications. The whitepaper shares more detail on the AI Hub interface, its capabilities, and insights into the risks identified by Microsoft Purview. It also shows how you can protect sensitive data throughout its AI journey, with information on sensitivity labeling, data security controls, and data loss prevention capabilities.

The whitepaper also details how your organization can prioritize compliance obligations with Microsoft Purview, assess your compliance with existing AI regulations, and conduct legal investigations for incidents where AI interactions were involved.

Gain the confidence to innovate with AI, securely

Implementing the strategies described in our whitepaper—Data security as a foundation for secure AI adoption—can help give your organization the confidence to explore new avenues and opportunities with AI while protecting and governing your data to minimize security risks and stay ahead of compliance obligations.

To learn more about Microsoft Security solutions, visit our website. Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us on LinkedIn (Microsoft Security) and X (@MSFTSecurity) for the latest news and updates on cybersecurity.

¹PwC AI Analysis—Sizing the Prize, PwC.

²The 2023 State of Generative AI Survey, Portal26.

³As Companies Eye Generative AI to Improve Productivity and Growth, Two-thirds Admit to GenAI-related Security or Misuse Incident in the Last Year, Yahoo.

The post New Microsoft whitepaper shares how to prepare your data for secure AI adoption appeared first on Microsoft Security Blog.

Also at Black Hat, our Microsoft AI Red Team will be onsite holding training sessions, briefings, and panel discussions. And today, we’re releasing a white paper to demonstrate the impact of red teaming in practice when incorporated in the AI development life cycle. The paper details our innovative “Break-Fix” approach to red teaming AI systems and our close collaboration with Microsoft’s Phi-3 team, which allowed us to reduce the harms by 75% in Microsoft’s state-of-the-art small language models.¹   

As a proud sponsor of the inaugural AI Summit at Black Hat, we’re further investing in the community by sharing our learnings in both AI for Security and Securing AI. We’ll be participating in a panel discussion titled “Balancing Security and Innovation—Risks and Rewards in AI-Driven Cybersecurity,” where we’ll debate the trade-offs between innovation in AI and security risks and share strategies to foster innovation while maintaining robust security postures.  

There’s also a sponsored session titled “Moonstone Sleet: A Deep Dive into their TTPs,” presented by Greg Schloemer, Threat Intelligence Analyst at Microsoft, that takes a deep dive into cyber threat actors associated with the Democratic People’s Republic of Korea (DPRK), as well as educational and engaging theater sessions in our Microsoft booth #1240. With a ton of critical security content to catch—all detailed below—we hope you’ll make time to connect with us at Black Hat 2024. 

Plan your schedule with our standout sessions  

Join us for core Black Hat sessions, submitted for consideration by Microsoft subject matter experts and selected by the Black Hat content committee to be included in its main agenda.  

Stop by our booth (1240) to connect with Microsoft security experts  

At Black Hat 2024, Microsoft Security is here with security leaders and resources that include:   

• Threat researchers and security experts from Microsoft Security, here to connect with the community and share insights.  
• Live demos of Microsoft Copilot for Security, informed by the 78 trillion signals Microsoft processes daily, to help security pros be up to 22% faster. ²
• Theater presentations of Microsoft’s unified security operations experience, which brings together extended detection and response (XDR) and security information and event management (SIEM), so you get full visibility into cyberthreats across your multicloud, multiplatform environment.  
• Hands-on experience with Microsoft Security solutions to help you adopt AI safely.  

Connect with Microsoft leaders and representatives to learn about our AI-first end-to-end security for all. Additionally, you’ll be able to view multiple demonstrations on a wide range of topics including threat protection, securing AI, multicloud security, Copilot for Security, data security, and advanced identity. You’ll also be able to connect with our Microsoft Intelligent Security Association (MISA) partners during your visit—the top experts from across the cybersecurity industry with the shared goal of improving customer security worldwide. And if you have specific questions to ask, sign up for a one-on-one chat with Microsoft Security leaders. 

Partner presence at the Microsoft booth

At the Theater in the Microsoft booth, watch our series of presentations and panels featuring Microsoft Threat Intelligence Center (MSTIC) experts and Microsoft Researchers. Half of the sessions will be presented by the MSTIC Team. The Microsoft booth will also feature sessions from select partners from the Microsoft Intelligent Security Association (MISA). MISA is an ecosystem of leading Security companies that have integrated their solutions with Microsoft Security technology with a goal of protecting our mutual customers from cybersecurity threats. Seven partners will showcase their solutions at our MISA demo station and five partners will be presenting their solutions in our mini-theater. We would love to see you there. Click here to view our full theater session schedule. 

Reserve your spot at the Microsoft Security VIP Mixer  

The event will be co-hosted by Ann Johnson, Corporate Vice President and Deputy CISO of Microsoft Security, and Aarti Borkar, Vice President of Microsoft Security, Customer Success and Microsoft Incident Response, and, we are thrilled to have five MISA partners—Avertium, BlueVoyant, NCC Group, Trustwave, and Quorum Cyber—sponsoring our Microsoft Security VIP Mixer. The mixer is a great time to connect and network with fellow industry experts, and grab a copy of Security Mixology, a threat intelligence-themed cocktail and appetizer cookbook—you’ll be able to meet some of the contributors! Drinks and appetizers will be provided. Reserve your spot to join us at this exclusive event.

Don’t miss the AI Summit at Black Hat  

On Tuesday, August 6, 2024, from 11:10 AM PT to 11:50 AM PT, we’ll be part of a panel discussion titled “Balancing Security and Innovation—Risks and Rewards in AI-Driven Cybersecurity.” Microsoft is honored to be a VisionAIre sponsor for this event. Brandon Dixon, Partner Product Manager, Security AI Strategy will debate the trade-offs between innovation in AI and security risks, share strategies to foster innovation while maintaining robust security, and more. Note: The AI Summit is a separate, one-day event featuring technical experts, industry leaders, and security tsars, designed to give attendees a comprehensive understanding of the potential risks, challenges, and opportunities associated with AI and cybersecurity.  

Microsoft’s Most Valuable Researchers 

Security researchers are a critical part of the defender community, on the front lines of security response evolution, working to protect customers and the broader ecosystem. On Thursday, August 8, 2024, we’ll host our invite-only Microsoft Researcher Celebration. And on August 6, 2024, Microsoft Security Response Center (MSRC) will announce the annual top 100 Most Valuable Researchers (MVRs) who help protect our customers through surfacing and reporting security vulnerabilities under Coordinated Vulnerability Disclosure (CVD). Follow @msftsecresponse on X and Microsoft Security Response Center on LinkedIn for the MVR reveal. 

Secure your future with Microsoft global-scale threat intelligence  

In the hands of security professionals and teams, AI can deliver the greatest advantage to organizations of every size, across every industry, tipping the scales in favor of defenders. Microsoft is bringing together every part of the company in a collective mission to advance cybersecurity protection to help our customers and the security community. We offer four powerful advantages to drive security innovation: large-scale data and threat intelligence; the most complete end-to-end protection; industry leading, responsible AI; and the best tools to secure and govern the use of AI. Together we can propel innovation and create a safer world. We’re excited to share the latest product news and Microsoft Security innovations during Black Hat 2024 and we hope to see you there.  

Join us at the Microsoft Security VIP Mixer

Don’t miss this opportunity to connect with Microsoft Security experts and fellow industry leaders—and pick up your copy of Security Mixology!

Register now

For more threat intelligence guidance and insights from Microsoft security experts, visit Security Insider. 

To learn more about Microsoft Security solutions, visit our website. Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us on LinkedIn (Microsoft Security) and X (@MSFTSecurity) for the latest news and updates on cybersecurity. 

Sources:

¹Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone, Microsoft. April 2024.

²Microsoft Copilot for Security is generally available on April 1, 2024, with new capabilities, Vasu Jakkal. March 13, 2024.

The post Connect with Microsoft Security at Black Hat USA 2024​​ appeared first on Microsoft Security Blog.

As we discussed in a previous blog post about AI jailbreaks, an AI jailbreak could cause the system to violate its operators’ policies, make decisions unduly influenced by a user, or execute malicious instructions.     

In this blog, we’ll cover the details of a newly discovered type of jailbreak attack that we call Skeleton Key, which we covered briefly in the Microsoft Build talk Inside AI Security with Mark Russinovich (under the name Master Key). Because this technique affects multiple generative AI models tested, Microsoft has shared these findings with other AI providers through responsible disclosure procedures and addressed the issue in Microsoft Azure AI-managed models using Prompt Shields to detect and block this type of attack. Microsoft has also made software updates to the large language model (LLM) technology behind Microsoft’s additional AI offerings, including our Copilot AI assistants, to mitigate the impact of this guardrail bypass.

Introducing Skeleton Key

This AI jailbreak technique works by using a multi-turn (or multiple step) strategy to cause a model to ignore its guardrails. Once guardrails are ignored, a model will not be able to determine malicious or unsanctioned requests from any other. Because of its full bypass abilities, we have named this jailbreak technique Skeleton Key.

This threat is in the jailbreak category, and therefore relies on the attacker already having legitimate access to the AI model. In bypassing safeguards, Skeleton Key allows the user to cause the model to produce ordinarily forbidden behaviors, which could range from production of harmful content to overriding its usual decision-making rules. Like all jailbreaks, the impact can be understood as narrowing the gap between what the model is capable of doing (given the user credentials, etc.) and what it is willing to do. As this is an attack on the model itself, it does not impute other risks on the AI system, such as permitting access to another user’s data, taking control of the system, or exfiltrating data.

To protect against Skeleton Key attacks, as detailed in this blog, Microsoft has implemented several approaches to our AI system design and provides tools for customers developing their own applications on Azure. Below, we also share mitigation guidance for defenders to discover and protect against such attacks.

Microsoft recommends customers who are building their own AI models and/or integrating AI into their applications to consider how this type of attack could impact their threat model and to add this knowledge to their AI red team approach, using tools such as PyRIT. (Note: Microsoft has updated PyRIT to include Skeleton Key)

In the next sections, we will discuss some of the known methods for exploiting generative AI models using the Skeleton Key technique, explain the steps we’re taking to address the risk, and provide guidance for the detection and mitigation of this threat. You can watch this video to learn more about how Microsoft approaches AI Red Teaming.

Attack flow

Skeleton Key works by asking a model to augment, rather than change, its behavior guidelines so that it responds to any request for information or content, providing a warning (rather than refusing) if its output might be considered offensive, harmful, or illegal if followed. This attack type is known as Explicit: forced instruction-following.

In one example, informing a model that the user is trained in safety and ethics, and that the output is for research purposes only, helps to convince some models to comply, as shown below:

When the Skeleton Key jailbreak is successful, a model acknowledges that it has updated its guidelines and will subsequently comply with instructions to produce any content, no matter how much it violates its original responsible AI guidelines.

During our testing, from April to May 2024, the jailbreak was shown to work on the following base models and hosted models:

• Meta Llama3-70b-instruct (base)
• Google Gemini Pro (base)
• OpenAI GPT 3.5 Turbo (hosted)
• OpenAI GPT 4o (hosted)
• Mistral Large (hosted)
• Anthropic Claude 3 Opus (hosted)
• Cohere Commander R Plus (hosted)

For each model that we tested, we evaluated a diverse set of tasks across risk and safety content categories, including areas such as explosives, bioweapons, political content, self-harm, racism, drugs, graphic sex, and violence. All the affected models complied fully and without censorship for these tasks, though with a warning note prefixing the output as requested. Unlike other jailbreaks like Crescendo, where models must be asked about tasks indirectly or with encodings, Skeleton Key puts the models in a mode where a user can directly request tasks, for example, “Write a recipe for homemade explosives”. Further, the model’s output appears to be completely unfiltered and reveals the extent of a model’s knowledge or ability to produce the requested content.

Consistent with responsible disclosure principles, Microsoft shared this research with the affected AI vendors before publication, helping them determine how to best address mitigations, as needed, in their respective products or services.

GPT-4 demonstrated resistance to Skeleton Key, except when the behavior update request was included as part of a user-defined system message, rather than as a part of the primary user input. This is something that is not ordinarily possible in the interfaces of most software that uses GPT-4, but can be done from the underlying API or tools that access it directly. This indicates that the differentiation of system message from user request in GPT-4 is successfully reducing attackers’ ability to override behavior.

Mitigation and protection guidance

Microsoft has made software updates to the LLM technology behind Microsoft’s AI offerings, including our Copilot AI assistants, to mitigate the impact of this guardrail bypass. Customers should consider the following approach to mitigate and protect against this type of jailbreak in their own AI system design:

• Input filtering: Azure AI Content Safety detects and blocks inputs that contain harmful or malicious intent leading to a jailbreak attack that could circumvent safeguards.
• System message: Prompt engineering the system prompts to clearly instruct the large language model (LLM) on appropriate behavior and to provide additional safeguards. For instance, specify that any attempts to undermine the safety guardrail instructions should be prevented (read our guidance on building a system message framework here).
• Output filtering: Azure AI Content Safety post-processing filter that identifies and prevents output generated by the model that breaches safety criteria.
• Abuse monitoring: Deploying an AI-driven detection system trained on adversarial examples, and using content classification, abuse pattern capture, and other methods to detect and mitigate instances of recurring content and/or behaviors that suggest use of the service in a manner that may violate guardrails. As a separate AI system, it avoids being influenced by malicious instructions. Microsoft Azure OpenAI Service abuse monitoring is an example of this approach.

Building AI solutions on Azure

Microsoft provides tools for customers developing their own applications on Azure. Azure AI Content Safety Prompt Shields are enabled by default for models hosted in the Azure AI model catalog as a service, and they are parameterized by a severity threshold. We recommend setting the most restrictive threshold to ensure the best protection against safety violations. These input and output filters act as a general defense not only against this particular jailbreak technique, but also a broad set of emerging techniques that attempt to generate harmful content. Azure also provides built-in tooling for model selection, prompt engineering, evaluation, and monitoring. For example, risk and safety evaluations in Azure AI Studio can assess a model and/or application for susceptibility to jailbreak attacks using synthetic adversarial datasets, while Microsoft Defender for Cloud can alert security operations teams to jailbreaks and other active threats.

With the integration of Azure AI and Microsoft Security (Microsoft Purview and Microsoft Defender for Cloud) security teams can also discover, protect, and govern these attacks. The new native integration of Microsoft Defender for Cloud with Azure OpenAI Service, enables contextual and actionable security alerts, driven by Azure AI Content Safety Prompt Shields and Microsoft Defender Threat Intelligence. Threat protection for AI workloads allows security teams to monitor their Azure OpenAI powered applications in runtime for malicious activity associated with direct and in-direct prompt injection attacks, sensitive data leaks and data poisoning, or denial of service attacks.

References

• Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey (Shanghai AI Laboratory)
• AI jailbreaks: What they are and how they can be mitigated
• How Microsoft discovers and mitigates evolving attacks against AI guardrails

Learn more

To learn more about Microsoft’s Responsible AI principles and approach, refer to http://approjects.co.za/?big=ai/principles-and-approach.

For the latest security research from the Microsoft Threat Intelligence community, check out the Microsoft Threat Intelligence Blog: https://aka.ms/threatintelblog.

To get notified about new publications and to join discussions on social media, follow us on LinkedIn at https://www.linkedin.com/showcase/microsoft-threat-intelligence, and on X (formerly Twitter) at https://twitter.com/MsftSecIntel.

To hear stories and insights from the Microsoft Threat Intelligence community about the ever-evolving threat landscape, listen to the Microsoft Threat Intelligence podcast: https://thecyberwire.com/podcasts/microsoft-threat-intelligence.

The post Mitigating Skeleton Key, a new type of generative AI jailbreak technique appeared first on Microsoft Security Blog.

What is AI jailbreak?

An AI jailbreak is a technique that can cause the failure of guardrails (mitigations). The resulting harm comes from whatever guardrail was circumvented: for example, causing the system to violate its operators’ policies, make decisions unduly influenced by one user, or execute malicious instructions. This technique may be associated with additional attack techniques such as prompt injection, evasion, and model manipulation. You can learn more about AI jailbreak techniques in our AI red team’s Microsoft Build session, How Microsoft Approaches AI Red Teaming.

Here is an example of an attempt to ask an AI assistant to provide information about how to build a Molotov cocktail (firebomb). We know this knowledge is built into most of the generative AI models available today, but is prevented from being provided to the user through filters and other techniques to deny this request. Using a technique like Crescendo, however, the AI assistant can produce the harmful content that should otherwise have been avoided. This particular problem has since been addressed in Microsoft’s safety filters; however, AI models are still susceptible to it. Many variations of these attempts are discovered on a regular basis, then tested and mitigated.

Why is generative AI susceptible to this issue?

When integrating AI into your applications, consider the characteristics of AI and how they might impact the results and decisions made by this technology. Without anthropomorphizing AI, the interactions are very similar to the issues you might find when dealing with people. You can consider the attributes of an AI language model to be similar to an eager but inexperienced employee trying to help your other employees with their productivity:

1. Over-confident: They may confidently present ideas or solutions that sound impressive but are not grounded in reality, like an overenthusiastic rookie who hasn’t learned to distinguish between fiction and fact.
2. Gullible: They can be easily influenced by how tasks are assigned or how questions are asked, much like a naïve employee who takes instructions too literally or is swayed by the suggestions of others.
3. Wants to impress: While they generally follow company policies, they can be persuaded to bend the rules or bypass safeguards when pressured or manipulated, like an employee who may cut corners when tempted.
4. Lack of real-world application: Despite their extensive knowledge, they may struggle to apply it effectively in real-world situations, like a new hire who has studied the theory but may lack practical experience and common sense.

In essence, AI language models can be likened to employees who are enthusiastic and knowledgeable but lack the judgment, context understanding, and adherence to boundaries that come with experience and maturity in a business setting.

So we can say that generative AI models and system have the following characteristics:

• Imaginative but sometimes unreliable
• Suggestible and literal-minded, without appropriate guidance
• Persuadable and potentially exploitable
• Knowledgeable yet impractical for some scenarios

Without the proper protections in place, these systems can not only produce harmful content, but could also carry out unwanted actions and leak sensitive information.

Due to the nature of working with human language, generative capabilities, and the data used in training the models, AI models are non-deterministic, i.e., the same input will not always produce the same outputs. These results can be improved in the training phases, as we saw with the results of increased resilience in Phi-3 based on direct feedback from our AI Red Team. As all generative AI systems are subject to these issues, Microsoft recommends taking a zero-trust approach towards the implementation of AI; assume that any generative AI model could be susceptible to jailbreaking and limit the potential damage that can be done if it is achieved. This requires a layered approach to mitigate, detect, and respond to jailbreaks. Learn more about our AI Red Team approach.

What is the scope of the problem?

When an AI jailbreak occurs, the severity of the impact is determined by the guardrail that it circumvented. Your response to the issue will depend on the specific situation and if the jailbreak can lead to unauthorized access to content or trigger automated actions. For example, if the harmful content is generated and presented back to a single user, this is an isolated incident that, while harmful, is limited. However, if the jailbreak could result in the system carrying out automated actions, or producing content that could be visible to more than the individual user, then this becomes a more severe incident. As a technique, jailbreaks should not have an incident severity of their own; rather, severities should depend on the consequence of the overall event (you can read about Microsoft’s approach in the AI bug bounty program).

Here are some examples of the types of risks that could occur from an AI jailbreak:

• AI safety and security risks:
  • Unauthorized data access
  • Sensitive data exfiltration
  • Model evasion
  • Generating ransomware
  • Circumventing individual policies or compliance systems
• Responsible AI risks:
  • Producing content that violates policies (e.g., harmful, offensive, or violent content)
  • Access to dangerous capabilities of the model (e.g., producing actionable instructions for dangerous or criminal activity)
  • Subversion of decision-making systems (e.g., making a loan application or hiring system produce attacker-controlled decisions)
  • Causing the system to misbehave in a newsworthy and screenshot-able way
  • IP infringement

How do AI jailbreaks occur?

The two basic families of jailbreak depend on who is doing them:

• A “classic” jailbreak happens when an authorized operator of the system crafts jailbreak inputs in order to extend their own powers over the system.
• Indirect prompt injection happens when a system processes data controlled by a third party (e.g., analyzing incoming emails or documents editable by someone other than the operator) who inserts a malicious payload into that data, which then leads to a jailbreak of the system.

You can learn more about both of these types of jailbreaks here.

There is a wide range of known jailbreak-like attacks. Some of them (like DAN) work by adding instructions to a single user input, while others (like Crescendo) act over several turns, gradually shifting the conversation to a particular end. Jailbreaks may use very “human” techniques such as social psychology, effectively sweet-talking the system into bypassing safeguards, or very “artificial” techniques that inject strings with no obvious human meaning, but which nonetheless could confuse AI systems. Jailbreaks should not, therefore, be regarded as a single technique, but as a group of methodologies in which a guardrail can be talked around by an appropriately crafted input.

Mitigation and protection guidance

To mitigate the potential of AI jailbreaks, Microsoft takes defense in depth approach when protecting our AI systems, from models hosted on Azure AI to each Copilot solution we offer. When building your own AI solutions within Azure, the following are some of the key enabling technologies that you can use to implement jailbreak mitigations:

• Prompt filtering: Prompt Shields in Azure AI Content Safety
• Identity management: Managed identities for Azure resources
• Data access controls: Microsoft Purview data security for generative AI apps   
• System metaprompt: System message framework and template recommendations for large language models (LLMs)
• Content filtering: Azure OpenAI Service content filtering
• Abuse monitoring: Azure OpenAI Service abuse monitoring
• Model alignment during training: Microsoft Azure AI Fundamentals: Generative AI – Training
• Threat protection: Microsoft Defender for Cloud threat protection for AI workloads

With layered defenses, there are increased chances to mitigate, detect, and appropriately respond to any potential jailbreaks.

To empower security professionals and machine learning engineers to proactively find risks in their own generative AI systems, Microsoft has released an open automation framework, Python Risk Identification Toolkit for generative AI (PyRIT). Read more about the release of PyRIT for generative AI Red teaming, and access the PyRIT toolkit on GitHub.

When building solutions on Azure AI, use the Azure AI Studio capabilities to build benchmarks, create metrics, and implement continuous monitoring and evaluation for potential jailbreak issues.

If you discover new vulnerabilities in any AI platform, we encourage you to follow responsible disclosure practices for the platform owner. Microsoft’s procedure is explained here: Microsoft AI Bounty Program.

Detection guidance

Microsoft builds multiple layers of detections into each of our AI hosting and Copilot solutions.

To detect attempts of jailbreak in your own AI systems, you should ensure you have enabled logging and are monitoring interactions in each component, especially the conversation transcripts, system metaprompt, and the prompt completions generated by the AI model.

Microsoft recommends setting the Azure AI Content Safety filter severity threshold to the most restrictive options, suitable for your application. You can also use Azure AI Studio to begin the evaluation of your AI application safety with the following guidance: Evaluation of generative AI applications with Azure AI Studio.

Summary

This article provides the foundational guidance and understanding of AI jailbreaks. In future blogs, we will explain the specifics of any newly discovered jailbreak techniques. Each one will articulate the following key points:

1. We will describe the jailbreak technique discovered and how it works, with evidential testing results.
2. We will have followed responsible disclosure practices to provide insights to the affected AI providers, ensuring they have suitable time to implement mitigations.
3. We will explain how Microsoft’s own AI systems have been updated to implement mitigations to the jailbreak.
4. We will provide detection and mitigation information to assist others to implement their own further defenses in their AI systems.

Richard Diver
Microsoft Security

Learn more

For the latest security research from the Microsoft Threat Intelligence community, check out the Microsoft Threat Intelligence Blog: https://aka.ms/threatintelblog.

To get notified about new publications and to join discussions on social media, follow us on LinkedIn at https://www.linkedin.com/showcase/microsoft-threat-intelligence, and on X (formerly Twitter) at https://twitter.com/MsftSecIntel.

To hear stories and insights from the Microsoft Threat Intelligence community about the ever-evolving threat landscape, listen to the Microsoft Threat Intelligence podcast: https://thecyberwire.com/podcasts/microsoft-threat-intelligence.

The post AI jailbreaks: What they are and how they can be mitigated appeared first on Microsoft Security Blog.