• General availability of fine-tuning for Azure OpenAI Service GPT-4o and GPT-4o mini.
• Availability of new models including Phi-3.5-MoE, Phi-3.5-vision through serverless endpoint, Meta’s Llama 3.2, The Saudi Data and AI Authority (SDAIA) ‘s ALLaM-2-7B, and updated Command R and Command R+ from Cohere. 
• New capabilities that expand on our enterprise promise including upcoming availability of Azure OpenAI Data Zones.
• New responsible AI features including Correction, a capability in Azure AI Content Safety’s groundedness detection feature, new evaluations to assess the quality and security of outputs, and Protected Material Detection for Code.
• Full Network Isolation and Private Endpoint Support for building and customizing generative AI apps in Azure AI Studio.

Unlock the power of custom LLMs with Azure AI 

Customization of LLMs has become an increasingly popular way for our users to gain the power of best-in-class generative AI models, combined with the unique value of proprietary data and domain expertise. Fine-tuning has become the preferred choice to create custom LLMs: faster, cheaper, and more reliable than training models from scratch.

Azure AI is proud to offer tooling to enable customers to fine-tune models across Azure OpenAI Service, the Phi family of models, and over 1,600 models in the model catalog. Today, we’re excited to announce the general availability of fine-tuning for both GPT-4o and GPT-4o mini on Azure OpenAI Service. Following a successful preview, these models are now fully available for customers to fine-tune. We’ve also enabled fine-tuning for SLMs with the Phi-3 family of models.

Whether you’re optimizing for specific industries, enhancing brand voice consistency, or improving response accuracy across different languages, GPT-4o and GPT-4o mini deliver robust solutions to meet your needs. 

Lionbridge, a leader in the field of translation automation, has been one of the early adopters of Azure OpenAI Service and has leveraged fine-tuning to further enhance translation accuracy. 

“At Lionbridge, we have been tracking the relative performance of available translation automation systems for many years. As a very early adopter of GPTs on a large scale, we have fine-tuned several generations of GPT models with very satisfactory results. We’re thrilled to now extend our portfolio of fine-tuned models to the newly available GPT-4o and GPT-4o mini on Azure OpenAI Service. Our data shows that fine-tuned GPT models outperform both baseline GPT and Neural Machine Translation engines in languages like Spanish, German, and Japanese in translation accuracy. With the general availability of these advanced models, we’re looking forward to further enhance our AI-driven translation services, delivering even greater alignment with our customers’ specific terminology and style preferences.”—Marcus Casal, Chief Technology Officer, Lionbridge.

Nuance, a Microsoft company, has been a pioneer in AI-enabled healthcare solutions since 1996, starting with the first clinical speech-to-text automation for healthcare. Today, Nuance continues to leverage generative AI to transform patient care. Anuj Shroff, General Manager of Clinical Solutions at Nuance, highlighted the impact of generative AI and customization: 

“Nuance has long recognized the potential of fine-tuning AI models to deliver highly specialized and accurate solutions for our healthcare clients. With the general availability of GPT-4o and GPT-4o mini on Azure OpenAI Service, we’re excited to further enhance our AI-driven services. The ability to tailor GPT-4o’s capabilities to specific workflows marks a significant advancement in AI-driven healthcare solutions”—Anuj Shroff, General Manager of Clinical Solutions at Nuance.

For customers focused on low costs, small compute footprints, and edge compatibility, Phi-3 SLM fine-tuning is proving to be a valuable approach. Khan Academy recently published a research paper showing their fine-tuned version of Phi-3 performed better at finding and fixing student math mistakes compared to other models.

A platform for customization quality 

Fine-tuning is about so much more than just training models. From data generation to model evaluation, and support for scaling your custom models to production workloads, Azure provides a unified platform: data generation via powerful LLMs, AI Studio Evaluation, built in safety guardrails for fine-tuned models, and more. As part of our GPT-4o and 4o-mini now generally available, we’ve recently shared an end-to-end distillation flow for retrieval augmented fine-tuning, showing how to leverage Azure AI for custom, domain-adapted models.

We are hosting a webinar on October 17, 2024, to unpack the essentials and practical recipes to get started with fine-tuning. We hope you will join us to learn more.

Expanding model choice

With over 1,600 models, Azure AI model catalog offers the broadest selection of models to build generative AI applications. Azure AI models are now also available through GitHub Models so developers can quickly prototype and evaluate the best model for their use case.

I am excited to share new model availability, including: 

• Phi-3.5-MoE-instruct, a Mixture-of-Experts (MoE) model and Phi-3.5-vision-instruct through serverless endpoint and also through GitHub Models. Phi-3.5-MoE-instruct, with 16 experts and 6.6B active parameters provides multi-lingual capability, competitive performance, and robust safety measures. Phi-3.5-vision-instruct (4.2B parameters), now available through managed compute enables reasoning across multiple input images, opening up new possibilities such as detecting differences between images.
• Meta’s Llama 3.2 11B Vision Instruct and Llama 3.2 90B Vision Instruct. These models are Llama’s first ever multi-modal models and are available via managed compute in the Azure AI model catalog. Inferencing through serverless endpoints is coming soon. 
• SDAIA’s ALLaM-2-7B. This new model is designed to facilitate natural language understanding in both Arabic and English. With 7 billion parameters, ALLaM-2-7B aims to serve as a critical tool for industries requiring advanced language processing capabilities.
• Updated Command R and Command R+ from Cohere available in Azure AI Studio and through Github Models. Known for their expertise in retrieval-augmented generation (RAG) with citations, multilingual support in over 10 languages, and workflow automation, the latest versions offer better efficiency, affordability, and user experience. They feature improvements in coding, math, reasoning, and latency, with Command R being the fastest and most efficient model yet.

Achieve AI transformation with confidence

Earlier this week, we unveiled Trustworthy AI, a set of commitments and capabilities to help build AI that is secure, safe, and private. Data privacy and security, core pillars of Trustworthy AI, are foundational to designing and implementing new solutions. To help meet regulatory and compliance standards, Azure OpenAI Service—an Azure service, provides robust enterprise controls so organization can build with confidence. We continue to invest to expand enterprise controls and recently announced upcoming availability of Azure OpenAI Data Zones to further enhance data privacy and security capabilities. With the new Data Zones feature that builds on the existing strength of Azure OpenAI Service’s data processing and storage options, Azure OpenAI Service now provides customers with options between Global, Data Zone, and regional deployments, allowing customers to store data at rest within the Azure chosen region of their resource. We are excited to bring this to customers soon.

Additionally, we recently announced full network isolation in Azure AI Studio, with private endpoints to storage, Azure AI Search, Azure AI services, and Azure OpenAI Service supported via managed virtual network (VNET). Developers can also chat with their enterprise data securely using private endpoints in the chat playground. Network isolation prevents entities outside the private network from accessing its resources. For additional control, customers can now enable Entra ID for credential-less access to Azure AI Search, Azure AI services, and Azure OpenAI Service connections in Azure AI Studio. These security capabilities are critical for enterprise customers, particularly those in regulated industries using sensitive data for model fine-tuning or retrieval augmented generation (RAG) workflows.

In addition to privacy and security, safety is top of mind. As part of our responsible AI commitment, we launched Azure AI Content Safety in 2023 to enable generative AI guardrail. Building on this work, Azure AI Content Safety features—including prompt shields and protected material detection—are on by default and available at no cost in Azure OpenAI Service. Further, these capabilities can be leveraged as content filters with any foundation model included in our model catalog, including Phi-3, Llama, and Cohere. We also announced new capabilities in Azure AI Content Safety including:

• Correction to help fix hallucination issues in real time before users see them, now available in preview.
• Protected Material Detection for Code to help detect pre-existing content and code. This feature helps developers explore public source code in GitHub repositories, fostering collaboration and transparency, while enabling more informed coding decisions.

Lastly, we announced new evaluations to help customers assess the quality and security of outputs and how often their AI application outputs protected material.

Get started with Azure AI

As a product builder it is exciting and humbling to bring new AI innovations to customers including models, customization, and safety features and to see real transformation that customers are driving. Whether an LLM or SLM, customizing generative AI model helps to boost their potential, allowing businesses to address specific challenges and innovate in their respective fields. Create the future today with Azure AI.

Additional resources 

• Get started with Azure OpenAI Service.
• Learn more about fine-tuning with Azure OpenAI Service.
• Get started with fine-tuning with Phi-3.

The post Announcing fine-tuning for customization and support for new models in Azure AI  appeared first on Microsoft AI Blogs.

As AI advances, we all have a role to play to unlock AI’s positive impact for organizations and communities around the world. That’s why we’re focused on helping customers use and build AI that is trustworthy, meaning AI that is secure, safe and private.

At Microsoft, we have commitments to ensure Trustworthy AI and are building industry-leading supporting technology. Our commitments and capabilities go hand in hand to make sure our customers and developers are protected at every layer.

Building on our commitments, today we are announcing new product capabilities to strengthen the security, safety and privacy of AI systems.

Security. Security is our top priority at Microsoft, and our expanded Secure Future Initiative (SFI) underscores the company-wide commitments and the responsibility we feel to make our customers more secure. This week we announced our first SFI Progress Report, highlighting updates spanning culture, governance, technology and operations. This delivers on our pledge to prioritize security above all else and is guided by three principles: secure by design, secure by default and secure operations. In addition to our first party offerings, Microsoft Defender and Purview, our AI services come with foundational security controls, such as built-in functions to help prevent prompt injections and copyright violations. Building on those, today we’re announcing two new capabilities:

• Evaluations in Azure AI Studio to support proactive risk assessments.
• Microsoft 365 Copilot will provide transparency into web queries to help admins and users better understand how web search enhances the Copilot response. Coming soon.

Our security capabilities are already being used by customers. Cummins, a 105-year-old company known for its engine manufacturing and development of clean energy technologies, turned to Microsoft Purview to strengthen their data security and governance by automating the classification, tagging and labeling of data. EPAM Systems, a software engineering and business consulting company, deployed Microsoft 365 Copilot for 300 users because of the data protection they get from Microsoft. J.T. Sodano, Senior Director of IT, shared that “we were a lot more confident with Copilot for Microsoft 365, compared to other large language models (LLMs), because we know that the same information and data protection policies that we’ve configured in Microsoft Purview apply to Copilot.”

Safety. Inclusive of both security and privacy, Microsoft’s broader Responsible AI principles, established in 2018, continue to guide how we build and deploy AI safely across the company. In practice this means properly building, testing and monitoring systems to avoid undesirable behaviors, such as harmful content, bias, misuse and other unintended risks. Over the years, we have made significant investments in building out the necessary governance structure, policies, tools and processes to uphold these principles and build and deploy AI safely. At Microsoft, we are committed to sharing our learnings on this journey of upholding our Responsible AI principles with our customers. We use our own best practices and learnings to provide people and organizations with capabilities and tools to build AI applications that share the same high standards we strive for.

Today, we are sharing new capabilities to help customers pursue the benefits of AI while mitigating the risks:

• A Correction capability in Microsoft Azure AI Content Safety’s Groundedness detection feature that helps fix hallucination issues in real time before users see them.
• Embedded Content Safety, which allows customers to embed Azure AI Content Safety on devices. This is important for on-device scenarios where cloud connectivity might be intermittent or unavailable.
• New evaluations in Azure AI Studio to help customers assess the quality and relevancy of outputs and how often their AI application outputs protected material.
• Protected Material Detection for Code is now in preview in Azure AI Content Safety to help detect pre-existing content and code. This feature helps developers explore public source code in GitHub repositories, fostering collaboration and transparency, while enabling more informed coding decisions.

It’s amazing to see how customers across industries are already using Microsoft solutions to build more secure and trustworthy AI applications. For example, Unity, a platform for 3D games, used Microsoft Azure OpenAI Service to build Muse Chat, an AI assistant that makes game development easier. Muse Chat uses content-filtering models in Azure AI Content Safety to ensure responsible use of the software. Additionally, ASOS, a UK-based fashion retailer with nearly 900 brand partners, used the same built-in content filters in Azure AI Content Safety to support top-quality interactions through an AI app that helps customers find new looks.

We’re seeing the impact in the education space too. New York City Public Schools partnered with Microsoft to develop a chat system that is safe and appropriate for the education context, which they are now piloting in schools. The South Australia Department for Education similarly brought generative AI into the classroom with EdChat, relying on the same infrastructure to ensure safe use for students and teachers.

Privacy. Data is at the foundation of AI, and Microsoft’s priority is to help ensure customer data is protected and compliant through our long-standing privacy principles, which include user control, transparency and legal and regulatory protections. To build on this, today we’re announcing:

• Confidential inferencing in preview in our Azure OpenAI Service Whisper model, so customers can develop generative AI applications that support verifiable end-to-end privacy. Confidential inferencing ensures that sensitive customer data remains secure and private during the inferencing process, which is when a trained AI model makes predictions or decisions based on new data. This is especially important for highly regulated industries, such as healthcare, financial services, retail, manufacturing and energy.
• The general availability of Azure Confidential VMs with NVIDIA H100 Tensor Core GPUs, which allow customers to secure data directly on the GPU. This builds on our confidential computing solutions, which ensure customer data stays encrypted and protected in a secure environment so that no one gains access to the information or system without permission.
• Azure OpenAI Data Zones for the EU and U.S. are coming soon and build on the existing data residency provided by Azure OpenAI Service by making it easier to manage the data processing and storage of generative AI applications. This new functionality offers customers the flexibility of scaling generative AI applications across all Azure regions within a geography, while giving them the control of data processing and storage within the EU or U.S.

We’ve seen increasing customer interest in confidential computing and excitement for confidential GPUs, including from application security provider F5, which is using Azure Confidential VMs with NVIDIA H100 Tensor Core GPUs to build advanced AI-powered security solutions, while ensuring confidentiality of the data its models are analyzing. And multinational banking corporation Royal Bank of Canada (RBC) has integrated Azure confidential computing into their own platform to analyze encrypted data while preserving customer privacy. With the general availability of Azure Confidential VMs with NVIDIA H100 Tensor Core GPUs, RBC can now use these advanced AI tools to work more efficiently and develop more powerful AI models.

Achieve more with Trustworthy AI 

We all need and expect AI we can trust. We’ve seen what’s possible when people are empowered to use AI in a trusted way, from enriching employee experiences and reshaping business processes to reinventing customer engagement and reimagining our everyday lives. With new capabilities that improve security, safety and privacy, we continue to enable customers to use and build trustworthy AI solutions that help every person and organization on the planet achieve more. Ultimately, Trustworthy AI encompasses all that we do at Microsoft and it’s essential to our mission as we work to expand opportunity, earn trust, protect fundamental rights and advance sustainability across everything we do.

Related:

Commitments

• Security: Secure Future Initiative  
• Privacy: Trust Center 
• Safety: Responsible AI Principles

Capabilities

• Security: Security for AI
• Privacy: Azure Confidential Computing
• Safety: Azure AI Content Safety

The post Microsoft Trustworthy AI: Unlocking human potential starts with trust appeared first on Microsoft AI Blogs.

We are excited to add OpenAI’s newest models o1-preview and o1-mini to Microsoft Azure OpenAI Service, Azure AI Studio, and GitHub Models. The o1 series enables complex coding, math reasoning, brainstorming, and comparative analysis capabilities, setting a new benchmark for AI-powered solutions. 

The o1-preview and o1-mini models are now accessible in Azure AI Studio and GitHub Models to a select group of Azure customers to collaboratively explore and identify the unique strengths of each model. The o1 series of advanced reasoning models excel at complex and nuanced problem spaces like these: 

• Complex code generation: Capable of algorithm generation and advanced coding tasks to help developers. 

• Advanced problem solving: Perfect for comprehensive brainstorming sessions and tackling multifaceted issues. 

• Complex document comparison: Ideal for analyzing contracts, case files, or legal documents to discern subtle differences. 

• Instruction following and workflow management: Particularly adept at handling workflows that require shorter context. 

Early adopters share their insights 

Our early adopters have already shared valuable insights and are excited to use these models to advance the capabilities of their applications. 

GitHub Copilot, the world’s most widely adopted AI developer tool, has been testing the o1 models, showing promising results in code analysis and optimization. 

“With o1 and its strong reasoning capabilities allows, GitHub Copilot will enable developers to build for the bigger picture, faster. Nothing beats the feeling when you solve a coding problem within minutes instead of hours. And through GitHub Models, we can’t wait to see what developers do with o1 in their apps.”—Thomas Dohmke, CEO, GitHub.

Harvey, a leader in generative AI for professional services, is inspiring customers by integrating its platform with Azure OpenAI Service, now available in Azure Marketplace.

“In our testing, we believe o1 marks a step change in legal reasoning and will enable us to create custom solutions that tackle more ambitious workflows like SPA and S-1 drafting or agents that assist in critical aspects of due diligence and e-discovery. This marks a transition from chatbots to collaborative platforms that allow professionals to work hand-in-hand with AI on complex use cases.”—Winston Weinberg, co-founder and CEO of Harvey.

Cognition, an applied AI lab building end-to-end software agents, is excited to integrate the new models into Devin, the world’s first fully autonomous AI software engineer.

“o1 is a significant advancement in reasoning models, and we’re excited for how innovations like this will improve Devin, allowing it to solve ever-more complex coding tasks.”—Scott Wu, CEO of Cognition. 

These examples highlight just a few of the ways the o1 series of models can bring advanced reasoning and analytical capabilities to your projects. 

New updates in our commitment to responsible AI and Safety by default

Safety continues to be our top priority in delivering new generative AI models, o1 and o1-mini models in Azure OpenAI Service have “on-by-default” Content Safety features, at no additional cost for you. 

OpenAI has invested in additional safety advancements with these models, including new methods that help the model refuse unsafe requests. This makes the o1 series of models among the most robust models we have deployed to date. You can assess the safety of their AI application and model deployments with Azure AI Studio safety evaluations. 

With our joint efforts with OpenAI to make the models safer, we continue to expand and improve the Content Safety systems for all models. We’re pleased to announce that Prompt Shields and Protected Materials for Text are now generally available in Azure OpenAI Service. These features help customers maintain the safety and integrity of their generative AI deployments, preventing potential misuse. Finally, we’ve introduced “Spotlighting,” a family of techniques designed to help models distinguish between valid instructions and potentially untrusted external inputs. We’ve updated our documentation explaining how to use “Spotlighting” and related techniques so customers can take advantage of Microsoft’s latest research. 

Trying the new models in Azure AI Studio and GitHub 

We’re excited to offer a preview of the o1-preview and o1-mini models in Azure AI Studio playground and GitHub Models, available to select customers. This exclusive access enables early experimentation with their advanced reasoning capabilities, with API access coming soon. To get started, visit the Azure AI Studio model catalog and apply for access today. 

Navigating our model lineup 

GPT-4o models have industry leading capabilities in text and image comprehension and GPT-4o mini offers high quality summarization and Q&A, with cost-effective deployment options and quick response times. At the same time, we have seen the limitations of these models when it comes to solving even simple math problems with strong guidance and examples of how to solve them. 

To better understand o1-preview and o1-mini’s place in our model lineup, here’s a quick overview of the key models powering Azure OpenAI Service: 

• o1-preview: Focused on advanced reasoning and solving complex problems, including math and science tasks. Ideal for applications that require deep contextual understanding and agentic workflows. 

• o1-mini: Smaller and faster, and 80% cheaper than o1-preview, performs well at code generation and small context operations. 

• GPT-4o: A versatile, multimodal model that excels in both text and image processing, with superior performance in non-English languages and vision tasks. Suitable for applications needing enhanced accuracy and multilingual capabilities. The model also features JSON Structured Outputs for consistent, well-defined data formats, reducing post-processing needs and improving application efficiency. Designed for real-time applications that require fast, reliable text responses at minimal cost. 

• GPT-4o Mini: A smaller, cost-effective version of GPT-4o, optimized for environments with limited resources or high cost constraints. Retains text and image processing capabilities, making it ideal for lightweight applications. 

• DALL-E: Generates images from text prompts with safety, ideal for creative content and marketing. 

• Whisper: Transcribes and translates speech to text, suitable for real-time transcription and multilingual communication. 

These models represent a progression of capabilities, from efficient text processing to advanced reasoning and multimodal functionality. Each model has paved the way for the innovations we continue to introduce with “o1.” 

Innovating at scale with Azure 

The latest OpenAI models are backed with all the capabilities of the Azure platform, including enterprise-grade security, flexible deployment options, and broad regional availability, which helps customers meet data residency and compliance needs. We have over 60,000 Azure AI customers with exciting production use cases across industries.

Join us in shaping the future of AI 

We are excited to invite a select group of customers to explore o1-preview and o1-mini in Azure AI Studio playground and experience its unique capabilities firsthand, by visiting the model catalog and applying for access. But don’t worry if you’re not part of the initial group—our upcoming API release will make o1 series models more broadly available. 

We believe in the power of collaboration and innovation, and we want everyone to have the opportunity to benefit from these cutting-edge advancements in AI. Whether you’re new to Azure AI or a seasoned user, there’s never been a better time to experiment, innovate, and grow with us. We look forward to welcoming a wider audience soon and working together to shape the future of AI. 

The post Introducing o1: OpenAI’s new reasoning model series for developers and enterprises on Azure appeared first on Microsoft AI Blogs.

One year ago, Microsoft and Oracle built upon our multi-year partnership by announcing a first of its kind solution—Oracle Database@Azure—to offer Oracle Database services running on Oracle Cloud Infrastructure (OCI) inside Microsoft Azure datacenters. This gives customers the ability to run or build new cloud applications that combine Oracle Database services with native Microsoft Azure services and benefit from extremely low latency and high performance.

We are excited to share important new enhancements to Oracle Database@Azure that unlock more value from data and pave the way to AI innovation:

• Microsoft Fabric plus Oracle Database@Azure integration to fuel customer data and AI innovations.
• Integration with Microsoft Sentinel and compliance certifications to provide industry-leading security and compliance for mission-critical workloads.
• Plans to expand offering to a total of 21 primary regions, each with at least two availability zones and support for Oracle’s Maximum Availability Architecture (MAA) to deliver the highest levels of availability and resilience.
• Examples from customers joining us on stage at Oracle CloudWorld this week.

Customers choose Oracle Database@Azure to advance their cloud strategy

Customer choice is at the heart of the Microsoft and Oracle partnership. Performance, scalability, security, and reliability are top of mind for our customers as they migrate and modernize mission-critical workloads in the cloud. 

Across industries, leading organizations such as Conduent, Liantis, MSCI, State Street Corporation, Sanofi, and SAS are choosing Oracle Database@Azure to run their Oracle workloads.

“With the continuing threat of ransomware, companies must adapt to rebuild their critical services and systems from scratch—not just reconstitute data into an environment that is compromised. Oracle Database@Azure is the only service that meets MSCI’s Cyber DR needs from a recovery time, security isolation, recovery point and cost perspective.”—John Rogers, Chief Information Security Officer, MSCI.

“Sanofi is advancing its cloud strategy and modernization of critical workloads on the cloud. By hosting Oracle Databases on Azure, we expect to significantly enhance the scalability, reliability, and performance of our critical workloads. This approach enables us to optimize costs, accelerate cloud growth, and achieve greater agility and flexibility across our operations. While currently in the initial phases of implementation, Oracle Database@Azure already shows promising potential to support our long-term digital transformation goal.”—Kevin Fuller, Global Head of Technology, Sanofi.

“Oracle Database@Azure provides us with a best-of-breed service—combining the performance of Exadata and power of Azure. Oracle Database@Azure allows us to accelerate our cloud journey for Azure workloads without major refactoring. It protects our existing investments in Exadata and allows for a seamless migration experience for our mission-critical workloads.”—Akshay Sharma, Managing Director, Cloud Engineering, State Street Corporation.

New Microsoft Fabric plus Oracle Database@Azure integration fuels customer data and AI innovations

Now in public preview, customers have the opportunity to use OCI GoldenGate—a database replication and heterogeneous data integration service—to sync their data estates with Microsoft Fabric. This integration unlocks new prospects for data analytics and AI applications by unifying diverse datasets, allowing teams to identify patterns and visualize opportunities.

Microsoft Fabric is an ever-evolving, AI-powered data analytics platform that empowers customers to unify and future-proof their data estate. Fabric keeps up with the trends and seamlessly integrates each new capability so businesses can spend less time integrating and managing their data estate and more time unlocking value from their data. OCI GoldenGate offers seamless support to integrate data across dozens of data sources and targets including OneLake in Microsoft Fabric, delivering enterprise-grade, real-time data to the Microsoft ecosystem. The combination of OCI GoldenGate’s continuous, low-latency data availability in Microsoft Fabric’s comprehensive data and analytics tools, like Power BI and Copilot, enables customers to connect their essential data sources—both Oracle and non-Oracle—to drive better insights and decision-making.

Oracle Database@Azure customer workloads now protected with industry-leading security and compliance integrations

With Azure’s industry-leading security and compliance, Oracle Database@Azure customers are protecting their data with enterprise-grade threat protection.

Now in public preview is the Oracle Database@Azure integration with Microsoft Sentinel, a cloud-native security information and event management system designed to identify and address cyberthreats across your entire enterprise with intelligent security analytics. Customers will be able to extend their threat analytics to their Oracle Exadata Database Service and infrastructure powering Oracle Database@Azure and the Oracle Database services that operate them.

To help customers address regulatory requirements, we’ve also extended Azure’s industry-leading compliance to Oracle Database@Azure through a comprehensive array of global certifications for privacy, healthcare, payment services, and more.

Securing mission-critical workloads is also essential for resiliency and business continuity.

Oracle Database@Azure enterprise-grade resiliency enhancements boost data protection and business continuity

Oracle Zero Data Loss Autonomous Recovery Service is now generally available, offering businesses a fully managed data protection service for organizations running Oracle Exadata Database on Oracle Database@Azure. Organizations can promptly restore business-critical data after ransomware attacks without losing any data and enhance operational efficiency to address financial regulations—all with the simplicity expected of cloud services.

Oracle Database@Azure is designed with a robust architecture to meet the high levels of availability and resilience required for business-critical applications. With the capability to deploy across at least two availability zones in all primary regions, Oracle Database@Azure is optimized to provide maximum uptime and fault tolerance. Currently, all regions provide support for Oracle’s Maximum Availability Architecture (MAA) Gold tier. For customers looking to achieve the highest level of availability, selecting a combination of regions such as UK South for primary and Germany West Central for disaster recovery allows them to collaborate with both Oracle and Microsoft for MAA Platinum certification.

Regional expansion underway to meet growing customer demand

Customer demand for Oracle Database@Azure continues to grow—today, we’re in six regions worldwide and we’re announcing plans to expand regional availability to a total of 21 regions around the world. In addition, we plan to add support for disaster recovery in a number of other Azure regions. Watch this page for more information on the new regions opening soon.

Updates for Oracle Autonomous Database on Oracle Database@Azure

We recently announced the general availability of Oracle Autonomous Database service is available on Oracle Database@Azure. Customers can purchase Oracle Autonomous Database on Oracle Database@Azure via a private offer from Oracle sales or with a new pay-as-you-go option available in the Azure Marketplace. The Oracle Autonomous Database service provides customers with a managed Oracle Database service, alongside the Oracle Exadata Database Service which provides the ultimate control and flexibility.

Getting started with Oracle Database@Azure

We are excited to offer Oracle Database@Azure as your solution to run Oracle workloads alongside non-Oracle workloads, and leverage Azure services to innovate and get applications AI-ready.

If you are here at Oracle CloudWorld, please join us at these sessions and stop at the Microsoft booth for more demos and to speak with an expert.

Learn how to migrate and manage your Oracle databases in Azure.

To get started, contact our sales team.

The post Microsoft and Oracle enhance Oracle Database@Azure with data and AI integration  appeared first on Microsoft AI Blogs.

• Improvements to the Phi family of models, including a new Mixture of Experts (MoE) model and 20+ languages.

• AI21 Jamba 1.5 Large and Jamba 1.5 on Azure AI models as a service.

• Integrated vectorization in Azure AI Search to create a streamlined retrieval augmented generation (RAG) pipeline with integrated data prep and embedding.

• Custom generative extraction models in Azure AI Document Intelligence, so you can now extract custom fields for unstructured documents with high accuracy.

• The general availability of Text to Speech (TTS) Avatar, a capability of Azure AI Speech service, which brings natural-sounding voices and photorealistic avatars to life, across diverse languages and voices, enhancing customer engagement and overall experience. 

• The general availability of the VS Code extension for Azure Machine Learning.

• The general availability of Conversational PII Detection Service in Azure AI Language.

Use the Phi model family with more languages and higher throughput 

We are introducing a new model to the Phi family, Phi-3.5-MoE, a Mixture of Experts (MoE) model. This new model combines 16 smaller experts into one, which delivers improvements in model quality and lower latency. While the model is 42B parameters, since it is an MoE model it only uses 6.6B active parameters at a time, by being able to specialize a subset of the parameters (experts) during training, and then at runtime use the relevant experts for the task. This approach gives customers the benefit of the speed and computational efficiency of a small model with the domain knowledge and higher quality outputs of a larger model. Read more about how we used a Mixture of Experts architecture to improve Azure AI translation performance and quality.

We are also announcing a new mini model, Phi-3.5-mini. Both the new MoE model and the mini model are multi-lingual, supporting over 20 languages. The additional languages allow people to interact with the model in the language they are most comfortable using.

Even with new languages the new mini model, Phi-3.5-mini, is still a tiny 3.8B parameters.

Companies like CallMiner, a conversational intelligence leader, are selecting and using Phi models for their speed, accuracy, and security.

“CallMiner is constantly innovating and evolving our conversation intelligence platform, and we’re excited about the value Phi models are bringing to our GenAI architecture. As we evaluate different models, we’ve continued to prioritize accuracy, speed, and security... The small size of Phi models makes them incredibly fast, and fine tuning has allowed us to tailor to the specific use cases that matter most to our customers at high accuracy and across multiple languages. Further, the transparent training process for Phi models empowers us to limit bias and implement GenAI securely. We look forward to expanding our application of Phi models across our suite of products“—Bruce McMahon, CallMiner’s Chief Product Officer.

To make outputs more predictable and define the structure needed by an application, we are bringing Guidance to the Phi-3.5-mini serverless endpoint. Guidance is a proven open-source Python library (with 18K plus GitHub stars) that enables developers to express in a single API call the precise programmatic constraints the model must follow for structured output in JSON, Python, HTML, SQL, whatever the use case requires. With Guidance, you can eliminate expensive retries, and can, for example, constrain the model to select from pre-defined lists (e.g., medical codes), restrict outputs to direct quotes from provided context, or follow in any regex. Guidance steers the model token by token in the inference stack, producing higher quality outputs and reducing cost and latency by as much as 30-50% when utilizing for highly structured scenarios. 

• Read more about the benefits of Guidance.

We are also updating the Phi vision model with multi-frame support. This means that Phi-3.5-vision (4.2B parameters) allows reasoning over multiple input images unlocking new scenarios like identifying differences between images.

At the core of our product strategy, Microsoft is dedicated to supporting the development of safe and responsible AI, and provides developers with a robust suite of tools and capabilities.  

Developers working with Phi models can assess quality and safety using both built-in and custom metrics using Azure AI evaluations, informing necessary mitigations. Azure AI Content Safety provides built-in controls and guardrails, such as prompt shields and protected material detection. These capabilities can be applied across models, including Phi, using content filters or can be easily integrated into applications through a single API. Once in production, developers can monitor their application for quality and safety, adversarial prompt attacks, and data integrity, making timely interventions with the help of real-time alerts. 

Introducing AI21 Jamba 1.5 Large and Jamba 1.5 on Azure AI models as a service

Furthering our goal to provide developers with access to the broadest selection of models, we are excited to also announce two new open models, Jamba 1.5 Large and Jamba 1.5, available in the Azure AI model catalog. These models use the Jamba architecture, blending Mamba, and Transformer layers for efficient long-context processing.

According to AI21, the Jamba 1.5 Large and Jamba 1.5 models are the most advanced in the Jamba series. These models utilize the Hybrid Mamba-Transformer architecture, which balances speed, memory, and quality by employing Mamba layers for short-range dependencies and Transformer layers for long-range dependencies. Consequently, this family of models excels in managing extended contexts ideal for industries including financial services, healthcare, and life sciences, as well as retail and CPG. 

“We are excited to deepen our collaboration with Microsoft, bringing the cutting-edge innovations of the Jamba Model family to Azure AI users…As an advanced hybrid SSM-Transformer (Structured State Space Model-Transformer) set of foundation models, the Jamba model family democratizes access to efficiency, low latency, high quality, and long-context handling. These models empower enterprises with enhanced performance and seamless integration with the Azure AI platform”— Pankaj Dugar, Senior Vice President and General Manger of North America at AI21

• Learn more on the Jamba model family.

Simplify RAG for generative AI applications

We are streamlining RAG pipelines with integrated, end to end data preparation and embedding. Organizations often use RAG in generative AI applications to incorporate knowledge on private organization specific data, without having to retrain the model. With RAG, you can use strategies like vector and hybrid retrieval to surface relevant, informed information to a query, grounded on your data. However, to perform vector search, significant data preparation is required. Your app must ingest, parse, enrich, embed, and index data of various types, often living in multiple sources, just so that it can be used in your copilot. 

Today we are announcing general availability of integrated vectorization in Azure AI Search. Integrated vectorization automates and streamlines these processes all into one flow. With automatic vector indexing and querying using integrated access to embedding models, your application unlocks the full potential of what your data offers.

In addition to improving developer productivity, integration vectorization enables organizations to offer turnkey RAG systems as solutions for new projects, so teams can quickly build an application specific to their datasets and need, without having to build a custom deployment each time.

Customers like SGS & Co, a global brand impact group, are streamlining their workflows with integrated vectorization.

“SGS AI Visual Search is a GenAI application built on Azure for our global production teams to more effectively find sourcing and research information pertinent to their project… The most significant advantage offered by SGS AI Visual Search is utilizing RAG, with Azure AI Search as the retrieval system, to accurately locate and retrieve relevant assets for project planning and production”—Laura Portelli, Product Manager, SGS & Co

• Learn more on vectorization.

Extract custom fields in Document Intelligence 

You can now extract custom fields for unstructured documents with high accuracy by building and training a custom generative model within Document Intelligence. This new ability uses generative AI to extract user specified fields from documents across a wide variety of visual templates and document types. You can get started with as few as five training documents. While building a custom generative model, automatic labeling saves time and effort on manual annotation, results will display as grounded where applicable, and confidence scores are available to quickly filter high quality extracted data for downstream processing and lower manual review time.

• Learn more on custom field extraction preview.

Create engaging experiences with prebuilt and custom avatars 

Today we are excited to announce that Text to Speech (TTS) Avatar, a capability of Azure AI Speech service, is now generally available. This service brings natural-sounding voices and photorealistic avatars to life, across diverse languages and voices, enhancing customer engagement and overall experience. With TTS Avatar, developers can create personalized and engaging experiences for their customers and employees, while also improving efficiency and providing innovative solutions.

The TTS Avatar service provides developers with a variety of pre-built avatars, featuring a diverse portfolio of natural-sounding voices, as well as an option to create custom synthetic voices using Azure Custom Neural Voice. Additionally, the photorealistic avatars can be customized to match a company’s branding. For example, Fujifilm is using TTS Avatar with NURA, the world’s first AI-powered health screening center.

“Embracing the Azure TTS Avatar at NURA as our 24-hour AI assistant marks a pivotal step in healthcare innovation. At NURA, we envision a future where AI-powered assistants redefine customer interactions, brand management, and healthcare delivery. Working with Microsoft, we’re honored to pioneer the next generation of digital experiences, revolutionizing how businesses connect with customers and elevate brand experiences, paving the way for a new era of personalized care and engagement. Let’s bring more smiles together”—Dr. Kasim, Executive Director and Chief Operating Officer, Nura AI Health Screening

As we bring this technology to market, ensuring responsible use and development of AI remains our top priority. Custom Text to Speech Avatar is a limited access service in which we have integrated safety and security features. For example, the system embeds invisible watermarks in avatar outputs. These watermarks allow approved users to verify if a video has been created using Azure AI Speech’s avatar feature.  Additionally, we provide guidelines for TTS avatar’s responsible use, including measures to promote transparency in user interactions, identify and mitigate potential bias or harmful synthetic content, and how to integrate with Azure AI Content Safety. In this transparency note, we describe the technology and capabilities for TTS Avatar, its approved use cases, considerations when choosing use cases, its limitations, fairness considerations and best practice for improving system performance. We also require all developers and content creators to apply for access and comply with our code of conduct when using TTS Avatar features including prebuilt and custom avatars.  

• Learn more about Azure TTS Avatar.

Use Azure Machine Learning resources in VS Code

We’re thrilled to announce the general availability of the VS Code extension for Azure Machine Learning. The extension allows you to build, train, deploy, debug, and manage machine learning models with Azure Machine Learning directly from your favorite VS Code setup, whether on desktop or web. With features like VNET support, IntelliSense and integration with Azure Machine Learning CLI, the extension is now ready for production use. Read this tech community blog to learn more about the extension.

Customers like Fashable have put this into production.

“We have been using the VS Code extension for Azure Machine Learning since its preview release, and it has significantly streamlined our workflow… The ability to manage everything from building to deploying models directly within our preferred VS Code environment has been a game-changer. The seamless integration and robust features like interactive debugging and VNET support have enhanced our productivity and collaboration. We are thrilled about its general availability and look forward to leveraging its full potential in our AI projects.”—Ornaldo Ribas Fernandes, Co-founder and CEO, Fashable

Protect users’ privacy 

Today we are excited to announce the general availability of Conversational PII Detection Service in Azure AI Language, enhancing Azure AI’s ability to identify and redact sensitive information in conversations, starting with English language. This service aims to improve data privacy and security for developers building generative AI apps for their enterprise. The Conversational PII redaction service expands upon the Text PII redaction service, supporting customers looking to identify, categorize, and redact sensitive information such as phone numbers and email addresses in unstructured text. This Conversational PII model is specialized for conversational style inputs, particularly those found in speech transcriptions from meetings and calls. 

• Learn more on Conversational PII redaction service.

Self-serve your Azure OpenAI Service PTUs  

We recently announced updates to Azure OpenAI Service, including the ability to manage your Azure OpenAI Service quota deployments without relying on support from your account team, allowing you to request Provisioned Throughput Units (PTUs) more flexibly and efficiently. We also released OpenAI’s latest model when they made it available on 8/7, which introduced Structured Outputs, like JSON Schemas, for the new GPT-4o and GPT-4o mini models. Structured outputs are particularly valuable for developers who need to validate and format AI outputs into structures like JSON Schemas. 

• Read more about Structured Outputs for GPT-4o and GPT-4o mini models on the Azure Blog.

We continue to invest across the Azure AI stack to bring state of the art innovation to our customers so you can build, deploy, and scale your AI solutions safely and confidently. We cannot wait to see what you build next.

Stay up to date with more Azure AI news 

• See the latest Azure OpenAI Service news.
• See the latest Azure AI news.
• Read more in our Azure AI services documentation.
• Read the latest AI and machine learning blogs.

The post Boost your AI with Azure’s new Phi model, streamlined RAG, and custom generative AI models appeared first on Microsoft AI Blogs.

What’s new? 

Self-Service Provisioning and Model Independent Quota Requests 

We are introducing self-service provisioning alongside standard tokens, allowing you to request Provisioned Throughput Units (PTUs) more flexibly and efficiently. This new feature empowers you to manage your Azure OpenAI Service quata deployments independently without relying on support from your account team. By decoupling quota requests from specific models, you can now allocate resources based on your immediate needs and adjust as your requirements evolve. This change simplifies the process and accelerates your ability to deploy and scale your applications. 

Visibility to service capacity and availability

Gain better visibility into service capacity and availability, helping you make informed decisions about your deployments. With this new feature, you can access real-time information about service capacity in different regions, ensuring that you can plan and manage your deployments more effectively. This transparency allows you to avoid potential capacity issues and optimize the distribution of your workloads across available resources, leading to improved performance and reliability for your applications. 

Provisioned hourly pricing and reservations 

We are excited to introduce two new self-service purchasing options for PTUs: 

1. Hourly no-commitment purchasing 
   • You can now create a Provisioned deployment for as little as an hour, with a flat hourly rate of $2 per unit per hour. This model-independent pricing makes it easy to deploy and tear down deployments as needed, offering maximum flexibility. This is ideal for testing scenarios or transitional periods without any long-term commitment. 

2. Monthly and yearly Azure reservations for Provisioned deployments
   • For production environments with steady request volumes, Azure OpenAI Service Provisioned Reservations offer significant cost savings. By committing to a monthly or yearly reservation, you can save up to 82% or 85%, respectively, over hourly rates. Reservations are now decoupled from specific models and deployments, providing unmatched flexibility. This approach allows enterprises to optimize costs while maintaining the ability to switch models and adjust deployments as needed. Read our technical blog on Reservations here.

Benefits for decision makers 

These updates are designed to provide flexibility, cost efficiency, and ease of use, making it simpler for decision-makers to manage AI deployments. 

• Flexibility: With self-service provisioning and hourly pricing, you can scale your deployments up or down based on immediate needs without long-term commitments. 

• Cost efficiency: Azure Reservations offer substantial savings for long-term use, enabling better budget planning and cost management. 

• Ease of use: Enhanced visibility and simplified provisioning processes reduce administrative burdens, allowing your team to focus on strategic initiatives rather than operational details. 

Customer success stories 

Before we made self-service available, select customers started achieving benefits of these options. 

• Visier Solutions: By leveraging Provisioned Throughput Units (PTUs) with Azure OpenAI Service, Visier Solutions has significantly enhanced their AI-powered people analytics tool, Vee. With PTUs, Visier guarantees rapid, consistent response times, crucial for handling the high volume of queries from their extensive customer base. This powerful synergy between Visier’s innovative solutions and Azure’s robust infrastructure not only boosts customer satisfaction by delivering swift and accurate insights but also underscores Visier’s commitment to using cutting-edge technology to drive transformational change in workforce analytics. Read the case study on Microsoft. 

• An analytics and insights company: Switched from Standard Deployments to GPT-4 Turbo PTUs and experienced a significant reduction in response times, from 10–20 seconds to just 2–3 seconds. 

• A Chatbot Services company: Reported improved stability and lower latency with Azure PTUs, enhancing the performance of their services. 

• A visual entertainment company: Noted a drastic latency improvement, from 12–13 seconds down to 2–3 seconds, enhancing user engagement. 

Empowering all customers to build with Azure OpenAI Service

These new updates do not alter the technical excellence of Provisioned deployments, which continue to deliver low and predictable latency. Instead, they introduce a more flexible and cost-effective procurement model, making Azure OpenAI Service more accessible than ever. With self-service Provisioned, model-independent units, and both hourly and reserved pricing options, the barriers to entry have been drastically lowered. 

To learn more about enhancing the reliability, security, and performance of your cloud and AI investments, explore the additional resources below.

Additional Resources 

• Azure Pricing Provisioned Reservations

• Azure OpenAI Service Pricing 

• More details about Provisioned

• Documentation for On-Boarding 

• PTU Calculator in Azure AI Studio 
• Unveiling Azure OpenAI Service Provisioned reservations blog

The post Elevate your AI deployments more efficiently with new deployment and cost management solutions for Azure OpenAI Service including self-service Provisioned appeared first on Microsoft AI Blogs.

An explosion of interest in generative AI across many industries has sparked, a direct result of the collaboration of Microsoft and NVIDIA and the breakthrough technology behind OpenAI’s ChatGPT. As a result, artificial intelligence (AI) is transforming the way we interact with digital products and services, from chatbots and voice assistants to smart cameras and recommendation systems. It’s now almost a daily demand to leverage the power of AI to create applications that can understand, interact, and learn from their users and environments.

However, building these AI applications presents significant challenges in terms of time, resources, access to AI infrastructure, and costs—which can be prohibitive for many developers and organizations.

To alleviate these challenges, developers can benefit from the combined benefits of Azure AI—a set of cloud-based AI and machine learning services that can help you build, train, and deploy AI-enabled applications with ease—and the NVIDIA AI platform to maximize application performance throughout every stage of development and deployment. To make access and entry even easier to build the best AI-enabled applications, Microsoft and NVIDIA have launched a collaborative resource for developers and organizations to experience the better together benefits.

In this blog, we’ll discuss how combining the power of both Azure AI and the NVIDIA AI Platform can help you create your most impactful AI-enabled applications, providing you with flexibility, productivity, efficiency, and innovation.

Better together: Microsoft and NVIDIA

Recognizing the barriers developers face, NVIDIA and Microsoft have worked closely to democratize access to the same core technology that powers ChatGPT to accelerate adoption. The partnership focuses on optimizing every layer of the generative AI stack—from highly performant and scalable AI infrastructure to developer-friendly tools and services—to reduce complexity and cost, making advanced AI capabilities more accessible and feasible for a broader range of applications and industries.

Used by more than 60,000 organizations, Azure AI integrates with popular developer environments Visual Studio Code and GitHub, allowing you to use your preferred tools and workflows to develop, test, and deploy your AI solutions. Whether you want to use pre-built models and APIs, or build and train your own custom models, Azure AI can support various AI scenarios including building your own copilot with enterprise chat, speech analytics, document processing automation, and more.

Azure’s leading cloud AI supercomputing infrastructure, leveraging both state of the art NVIDIA GPUs and NVIDIA InfiniBand networking, provides the best performance, scalability, and built-in security needed to build, train, and deploy the most demanding AI workloads with confidence, at any scale. This combination accelerates time to solution, lowers deployment costs by supporting more users with fewer compute resources, and enhances user experience through optimized performance and faster data throughput.

Benefits for developers in GitHub and Visual Studio Code

Whether a developer that uses Visual Studio Code or GitHub, Azure AI integrates with your existing development environment, allowing you to use the same tools and workflows that you’re already familiar with.

Some of the benefits of using these AI tools and services for developers in GitHub and Visual Studio Code include:

• Flexibility and choice: Choose the AI solution that best suits you, whether using pre-built models and APIs or building and training your own custom models. Choose the framework and language that you prefer, such as LangChain, Semantic Kernel, TensorFlow, PyTorch, Scikit-learn, Python, or R. You can even use the Azure OpenAI Service to access the latest GPT models from OpenAI. Additionally, folks can use the new Prompty format to work with prompts in their preferred environments (like Visual Studio Code and GitHub) all while using the trusted platform.
• Productivity and efficiency: Simplify and accelerate the AI development process by using Visual Studio Code extensions and GitHub Actions. For example, use prompt flow to manage various versions of your flow assets like prompts, code, configurations, and environments via code repo, with tracked changes and rollback to previous versions, promoting a collaborative LLMOps ethos. For machine learning workloads, use GitHub Actions for Azure Machine Learning to automate model training, testing, and deployment.
• Performance and scalability: Harness NVIDIA-optimized software from NVIDIA AI Enterprise, available in the Azure marketplace, to streamline workflows and embrace powerful AI capabilities. With support for remote development using Visual Studio Code extensions, you can write, debug, and optimize GPU-accelerated applications—including AI models—while using NVIDIA GPU-powered Azure Virtual Machines.
• Innovation and creativity: Build applications that understand, interact, and learn from their users and environments, and that deliver personalized and engaging experiences. Use Azure AI to build a comprehensive generative AI stack and enrich your applications with retrieval-augmented generation, natural language processing, machine learning, and more.

Start building your most innovative applications

The strategic partnership between Microsoft and NVIDIA has significantly enhanced the Azure AI ecosystem. The integration of the NVIDIA AI Enterprise software platform combined with Azure’s AI toolsets and libraries ensures a robust and efficient environment for advancing your AI projects. Accelerate your time to deployment with the optimized NVIDIA Nemotron models, NVIDIA NIM inference microservices, Langchain and Hugging Face integrations, and APIs, inside of your Azure AI environment.

By building AI-enabled applications within the Microsoft ecosystem, developers can benefit from the productivity and efficiency gains that come from using a single, integrated set of tools and services. This can help reduce development time, support costs, and enhance collaboration and communication among team members. You can also benefit from the innovation and creativity that Azure AI enables, allowing you to create applications that understand, interact, and learn from users and environments, and deliver personalized and engaging experiences.

Learn more about how you can streamline development and build AI-enabled applications faster and easier with the combined power of Microsoft and NVIDIA.

The post Build AI-enabled applications with Azure AI and NVIDIA appeared first on Microsoft AI Blogs.

More than 60,000 organizations use Microsoft Azure AI today to explore the power of custom AI applications. However, the market is quickly moving from experimentation to scale, and we see more developers around the world becoming AI developers. With this natural evolution, the needs of developers and their requirements to access and build with AI models and tools are transforming as well.

To support this shift to scale, we are excited to partner with GitHub to empower their more than 100 million developers to build AI applications directly from GitHub.com with seamless integrations with Codespaces and Microsoft Visual Studio Code. Our collaboration starts today as we bring Azure AI’s leading model selection to developers through GitHub Models, along with simple APIs to empower responsible, production-ready AI applications.

For more insights into how GitHub Models can help you increase experimentation and accelerate your development cycles, all in GitHub, please read the blog from GitHub CEO Thomas Dohmke.

Simplifying AI development 

As AI model innovation accelerates, Azure remains committed to delivering the leading model selection and greatest model diversity to meet the unique cost, latency, design, and safety needs of AI developers. Today, we offer the largest and most complete model library in the market, including the latest models from OpenAI, Meta, Mistral and Cohere and updates to our own Phi-3 family of small language models. With GitHub Models, developers can now explore and utilize the latest models along with AI innovations and next-generation frontier models. This offering gives every developer the flexibility to choose the best combination of unique capabilities, performance metrics, and cost efficiencies.

While continuous model innovation brings more choice, it also brings complexity when selecting the right model for the right scenario. Today, developers have a range of options for cloud vs. edge, general-purpose vs. task specific, and more. On top of that, organizations often need multiple models to enable better quality, lower cost of goods sold, and to address complex use cases for each industry. GitHub Models opens the door for developers to experiment with multiple models, simplifying model experimentation and selection across the best of the Azure AI catalog, quickly comparing models, parameters, and prompts.

By making Azure AI an open, modular platform, we aim to help our customers rapidly go from idea to code to cloud. With Azure AI on GitHub, developers can do just that by utilizing Codespaces to set up a prototype or use the Prompty extension to generate code with GitHub Models directly in Microsoft Visual Studio Code.

In the coming months, we will expand our integration even further, bringing Azure AI’s language, vision, and multi-modal services to GitHub, along with additional Azure AI toolchain elements, further streamlining the AI application development process.

Integrating safety by default 

Developers building with AI want to be confident their AI applications are trustworthy, safe, and secure. GitHub Models gives developers a strong foundation from the start with built-in safety and security controls from Azure AI.

Azure AI works with model providers and other partners such as HiddenLayer to reduce emerging threats, from cybersecurity vulnerabilities, to malware, and other signs of tampering. And we have taken this further in GitHub Models by integrating Azure AI Content Safety for top foundation models including Azure OpenAI Service, Llama, and Mistral. Azure AI Content Safety enables built-in, real time protection for risks such as the generation of harmful content, copyright materials, hallucination, and new AI specific attacks such as jailbreaks and prompt injection attacks.

If developers want to go deeper, they can customize these controls in Azure AI, using evaluations to test and monitor their applications for ongoing quality and safety.

AI simplicity with a single API

Increased model selection gives developers the broadest range of options for the individual applications they are building. But each model naturally brings with it increased complexity. To counteract this, we’re making it incredibly easy for every developer to experiment with a range of models through the Azure AI model inference API. Using this single API, GitHub developers can now access a common set of capabilities to compare performance across a diverse set of foundational models in a uniform and consistent way, easily switching between models to compare performance without changing the underlying code.

The Azure AI Inference SDK provides client libraries in Python and JavaScript with support for C# and .NET coming soon. This SDK makes it easy to integrate AI into your applications by simplifying common tasks related to authentication, security and retries in your programming language of choice. You can get started today with Python and JavaScript samples.

Streamlining GitHub Enterprise access through Microsoft Azure 

Beyond these new integrations, we are also making it easier than ever for organizations to access GitHub Enterprise through Azure, combining GitHub’s cloud-native platform with Azure’s robust enterprise-grade security and scalability.

Organizations with an existing Azure subscription can purchase GitHub products via self-service, directly through Microsoft Sales or via Microsoft Cloud Solution Providers and can adjust the number of GitHub seats as needed to ensure efficient usage. Additionally, eligible organizations may take advantage of the Microsoft Azure Consumption Commitment (MACC) and Azure Commitment Discount (ACD). 

Companies can now spin-up a GitHub instance directly from the Azure Portal and connect their Microsoft Entra ID with GitHub to facilitate user management and access control. With an Azure subscription, you have all the necessary tools for creating an intelligent AI application, including access to GitHub’s complete range of services like repositories, Actions, Advanced Security, and Copilot. This makes it incredibly simple and efficient to give developers everything they need to build and deploy AI applications at scale.

We invite you to experience the power of this integrated end-to-end development experience. New customers can explore these capabilities with a free 30-day trial of GitHub Enterprise. 

We can’t wait to see what you will build with GitHub and Azure. 

• Learn more about GitHub Models Launch. 
• Explore and experiment with Azure AI models in GitHub. 
• Get deeper technical details on GitHub Models. 
• Use GitHub Codespaces to setup a prototype from your repo. 
• Learn more about the free 30-day trial of GitHub Enterprise. 

The post Accelerating AI app development with Azure AI and GitHub appeared first on Microsoft AI Blogs.

• Serverless fine-tuning for Phi-3-mini and Phi-3-medium models enables developers to quickly and easily customize the models for cloud and edge scenarios without having to arrange for compute.
• Updates to Phi-3-mini including significant improvement in core quality, instruction-following, and structured output, enabling developers to build with a more performant model without additional cost.
• Same day shipping earlier this month of the latest models from OpenAI (GPT-4o mini), Meta (Llama 3.1 405B), Mistral (Large 2) to Azure AI to provide customers greater choice and flexibility.

Unlocking value through model innovation and customization  

In April, we introduced the Phi-3 family of small, open models developed by Microsoft. Phi-3 models are our most capable and cost-effective small language models (SLMs) available, outperforming models of the same size and next size up. As developers look to tailor AI solutions to meet specific business needs and improve quality of responses, fine-tuning a small model is a great alternative without sacrificing performance. Starting today, developers can fine-tune Phi-3-mini and Phi-3-medium with their data to build AI experiences that are more relevant to their users, safely, and economically.

Given their small compute footprint, cloud and edge compatibility, Phi-3 models are well suited for fine-tuning to improve base model performance across a variety of scenarios including learning a new skill or a task (e.g. tutoring) or enhancing consistency and quality of the response (e.g. tone or style of responses in chat/Q&A). We’re already seeing adaptations of Phi-3 for new use cases.

Microsoft and Khan Academy are working together to help improve solutions for teachers and students across the globe. As part of the collaboration, Khan Academy uses Azure OpenAI Service to power Khanmigo for Teachers, a pilot AI-powered teaching assistant for educators across 44 countries and is experimenting with Phi-3 to improve math tutoring. Khan Academy recently published a research paper highlighting how different AI models perform when evaluating mathematical accuracy in tutoring scenarios, including benchmarks from a fine-tuned version of Phi-3. Initial data shows that when a student makes a mathematical error, Phi-3 outperformed most other leading generative AI models at correcting and identifying student mistakes.

And we’ve fine-tuned Phi-3 for the device too. In June, we introduced Phi Silica to empower developers with a powerful, trustworthy model for building apps with safe, secure AI experiences. Phi Silica builds on the Phi family of models and is designed specifically for the NPUs in Copilot+ PCs. Microsoft Windows is the first platform to have a state-of-the-art small language model (SLM) custom built for the Neural Processing Unit (NPU) and shipping inbox.

You can try fine-tuning for Phi-3 models today in Azure AI.

I am also excited to share that our Models-as-a-Service (serverless endpoint) capability in Azure AI is now generally available. Additionally, Phi-3-small is now available via a serverless endpoint so developers can quickly and easily get started with AI development without having to manage underlying infrastructure. Phi-3-vision, the multi-modal model in the Phi-3 family, was announced at Microsoft Build and is available through Azure AI model catalog. It will soon be available via a serverless endpoint as well. Phi-3-small (7B parameter) is available in two context lengths 128K and 8K whereas Phi-3-vision (4.2B parameter) has also been optimized for chart and diagram understanding and can be used to generate insights and answer questions.

We are seeing great response from the community on Phi-3. We released an update for Phi-3-mini last month that brings significant improvement in core quality and instruction following. The model was re-trained leading to substantial improvement in instruction following and support for structured output. We also improved multi-turn conversation quality, introduced support for prompts, and significantly improved reasoning capability.

The table below highlights improvements across instruction following, structured output, and reasoning.

We continue to make improvements to Phi-3 safety too. A recent research paper highlighted Microsoft’s iterative “break-fix” approach to improving the safety of the Phi-3 models which involved multiple rounds of testing and refinement, red teaming, and vulnerability identification. This method significantly reduced harmful content by 75% and enhanced the models’ performance on responsible AI benchmarks. 

Expanding model choice, now with over 1600 models available in Azure AI

With Azure AI, we’re committed to bringing the most comprehensive selection of open and frontier models and state-of-the-art tooling to help meet customers’ unique cost, latency, and design needs. Last year we launched the Azure AI model catalog where we now have the broadest selection of models with over 1,600 models from providers including AI21, Cohere, Databricks, Hugging Face, Meta, Mistral, Microsoft Research, OpenAI, Snowflake, Stability AI and others. This month we added—OpenAI’s GPT-4o mini through Azure OpenAI Service, Meta Llama 3.1 405B, and Mistral Large 2.

Continuing the momentum today we are excited to share that Cohere Rerank is now available on Azure. Accessing Cohere’s enterprise-ready language models on Azure AI’s robust infrastructure enables businesses to seamlessly, reliably, and safely incorporate cutting-edge semantic search technology into their applications. This integration allows users to leverage the flexibility and scalability of Azure, combined with Cohere’s highly performant and efficient language models, to deliver superior search results in production.

TD Bank Group, one of the largest banks in North America, recently signed an agreement with Cohere to explore its full suite of large language models (LLMs), including Cohere Rerank.

At TD, we’ve seen the transformative potential of AI to deliver more personalized and intuitive experiences for our customers, colleagues and communities, we’re excited to be working alongside Cohere to explore how its language models perform on Microsoft Azure to help support our innovation journey at the Bank.”

Kirsti Racine, VP, AI Technology Lead, TD.

Atomicwork, a digital workplace experience platform and longtime Azure customer, has significantly enhanced its IT service management platform with Cohere Rerank. By integrating the model into their AI digital assistant, Atom AI, Atomicwork has improved search accuracy and relevance, providing faster, more precise answers to complex IT support queries. This integration has streamlined IT operations and boosted productivity across the enterprise. 

The driving force behind Atomicwork’s digital workplace experience solution is Cohere’s Rerank model and Azure AI Studio, which empowers Atom AI, our digital assistant, with the precision and performance required to deliver real-world results. This strategic collaboration underscores our commitment to providing businesses with advanced, secure, and reliable enterprise AI capabilities.”

Vijay Rayapati, CEO of Atomicwork

Command R+, Cohere’s flagship generative model which is also available on Azure AI, is purpose-built to work well with Cohere Rerank within a Retrieval Augmented Generation (RAG) system. Together they are capable of serving some of the most demanding enterprise workloads in production. 

Earlier this week, we announced that Meta Llama 3.1 405B along with the latest fine-tuned Llama 3.1 models, including 8B and 70B, are now available via a serverless endpoint in Azure AI. Llama 3.1 405B can be used for advanced synthetic data generation and distillation, with 405B-Instruct serving as a teacher model and 8B-Instruct/70B-Instruct models acting as student models. Learn more about this announcement here.

Mistral Large 2 is now available on Azure, making Azure the first leading cloud provider to offer this next-gen model. Mistral Large 2 outperforms previous versions in coding, reasoning, and agentic behavior, standing on par with other leading models. Additionally, Mistral Nemo, developed in collaboration with NVIDIA, brings a powerful 12B model that pushes the boundaries of language understanding and generation. Learn More.

And last week, we brought GPT-4o mini to Azure AI alongside other updates to Azure OpenAI Service, enabling customers to expand their range of AI applications at a lower cost and latency with improved safety and data deployment options. We will announce more capabilities for GPT-4o mini in coming weeks. We are also happy to introduce a new feature to deploy chatbots built with Azure OpenAI Service into Microsoft Teams.  

Enabling AI innovation safely and responsibly  

Building AI solutions responsibly is at the core of AI development at Microsoft. We have a robust set of capabilities to help organizations measure, mitigate, and manage AI risks across the AI development lifecycle for traditional machine learning and generative AI applications. Azure AI evaluations enable developers to iteratively assess the quality and safety of models and applications using built-in and custom metrics to inform mitigations. Additional Azure AI Content Safety features—including prompt shields and protected material detection—are now “on by default” in Azure OpenAI Service. These capabilities can be leveraged as content filters with any foundation model included in our model catalog, including Phi-3, Llama, and Mistral. Developers can also integrate these capabilities into their application easily through a single API. Once in production, developers can monitor their application for quality and safety, adversarial prompt attacks, and data integrity, making timely interventions with the help of real-time alerts.

Azure AI uses HiddenLayer Model Scanner to scan third-party and open models for emerging threats, such as cybersecurity vulnerabilities, malware, and other signs of tampering, before onboarding them to the Azure AI model catalog. The resulting verifications from Model Scanner, provided within each model card, can give developer teams greater confidence as they select, fine-tune, and deploy open models for their application. 

We continue to invest across the Azure AI stack to bring state of the art innovation to our customers so you can build, deploy, and scale your AI solutions safely and confidently. We cannot wait to see what you build next.

Stay up to date with more Azure AI news

• Watch this video to learn more about Azure AI model catalog.
• Listen to the podcast on Phi-3 with lead Microsoft researcher Sebastien Bubeck.

The post Announcing Phi-3 fine-tuning, new generative AI models, and other Azure AI updates to empower organizations to customize and scale AI applications appeared first on Microsoft AI Blogs.

GPT-4o mini allows customers to deliver stunning applications at a lower cost with blazing speed. GPT-4o mini is significantly smarter than GPT-3.5 Turbo—scoring 82% on Measuring Massive Multitask Language Understanding (MMLU) compared to 70%—and is more than 60% cheaper.¹ The model delivers an expanded 128K context window and integrates the improved multilingual capabilities of GPT-4o, bringing greater quality to languages from around the world.

GPT-4o mini, announced by OpenAI today, is available simultaneously on Azure AI, supporting text processing capabilities with excellent speed and with image, audio, and video coming later. Try it at no cost in the Azure OpenAI Studio Playground.

We’re most excited about the new customer experiences that can be enhanced with GPT-4o mini, particularly streaming scenarios such as assistants, code interpreter, and retrieval which will benefit from this model’s capabilities. For instance, we observed the incredible speed while testing GPT-4o mini on GitHub Copilot, an AI pair programmer that assists you by delivering code completion suggestions in the tiny pauses between keystrokes, rapidly updating recommendations with each new character typed.

We are also announcing updates to Azure OpenAI Service, including extending safety by default for GPT-4o mini, expanded data residency, and worldwide pay-as-you-go availability, plus performance upgrades. 

Azure AI brings safety by default to GPT-4o mini

Safety continues to be paramount to the productive use and trust that we and our customers expect.

We’re pleased to confirm that our Azure AI Content Safety features—including prompt shields and protected material detection— are now ‘on by default’ for you to use with GPT-4o mini on Azure OpenAI Service.

We have invested in improving the throughput and speed of the Azure AI Content Safety capabilities—including the introduction of an asynchronous filter—so you can maximize the advancements in model speed while not compromising safety. Azure AI Content Safety is already supporting developers across industries to safeguard their generative AI applications, including game development (Unity), tax filing (H&R Block), and education (South Australia Department for Education).

In addition, our Customer Copyright Commitment will apply to GPT-4o mini, giving peace of mind that Microsoft will defend customers against third-party intellectual property claims for output content.

Azure AI now offers data residency for all 27 regions

From day one, Azure OpenAI Service has been covered by Azure’s data residency commitments.

Azure AI gives customers both flexibility and control over where their data is stored and where their data is processed, offering a complete data residency solution that helps customers meet their unique compliance requirements. We also provide choice over the hosting structure that meets business, application, and compliance requirements. Regional pay-as-you-go and Provisioned Throughput Units (PTUs) offer control over both data processing and data storage.

We’re excited to share that Azure OpenAI Service is now available in 27 regions including Spain, which launched earlier this month as our ninth region in Europe.

Azure AI announces global pay-as-you-go with the highest throughput limits for GPT-4o mini

GPT-4o mini is now available using our global pay-as-you-go deployment at 15 cents per million input tokens and 60 cents per million output tokens, which is significantly cheaper than previous frontier models.

We are pleased to announce that the global pay-as-you-go deployment option is generally available this month, allowing customers to pay for the resources they consume, making it flexible for variable workloads, while traffic is routed globally to provide higher throughput, and still offering control over where data resides at rest.

Additionally, we recognize that one of the challenges customers face with new models is not being able to upgrade between model versions in the same region as their existing deployments. Now, with global pay-as-you-go deployments, customers will be able to upgrade from existing models to the latest models.

Global pay-as-you-go offers customers the highest possible scale, offering 15M tokens per minute (TPM) throughput for GPT-4o mini and 30M TPM throughput for GPT-4o. Azure OpenAI Service offers GPT-4o mini with 99.99% availability and the same industry leading speed as our partner OpenAI.

Azure AI offers leading performance and flexibility for GPT-4o mini

Azure AI is continuing to invest in driving efficiencies for AI workloads across Azure OpenAI Service.

GPT-4o mini comes to Azure AI with availability on our Batch service this month. Batch delivers high throughput jobs with a 24-hour turnaround at a 50% discount rate by using off-peak capacity. This is only possible because Microsoft runs on Azure AI, which allows us to make off-peak capacity available to customers.

We are also releasing fine-tuning for GPT-4o mini this month which allows customers to further customize the model for your specific use case and scenario to deliver exceptional value and quality at unprecedented speeds. Following our update last month to switch to token based billing for training, we’ve reduced the hosting charges by up to 43%. Paired with our low price for inferencing, this makes Azure OpenAI Service fine-tuned deployments the most cost-effective offering for customers with production workloads.

With more than 53,000 customers turning to Azure AI to deliver breakthrough experiences at impressive scale, we’re excited to see the innovation from companies like Vodafone (customer agent solution), the University of Sydney (AI assistants), and GigXR (AI virtual patients). More than 50% of the Fortune 500 are building their applications with Azure OpenAI Service.

We can’t wait to see what our customers do with GPT-4o mini on Azure AI!

¹GPT-4o mini: advancing cost-efficient intelligence | OpenAI

The post OpenAI’s fastest model, GPT-4o mini is now available on Azure AI appeared first on Microsoft AI Blogs.