Improve AI generated responses by grounding prompts with your enterprise data tables, now in Public Preview

As companies are rapidly adopting Copilot and conversational AI tools, they are increasingly introducing prompts as a foundational element to integrate generative AI capabilities within Microsoft Power Platform solutions, including Power Apps, Power Automate, and Copilot Studio. With Prompt Builder, already generally available, anyone can use natural language to instruct the GPT model to behave in a certain way or to perform a specific task. Typical scenarios include the ability to generate content analysis, summarize, classify, extract or transform information data.

Today, we are excited to announce the public preview of data grounding with Dataverse tables for Prompt Builder in Microsoft Copilot Studio, a new capability that helps makers create reliable and efficient AI solutions at enterprise scale.  With data grounding, customers can add a dynamic knowledge source to their prompts, using the model of Retrieval Augmented Generation (RAG), to enhance the underlying AI model’s capabilities from being a general-purpose model to an enterprise specific tool that operates within the business context of an enterprise customer.

Learn more about prompt builder:

• Get started with creating your own prompt to generate responses that suit your specific business need.
• Use your prompt in Power Apps | Microsoft Learn
• Use your prompt in Power Automate | Microsoft Learn

Infuse AI into your application’s business logic with AI-powered low-code plugins

Low-code plugins enable any maker to build complex AI-powered solutions in the Power Platform using the PowerFx expression language.  Capgemini New Zealand, A global leader in consulting, technology services and digital transformation, saw the potential of low-code plugins from their announcement at Microsoft Build 2023 and quickly began upskilling on them in their local Dynamics 365/Power Platform practice. Low-code plugins provide the ability to create real-time, secure and scalable business logic for their customers, all without the underlying cost and added infrastructure of writing traditional code. As a result, Capgemini NZ can create value for their clients, solving their business challenges faster than ever before.

I’m particularly excited by the integration potential with Copilot Studio. Our clients are asking us to help them apply generative AI to their business challenges. Low-code plugins fill a crucial gap of automating business logic between the large language model in Copilot and the client’s underlying data. As such we’ve been showcasing our learnings from low-code tools such as low-code plugins in our regular internal Communities of Practice so that we’re ready to hit the ground running when this feature is generally available.
-Richard Anness, Power Platform Practice Lead, Capgemini NZ 
With low-code plugins now available in Microsoft Copilot Studio, customers can embed advanced business logic into their Copilots and AI-powered applications including triggering workflows & data updates within business processes modeled within Microsoft Dataverse. In addition to the existing set of supported Power Fx operators, variables and formulas, we are also excited to introduce low-code plugin support for 5 new Power Fx AI functions:

• AIReply: Drafts replies to messages (e.g., customer reviews of a product).
• AISummarize: Summarizes text (e.g., email messages or document content).
• AISentiment: Detects sentiment (positive, negative, neutral) in text (e.g., customer reviews).
• AIClassify: Classifies text into predefined categories (e.g., Problem, Billing, How To, Licensing).
• AITranslate: Translates text from one language to another (source language is automatically detected).

After the development of low-code solutions using low-code plugins, monitoring their execution becomes an integral part of maintaining a high-quality, reliable, and performant solution. It empowers developers and administrators with the insights needed to ensure the best possible experience for end-users. We are delighted to announce that we are now enabling the preview of the new plugin monitoring report  inside Dataverse Accelerator with two key capabilities:

• Centralized log viewer: Access and view trace logs from Dataverse Custom APIs, low-code plugins, and pro-code plugin executions in an environment from one central location.
• Filtering capabilities: Conveniently filter log history to quickly find logs relevant to debugging needs.

For more information on how to try out low-code plugins:

• In Power Apps: Use a prebuilt prompt in a Dataverse low-code plug-in in Power Apps (preview) | Microsoft Learn
• In Copilot Studio: Create Dataverse low-code plug-ins to use with a copilot (preview) – Power Apps | Microsoft Learn

The post Infuse any AI solution with your enterprise data, prompts and plugins appeared first on Microsoft Power Platform Blog.

Today, we are excited to launch a new set of prebuilt AI functions that can be easily added to a low-code solution without having to engineer a custom prompt. These prebuilt functions provide a great starting point to combine the power of LLMs and power of low-code, but they are just scratching the surface of what is possible. We will also cover how Projectum, a Microsoft partner, was able to leverage prompt builder, Microsoft Power Platform no-code prompt engineering interface, to build and incorporate custom, generative AI actions into their low-code project management solution. If you do choose to try out custom prompts, then please check out the newly published AI Builder prompt engineering guide for some great strategies on how to get started.

Start with prebuilt functions to quickly implement generative AI capabilities 

Starting today, prebuilt AI functions can be used to quickly incorporate generative AI into your Microsoft Power Apps solutions or your Microsoft Power Automate flows without having to engineer your own prompt. Then, if needed, makers can utilize the prebuilt prompt template and customize it to their needs in Microsoft Copilot Studio. 

These readily accessible AI functions are designed to facilitate common AI-driven tasks across the platform. Whether it’s summarizing emails or conversations, categorizing complaints or reviews, extracting critical information from extensive text, crafting responses or drafts to specific messages, or discerning the sentiment of product evaluations, AI functions provide quick-start access to generative AI capabilities.

Here are some examples of how you can use these ready-made prebuilt prompts today:

• AI Summarize: Summarize text from an email or document and add the summary to a Microsoft Dataverse table record. Your application end users would benefit from the summary and can quickly evaluate the next steps.
• AI Classify: Quickly classify customer inquiries into your own categories to ensure the inquiry is assigned to the correct team. You could use this from your custom copilot to route the inquiry and reduce customer frustration from being redirected multiple times.
• AI Reply: Help staff move past writer’s block by drafting a reply message. For example, draft a reply to a customer’s review of a product.
• AI Extract: Automate data augmentation by extracting data like phone numbers or names of people from incoming correspondence. This saves time and improves data quality by prepopulating the data.
• AI Sentiment: Build in the ability to your apps or automations to quickly check if text is positive, negative, or neutral. For example, for the staff reviewing customer feedback, you could provide an indicator in the application of the sentiment.

Projectum: Empowering project management with AI-driven insights

Projectum is a leader in strategic portfolio management, delivering solutions that streamline project management and yield superior outcomes. Specializing in portfolio management, resource and capacity demand management, and time registration, Projectum is the go-to partner for businesses aiming to turn strategy into successful execution. With Projectum, organizations enhance project delivery, optimize efficiency, and boost their bottom line. 
Last year, Projectum unlocked new product opportunities with generative AI capabilities in its Power PPM (project portfolio management) product. Using AI Builder’s prompt builder, Projectum allows project managers to create strengths, weaknesses, opportunities, and threats (SWOT) analysis, identify project risks, summarize project reports, and generate real-time project status updates—all using natural language. With AI Builder’s custom prompts, Projectum can easily control the generative response, invoke prompts via Power Apps, and ensure governance through integrated monitoring. AI Builder also enables Projectum to ground the generative AI’s responses in enterprise knowledge, thereby delivering accurate and relevant responses to its customers. 

“AI Builder and Copilot in Power Platform products have been transformative for Projectum. Our users can now wield powerful AI capabilities right within our Power PPM solution. The ability to create custom prompts, invoke AI-driven SWOT analysis, and generate new types of insights has elevated our product’s value proposition.
It’s a win-win for us and our customers, and we see AI Builder supporting all three categories of Project Management AI use cases: automation (improving approval flows and data summarization), assistance (draft schedules and risks logs), and augmentation (business case validation)” 
— Peter Charquero Kestenholz, Founder and Head of Innovation & AI at Projectum 

Best practices for combining prompts and low-code: AI Builder prompt engineering guide

Prompts act as the foundational elements for integrating generative AI capabilities within Microsoft Power Platform, embedding seamlessly into its comprehensive suite, including Power Apps, Power Automate, and Copilot Studio. Facing unlimited possibilities for content generation and data transformation, low-code makers and developers can reference the AI Builder prompt engineering guide for creating efficient and accurate GPT prompts. The following example visualizes a GPT prompt that makers could use to analyze a complaint from a customer on delivery of their order. Makers can incorporate the customer complaint and the shipping policy as input parameters.

Prompts empower makers to develop bespoke AI functions tailored to business needs and binds them as actions to controls in Power Apps. Leveraging Microsoft Power Fx, makers can effortlessly activate these prompts through various user interactions, such as page refreshes or button clicks. The new prompt engineering experience allows the addition of input parameters to prompts, providing a more contextually relevant data input to LLMs at runtime, thereby eliciting more precisely aligned responses from GPT models.

Prompts find significant utility in Copilot Studio, where they can operate as custom AI plugin actions using just natural language. These plugins serve as a powerful method to extend the native capabilities of your copilots, ensuring their responses are finely tuned to meet specific business requirements. Copilots are designed to call upon these custom plugins to fulfill tasks in response to end-user prompts, enabling a custom copilot experience that leverages these uniquely defined plugin actions for relevant user queries.

Furthermore, prompts can play a crucial role in Power Automate, where they simplify the creation of AI-driven actions for intelligent workflow design. By enabling the extraction of data from unstructured sources and transforming it into structured output, prompts enhance the efficiency and intelligence of downstream processing, making them an indispensable tool for building and infusing generative AI capabilities throughout the Microsoft Power Platform ecosystem.

How you can get started with prompt builder in AI Builder

• Try prompt builder
• Prompt engineering guide
• Prompts overview documentation
• Learning videos:
  • Overview video
  • GPT prompts with Excel video
• Governance: learn about administrating generative AI features and cross region availability

The post Prompt engineering made easier with AI Builder appeared first on Microsoft Power Platform Blog.

• There were several live sessions (now available on-demand) for participants to learn about the Copilot and AI features of Power Platform.
• Participants followed a Learn Collection for help in getting started.
• Over 100 project submissions were received with 4 winners chosen across the different categories.

Summary of the participants and project submissions from around the world.

The categories for the hackathon were:

Grand prize winner – The best of the best! Rewards the solution that meets all judging criteria, wows judges, and has potential real-world value for Power Platform Users.

Best AI-powered solution – Rewards the solution that integrates AI in an innovative way.

Best Better Together Use Case – Rewards the solution that uses the Power Platform and other Microsoft products in an interesting way.

Best Diversity, Equity, and Inclusion (DEI) Use Case – Rewards the solution that helps make the Power Platform community more diverse, equitable and inclusive.

The submissions were innovative and demonstrated how AI can be helpful across numerous use cases. Be inspired this year in building AI-powered solutions with Power Platform by checking out these community submissions below:

• Education – Teaching Accelerator
• Family Support – Make Life Easy | Parents Advisor
• Healthcare – Romão’s Homecare | Medical Assistant
• Human Resources – The Relocation Game | Seeker
• Not-for-profit – Migrant Worker Case Management
• Information Technology – Prompt Wagon | CodeScribe

Education

Teaching Accelerator

Teachers spend thousands of hours each year planning the curriculum for their students. Elliot Fraser developed a Power Platform solution, known as the Teaching Accelerator, to expedite the process for teachers in creating a curriculum for their students. The solution utilizes OpenAI’s ChatGPT service where teachers can interact with it through a Copilot embedded in the model-driven app to create lessons, modify the lessons created, and create an entire curriculum of lessons for a topic.

Teachers can interact with a Copilot within the model-driven app for creating lessons and a curriculum for students.

The solution consists of the following main features:

• Model-driven app which is the primary end user interface for the teachers to plan their lessons and a curriculum for students.
• Custom pages built using Power Apps canvas apps are embedded in the model-driven app to help guide teachers to create a curriculum of lessons, create lesson plans and view, modify and assign existing lessons.
• Teachers enter their questions into a Copilot that was built using Microsoft Copilot Studio to receive AI-generated suggestions for a curriculum and lesson plans. The Copilot appears in a pane within the model-driven app through a custom page.
• Power Automate cloud flows in the background interact with OpenAI’s ChatGPT service to relay the suggestions to the teacher.
• When the teacher is satisfied with the suggestions from Copilot, they can ask Copilot to generate the curriculum and lesson plans in the model-driven app. Depending on where the teacher executes this in the app, Copilot calls a function in OpenAI to create an array of the curriculum or lesson plans as a JSON object. This output is then saved in Dataverse using a cloud flow by transforming it into rows of data which the teacher sees as the curriculum or lesson plans in the model-driven app.

What we loved

Elliot’s Teaching Accelerator solution won the Grand Prize winner category of the hackathon due to the impact it would have in the education space by reducing the hours teachers spend annually. Teachers can quickly create a curriculum and lesson plans for students in minutes with the help of generative AI and Power Platform.

Since the hack, Elliot has an updated version of the Teaching Accelerator where he’s made enhancements based on feedback from teachers who have been testing the solution. For an overview of the updated solution version, watch Elliot’s presentation from a previous The Low Code Revolution episode.

Family support

Make Life Easy

To provide everyday support for parents of autistic children, Raghav Mishra developed a Power Apps canvas app to create structured daily routines. As a parent to an autistic child, Raghav recognized their son thrived when steps of their daily routines were illustrated visually by his wife with the corresponding written text. Inspired by his wife’s dedication in helping their son, Raghav turned to the Power Platform combined with OpenAI’s services to develop a Power Apps canvas app that uses generative AI to help create the text-based steps of the tasks and design the corresponding images of the steps. The Make Life Easy app can be used by parents to establish routines the children can follow.

Parents interact with generative AI in a canvas app to create written and visual steps for tasks children follow.

The solution consists of the following main features:

• Parents use the canvas app to interact with OpenAI’s ChatGPT service to generate a list of text-based steps for tasks, such as the steps children follow to put their shoes on.
• The text-based steps for each task are saved in Dataverse, which are displayed in the gallery of the app.
• Parents can select a step from the gallery and interact with OpenAI’s DALL-E service to create the corresponding visual images for all steps, such as asking for an image of a child sitting on a chair as a step the child follows to tie their shoelaces.
• Once all images of the steps for the tasks are created, they can be added to the schedule board for children to follow. Parents can then assign the tasks to their children.
• Children see their assigned tasks in their personal daily schedule board within the app. They can follow each of the steps through the visual and written instructions generated by AI.

What we loved

Raghav’s Make Life Easy solution won the Best Diversity, Equity and Inclusion (DEI) Use Case category for designing a solution that supports parents of autistic children and their daily routines. Parents can interact with generative AI in a Power Apps canvas app to define the steps of a task and create the corresponding visuals. Children can follow these AI-generated steps through written and visual comprehension in the app.

Parents Advisor

To help parents encourage and teach children every day, Richard Li created a Power Apps canvas app known as Parents Advisor, where parents can seek additional support for advice on educating their children. Several Azure OpenAI services and OpenAI services were used together with Power Platform to provide tailored recommendations based on the individual in-app user profiles. Children can also have one-on-one interaction with AI, where the AI-generated answers are verbally read to them from the app itself. The Parents Advisor app can be used daily by both parents and children for written and aural comprehension.

Parents and children interact with generative AI to create tailored learning experiences in the canvas app.

The solution consists of the following main features:

• Parents can select from built-in prompts in the canvas app such as how to encourage their child to sleep independently. They can also enter their own custom prompts directly in the app, where Azure OpenAI’s ChatGPT service will respond with an answer which the parent can save for future reference.
• Parents also have the option to provide their prompt verbally through the built-in microphone control of canvas apps. Their verbal prompt is then transcribed to text using OpenAI’s Whisper speech-to-text service and can also be translated to more than 50 languages.
• All responses from Azure OpenAI’s ChatGPT service can be transcribed from text-to-speech using the Speech service from Azure AI. The style of the voice can be altered using Speech Synthesis Markup Language (SSML) such as the effect, style and multilingual voices. This means the AI advisor can “talk” to children by reading the responses to prompts through the built-in audio control of canvas apps.
• Most of the parent end-user features are also replicated for the children end-users where they too can interact with the Parent Advisor app and access the list of built-in topic prompts. Or they can enter their own custom prompts and have the app read the responses generated.
• One of the fun features of the app is using Azure OpenAI’s ChatGPT service to author bedtime stories for children based on pre-selected criteria of the character (e.g. Cinderella), language, genre, length and the requested story details. The parent or child also has an option to create their own bedtime story by adding their own prompts. Based on the prompts selected, the app will generate a bedtime story which can be read by the parent or child, or the app can read it to them through the native audio control of canvas apps.

What we loved

It was very cool to see the solution utilize the Speech service from Azure AI to transcribe the responses from text-to-speech in relation to the prompt entered by the parent or child. We also liked it was diverse and inclusive in providing multi-language support to cater for different languages of families.

Healthcare

DISCLAIMER: Applications built using Microsoft Power Platform are not designed or intended to be a substitute for professional medical advice, diagnosis, treatment, or judgment and should not be used to replace or as a substitute for professional medical advice, diagnosis, treatment, or judgment.

Romão’s Homecare

To assist doctors and nurses with scheduled visits to patient’s homes, the Romão brothers – Douglas and Renato, built several apps by combining low code and pro-code techniques. A Power Apps canvas app, known as the Romão’s Homecare mobile app, is used by the doctors and nurses for visibility of daily scheduled in-home patient visits and to record details of their assessment from their devices. The information entered in the Romão Homecare app is saved in Dataverse and is accessible through a Power Apps model-driven app.

Doctors and nurses capture their assessment of patients during in-home visits with assistance from generative AI in a canvas app.

The solution consists of the following main features:

• The canvas app is the primary end user interface for doctors and nurses where they can see a list of their upcoming scheduled in-home visits. The built-in interactive map control displays the location of their patient visits.
• The native camera control of canvas apps enables them to capture photos during their assessment with the patient. These photos are uploaded into Azure Blob Storage and a sentiment score based on the images uploaded is generated using a Python web app via Azure App service.
• Audio recordings can also be captured using the native microphone control in canvas apps where the audio is transcribed to text using OpenAI Whisper’s speech-to-text service.
• A simple Yes/No checklist is used which is submitted to OpenAI to provide a score of the visit.
• On completion of the visit, the overall score calculated by OpenAI indicates the wellbeing of the patient with a high score being good and a low score requiring further attention.
• A model-driven app displays the stored patient data in Dataverse where information can be modified if needed.

What we loved

Doctors and nurses taking notes during or after a patient visit takes time and the solution reduces this administrative effort by capturing audio recordings and transcribing them to text using OpenAI Whisper’s speech-to-text service. There’s no duplicate time spent re-entering their notes by typing into the app. The data being accessible through Dataverse is also convenient for the information to be modified if additional details need to be edited.

Medical Assistant

To help the medical industry interpret computerized tomography (CT) scans and magnetic resonance imaging (MRI) scans, Yevhenii Dementiev, Valentin Gasenko and Surkho Salamov built an app, known as the Detect Cancer app, to support physicians with reviewal of patient scans. The aim of the solution is to help physicians to distinguish between possible tumors and other anomalies, improving the precision of diagnosis and identifying tumors at various stages.

AI technologies used in their solution identifies potential tumors and anomalies to help physicians with diagnosis.

The solution consists of the following main features:

• Physicians use a canvas app where images from the CT and MRI scans can be uploaded using the built-in attachments control of canvas apps.
• Using AI Builder Object Detection Model, the uploaded images are analyzed and highlight the detected possible cancer regions on the CT scan and MRI scan.
• A Power Automate cloud flow sends the detected information from the AI model to OpenAI’s Completions service to generate medical conclusions for the physicians to predict the recovery of the patient.
• The physician can also dive further into the regions detected for analysis and comparison from previous CT and MRI scans of the patient. Based on the historic data, OpenAI can provide a prediction of the patient’s recovery from cancer.
• Dataverse is used to store all information about the patient scans and conclusions of the detected cancer regions.

What we loved

This is an incredible solution that supports a healthcare use case by assisting detection of cancer regions and medical conclusions using AI Builder Object Detection Model combined with OpenAI services. It was also nice to see the team include feedback in their video submission from their target audience of physicians. Their solution has demonstrated how Power Platform and AI together can be of secondary help to healthcare professionals.

Human Resources

The Relocation Game

Employee relocation can be overwhelming for employees in two ways: understanding what their organization’s local policies are and adjusting to a new city – especially if the employee is unfamiliar with the local surroundings. Denisa Mihai built a Power Apps canvas app known as The Relocation Game, to help employees with their move by interacting with a Copilot to learn about the local policies and help identify the city’s landmarks through AI Builder.

Employees interact with a Copilot embedded in a canvas app to learn more about their work benefits, the required documentation for relocation and can be connected to the HR team.

The solution consists of the following main features:

• A Copilot is embedded in a Power Apps canvas app which uses the built-in boost conversation capabilities where natural language processing is applied to find, collate and parse relevant information from the sites and summarize the search results into plain language for the employee.
• Employees interact with the Copilot within the Relocation app where they can ask about the company benefits, documentation required for their move to a new country, and request to be connected to the HR team for additional questions.
• An AI Builder object detection custom model was developed and trained to recognize city landmarks for employees to become familiar with their city’s surroundings. The trained object detection custom model is embedded into the Relocation app.
• Using the built-in camera and image controls of canvas apps, the employee takes photos and uploads these into the object detection custom model for it to recognize and present the name of the landmark to the employee.
• Employees can also create their own collection of landmarks by taking photos and adding them to their list of landmarks in the Relocation app.

What we loved

Employees can feel supported for their relocation to another city by interacting with a Copilot with boost conversation capabilities enabled to help with HR related enquiries. We liked that there was an element of “discovery” built into the app for employees to take photos and verify a landmark of the city using the AI Builder object detection custom model.

Seeker

Recruiting new talent is often time-consuming during the phases of advertising, finding candidates and interviewing candidates. Isabelle Gaboc, Daniel Kerridge, and Jady Mulqueeney developed an end-to-end solution to reduce the administrative hours spent. A Power Pages site was built to handle the job listings and the initial interview process with candidates. Candidates can choose to have an automated interview within the Power Pages site where interviews are tailored to the candidate. Azure OpenAI’s ChatGPT capabilities in AI Builder generates open ended questions in alignment to the job description and information provided in candidate’s resumes, and OpenAI’s ChatGPT service is utilized during the interview for an ongoing conversational dialog with the candidate.

Open ended interview questions generated with the AI Builder connector in Power Automate are used in the AI-led interviews with candidates.

The solution consists of the following main features:

• An AI-generated Power Pages site was created using Copilot that enables candidates to review all available positions. They can apply by uploading their resumes and begin the automated interview process shortly after.
• Several cloud flows were built to extract text from the uploaded resume where the data is parsed and analyzed using the AI Builder text recognition prebuilt model to convert the resume to text.
• The automation continues by converting the job description to text using Azure OpenAI GPT capabilities in AI Builder to format the resume text from the previous step by removing unnecessary information and only keep the information aligned to the job description. As the last automation step, AI Builder’s OpenAI GPT capabilities are used again to generate open ended interview questions based on the refined resume text and job description.
• The candidate interviews are conducted within a webpage in the Power Pages site through a virtual call with a .NET bot integrated with Azure Communication Services. The interview questions generated by AI Builder from the cloud flow are retrieved by the .NET bot from the webpage using liquid, a markup language in Power Pages.
• Using Azure AI services text-to-speech, the .NET bot asks the candidate the interview questions generated by AI Builder.
• In return, the candidate can respond verbally to the .NET bot where the audio dialog is translated from speech-to-text using Azure AI services.
• The .NET bot will then send the translated text to OpenAI’s ChatGPT service with a prompt to respond accordingly for the interview. The output used for the response is translated once again to speech and relayed aurally back to the candidate. This enables a constant conversational AI dialog loop between the bot and the candidate.

What we loved

Their Seeker site solution won the Best AI-powered solution category of the hackathon due to the impact it would have for Human Resources in allowing candidates to use a Power Pages site to upload their resume for a job application and use OpenAI’s ChatGPT services to create a list of interview questions, followed by conducting the interview through a virtual voice call – all without any human intervention.

For a more detailed overview of the solution watch the Seeker team’s presentation from a previous Microsoft 365 & Power Platform weekly call.

Not-for-profit

Migrant Worker Case Management

Filipino citizens who work abroad sometimes face difficulties in understanding their rights in the country they work in. A case management solution was developed by Carmina Symaco, Marcos Antonio Abrematia and Jayson Espadero to support migrant workers in solving their issues by interacting with a Copilot embedded in a Power Pages site. All questions are created as a case in Dataverse where additional support personnel can review and respond to the person working abroad.

Migrant workers can paste an excerpt from their employment contract into the page in the Power Pages site to generate details that are more easily understood.

The solution consists of the following main features:

• A Copilot built using Microsoft Copilot Studio utilizes the built-in boost conversation capabilities where local websites of a country are associated to the Copilot to generate responses to questions asked by migrant workers.
• The Copilot is embedded in a Power Pages site where the user provides details of their role, the country they are working in and selects from a list of topics that resemble their issue to gain further assistance. Based on the information provided, Copilot returns the desired information extracted and collated from local websites of the country the Filipino citizen works in.
• Excerpts from their employee contract can also be parsed and analyzed using Power Automate cloud flows combined with OpenAI’s ChatGPT service to interpret the details and generate an answer that can be easily understood by them.
• All answers are captured as a case where support personnel can review them in a canvas app. An embedded Copilot in the app assists the support personnel to generate responses for the cases.
• AI Builder text recognition prebuilt model was used to classify the cases based on the description the migrant worker provided to the Copilot in Power Pages. In the app, there is a view for cases where the tag identified by AI Builder and the selected issue type by the migrant worker do not match. The administrator reviews the conversation history between the migrant worker and Copilot to correct the classification of the case.

What we loved

The solution was supportive of migrant workers by using a Copilot with boost conversation capabilities to answer queries and use OpenAI’s ChatGPT service to help them understand their employment contracts. Since it’s embedded in a Power Pages site, they’re able to access the Copilot 24/7 which is of benefit to workers across different time zones. The team also included some neat features for the support personnel and one that stood out was using AI Builder text recognition prebuilt model to verify the classification of the case was correct which will help with the long-term quality of the data.

Information Technology

Prompt Wagon

Artem Chernevskiy, Katerina Chernevskaya, and Nikita Chernevskiy recognized that prompt engineering is a growing new skill for organizations of all industries to utilize. Prompt engineering is natural language inputs or queries users provide to Large Language Models (LLMs) to produce the desired outputs or responses. To help organizations become skilled in prompt engineering, a Power Apps canvas app known as Prompt Wagon, was developed where users can select from a list of more than 100 pre-defined prompts and practice their knowledge of prompts by interacting with Azure OpenAI’s ChatGPT service within the app. The aim of the app is to help end users of all levels – beginners or advanced to learn how to construct their prompts and grow their prompt engineering skills.

Users can practice prompt engineering with the canvas app and gain more confidence in using generative AI.

The solution consists of the following main features:

• Users can select from 100+ pre-defined prompts across three categories of Business, Data and Education in the canvas app to help them exercise prompt engineering with the Azure OpenAI ChatGPT service. It also includes multi-language support in English, German and French.
• A custom connector was built to query the Azure OpenAI ChatGPT service from the canvas app. This enables users to interact with the services from within the app.
• Dataverse is used to store the text, images, prompts and translations of the German and French languages. All of these are surfaced onto the canvas app from Dataverse.
• Built-in app settings are available to modify the prompts and query parameters directly within the app. The updated settings are adhered to when end users next enter their prompts into the app.

What we loved

The app will help organizations be better equipped with prompt engineering through interacting with generative AI in the app. It was great to see the team build a custom connector to make the actions reusable across their solution that query the Azure OpenAI ChatGPT service from the canvas app. The use of Dataverse for the high volume of text, images, prompts and translations of several languages will support the solution to scale as the library of prompts grow over time with use.

CodeScribe

Developers often review code authored by others, with a common scenario being legacy code, where there can be little documentation or comments in the source code. To help automate code reviews, Robert Perillo built a Power Apps canvas app known as CodeScribe for developers to copy and paste code snippets into the app. A summary of the code is provided using Azure OpenAI’s Completions service to assist developers with understanding the code.

Developers can perform code reviews instantly by entering code snippets into a Power Apps canvas app and Azure OpenAI will summarize the code.

The solution consists of the following main features:

• Developers can paste code snippets into the canvas app and when they submit the code for review within the app, a Power Automate cloud flow executes to call the Azure OpenAI Completions service which analyzes the code snippet.
• Azure OpenAI then formats the code into two outputs, the first being a code summary that provides an overview of what the code does, and the second is a commented code block that explains each step of the code in detail for documentation purposes.
• The Power Automate cloud flow sends the formatted code to the canvas app for developers to review the summarized code and view a running commentary on the lines of code.

What we loved

Robert’s CodeScribe app won the Best Better Together Use Case category to help developers with the task of reviewing and documenting code by combining Power Platform and Azure OpenAI Completions service. We liked that the summaries are produced within minutes, reducing the time spent on code reviews. The solution breaks down what the code does by first summarizing it in a short paragraph, followed by a detailed explanation for each of the steps in the code.

Share your story with us!

Do you have a story or experience to share? We are excited to learn more about how individuals and organizations are using Power Platform with generative AI and Copilot capabilities! Submit your story at https://aka.ms/ShareAIStory

Learn more about Microsoft Power Platform and Copilot

• Understanding prompt engineering
• Create an app using AI Copilot
• Get started with Copilot in cloud flows
• AI-powered and Copilot features in Power Pages
• Build an AI-based Copilot with Microsoft Copilot Studio
• Quickstarts with Azure OpenAI Services

The post How Power Platform Global AI #HackTogether inspired AI-powered solutions for real-world use cases appeared first on Microsoft Power Platform Blog.

A simpler way to manage diabetes

In just a few days, Haniel had come up with a solution. He created a mobile app using Power Apps that simplifies many of the time-consuming tasks required to manage diabetes. The app is called “Munching Red Panda”, named after Eden’s favorite animal. (She came up with the ‘munching’ part.)

A simple interface makes it easy for Eden to enter blog sugar levels on her phone (date and time are tracked automatically). This includes a Power Automate flow that triggers an alert if Eden enters a dangerously low blood sugar level. The alert is sent to Haniel and his wife as an email or text message, along with a map that shows Eden’s location. Another function tells Eden which type of insulin to take (she has two types, depending on the time of day). The app also tracks when an insulin pen is about to expire and provides a reminder when a pen needs to be changed.

The app also helped simplify an even more tedious task – food monitoring and carb calculations. Initially, Haniel would have to manually enter several pieces of information into the app for each meal: the food Eden was about to eat, the amount, and the units. Eventually, he built up a catalog of this data which he displayed in the app. In this way, Eden only needed to select the food, enter an amount – and the app would calculate total carbs. Time saved.

Saving time with generative AI

The app turned out to be an ideal solution, especially when Eden was younger – and her diet was simpler. But as she grew older, went out more to restaurants, and tried new foods, the food catalog that Haniel created needed to be manually updated with each new food. So, Haniel added another innovative update – this time, using the generative AI features in Power Automate that automates the carb count for just about any food.

Haniel started by selecting a pre-built text generation model. He then configured the ‘Create text with GPT’ action by giving it a prompt with the food entered by Eden. The flow takes the data, structures it as a JSON object, and sends it to Edamam, a nutrition website which looks up each food and returns many nutritional parameters, including the total carb count. What’s even better is that the action will work off data entered by either voice or text. “AI Builder made it surprisingly easy to add exactly the AI model and functionality I needed. In just a few minutes, I was able to dramatically improve the way my app works,” says Haniel.

Now, all Eden has to do is press the microphone on her phone and say the food and quantity that she wants to eat. For example, she could say: I want one slice of watermelon, a banana and 300 grams of cheese – and the app automatically displays the total carbs.

Benefits like this had a huge impact on the family’s everyday lifestyle. As Haniel explains, “Recording and calculating data – something that could take up to 10 minutes per meal — could now be done in a few seconds. That’s a significant time savings – especially when you have to do this for every meal.”

Eden was particularly impressed with the AI capabilities of the app. “I have seen AI work in my robotics courses – but was surprised at how well it worked within the app.” She also appreciates the time that the app gives back to her. As she points out, “I would often start eating later than everyone else because we would have to calculate all the carbs manually.” With the app, she can eat right on time with her family and friends.

Sharing their success with others

Another happy ending to this story is that, now that Eden has access to a glucose monitor and insulin pump, she no longer needs many of the functions built into the app. But as Haniel says, “There are thousands of families who simply can’t afford this equipment – even with insurance, which often doesn’t cover the full cost.” This is why Haniel is working to make the app more widely available.

In addition, Haniel is talking to doctors at the Hospital for Sick Children in Toronto, Canada (where insulin was invented 101 years ago) about how the app could be used to support their diabetes research. In addition to collecting food information, Haniel envisions using the app to collect other data such as demographics, activity levels, health conditions – all information that could help generate new insights and breakthroughs.

In the meantime, he’s just pleased that he could help make life easier for his daughter, Eden – all thanks to some inventive thinking and a simple app on Power Platform.

Thanks 

Share your story with us!

Do you have a story or experience to share? We are excited to learn more about how individuals and organizations are using Power Platform with generative AI and Copilot capabilities! Submit your story at https://aka.ms/ShareAIStory

MEDICAL DEVICE DISCLAIMER. Applications built using Microsoft Power Platform are not designed or intended to be a substitute for professional medical advice, diagnosis, treatment, or judgment and should not be used to replace or as a substitute for professional medical advice, diagnosis, treatment, or judgment. 

The post Maker uses Power Platform to help his daughter manage diabetes appeared first on Microsoft Power Platform Blog.

Equinor, a forward thinking energy company based in Norway, has forecasted the estimated annual value from a solution delivered by their DigiTEAM and citizen developers at more than 500 million NOK. As of the currency exchange rate on November 16^th, 500 million NOK equals to more than 49 million US Dollars. These savings do not include the Power Platform projects delivered by their IT Team.

The company is on a journey to net-zero emissions through optimizing its oil and gas portfolio, accelerating growth in renewables, and pioneering developments in carbon capture and hydrogen. Given the significant changes required in operations to move to renewable energy, the technology leadership at Equinor turned to Power Platform to enable rapid development of enterprise-level solutions that would deliver immediate business value.

“The business impact and benefits of the resulting Power Platform solutions truly speak for themselves, so our focus is to promote them effectively.” -Rebecca Brekke: Center for Enablement Task Lead

The teams at Equinor have digitally transformed multiple use cases including those in the logistics and supply chain area.

Svein Inge Heggøy, Discipline Responsible Logistics and crane operator at Equinor, had no prior Power Platform experience and built a “Crane and Lifting” Power Apps app. This app enabled users to easily make crane lift requests, a process that had historically been ad hoc and decentralized with requests coming via email using Excel or SharePoint lists, and via phone calls and handwritten notes. Hans Martin Berge, who has an engineering background and is a Task Lead at Equinor’s DigiTEAM, partnered with Svein Inge to integrate Dataverse and replace SharePoint for data storage so it could be scaled and deployed to other Equinor installations. Now, crane lift operations run more efficiently and effectively with a centralized tool to manage requests.

Another example is the “Oil Country Tubular Goods (OCTG) Dashboard” solution that started out as single Power App utilizing AI Builder capabilities to automatically count pipes in photos uploaded by users from tablets and mobile phones. The solution expanded to centralize the critical data in the supply chain in Dataverse with orders’ creation and updates via a Power Apps app, automated notifications and reminders on pending/delayed/confirmed orders via Power Automate and visualization of logistical trends, risks, and areas for improvement in Power BI.

Read more about how Equinor approached and implemented governance, provided training and best practice guidance on Power Platform and, are furthering fusion development in their organization.

“Power Platform’s low-code technology fosters better innovative relationships between business experts and IT experts and avoids the ‘shadow IT’ pitfall many organizations operate within.” -Per Kåre Foss: Vice President Enterprise Infrastructure Platforms 

The post Equinor embraces Power Platform to increase process efficiency while cutting costs appeared first on Microsoft Power Platform Blog.

1. Use AI in your apps with Power Fx (Preview). Using Power Fx, makers can now use any AI model on any control in canvas apps.
2. Bring Your own AI model (GA). Companies can leverage existing investments in data science. Any AI model, built with any language, framework, or tooling, can now be used in Power Platform via AI Builder.
3. Image classification with Lobe (Preview). Domain experts can train image classification models on their computers with Lobe and upload those models to AI Builder to be used in Power Platform.
4. AI Builder starter capacity is now included in select Power Apps and Power Automate license plans.

Let’s walk through these in more detail.

Use AI in your apps with Power Fx

Until now, the only way you could use AI in your canvas app was through our five AI Builder controls. This worked, but the controls were limited in the ways you could customize them, and only supported five of our sixteen AI model types.

Now we are making it easier than ever to add intelligence to your Power Apps directly through Power Fx. This gives you access to all the pre-trained and custom AI models, and can be used with any control within your app.

Let me show how you can use AI with Power Fx with an example app I made to report car damage claims with a picture and description.

First, add the AI model you’d like to use as a data source to your app. For this example, I’m adding the model ‘Car Damage Classifier’ that I trained with Lobe.

Figure 1:  Add an AI model as a data source

Once the AI model has been added as a data source to your app, select any control and call the AI model through Power Fx to make a prediction. In this example, I’ve uploaded a car image for the AI model to classify the type of damage.

'Model Name'.Predict(Image).Prediction

Figure 2: Use AI through Power Fx 

Once you’ve set up your model and Power Fx you can test it out. In my example app, I can add an image to my app and it will predict what type of car damage the image contains.

Figure 3: See real-time predictions in canvas apps

Using AI models through Power Fx will be available in preview in all regions later this month. Learn more here.

Bring your own AI model

When we first built AI Builder, we designed it to empower citizen developers to leverage AI without needing data science or machine learning experience. But over time, and after talking with customers, we’ve learned that companies who have data scientists or machine learning engineers could not easily enable citizen developers to leverage their custom AI models in the Power Platform.

Now, anyone can bring their own custom AI models into AI Builder – enabling you to build solutions around your company’s investments in AI. You can build custom models using any developer tool chain and deploy your models to any cloud service, like Azure. All you need to do is register your model with AI Builder, and it can be used alongside any other pre-trained or custom models.

Figure 4: You can now bring your own model to AI Builder

We’re excited to see what business solutions you will build around your own AI models. Bring your own model is now generally available. Learn more here. 
NOTE: Generally available products and services are considered production ready.  By moving out of preview, these capabilities will become premium. You will need an AI Builder license or an active trial to continue using them. 

Image classification with Lobe

There is an ever-growing list of tools to build custom machine learning models. We strive to make these tools easier to use with AI Builder – starting with Lobe.

Lobe is an easy-to-use app from Microsoft that helps you build image classification models. It is free to use and trains models on your own computer. All you need to do is add and label images and Lobe will automatically train a custom machine learning model for you. And now you can easily upload your model from Lobe directly to AI Builder to use in Power Apps and Power Automate.

Figure 5: Use Lobe to train models and upload to AI Builder

Using Lobe with Power Platform will be available in preview in select regions starting next week.

AI Builder starter capacity

We want to make it easy for everyone to get started with AI Builder, a premium service in the Power Platform. Today, you can try out all the AI Builder capabilities with a free trial, but adding AI to apps and flows requires purchasing AI Builder credits.

To help make it easier to get started, we have added 250 credits for Power Apps per App license and 500 credits for Power Apps User license. These credits are visible in the Power Platform Admin Center.

Today, it must be assigned to an environment before it can be used. In December, this process will become totally seamless. All non-assigned credits will be consumable on any environment without any admin action required.

This means you can more easily start adding intelligence to your solutions today.

Learn more and share your feedback

We can’t wait to see how you add intelligence to your business solutions. Over time, we will share more blog posts about these new features and capabilities for AI Builder. Make sure to watch our Ignite session for AI Builder and read about more updates for AI Builder for Power Automate here.

We are always working to improve AI Builder to meet your needs. Please  share your feedback with us in the AI Builder forum or email us directly at aihelpen@microsoft.com.

The post New ways to add intelligence to your Power Apps appeared first on Microsoft Power Platform Blog.

Both new capabilities can be found under the Prediction model card in AI Builder. One of them is prediction of multiple outcomes, and the other is prediction of a numerical field. These new features make the prediction model richer, and be able to solve more business problems for you. Previously, AI Builder prediction model could predict binary outcomes, such as whether a shipment would arrive on time or not. Now you can use it to predict more than 3 outcomes, like will a shipment arrive early, on-time or late. You can also predict a number, like how many days it would take to close a customer ticket.

More information: Overview of the prediction model – AI Builder | Microsoft Docs

With the release of these capabilities, they’re now considered premium features. You will need an AI Builder add-on license or an active trial to use them.

Multiple outcome prediction

To predict multiple outcomes, you just need to create a prediction model, and when picking the target column, the Choice columns are available in the dropdown list.  Select one column to see the available prediction outcomes. You could select all the outcomes, or just the most relevant  to your business problem. You could also map them to two outcomes if you want to do a binary prediction by clicking the Manage Outcome button in the top navigation bar.

Numerical prediction

To predict a number, go ahead and create a prediction model. At the select outcome step, select a numerical field as your target column. We support all types of numerical columns in Dataverse, including whole number, decimal number, floating point number, and currency.

Use your prediction model in Power Apps and Power Automate

To use the model, follow the steps in AI Builder to finish training, and if the result looks good, publish the model. Once it’s published, you can use your model in Power Apps or Power Automate.

More information: Use your prediction model – AI Builder | Microsoft Docs

For additional  AI Builder resources, visit Power Apps and learn how your users can make their apps AI enabled, or go to Power Automate and learn how your users make their workflow solutions better with AI. Please visit our pricing pages to learn more about the AI Builder capacity add-on for your apps and workflows. Visit our Licensing page for more information on licenses and trials.

The post New AI Builder capabilities are now generally available appeared first on Microsoft Power Platform Blog.

Expense reports can be a very cumbersome and time-consuming task. Between all the manual data entry, approval workflows, and auditing, there are many pain points across the end-to-end process. With the AI Builder receipt processing prebuilt model, you can minimize those pain points and increase the productivity of your employees, delivering real value back to your business.

Receipt processing lets you read and save key information from common sales receipts, like those used in restaurants, gas stations, retail, and more. Using this information, you can automatically pre-populate expense reports simply by scanning photos of your receipts. And when you automate the process at a large scale, there is the potential to save you and your business valuable time and money.

The prebuilt model uses state-of-the-art optical character recognition (OCR) to extract both printed and handwritten text from receipts. You can retrieve valuable information such as the merchant details, transaction date and time, list of purchased items, tax, and totals.

No training or prior configuration is required to use this prebuilt model. Start processing receipts right away in your apps and flows using the new canvas app component and AI Builder flow action.

Learn more about Receipt processing.

The post Process receipts with AI Builder appeared first on Microsoft Power Platform Blog.

The Entity extraction prebuilt model identifies and classifies key elements from text into pre-defined entities. You can use the prebuilt model out of the box in power apps and power automate.

More information: Entity extraction prebuilt in AI Builder

The Entity extraction custom model enables you to identify and classify key elements from text into your domain-specific custom entities that you define. You can train the model by creating custom entities with a few examples or tailor prebuilt entities to suit your business needs.

More information: Entity extraction in AI Builder

The Category classification custom model helps you classify text into your own domain-specific categories. You can train the model with a small set of examples.

More information: Category classification in AI Builder

With the release of these models, they are now considered premium features. You will need an AI Builder add-on license or an active trial to use them.

Entity extraction prebuilt model

Prebuilt models can be used out of the box and require no effort to build, train, or publish. They help you add intelligence without having to gather data and then build, train, and publish your own models.

With the AI Builder entity extraction prebuilt model, you can identify and classify key elements from text into pre-defined categories

The ‘Try it out’ feature includes prepopulated examples in different languages. You can also enter your own custom text when you try out the model.

You can go to maker portal and try out the entity extraction model. You can go to Power apps or Power automate to start using it.

Entity extraction custom model

With the AI Builder entity extraction customization, you can build a model to identify and classify key elements from text into your domain specific pre-defined entities.

You can create new custom entity types by providing a few examples. You can also modify an existing entity type to be more tailored to your business.

After you publish the model, you can go to Power apps or Power automate to start using it.

Category classification custom model

With the AI Builder category classification custom model, you can build a model to classify text into your domain specific pre-defined categories. You can train the model with a small set of examples.

After you publish the model, you can go to Power apps or Power automate to start using it.

For additional  AI Builder resources, visit Power Apps and learn how your users can make their apps AI enabled, or go to Power Automate and learn how your users make their workflow solutions better with AI. Please visit our pricing pages to learn more about the AI Builder capacity add-on for your apps and workflows. Visit our Licensing page for more information on licenses and trials.

The post New AI Builder models are now generally available appeared first on Microsoft Power Platform Blog.

REBROADCAST: Empowering every developer, with Satya Nadella
Speaker: Satya Nadella

In this time of uncertainty, developers will play a central role in reimagining the world we live in and accelerating our path to recovery.

REBROADCAST: Azure: Invent with Purpose
Speakers: Scott Guthrie, Julia White, Donovan Brown, Jeff Hollan, Rohan Kumar, Amanda Silver

Microsoft is unique in the industry and provides the world’s most comprehensive developer toolkit and platform to support developers in their journey to build amazing cloud apps with Azure.

REBROADCAST: Accelerating enterprise app development with Power Platform
Speakers: James Phillips, Charles Lamanna, Kim Manis, Julie Strauss

Join James Phillips to see how the Power Platform complements existing development tools, radically accelerating application development. This custom list of Microsoft Learn modules was built to complement the content in the session.

Accelerating Developers with the Microsoft Power Platform
Speakers: Charles Lamanna, Arun Ulag

The Microsoft Power Platform is an industry leading low-code development platform. However, Power Platform is also rich with capabilities enabling greater speed and effectiveness for pro developers. In this session, we’ll focus on the variety of Pro Developer features available in the Power Platform, and how Pro Developers can harness these capabilities to rapidly create applications. We will also discuss the unique ability of the Power Platform to support a deep synergy between Pro Developers and Citizen Developers who can more easily work together to accelerate an organization’s development efforts.

Building Azure Apps using the Common Data Service
Speaker: Marc Mercuri

In this session, we’ll discuss why to use the Common Data Service for building business apps, vs starting from scratch. From there, we’ll dig into how you can use the Common Data Service as a part of your Azure application – we’ll be digging into core CDS capabilities, as well as dive into the CDS APIs and SDKs.

Amplify your pro developer skills with the Power Platform
Speakers: Pierce Boggan, Julie Strauss

Are you a pro developer wondering what Power Apps can do for you? Join us to learn how you can use our rich extensibility model, to deliver Azure based building blocks accelerating the digital transformation efforts in your organization.

Mixed Reality in Power Apps
Speakers: Michael O’Connor, Ken Oien

Power Apps recently released a new set of features that allow citizen developers to incorporate Mixed Reality experiences into their apps with little to no code required. Experiences like placing 3D objects of your products in a room or taking measurements to see if they will fit. Come to see what’s possible and bring questions!

Learn Studio Session: Build your own Power App Components
Speaker: Cassie Breviu

Microsoft Learn Studio Sessions are designed to be interactive and hands on. To ensure everyone attending has the best experience, these sessions have limited capacity. But don’t worry, they are being recorded and will be available on demand. Learn Studio Session rooms will open 10 minutes early for anyone needing prerequisite set up support.

Build apps for Teams with low-code/no-code tools
Speakers: Loki Meyburg, Efesa Origbo

Build apps for Teams with tools for every developer project – from Power Platform, to Visual Studio, to the command line.

Accelerating Developers with the Microsoft Power Platform
Speakers: Charles Lamanna, Dona Sarkar

The Microsoft Power Platform is an industry leading low-code development platform. However, Power Platform is also rich with capabilities enabling greater speed and effectiveness for pro developers.

Hack4Good: Quick Apps for your Code
Speakers: Amy Boyd; Microsoft, Chris Huntingford; Microsoft, Alison Mulligan; Maximus IT, Keith Whatling; QUANITQ

See solutions created during the Hack4Good Hackathon from 3 perspectives: citizen developer, functional consultant & professional developer. Providing amazing platforms for developers of all types.

Ask the Team: Power Platform
Speakers: Ryan Cunningham, Stephen Siciliano

Community Connections experience – What are developers doing with low code? Come ask the team who build and manage Power Apps, Power Automate, Power BI, and Power Virtual Agents.

Application Lifecycle Management (ALM) on the Power Platform

Speakers: Casey Burke, Shan McArthur

Want to extend Cloud Native Apps with low code UI without giving up on familiar tools? No Problem! With the new Power Platform Build Tools, integrated with Azure DevOps you will enjoy a fully automatable DevOps process.

Low code, no code, more power – Power Platform

Speakers: Phantip Kokilanon, Haimantika Mitra, Justin Yoo

Power Platform is a boon to all those developers who have a lot in their mind but are unable to put that into use. We will see how we can use the Power Platform to quickly leverage our ideas into action. View this session on the MyBuild home page.

Australia highlights Horizon & Power Platform keynotes

Speakers: Rick Clau, Sonia Cuf, Aaron Powell

Join Aaron Powell and Sonia Cuff for the keynote reactions and highlights. View this session on the MyBuild home page.

Learn more about Power Apps: https://docs.microsoft.com/powerapps/

The post Power Apps at Build Developer Virtual Conference, 2020 appeared first on Microsoft Power Platform Blog.