Get introduced to AI Builder

We have added a video presenting AI Builder and how it can help to enhance your business.

Simpler to discover models

It’s now simpler to find the AI model that suits your needs:

• We improved readability to start from the type of content you need to process: Documents, Text, Structured data, and Images.

• The scenario-oriented description allows you to understand which model applies to your use case.

• Distinction between models prebuilt by Microsoft that can be used right away, and models requiring training data, is more clearly identifiable.

• We also made it faster to assess the model capabilities by using your own data or the samples provided.

Learn by example

You can also learn how to get started with AI Builder by accessing the tutorials or videos created by the community.

Get inspiration on how to apply AI Builder in your business

Finally, you can get inspiration from other customers who enhanced their processes with AI Builder and be informed about the latest additions made to the product.

We’d like to hear from you

We are always working to improve AI Builder to meet your needs. Feel free to share your feedback with us in the AI Builder forum, raise an idea in the Power Automate Ideas, or email us directly at aihelpen@microsoft.com.

Try AI Builder https://aka.ms/tryaibuilder

The post Explore the new AI Builder homepage appeared first on Microsoft Power Platform Blog.

Background and challenge

Usually, customers request quotes by email which are processed with repetitive actions, identifying the product concerned, the quantity, checking the pricing applicable and potential rebates, finally answering the requester.

AI can help to reduce the manual steps required by semi-automating the information gathering, lookups to the ERP and CRM system and preparing the response. This way you can save time and energy for more business impactful tasks like identifying upsell or cross-sell opportunities and more.

What AI Builder can do

Microsoft AI Builder offers a wide range of natural language processing capabilities allowing to extract the relevant data in emails together with Microsoft Power Automate enabling to orchestrate the process’ automation.

In this context, Microsoft Power Automate can extract the raw text when an email arrives, send it to a prebuilt or customized AI Builder Entity Extraction model to extract the product name and quantity, and connect to your system matching the information required from the requestor to prepare the quotes.

The agent must only perform the last mile by generating a document and replying to the requestor.

NOTE: Generating documents is in our roadmap so even this last step could be automated soon.

Implement the solution

The initial step is to build a Power Automate flow that will orchestrate the process as summarized in the figure below:

Building the orchestration logic with Microsoft Power Automate

For this you can use built-in actions and connectors. A basic Microsoft Power Automate flow would look like this:

• The flow will run when an email is received in a shared mailbox,
• Customer identifier is retrieved from the CRM as well as the account manager in charge of processing the request,
• Content is converted to plain text,
• Requested product and quantity are extracted by the AI model,
• A lookup is performed to the ERP system to get the price and discount applicable to this customer,
• The proposal is sent to the operator.

Choosing a Microsoft AI Builder model to extract the information

Now, you must decide wherever the default Entity Extraction model suits your scenario or if you need to build a custom one by providing sample data from past quotes requests to help the model learn your business context.

Leveraging the prebuilt entity extraction model

The AI Builder prebuilt entity extraction supports a list of predefined entities which may work in the most common cases. Follow this link to consult the list of entities extracted.

In the below illustration we can see an example of what the prebuilt can achieve.

Building a custom entity extraction model

If you need additional information than the one extracted by the prebuilt, follow the steps below or read this step-by-step guide to get started building your very own custom entity extraction model.

1. Create a dataset that must be uploaded in Microsoft Dataverse. You can refer to https://docs.microsoft.com/ai-builder/entity-extraction-sample-data.
2. Create a custom AI Builder custom Entity extraction model from the explore page https://make.powerapps.com/aibuilder/build and clicking on “Get started”
   
3. Select the Microsoft Dataverse table where the training set has been uploaded and the column containing the sample data
4. Select from the list of supported languages and “Analyze” the dataset
5. Use the existing default entities or create new ones by clicking on “+New entity”
6. Tag in the left pane the text that matches your entities
7. Train, test and publish your model to use in the above Microsoft Power Automate flow.

Outcomes

Now, each time a new quotes request arrives in the shared mailbox, the content will be analyzed, processed and a proposal will be sent to the processor contact.

You can read this story Ingram Micro frees thousands of hours for sales teams, unlocks new opportunities with automated sales quotes to get more inspiration on a real customer case.

Expected ROI

• Automate part of the quotes request response process.
• Reduce delay answering customers.
• Increased the likelihood of generating additional revenue.
• Minimize error rate.
• Save your workforce from tedious tasks to maximize their efficiency.

Want to know more?

AI Builder public documentation is the place to start to learn more about Entity Extraction custom model and Entity Extraction prebuilt model

The post Simplify quote requests management with AI Builder appeared first on Microsoft Power Platform Blog.

Scenarios becoming generally available

Several AI Builder models have graduated out of preview and reached general availability.

In past months, we enhanced AI Builder with new capabilities which are now production-ready:

• Invoice processing – Read and save information from invoices which now also include the ability to extract line items.
  
• Receipt processing – Read and save information from receipts, with support now for sales receipts from Australia, Canada, United States, Great Britain, and India.
• Identity document reader – Read and save information from identity documents.

You can try out these AI models today directly from the AI Builder page.

Generally available products and services are considered production ready.
By moving out of preview, these models become premium. You will need an AI Builder license or an active trial to continue using them.

In addition, new features on existing generally available scenarios will now also be production ready.

• Train using documents that have different layouts – This feature was released in public preview last November and allows users to create a unique form processing model that will extract the same information from up to 100 different document layouts.
• Ability to specify the page to be analyzed (available mid July) – Form processing, invoice processing and receipt processing models can now specify a page or page range to be analyzed in the Power Automate action. This is useful to process only the relevant parts of the documents reducing prediction costs.

For additional resources on AI Builder, visit Power Apps and learn how your users can make their apps AI enabled, or go to Power Automate and learn how your users can make their workflow solutions better with AI. Please visit our pricing pages to learn more about the AI Builder capacity add-on for your apps and workflows. Visit our Licensing page for more information on licenses and trials.

Support of additional languages

We have added the support of new languages for sentiment analysis and key phrase extraction. See Language support – Text Analytics Sentiment Analysis and for Text Recognition, see Language Support – Text recognition

For text recognition, this also includes some OCR performance improvements. AI Builder leverages Cognitive Services Computer Vision. You can access the list of all languages supported with version 3.2 here – Language support – Computer Vision.

Coming soon: Form processing will support documents from these 73 different languages later this summer including Japanese and Chinese.

Making it easier to use your form processing model in cloud flows

One popular scenario that customers are using form processing for is to automate data extraction from documents received by email. Now, after training and publishing a form processing model and selecting Use model a New flow in Power Automate, you will land in an end-to-end functional cloud flow that will process documents from email attachments. You can test the flow right away to see it in action and then tailor it to meet your business needs.

Confidence scores and automatic validation in document automation

The prebuilt end-to-end solution for automating the processing of documents has been updated with new capabilities to enrich and simplify the pipeline.

When the data has been extracted from documents, some basic validation rules are now executed to check empty or not recognized fields and detect low confidence.

In this case, the documents must be reviewed by a human, the confidence of each field being now visible in the validation application. Otherwise, the data extracted can be automatically exported to the target system.

Availability in US GCC, GCC High, and Switzerland

We have added the support of environments created in the US government cloud GCC and GCC High.

Support for Switzerland will also be available by mid-July. Customers in these regions can now leverage AI Builder on their current environment without the need to take any action.

They can be assured that their data will be processed in these boundaries.

The list of available scenarios on these new regions can be reviewed in the feature availability by region matrix.

Other enhancements

• We have improved the export and import reliability of solutions containing models retrained multiple times.
• Makers on the tenant’s default environment can now enable AI Builder and Dataverse from the AI Builder home page if missing.

We are always working to improve AI Builder to meet your needs. As always, feel free to share your feedback with us in the AI Builder forum or email us directly at aihelpen@microsoft.com.

The post AI Builder July 2021 update appeared first on Microsoft Power Platform Blog.

Improved table extraction in form processing

We can now train a form processing model to recognize tables that were not supported until today, available in public preview. Tables can range from simple ones with clear delimitations between rows and columns, to more complex ones without borders, that have merged cells, or even nested items. Just tag the tables you want to extract on the document the same way you tag fields. This way, you’ll teach the AI model how to extract the tables.

Once your form processing model has been trained and published, you can use your model in your automation flows in Microsoft Power Automate or apps in Microsoft Power Apps. You can also leverage it in the recently released end-to-end document automation solution.

Learn more about improved table extraction in AI Builder form processing.

Support of table data in document automation

Document automation enables you to build a rich and robust solution to process and extract data from documents allowing users to manually review and approve the result of the processing.

In March 2021, we have added support of table data extraction. This is used for instance in accounts payable automation to retrieve invoice line items.

Thanks to the improved extraction capability of form processing, you can create a model extracting data for tables and use it in the document automation pipeline. Once the processing is complete, you can review the data extracted in a new tab of the side-by-side viewer:

We have also published a whitepaper explaining how to implement document automation in your business with Microsoft Power Platform.

Identity document reader public preview release

We announced the availability of the new identity document reader model in public preview on March 26, 2021.

 This model allows you to extract information (such as first name, last name, or date of birth) from passports from all over the world and state-issued driver’s licenses.

AI Builder makes it easy to read and export this data in your business processes if you use either passports or driver’s licenses as proof of residency or for hotel check-in purposes, for instance.

Leveraging the latest in Microsoft AI Form Recognizer, this new model doesn’t require training nor prior configuration. It’s able to perform high quality data extraction even with imperfect photos or scans (glossy or tilted images, for instance).

Learn more about the identity document reader.

AI Builder in Teams

It’s now possible to use AI Builder through the dedicated Power Automate application.

You will be able, for example, to analyze the sentiment of emails received with AI Builder and send results to Teams; or add channel messages into Microsoft Excel after extracting categories and key phrases using AI Builder.

After installation of the Power Automate application in Teams, you can go to the create tab and select one of the AI Builder templates:

Just click on a template and follow the simplified experience to create, in few seconds, a flow that will leverage the power of AI Builder models.

Several templates are already available, and more will be added in the coming months. You will also have the possibility to edit templates in advanced mode to tweak them at your convenience.

Availability in India, France, Germany, and UAE

AI Builder is now available in environments created in India, France, Germany, and United Arab Emirates in March and April 2021. Customers in these regions can now leverage AI Builder on their current environment without the need to take any action and be assured that their data will be processed and will remain in these boundaries.

The list of available scenarios on these new regions can be reviewed in the feature availability by region matrix.

We are always working to improve AI Builder to meet your needs. Feel free to share your feedback with us in the AI Builder forum or email us directly at aihelpen@microsoft.com.

The post AI Builder April 2021 update appeared first on Microsoft Power Platform Blog.

Microsoft Power Platform offers to extract data from any kind of document (see Improve business performance with AI Builder) and a reference document automation pipeline (see Get started with AI Builder document automation).

Implementing such a solution is usually an iterative  journey which could be summarized in 4 phases :

1. Building the initial AI model
2. Prototyping
3. Initial rollout in production
4. Mature production

Build the initial AI model

During this phase, a champion (commonly named as “maker”) with some technical background, will train an AI model using samples from the most common documents received to assess wherever the AI technology is able to properly extract data from these samples.

Select samples

To start, the maker should gather at least 5 sample documents for each document layout to process. The AI model will use these samples to learn how to extract the desired data. Usually, documents coming from different suppliers have different layouts.

The form processing model currently supports up to 200 collections which should in theory handle up to 200 suppliers. Given that there is a limited number of documents generating software, it’s likely that several suppliers will provide similar layout which could be extracted by the same collection.

This will be fine-tuned during the prototyping and initial production phase (see Prototyping section in this article ).

For each provider sample, the maker will identify which fields data and which table data must be extracted by the AI model.

Use the training wizard

Now that the maker has gathered sample documents and has identified the fields and tables to extract, the next action is to create a new Form Processing model (see Create a form processing model). The first step of the training wizard is to define the fields to define in the “Fields” tab:

And the tables in the “Tables (preview)” tab

The maker will define one collection per layout, so one per supplier.

The combined file size of the documents used for training per collection must not exceed 50 megabytes (MB), and PDF documents shouldn’t have more than 500 pages (see Requirements and limitations).

Test the model

After going through the model tagging and training, the model is ready to be tested. The maker will use the “Quick test” to check that the data is properly extracted.

When necessary, the maker will use the “Edit model” to improve the performance by providing more samples or retagging (see Improve the performance of your form processing model).

Once satisfied, the maker can publish the model (see Train and publish your form processing model) and is now ready to step into prototyping.

Prototyping

During this phase, the maker will setup the document automation pipeline to adapt it to his business needs and will test it with selected documents.

The maker will setup the pipeline provided with Power Automate document automation base kit by configuring the previously created model to handle new documents received (see Configure the document automation base kit).

The result of the extraction is stored in Dataverse (see What is Microsoft Dataverse?) and manually reviewed in the document automation application by the maker.

This data can be manually imported to the target system or the maker can build an export flow.

The state of all documents entering the pipeline can be monitored in the document automation application.

• When the data is not properly extracted because of a specific layout, the maker can manually fill or fix the missing parts, then download the document to retrain the model manually and improve the accuracy for further processing.
• When the format of the document is not supported, the maker can take further actions like contacting the supplier or adding some conversion steps in the attachment extraction flow.

At this stage, the process continues to include a large part of human intervention as the organization needs to gain trust in AI to manage the sensitive business processes. The maker is playing a critical role fine-tuning the pipeline to fit with the organization’s needs.

Initial rollout in production

During this phase, the organization has confirmed that the solution is able to automate the processing of the documents and can expect significant productivity gains. The prototype will be rolled out to production for selected business scenarios during an observation period.

The model should have been optimized during the prototyping phase and should handle input documents properly in most cases.

The maker will implement in Power Automate some automatic validation rules after the data extraction to check if the data has been properly extracted. Examples of some validation rules include checking the model confidence, that all fields are filled in, and performing data consistency lookup on the target system.

The maker will also build a Power Automate flow to export the result of the extraction to the target system (CRM, ERP, or other) that will be triggered when the processed documents are validated.

If the data didn’t pass the automatic validation, it will be reviewed, edited, and validated in the document automation application like during the prototyping phase. This will happen when a supplier has changed the layout of the documents or a new supplier with unknown document layout has been added.

The maker will manually update and retrain the model to address these cases and ensure an automatic extraction in further processing. In the meantime, documents received from this new supplier will be manually handled in the document automation application.

Eventually, the business user will see documents flowing in the target system after the data export flow has been triggered and will be able to approve it.

NOTE: The export to target system in the above diagram is not delivered with the built-in document automation pipeline. Each customer can easily customize it with Power Automate using one of the prebuilt connectors.
At this stage, the process is mostly automated but requires the expertise of the maker who performed the implementation to adapt to supplier’s changes and to manually handle corner cases.

Mature production

This is a phase that the organization will reach after months of productive usage. They will have defined the processes for the governance of this pipeline. This includes training manual reviewers who will replace the maker in managing rare cases where documents have not been properly extracted, and a process administrator who will ensure the efficiency of the pipeline.

Manual reviewers will be trained to review, edit, and manually validate documents that didn’t pass the automatic validation rules. In this process, they will tag these documents by drawing bounding boxes like what is done in the model authoring phase.

The pipeline will be able to handle this input to automatically retrain the model, improving the accuracy for existing suppliers, addressing layout changes, and supporting new suppliers.

NOTE: Defining data validation rules and having an automatic retraining logic is not yet implemented in the current version of document automation. This is part of the roadmap and should be available soon.
At this stage, the pipeline will maximize productivity gains by removing most of the need for human involvement in the process. Manual reviewers will rarely review documents thanks to the automatic retraining that will ensure an optimal accuracy of the model. The process administrator will be in a monitoring perspective only acting in case of outages.

If you have any feedback or questions about document automation, feel free to share it with us in the AI Builder forum or email us directly at mailto:aihelpen@microsoft.com.

The post How to implement document automation in your business with Microsoft Power Platform appeared first on Microsoft Power Platform Blog.

• Power Automate to orchestrate the overall process
• AI Builder to bring the Intelligence required to efficient extract information from documents
• Power Apps to allow users to manually review and approve documents, and
• Dataverse to manage the document queue and store all the data, files and configuration required.

Today we excited to release ‘document automation’ which provides this standard end-to-end solution and includes the following components:

• Manage documents received in emails orchestrated by Power Automate,
• Use AI Builder form processing solution to extract data from those documents,
• Process the data extracted and allow users to review and approve data, through a central manual validation Power App.

For example, if your company receives delivery notes by email where today somebody needs to open the attachment and manually enter the data contained in the delivery notes in your ERP. Now with ‘document automation’ you can easily set up a robust automation that will read and extract the data using AI which can easily be reviewed before triggering the next steps in your process, saving you valuable time.

A Simple Installation

By navigating to Power Automate you can access the new document automation page to install the solution in a couple of minutes.

Select the “Custom documents” solution to manage any kind of documents like invoices, receipts and more.

Follow the wizard steps to install it on your environment and configure your automation including permissions and the Power Automate flow connections to your Outlook service mailbox and Dataverse.

Stay in control of the process

Configure the AI Builder model to use for the data extraction and monitor the incoming documents thanks to the “Document Automation Application”.

Review and approve the data extracted when required.

Learn more

Thanks to this automation solutions, organizations can accelerate and optimize the processing of any kind of printed or digital form so humans can be saved from repetitive, low value and error prone tasks.

To learn more about document automation, you can refer to this documentation page.

The Get started with document automation module in Microsoft Learn is also a great way to learn how you can easily automate the processing of your documents.

If you have any feedback on how we can make it even simpler for you to automate your document processing successfully, feel free to share it with us in the AI Builder forum or email us directly at aihelpen@microsoft.com.

The post Automate document processing with Power Automate appeared first on Microsoft Power Platform Blog.

Report in Power Platform Admin Center

Are you wondering how many AI Builder add-on you need to purchase for your business? The AI Builder calculator allows you to perform an upfront assessment. Administrators need to compare this with the actual usage of their organization so they can adjust along the way.

Sign in to the  Power Platform Admin Center and open the Capacity menu. Here,  you can access a new feature that allows you to download an Excel report containing the AI Builder consumption.

You can select the target date, which means you can get the data for the 30 days preceding.

Once the report is ready, you can download the report as an Excel file.

The report will show your credits consumed by day and by environment.

You can compare the aggregated consumption for current calendar month on each environment with what has been allocated. Using this information, you can take action to allocate more credits to environments that are in overage.

Remember, the number of allocated credits by environment is visible and can be changed from the Add-ons page in the Power Platform Admin Center.

Learn more about AI Builder licensing and AI Builder Consumption Reporting.

The post AI Builder consumption reporting is now available appeared first on Microsoft Power Platform Blog.

The Invoice Processing Starter Kit leverages AI Builder to extract data from invoices, Power Automate flows for the orchestration, Common Data Service entities to store data and a Power App to configure and monitor the process as well as to review and edit the extracted data. This solution offers a role-based workflow.

Business scenario

Once an email is received, the attachment is retrieved by Power Automate, the corresponding provider is determined.

The associated AI Builder model is called. Then, Power Automate stores the extracted information in an invoice entity.

Get Started

You can define the permissions and accessible features based on the profile, validator or power user, to leverage the Power Platform teams and roles.

In AI Builder, you can build form processing models to extract data from each invoice type and take advantage of the new form processing supervised capability. For more information see:

–  AI Builder Form processing now lets you recognize undetected fields
–  Create a form processing model

In Power Apps, you can configure the sender address for each provider, and the specified AI Builder form processing model to extract the information. Then, you can map the keys of your invoice to the invoice fields provided by Microsoft.

You can also monitor this process:

Users can review the extracted fields, perform edits and validate the invoice before the data is sent to the target system.

This solution is provided at no additional cost when you have an AI Builder, Power Apps and Power Automate license. By leveraging the products of the Power Platform, it can be totally customized to fit to your business.

We are considering adding additional features in the future so stay tuned.

Learn more

You can download the package from App Source  and read the documentation.

The post AI Builder Invoice Processing Automation Starter Kit is available on App Source appeared first on Microsoft Power Platform Blog.

The Invoice Processing Starter Kit leverages AI Builder to extract data from invoices, Power Automate flows for the orchestration, Common Data Service entities to store data and a Power App to configure and monitor the process as well as to review and edit the extracted data. This solution offers a role-based workflow.

Business scenario

Once an email is received, the attachment is retrieved by Power Automate, the corresponding provider is determined, and the associated AI Builder model is called. Then, it stores the extracted information in an invoice entity.

Get Started

You can define the permissions and accessible features based on the profile, validator or power user, to leverage the Power Platform teams and roles.

In AI Builder, you can build form processing models to extract data from each invoice type and take advantage of the new form processing supervised capability. For more information see:

–  AI Builder Form processing now lets you recognize undetected fields
–  Create a form processing model

In Power Apps, you can configure the sender address for each provider, and the specified AI Builder form processing model to extract the information. Then, you can map the keys of your invoice to the invoice fields provided by Microsoft.

You can also monitor this process

Users can review the extracted fields, perform edits and validate the invoice before the data is sent to the target system.

This solution is provided at no additional cost when you have an AI Builder, Power Apps and Power Automate license. By leveraging the products of the Power Platform, it can be totally customized to fit to your business.

We are considering adding additional features in the future so stay tuned.

Learn more

You can download the package from App Source  and read the documentation.

The post AI Builder Invoice Processing Automation Starter Kit is available on App Source appeared first on Microsoft Power Platform Blog.