Following this feedback, we’re thrilled to announce the Copilot Studio implementation guide. This detailed guide is designed to help customers, partners, and Microsoft teams review copilot projects and architectures for a smooth and effective implementation journey.

Get ready for a comprehensive review of your Copilot Studio project

The Copilot Studio implementation guide provides a framework to do a 360-degree review of your project. Through probing questions, it highlights potential risks and gaps, aims at aligning the project with the product roadmap, and shares guidance, best practices and reference architecture examples.

Users of the implementation review are guided at each step of the journey with the side pane that provides more context on the questions or features, shares best practices, and includes links to additional resources.

Inspired by the Success by Design framework – a tried and tested Microsoft methodology used for Business Applications implementations – this review aims to detect bad patterns, identify risks, share best practices, and showcase example implementations.

Key principles behind the review

The Success by Design framework, the backbone of this review process, is centered around three critical principles:

1. Early Discovery: Identifying and dealing with potential issues at the earliest stage.
2. Proactive Guidance: Giving robust advice ahead of issues emerging, preventing potential problems.
3. Predictable Success: Providing a roadmap for success, using tested strategies and methods, and avoiding common pitfalls and anti-patterns.

Areas covered by the review

The Copilot Studio implementation guide covers these chapters:

• An overview of the project
• Architecture overview
• Language
• AI functionalities
• Integrations & channels
• Security, monitoring & governance specifications
• Application lifecycle management
• Analytics & KPIs
• Gaps & top requests
• Dynamics 365 Omnichannel (optional)

How is the implementation review typically delivered?

While it was initially delivered by Microsoft, the Copilot Studio implementation guide can now be used autonomously by customers and partners. It is designed to be a living document that can be updated throughout the project lifecycle.

Get the Copilot Studio implementation review today!

Download the Copilot Studio implementation review and provide feedback directly on GitHub, using this issue template.

We are confident that this tool will be beneficial to those implementing Copilot Studio and eagerly anticipate your feedback. Here’s to empowering more successful transformations with Microsoft Copilot Studio!

The post New Microsoft Copilot Studio implementation guide appeared first on Microsoft Copilot Blog.

This involves testing user utterances to confirm that the bot understands them correctly and triggers the appropriate topics. It also entails verifying that the full conversation leads to the intended outcome, such as resolution or escalation. Customers might also want to ensure that their chatbots meet quality requirements before deploying to production, which can be done automatically using Continuous Integration/Continuous Delivery (CI/CD) approaches like Azure DevOps.

The Power Virtual Agents teams have recently released new code samples to help automate chatbot tests, including testing utterances, validating topic triggering, intent candidates (multiple topics matched), playback of full conversation transcripts, and integration into a CI/CD pipeline.

What can be tested with the PVA Test Framework sample solution?

The PVA Test Framework is a sample solution available on GitHub that showcases how to execute tests against a Power Virtual Agents chatbot through APIs, using the Direct Line channel. It verifies that the bot performs as expected in various scenarios, such as:

• Testing the Natural Language Understanding model (topic triggering)
• Validating the multiple topics matched options (“did you mean…”)
• Performing scale load testing
• Testing full end-to-end conversations
• Testing adaptive cards
• Including a CI/CD test step in the deployment pipeline to prevent deployment if tests fail

How does the PVA Test Framework sample solution work?

The PVA Test Framework tool is a console application that can run tests against the Power Virtual Agents Direct Line API. It uses JSON as the format to play conversations back and verify outcomes, and it enables you to connect to Dataverse to download past conversation transcripts to run them again.

How to use the PVA Test Framework tool

There are different ways you can use the PVA Test Framework sample solution. The easiest one is to run it locally with the Command Prompt in Windows.

You first need to download or build the PVATestFramework.Console.exe file.
It then works as a console application, with command lines.

To run tests, you need conversation files in a .JSON format. You can either easily create these files from scratch, by writing your test cases in a .CHAT file format and converting them to .JSON, or you can download conversation transcripts from Dataverse.

For Natural Language Understanding testing, where you test many user utterances and verify that the bot answers as expected, it’s usually easier to start from your own set of user and bot utterances in a .CHAT format.

This command transforms a .CHAT file into a .JSON file:

.\PVATestFramework.exe convertChatFile --path test-set-sample.chat --outputFile test-set-sample.json

Here is what conversations look like a .CHAT format:

Each test conversation is separated by the <EOC> tag (end of conversation).
Messages following the user: tag are sent by the tool to the bot, and the messages received back from the bot are compared with the expected response, documented after the bot: tag, to determine if the test passes or fails.

user: Good morning
bot: Hello, how can I help you today?
<EOC>
user: When are you closed
bot: I'm happy to help with store hours.
<EOC>
user: Find me your nearest location
bot: I'd be glad to help find a store near you.
<EOC>
user: Where can I find your address?
bot: I'd be glad to help find a store near you.
<EOC>
user: I'd like to order a product
bot: I am happy to help you place your order.
<EOC>
user: what are you store hours and locations?
suggested: Lesson 2 - A simple topic with a condition and variable|Lesson 1 - A simple topic||None of these
bot: To clarify, did you mean:
user: Lesson 1 - A simple topic
bot: I'm happy to help with store hours.
<EOC>
user: What is weather today?
bot: I'm sorry, I'm not sure how to help with that. Can you try rephrasing?
<EOC>
user: What is weather today?
bot: I'm sorry, I'm not sure how to help with that. Can you try rephrasing?
<EOC>

Once converted to .JSON, you can run tests by using the below sample command. You can obtain the chatbot token endpoint from the Mobile app channel in Power Virtual Agents.

.\PVATestFramework.exe test --path test-set-sample.json --tokenEndpoint https://c53bf00279234d1cb5ae3265093d59.e1.environment.api.powerplatform.com/powervirtualagents/botsbyschema/cr507_testChatbot/directline/token?api-version=2022-03-01-preview --log --verbose

Test outcomes are then available in a CSV format that you can then use for further analysis.

Example analysis of the outputs of the PVA Test Framework:

You can learn more about the PVA Test Framework sample solution’s different settings and deployment options in the GitHub documentation.

Frequently Asked Questions

What is the cost of using the PVA Test Framework sample solution?

Conversations generated using the PVA Test Framework sample solution would typically generate billed sessions. See: Power Virtual Agents pricing.

How is the PVA Test Framework supported by Microsoft?

Although the underlying features and components used to build the PVA Test Framework are fully supported, the code itself represents a sample implementation of these features.

Our customers and community can use and customize these features to meet their requirements. Any issue with the PVA Test Framework sample solution should be raised on the Power Virtual Agents Samples GitHub repository and not through Microsoft Support.

The post Automate testing of your Power Virtual Agents chatbots with the PVA Test Framework sample solution appeared first on Microsoft Copilot Blog.

What is the Center of Excellence Starter Kit?

The Microsoft Power Platform CoE Starter Kit is a collection of components and tools that are designed to help organizations develop a strategy for adopting and supporting Microsoft Power Platform, with a focus on Power Apps, Power Automate, Power Pages, and Power Virtual Agents.

What is new for Power Virtual Agents in this release?

The CoE Starter Kit in its current version already included an inventory of bots across environments. This release introduces new capabilities that allow tenant administrators to:

• Keep track of the number of conversations per bot over time, with daily aggregates. While conversation transcripts are kept for a default duration of 30 days in Dataverse, the CoE Starter Kit generates aggregates for each bot, every day. A ‘last month’ aggregate is now also available at the bot and environment levels.
• Allocate a number of conversations per environment, as part of the environment add-ons.
• Receive capacity email alerts when environment add-ons get near (above 80%) or over their allocated number of conversations.
• Get an overview dashboard of Power Virtual Agents usage in their tenant over time as well as an all-up conversation consumption versus allocation.
• Identify bots that have been created using the new unified authoring canvas, now available in public preview.

Screenshot of the Power Virtual Agents dashboard in the CoE Starter Kit.

In the Power Virtual Agents Dashboard, administrators can see the volume of conversations over time, per bot or per environment, and keep track of actual usage versus allocation at the environment level.

Screenshot of a Power Virtual Agents bot in the CoE Starter Kit

When looking at an individual bot, an administrator can also access usage history, with a number of conversations per month.

Screenshot of an Environment Add-on record in the CoE Starter Kit

At the environment level, administrators can leverage the ‘Environment Add-on’ record to set an allocated number of conversations for it. This allocation isn’t technically enforced but is used to trigger capacity email alerts.

Screenshot of a capacity email alert notification

When an environment goes near or over their monthly allocated number of conversations, an email is sent to the configured administrators.

Frequently Asked Questions

Does the CoE Starter Kit cover bots that are created in the Power Virtual Agents Microsoft Teams app?

No, this doesn’t cover bots created in the context of PVA for Teams (https://aka.ms/PVAforTeams). This is because these bots reside in Dataverse for Teams environments, and data in these types of environments can’t be accessed from Power Automate, which the CoE starter kit uses to gather inventory data.

Are conversations the same thing as billed sessions?

This extension doesn’t use the concept of ‘billed sessions’ but counts the number of conversation transcript records instead. Billed sessions are still available to customers as part of their bot analytics, in Power Virtual Agents.

How can I deploy the CoE Starter Kit in my tenant?

Installation instructions are available here.

I have setup the CoE Starter Kit but only see aggregates for the past day

By default, the CoE Starter Kit only starts creating daily aggregate records for bot conversations the day the latest version is set up. If you need to rebuild aggregates for past conversation transcript records data, you can install the CenterofExcellencePVAUsageFirstRunAllData_1_0_0_0_managed.zip solution, available here, and run its flow once.

How is the CoE Starter Kit supported by Microsoft?

Although the underlying features and components used to build the CoE Starter Kit (such as Microsoft Dataverse, admin APIs, and connectors) are fully supported, the kit itself represents sample implementations of these features. Our customers and community can use and customize these features to implement admin and governance capabilities in their organizations. If you face issues with:

• Using the kit: report your issue here: https://aka.ms/coe-starter-kit-issues. (Microsoft Support won’t help you with issues related to this kit, but they will help with related, underlying platform and feature issues.)
• The core features in Microsoft Power Platform: use your standard channel to contact Microsoft Support.

The post Center of Excellence Starter Kit with Power Virtual Agents: Govern, Monitor and Manage appeared first on Microsoft Copilot Blog.

But what truly made this feature release a success? The power of COMMUNITY!

We love working with our global community members who bring best practices, technical skills and opportunities to shine, share and celebrate Power Virtual Agents.

Keep reading to see all of the fantastic resources that the Power Platform community members have created.

Kevin Stratvert: Power Virtual Agents Chatbot Tutorial

In this step-by-step tutorial, Kevin shows how you can build your own intelligent chatbots using the new Power Virtual Agents unified authoring canvas.

Follow Kevin Stratvert on YouTube, Twitter, LinkedIn, website.

Rishona Elijah (MVP) gives a tour PVA unified authoring canvas

In this article, Rishona gives an overview of changes and new features:

• Use Power Fx to handle additional logic
• Create adaptive cards from the PVA interface
• Add multiple message variations
• Easily insert images & videos

Article: Power Virtual Agents unified canvas: overview of changes and new features.

Follow Rishona Elijah on Twitter, LinkedIn, website, Microsoft MVP profile.

Dian Taylor (MVP) advanced authoring canvas preview

Dian’s article and video gives the new unified authoring canvas a deep-dive on setting variable nodes, using Power Fx formulas, handling list variables, and how to dynamically retrieve and use values from Dataverse and Dynamics 365 in her chatbot.

Article: Power Virtual Agents: advanced authoring canvas preview.

Follow Dian Taylor on YouTube, Twitter, LinkedIn, website, Microsoft MVP profile.

Ben den Blanken blogs about adding vertical buttons with the new canvas

Ben shows you how to add Vertical Buttons to a Power Virtual Agents question by using the new advanced authoring canvas to do so. This used to require Bot Framework Composer, but not anymore!

Article: Adding Vertical Buttons to Power Virtual Agents – Revisited.

Follow Ben den Blanken on Twitter, LinkedIn, website.

Daniel Christian (MVP) gives an overview of the new PVA features

In this video, Daniel demonstrates some of the new unified authoring features, with a look at the overall new design changes, the enhancements in the messages and controls focusing on the basic and adaptive cards, the Power Fx functions available and the new topic management capabilities.

Follow Daniel Christian on YouTube, Twitter, LinkedIn, Microsoft MVP profile.

Joe Gill (MVP) explains how to use Power Fx in the latest version of Power Virtual Agents

In Joe’s article, it includes how to use Power Fx in the authoring canvas and how it greatly enhances the ability to add capabilities like data validation and manipulation directly in your bot rather than relying on Power Automate.

Article: Power Fx -Power Virtual Agents.

Follow Joe Gill on Twitter, LinkedIn, website, Microsoft MVP profile.

Kent Weare (Microsoft) checks out the new PVA unified experience

In this video, Kent checks out the new PVA unified experience that includes support for Adaptive Cards, Setting Variables, message variation and much more.

Follow Kent Weare on YouTube, Twitter, LinkedIn, website.

Gary Pretty (Microsoft) showcases the new PVA on Power CAT Live

In this discussion between Gary Pretty and Phil Topness, Gary shows what’s new and what’s coming to give you more control over your bots and include the power of Bot Framework Composer in Power Virtual Agents. Power Fx is now integrated across the Power Virtual Agents canvas and gives you more power than ever.

Follow Gary Pretty on Twitter, LinkedIn, website.

Power Virtual Agents featured at the Power Platform French Summit

Power Virtual Agents also got a lot of coverage at the Power Platform French Summit with customer testimonials and deep-dives and demos of the new unified authoring canvas.

More no-code/low-code power in the new version of PVA (in French)

Follow Henry Jammes on Twitter, LinkedIn.

PVA used for employee gamification at Bolloré Transport & Logistics (in French)

Follow Clément Aumeunier on LinkedIn.

Follow Laurent Joseph on Twitter, LinkedIn.

Follow Cheick Diallo on LinkedIn.

Use virtual agents to optimize employee and customer service (in French)

Follow Henry Jammes on Twitter, LinkedIn.

PVA as one of the use cases to integrate the Power Platform with SAP (in French)

Follow Rémi Chambard on LinkedIn.

Follow Raphaël Botzon on LinkedIn.

Want to be part of the Power Virtual Agents community? check out the forum to connect with peers, share ideas, and learn from experts

The post Power Virtual Agents + Community = Best practices for the new advanced authoring canvas appeared first on Microsoft Copilot Blog.

Process Advisor is a feature of Power Automate that allows you to gain a better understanding of your business processes so you can optimize them. Process advisor offers task mining and process mining capabilities to do this.

To help get you started, process advisor offers several templates, including one for Power Virtual Agents.

What’s new compared to the previous template?

The process advisor offers a dialog map of how conversations have flown between topics, redirects, and their outcome. It focuses on activities that can be easily isolated versus the previous template that focused on individual, truncated, user and bot text messages.

The new template also comes with performance improvements, allowing it to run on large volumes of conversation transcripts.

The analytics tab of the Process Advisor Power Virtual Agents Template

What data does it use?

When using the Power Virtual Agents template, you can optionally provide a bot id. Process advisor then retrieves the conversation transcripts that are stored in the current Dataverse environment you are deploying the process advisor template in.

What are the available metrics?

The new power Virtual Agents process advisor template surfaces the session indicators below from your conversation transcripts.

• Triggered topics
• Redirects (either dialog or topic)
• Events when no topic is identified or when a “did you mean” is thrown.
• Confirmed success or failure
• CSAT scores
• Outcome of the session (resolved, escalated, abandoned).

A few additional attributes are also surfaced to further slide and dice:

• Activity Type
• Bot ID
• Bot Name
• Component Name
• Component Schema Name
• Component Type
• Conversation ID
• Conversation Start
• Conversation Transcript ID
• Session End
• Session Outcome
• Session Start
• Session Turns
• User ID

Process advisor also offers additional out-of-the-box insights and metrics that you can use to identify the activities that take the most time with your users.

Join our Power Virtual Agents community to share your ideas, provide your comments, and help drive the future direction of our product development.

The post New Process Advisor template enables deeper insight into bot conversation flows appeared first on Microsoft Copilot Blog.