Maker Deployment page

Makers can now view all their deployments in one place, regardless of solution or pipeline. If their deployment began from (or was deployed to) the current environment, they will be able to view the run history status of it. Failed Deployments and Active Deployments are highlighted in a convenient overview, so you don’t have to go looking for them either!

In addition to deployment visibility, we’ve included a Get started section that will evolve over time. For now, it will link to documentation to help a maker begin their journey to healthy application lifecycle management (ALM).

To further improve understanding of what healthy ALM looks like, we’ll soon be adding recommendations, where makers are alerted to resolve any potentially unhealthy ALM-related behavior, such as housing high-use apps outside of a solution.

Admin Deployment page

The Deployment page in the new admin center provides a streamlined experience to help administrators learn about, setup, and operation best-in-class ALM. The initial preview provides a central location to view all deployments in the tenant, approve deployment requests, and troubleshoot failures.

Select a pipelines host from the picker to see all the pipelines and deployment history managed with that host. A dedicated Failed deployments view helps admins quickly identify and troubleshoot failures and trending issues.

Admins can approve or reject deployment requests assigned to them. First setup Delegated deployments with service principals as the recommended way to deploy securely.

It is important that admins review changes in the solution and the sharing request. With the help of Copilot-generated deployment notes visible in the request, this becomes easier, but if you want a more granular look, be sure to set up Source control integration and link the repository to the approval.

Managed Operations

The Deployment pages are an offering of Managed Operations, allowing all who make or administrate to gain further insight into their Application Lifecycle Management to ensure that business solutions are reliable and performant in production. Aside from just deployment pipelines, these maker and admin experiences will evolve over the coming months and include many more Managed Operations intersections to take advantage of. So, stay tuned for more updates and please leave your feedback below!

Learn more

Admin Deployment page

Maker Deployment page

Managed Operations

New Power Platform Admin Center

The post Introducing Maker and Admin Deployment Pages appeared first on Microsoft Power Platform Blog.

With these updates, we’re excited to announce the general availability of delegated deployments for pipelines in Power Platform!

Additionally, admins can now enable pipelines for multi-geo support, making it easy to centrally administer global deployments within a single management plane.

Build

As makers develop, customizations are automatically saved to their preferred solution (no need to understand or navigate to solutions). This avoids downstream deployment issues, painful debugging steps, and other common mishaps.

 When authoring solution cloud flows, drafts and versioning provides a history for each published version of the flow. Makers can view the version history, restore prior versions, and save draft changes without publishing them live – even if the flow has errors! Then, the flow can be published when they are ready to run the flow. Please leave your drafts and versioning comments and feedback here.

Test

Admins can improve quality by ensuring solution checker runs on every deployment, and configure issue tolerance levels for different environments. Pipelines can be extended to run additional code and security scanning tools, or automated tests by integrating your source control systems. Functional user acceptance testing is also recommended after deployment to test environments.

We recommend starting with pipelines in Power Platform and using extensions if you need to integrate more advanced workloads running in Azure DevOps, GitHub, etc. Similarly, if taking dependencies on test automation, we recommend Test Engine.

Release

Makers are guided to deploy at the right time. For example, when manually exporting and importing solutions or blocked by sharing limits.

Instead, sharing is requested during deployment to target environments, like production, where it’s appropriate to share broadly. Admins simply need to approve the combined deployment and sharing request, and the rest happens automatically. Sharing is rolling out for canvas apps and soon for cloud flows and security roles.

Makers can also update existing environment variables and connection references in pipelines and solution import. Combined, these capabilities protect production assets with least privileged access, reduce admin burden, and train makers to submit all updates via your organization’s change management process. If admins haven’t configured pipelines and governance policies, makers are empowered to create their own pipeline to environments (they must already have access to manually import solutions. Pipelines doesn’t escalate permissions). Admins can apply additional governance controls and manage security within the default host and custom hosts. There’s a new entry point within the solution experience to Manage pipelines and a Security Teams section in the app to manage pipelines access. Note: we currently recommend using a custom pipelines host when more advanced control is desired.

 

Why pipelines vs manual export/import? Unlike manual export/import, pipelines stores backups for every version of every solution deployed, and admin accessible audit logs describing who, what, when, where, and why (AI provided). Pipelines deployments enjoy higher success rates by pre-validating solutions against target environments and sequencing multiple deployments to avoid conflicts. 

 

Solution simplification remains an ongoing effort. Recent improvements include streamlined dependency management and automatic conversion of unmanaged to managed solution objects (see moving from unmanaged to managed solutions).

 

What if breaking changes are inadvertently deployed? Now makers and admins can now re-deploy prior solution versions using pipelines or developer tooling.

 

Documentation

• ALM Guide
• Pipelines in Power Platform
• Managed environments
• Power Platform Developer tools
   

The post April 2024 ALM blog: What’s new and what it means for you appeared first on Microsoft Power Platform Blog.

We are excited to announce the public preview of Default Environment Routing, a new Managed Environments feature that allows Power Platform admins to automatically direct new makers into their own personal developer environment when they visit make.powerapps.com for the first time. Default environment routing offers new makers a personal, safe space to build with Microsoft Dataverse, without the fear of others accessing their apps or data.

Default Environment Routing will allow admins to place their makers in a healthy Application Lifecycle Management (ALM) state from day one by directing them into their own personal developer environment instead of the default environment. With Power Platform pipelines, admins can also create a release pipeline from the created personal developer environment to production. This feature will help customers streamline their environment strategy and offer administrators more control over their makers.

For new makers up until now, the default environment serves as their home environment, which offers a seamless experience and accelerates innovation with the Power Platform. Establishing the same governance guardrails for all the makers in a single shared environment, however, gets more challenging as the maker base of any business grows. Administrators also have a challenge monitoring and managing all the assets in the default environment at scale.

Default Environment Routing is an essential tool for tenant administrators, providing them with the ability to manage their makers’ environment according to their specific governance requirements. This feature will help them to optimize their environment strategy, making it more efficient and secure for both makers and the organization. Admins can customize the governance rules and policies for each individual maker or business unit in their own developer environment as per their needs.

Prerequisites

Default environment routing is a tenant-level, admin setting that:

• Is enabled by Power Platform admins only.
• Requires that the Developer environment assignment setting is enabled for Everyone

• Requires the use of Managed Environment, since all of the newly created environments will be managed. Users in a managed developer environment will require premium licenses to run Power Platform assets.

Enable the Default environment routing setting

The Default environment routing setting is disabled by default and must be enabled using Power Platform admin center or PowerShell.

Enable the feature in Power Platform admin center

1. In the Power Platform admin center, in navigation pane, select Settings.
2. On the Tenant settings page, select Environment routing (preview).
3. In the Environment routing pane, turn on the Create personal developer environments for new makers option.

This feature enables the automatic creation of a personal, developer environment for all of your Power Apps new makers when they first launch make.powerapps.com. The definition of a new maker is limited to the makers who visit make.powerapps.com for the first time. Returning makers who have visited the site before aren’t impacted. The created developer environment will be managed by default and the new makers are assigned the admin role in their newly created developer environments.

IMPORTANT

• Any maker can build apps in the managed developer environments created through default environment routing without requiring a premium license. Nevertheless, since Managed Environments is not included as an entitlement in the Developer Plan, every user who runs assets in these environments will require a premium license. More information on the Managed environments and the Developer plan can be found here
• Non-managed Developer environments are unaffected by the above stated premium license requirements. You can learn more about the developer environment and developer plan here.

Looking forward

Our roadmap includes configuring user roles as well as the automatic configuration of Managed Environments and DLP settings for all new Development environments created through environment routing. Keep an eye out for our release notes where we announce Power Platform new features. We are committed to providing our customers with the best possible experience on the Power Platform, and we are excited to see how our customers will use Default Environment Routing to optimize their environment strategy. To learn more, please check out our updated documentation for the Default environment routing here

See related

Managed Environments Overview

Overview of pipelines in Power Platform

About the Power Apps Developer Plan

The post Streamline your governance and environment strategy using Default Environment Routing (preview) appeared first on Microsoft Power Platform Blog.

Not all customers operate within the strict definition of day-time business hours; shifting the Maintenance from the default settings today to a time that best suits business needs for a given Environment will lend customers desired flexibility in scheduling Application and Database updates.

Please note Maintenance windows are not intended to signal downtime. We offer an online platform where customers can run mission critical applications/workloads 24×7, all the while bringing new value frequently without incurring any downtime or performance degradation. Maintenance windows allow customers to make sure that any perturbations to the service, even though done in a fully online fashion, can happen only during the configured time window.

For more information on the types of activities in scope of this feature and how to utilize it, please refer Maintenance Window Management documentation

The post Announcing Public Preview of Maintenance Window Management! appeared first on Microsoft Power Platform Blog.

Today, we are excited to announce that new DLP capabilities are now available for you to use worldwide in Public Preview:

• Connector Action Control enables admins to easily allow/block specific connector actions for each connector. For example, you could block the Delete row (V2) action for the SQL Server connector.
• Endpoint Filtering enables admins to properly secure connection endpoints by configuring an ordered list of endpoint patterns to allow/block. Continuing the SQL Server connector example above, you could configure a policy to allow only connections to testserver.contoso.com endpoint, and block everything else.
• Custom Connector Parity allows tenant admins to apply DLP classification on custom connector URL patterns and allows environment admins to apply DLP on custom connectors in their environment. For example, admins can create an additional rule for a custom connector by creating a new pattern for https://*.contoso.com URL and associate it with the connector.

Connection Action Control

Power Platform already allows admins to govern access to various data sources using DLP policies to classify data connectors into well-known buckets such Business, Non-business or Blocked. This defines if a given connector can be used within an app or flow and which connectors can be used together within these resources As organizations further adopted the Power Platform to solve more complex and diversified business use cases, it became apparent that admins needed ways to further configure DLP policies beyond the classification of connectors. This led to the new capabilities we are announcing today.

Connection action controls are designed to give you fine-grained control over specific actions that are allowed or blocked within a connector. Now you can revisit some of the above heavy-handed settings and leverage the granular control over specific actions to enable reads and block only writes for instance.

For example, an organization may consider a social media connector like Twitter risky, and they would place it in the Blocked category. However, with connector action controls, they can disable the riskier write operations but allow for read actions, safely enabling the connector for use cases that do not involve the threat of data exfiltration.

Endpoint filtering

By leveraging endpoint filtering, to configure which specific endpoints a connector can interact with. With this release, we enabled endpoint filtering support for commonly used connectors such as HTTP, SQL, Dataverse, SMTP, Azure Blob Storage, and more. Ability to configure an ordered list of endpoint patterns to allow/deny and pattern matching with ‘*’ support are also available.

Like connection action control capabilities, endpoint filtering allows customers to not only turn connector usage on or off but turn it on with restrictions in place so they can use available data sources in a controlled, more secure way. For example, you can use endpoint filtering to govern usage of specific SQL Server instances. Depending on the sensitivity of data stored on various servers, you can allow access to some SQL Server endpoints and deny access to others.

Custom connector parity

Also included in this release are new environment and tenant-level policy support, as well as enhanced user experiences for custom connectors.

Environment admins can now use the intuitive experience in Power Platform admin center, in addition to existing PowerShell commands, to classify individual custom connectors by name for environment level DLP policies. All custom connectors are listed in line with pre-built connectors in the Connectors tab of the Data Policies wizard.

Tenant admins can leverage either Power Platform admin center or PowerShell to classify custom connectors by their Host URL endpoints using a pattern matching construct with ‘*’ support for tenant level DLP policies. A new tab was added in the Data Policies wizard called ‘Custom connectors’, to allow you to specify an ordered list of allow and deny URL patterns for custom connectors.

In addition to above capabilities for enhanced custom connector experience, we also introduced support for admins to classify individual custom connectors by name for environment level DLP policies.  This will enable you to apply the same policy to the services regardless of how many customer connector definitions there are or how they are named.

Get Started

We are confident that this new set of DLP capabilities will help admins to effectively govern the movement of tenant data from AAD authorized data sources to and from Power Platform to effectively guard against data exfiltration risks. Start using these new capabilities today! Here are a few pointers to help you get started:

• Read the online documentation on granular connector action control, endpoint filtering, and custom connector parity patterns
• Understand what best practices to follow when establishing a DLP strategy
• Learn more about Data Loss Prevention policies and Power Platform governance.

The post New Data Loss Prevention policies for enhanced Power Platform governance available for Public Preview appeared first on Microsoft Power Platform Blog.

Newly added content includes:

• Administer Application Users, Security roles, Teams, and Users in the Power Platform admin center
• Adding users to Dataverse
• Assigning security roles in the Power Platform admin center

This brings us to 15 total videos with more on the way!

Check them out and let us know what you think of the content or other topics you’d like to see.

The post Security and Administration topics added to the Dataverse task based and overview video series appeared first on Microsoft Power Platform Blog.

Developing and maintaining a license management system takes time. So we are pleased to announce  a new capability called third-party app license management that enables our partners to manage and enforce licenses for their products using the same systems and technologies Microsoft uses for its own licensed products.

License assignment through the Microsoft 365 admin center

Admins can assign licenses of partner products using the Microsoft 365 admin center, a familiar portal which admins use today to manage Office and Dynamics licenses:

License enforcement built into the platform

The Power Platform will enforce partner product licenses at runtime to ensure that only licensed users can access a partner’s solutions. If a user has not been assigned a license for a solution then that solution will not appear in the user’s Dynamics Home Hub. Furthermore if that user is sent a direct link to the solution then they will be presented with a message indicating that a license is required:

Analytics and insights for partners

In Partner Center, partners can view aggregated reports of the types and quantities of licenses their customers own and whether those licenses have been assigned – a useful indicator to help understand whether customers are getting value from their license purchases:

Learn more and try it out

This new capability is available now. A few things to note: third-party app license management supports the Dynamics CE/Power Platform offer type in Partner Center and supports license enforcement of model-driven apps. Looking ahead we intend to build support for other Partner Center offer types and for the ability to secure Dataverse tables (entities) and SDK messages.

To get started:

• Watch this 30min webinar which provides an end to end overview
• Review the step-by-step walkthrough guide
• Post any questions you have to the Pro Dev and ISV Forum
• Get in touch with us at ISVLicenseManagement at microsoft.com to share your thoughts on this new capability

We look forward to your feedback!

Arif Gani – Senior PM, Commerce

Shawn Nandi  – Principal PM, Power Platform

The post Partners: Introducing a new way to manage and enforce licenses for your products appeared first on Microsoft Power Platform Blog.

• Data Transformations and Query Editor enhancements:
  • Diagram View for authoring of queries
  • Schema view
  • Add Column from Examples
  • Web Connector – By Example Data Extraction
  • Fuzzy Matching:
    • Group By – Fuzzy Matching options
    • Cluster Values
  • Copy/Paste Queries between PQ Online and PQ Desktop
  • Copy data from Query Editor preview
  • Formula bar – Query Script mode
  • Multi-select support in Queries pane
  • Status Bar & Evaluation Performance Counters
• New and Enhanced Connectors:
  • Parquet files
  • SAP HANA
  • SAP BusinessWarehouse
  • Snowflake
  • Impala
  • FHIR
  • Google Analytics
  • Azure Tables
  • Azure Blobs
  • SQL Server – Advanced Options in Get Data UX
  • Privacy Levels support within Get Data UX
• Dataflows Management:
  • Admin Take-over
  • Incremental Refresh for Dataflows loading data into CDS
  • Create a new dataflow from a Power Query template
• New Power Query & Dataflows documentation

You can continue reading each of the sections below for more details about these new capabilities.

Diagram View

With Power Query customers can easily create complex dependencies between data sources, queries and data transformations. While the Power Query Editor provides ways for users to list queries (“Queries” pane) and steps within them (“Applied Steps” pane), customers have demanded a more visual representation of all queries and their dependencies. In Power Query Desktop, the Query Dependencies view provides a read-only view allowing users to understand the dependencies between data sources and queries, or between multiple queries. However, feedback from customers has been that the Query Dependencies view felt short in a couple of aspects:

• Providing only query-level visibility for dependencies, instead of step-level granularity.
• Not allowing users to make edits (such as adding/deleting/editing) to their queries and steps within those queries.
• Because the Query Dependencies view is a modal dialog experience, there is friction in transitioning between using the Query Dependencies view and authoring queries within the main Power Query Editor dialog, which is a highly iterative process.

Based on this feedback, we decided to invest on a new Diagram View experience that is more deeply integrated within the Power Query Editor experience; becoming a new pane within this dialog that users can interact with at any point in time during their Power Query authoring experience. Conceptually, this new Diagram View provides a graphical representation of the Queries & Steps panes together, with the ability to both get a high-level view of an entire graph of queries (e.g. all queries within a dataflow) as well as digging into query-level and step-level information in a highly visual interactive way.

We’re very happy to announce that the Diagram View is now available as a Public Preview feature within Power Query Online to author dataflows, which customers can enable from within the View tab in the ribbon.

Upon enabling this option, the new Diagram View pane appears on top of the main preview area. At the same time, the Queries & Steps panes are automatically collapsed (since the Diagram View now provides a more visual representation of queries and steps, as well as the same functional capabilities via context menu option on each of these object types).

Please note that, currently, Diagram View is in Preview and as such it provides limited capabilities compared to other Generally Available features – In particular, support for Accessibility features (keyboarding, high-contrast mode, screen readers support) is limited or not yet available. Providing an accessible feature set within the Diagram View is our top priority and area of investment within this feature, along with incorporating new capabilities and addressing feedback from Preview users.

You can learn more about Diagram View in this deep dive blog post: https://powerapps.microsoft.com/en-us/blog/announcing-visual-data-prep-public-preview-diagram-view-in-power-platform-dataflows/

Schema view

When working in the Power Query Editor with tables that have many columns, simple tasks can become incredibly cumbersome because even finding the right column by horizontally scrolling and parsing through all the data is inefficient.

The newly released Schema view displays your column information in a list that’s easy to parse and interact with, making it easier than ever to work on your table schema within Power Query.

In addition to an optimized column management experience, another key benefit of schema view is that transforms tend to yield results faster. These results are faster because this view only requires the columns information to be computed instead of a preview of the data rows. So even working with long-running queries with a few columns will benefit from using the Schema view.

You can turn on the new Schema view by selecting Schema view in the View tab within the ribbon. When you’re ready to work on your data again, you can switch back to the Data view from the same location.

Upon enabling the Schema view, the main preview area switches from Data view to displaying the list of columns in the table results, including their names, data types and several contextual operations, can also be accessed from the ribbon’s new “Schema tools” tab.

You can learn more about the Schema view in this documentation article: https://docs.microsoft.com/en-us/power-query/schema-view

By Example Data Extraction

One of the most exciting and differentiating capabilities within Power Query is the ability to extract and reshape data by example from a variety of sources, including existing tables in the Query Editor, Web pages, Text & CSV files.

This month we are enabling two of these capabilities in Power Query Online: “Add Column From Examples” and “Web By Example”.

Add Column From Examples

“Add Column From Examples” enables users to seamlessly extract and reshape data from existing columns in the Power Query Editor without having to think about the (one or multiple) operations to apply on top of those columns. Instead, using Add Column From Examples (available within the “Add Column” tab in the ribbon) brings users into an interactive experience where they can specify one or more sample output values that they would like to obtain. With just a few examples, Power Query is able to apply smart AI heuristics to infer the best combination of data transformations to get to that result.

You can learn more about Add Column From Examples in this documentation article: https://docs.microsoft.com/en-us/power-query/column-from-example
Web By Example

The Web Page connector in Power Query allows users to extract data formatted as HTML tables from a Web Page. Often however, data on Web pages is not in tidy tables that are easy to extract. Getting data from such pages can be challenging, even if the data is structured and consistent.

Web By Example makes it very easy to extract data from such Web pages, by allowing users to specify a few sample output values from a web page and letting Power Query figure out the right transformation steps in order to extract data from the web page.

Web By Example can be accessed from the “Choose Data” screen when using the Web Page connector.

Upon selecting this option, users will be taken into the Web By Example dialog where they can specify sample output values to extract from this page. Note that, unlike in Power Query Desktop where an inline preview of the web page is displayed, this inline preview is not available within Power Query Online, but users can open it in a different browser window/tab.

Fuzzy Matching Enhancements

Often when trying to get data into the right shape, users need to deal with reconciling values that are conceptually the same but represented slightly different in their text form. Examples of this include company names, people names, cities, etc. with different representations or, simply, misspellings.

Last year, we introduced Fuzzy Matching options within the Merge Queries transformation, allowing users to match similar values when joining tables.

We’re now extending those capabilities, based on customer feedback, to support Fuzzy Matching capabilities within two other data transformations:

Fuzzy Group By

In Power Query, you can group values in various rows into a single value by grouping the rows according to the values in one or more columns. Up until now, the only matching criteria for grouping values was an exact match.

We have recently added new Fuzzy Matching options to the Group By dialog, allowing users to match similar values when trying to perform a Group By operation. In the example below, grouping rows to calculate total sales by person, and matching similar Name values.

Which results in the following summarized table:

You can learn more about the new Fuzzy Matching capabilities within Group By in this documentation article: https://docs.microsoft.com/en-us/power-query/group-by#fuzzy-grouping
Cluster Values

In many cases when dealing with slightly similar values that need to be reconciled, users may not want to merge tables nor group rows within a single table, but rather simply have a new column in their tables with the canonical value that is common across multiple slightly different values – what is known as a “cluster” value.

The newly added “Cluster values” transform does exactly that. It can be found within the “Add Column” tab in the ribbon and provides Fuzzy Matching options to give users the most flexibility in clustering values, including the ability to obtain the similarity score between the input and canonical values for each row of data.

The output from this transformation is a new column with the clustered values and (optionally) another column with the similarity score between values.

Copy/Paste Queries between PQ Online and PQ Desktop

With Power Query being available across several products, including Power BI (Desktop & Dataflows in PowerBI.com), Excel, Power Apps (Dataflows), Dynamics 365 Customer Insights, Azure Data Factory, Power Automate and more, it’s very common for users to want to move their queries across products. While it is possible to copy-paste M code within the Advanced Editor, this can be complex when trying to copy multiple queries with dependencies.

In Power Query Desktop, it has been possible for some time to right-click (or CTRL+C) queries from the Queries pane, which would take care of copying the selected query (or multiple queries) and all of their upstream dependencies needed in order to the selected queries to work, then paste it into another Power Query Desktop instance (whether in Power BI, Excel or SQL Server Data Tools).

Recently we enabled the same capability within Power Query Online, in order to make it extremely easy to move queries between Power Query Desktop and Online, or between multiple Power Query Online instances.

Copy data from Query Editor preview

An even more basic scenario than copy-pasting queries for users is to copy-paste data results from the Power Query Online Editor – whether it is copying an entire table, a column, a cell value, etc. – These options have all been recently added to Power Query Online; simply right-click on the desired element to copy it to the clipboard.

Multi-select support in Queries pane

It is now possible to perform actions over multiple queries at once by multi-selecting them (CTRL+Click or Shift+Click) in the Queries pane.

Multi-select actions include the ability to Copy, Paste, Delete or move queries to a query group.

Formula bar – Query Script mode

We have introduced a new mode in the Formula Bar, allowing users to switch from step-level script to the full query script. Users can toggle between the two modes from the View tab in the ribbon, or the new shortcut in the Status Bar.

Status Bar & Evaluation Performance Counters

We recently added a new status bar to the Power Query Online Editor dialog, enabling users to:

• Get a summary view of any validation warnings in their queries.
• See column & row counts for the currently selected query.
• When query evaluations are in progress, see performance counters and progress indicators to better understand where time is spent during a query evaluation.
• When Data Profiling is enabled, allow users to switch between “Top 1,000 rows” and “Entire dataset” for their data profiles.
• Toggle between Step & Query script modes for the formula bar.
• Switch between Data & Schema views

Parquet Files connector

The new Parquet Files connector allows users to seamlessly import data from Parquet files into Power Query Online. With this connector users can point at specific files or use other connectors (such as Azure Data Lake Storage Gen2) to connect to a folder with multiple Parquet files and easily combine them together.

SAP HANA

A few weeks ago, the SAP HANA connector was released in Power Query Online. This connector allows customers to import data from SAP HANA Analytic and Calculation views from SAP HANA databases in the cloud and on-premises, via the On-premises data gateway, into Power Query.

Note that Advanced Options for this connector (such as Native Database Queries, Connection Timeout options, etc.) are supported currently at the M Engine level. Support for these options in the Get Data UX will light up later this year, in the interim they can be configured via the formula bar or Advanced Editor.

SAP BusinessWarehouse

Just like with SAP HANA, the existing SAP Business Warehouse connectors for Application and Message Server have been recently enabled in Power Query Online.

Snowflake

Recently we enabled the Snowflake connector in Power Query Online, enabling customers to import data from Snowflake Data Warehouse into Power Query Online.

Impala

The new Impala connector allows customers to import data from Impala clusters into Power Query Online, by either using a cloud-to-cloud connection or the On-premises data gateway for on-premises clusters.

FHIR

We recently added a new connector to Power Query Online that allows users to import data from FHIR. FHIR is a standard describing data formats and APIs for exchanging electronic health records. The FHIR connector has been available for a few months in Power BI Desktop and we’re very excited to bring now into Power Query Online. You can learn more about the FHIR connector in this documentation article: https://docs.microsoft.com/en-us/power-query/connectors/fhir/fhir

Azure Tables & Blobs

The new Azure Tables & Blobs connectors enable customers to seamlessly ingest data from their Azure Tables and Blob storage accounts using Power Query Online.

SQL Server – Advanced Options in Get Data UX

We have enhanced the SQL Server connector to include Get Data UX support for some of the most frequently used Advanced Options provided by this connector, including the ability to specify a native SQL statement, to include relationship columns, navigating using full hierarchy (e.g. including database schemas) and support for SQL Server failover.

These new options can be found under the “Advanced Options” section within the SQL Server connector dialog.

Privacy Levels support within Get Data UX

Privacy levels allow users to combine data securely across multiple data sources without unintentionally leak data between sources. You can learn more about Privacy Levels in this documentation article: https://docs.microsoft.com/en-us/power-query/dataprivacyfirewall

Recently, we introduced support for (optionally) specifying the Privacy Level for a given data source while configuring its connection settings in the Get Data UX.

Admin Take-over

One of the most requested capabilities within Dataflows management has been the ability to take over or reassign an existing dataflow to a different owner. We have recently introduced this capability so that Environment Admins can search for dataflows created by another user within the environment and reassign as needed, either to themselves or to someone else within their organization.

Environment admins can now see a new “All Dataflows” tab within the Dataflows page and perform these management tasks.

Incremental Refresh for Standard Dataflows

We have recently introduced the ability to configure partition-based incremental refresh for dataflows ingesting data into CDS. This allows users to only refresh data from the last N days/weeks/months/years based on an existing date or time column within their query results.

The new Incremental Refresh option can be found under the “…” menu for a dataflow.

You can learn more about Dataflows Incremental Refresh capabilities in this documentation page: https://docs.microsoft.com/en-us/power-query/dataflows/incremental-refresh

Create a new dataflow from Power Query template

We built an easy way for you to export your Power Query queries from Excel into a Power Query template. Then, you can use that template to quickly create a new dataflow in Power Apps. This saves you from manually moving queries over.

This new capability to “Export Power Query Template” is currently available to Office Insiders – You can find more about the steps to try it out in this Office blog announcement: https://insider.office.com/en-us/blog/power-platform-dataflows-from-excel-queries

This is just the first step in our journey to allow broad reusability of Power Query templates across all Power Query product integrations.

New Power Query & Dataflows documentation

We have added over 50 new Power Query & Dataflows documentation articles, including getting started and quick how-to guides for common data transformations, connector reference articles and best practices for working with Power Query & Dataflows.

You can access them at https://docs.microsoft.com/en-us/power-query/

That’s all for this month! We hope that you find these features useful and continue sending us your feedback and suggestions so we can improve and add new capabilities to Power Query & Dataflows.

The post What’s New in Power Query and Dataflows in Power Apps – November 2020 appeared first on Microsoft Power Platform Blog.

A Microsoft Power Platform Center of Excellence (CoE), as talked about in this blog post, refers to an entity that is responsible for nurturing the growth of Power Apps and Power Automate in their organization, while applying the right administrative guardrails.

The responsibilities of a CoE typically fall into these four buckets:

1. Administration & Governance
2. Nurture
3. Support
4. Operations

A CoE is designed to drive innovation and improvement, and through its central function can break down geographic and organizational silos in order to bring together like minded people with similar business goals to share knowledge and success, whilst at the same time providing standards, consistency, and governance to the organization.

For a more detailed explanation of the functions and best practices to establish a CoE – see the talk titled “Tried and tested techniques for establishing a CoE”, that was presented at the Microsoft Business Applications Summit in June 2019. You can download the ppt here.

As the talk lays out, a CoE could start off quite simple with a single individual using the provided tools and best practices to get a view into Power Apps activity in their organization, or may grow into a more mature investment with multiple functions and roles, such as in the case of Chevron, where they’ve established a 20+ person CoE to manage multiple aspects of governance, training, support and automated app deployment across the organization.

We would encourage the reader to understand where they are in their adoption journey and invest accordingly. A key principle is to get clear about why you’re setting up a CoE, what you aim to accomplish and the business outcomes you hope to achieve. Then get started, and learn and evolve along the way. For many, the CoE is a first step in fostering greater creativity and innovation across the organization by empowering business units to digitize and automate their business processes, while maintaining the necessary level of central oversight and governance. In its fullness, a CoE typically encompasses all the functions mentioned above, in some form.

Center of Excellence Starter Kit

The Center of Excellence (CoE) Starter Kit is a collection of templatized best practices, that are designed to help organizations get started with tools needed to set up a CoE. The kit focuses on both Administration and Governance function, as well as Nurture functions of the CoE.

The download link for the CoE package, including setup instructions, documentation  and source code can be found at the end of this blog post.

Before we dig into CoE Starter Kit features, we did want to take a slight detour to explain our philosophy around rolling out Admin & Governance features and point out where the CoE Starter Kit fits in that context.

Why the need for a CoE Starter Kit

Our Admin and Governance capabilities can be broadly categorized into three categories.

Out of the Box features:  These core capabilities for admins and makers exist in the product admin portals and are the easiest and most robust way to complete tasks. For example, Environment and DLP Policy creation can be executed in the PowerApps and Flow admin centers.

Platform extensions: There are four ‘admin connectors’ that provide access to the same APIs that the out-of-box product use. These have been exposed through the connectors library to give users the ability to create custom solutions to execute administrative or governance related tasks. Please refer to the Admin connectors blogs for details. For example, provisioning a new Environment can be automated using the admin connectors.

The use of the extensions (and templates below) represents a feedback loop, that we use to help prioritize our out of the box product admin  features.

Templates and Customization: The CoE Starter Kit falls into this category. It is a set of templates that use the admin connectors in combination with other connectors and formulas to achieve some specified goal. The nature of a template is to provide a good solution for that specific task, but it might not have the exact functionality that everyone needs and could require adjustment to achieve what others might be looking for. For example, admins could use the Environment creation template that comes with the  Starter Kit, or they can use it as a starting point and modify it as appropriate.

CoE Starter Kit Features

To delve into the administration and governance steps in more detail, watch the admin breakout sessions from the 2019 Microsoft Business Applications Summit and the Managing and supporting Power Apps and Power Automate at scale session from 2019 Ignite.

The starter kit consists of multiple tools that are designed to help facilitate some of the responsibilities of a CoE – the tools and components are split into three solutions:

1. Center of Excellence – Core Components
   These components provide the core to get started with setting up a CoE – they sync all your resources into entities and build admin apps on top of that to help you get more visibility of what apps, flows and makers are in your environment. Additionally, apps like the DLP Editor and Set New App Owner help with daily admin tasks.
   The Core Components solution only contains assets relevant to admins. No assets need to be shared with other makers or end users.
   Requirement: User(s) will require a Per User license, as well as Global or Power Platform Admin permissions

   Scenario	Toolkit component
   Catalog tenant resources	1.CDS Entities: Environments, Apps, Flows 2.Admin | Sync Template v2 (Flows) – all, Apps, Flows, Custom Connectors, Connectors, Model Driven Apps 3.Admin | Sync Audit Log (Flow) 4.Power BI Dashboard 5.Custom Connector for Office 365 Audit Logs 6.Power Platform Admin View (Model Driven App)
   DLP Strategy + Visibility	7.DLP Editor (Canvas App) 8.DLP Customizer (Canvas App)
   Change App Ownership	9.Set New App Owner (Canvas App)

2. Center of Excellence – Governance Components
   Once you are familiar with your environments and resources, you might start thinking about audit and compliance processes for your apps. You might want to gather additional information about your apps from your makers, you might want to audit specific connectors or app usage – apps like the Developer Compliance Center and flows to identify connector usage part of this solution will help with that.
   The Compliance Components solution contains assets relevant to admins and existing makers.
   The Compliance Components provide a layer on top of the Core Components, it is required to install the Core Components prior to using the Audit Components.
   License Requirement: Makers participating in the audit and compliance workflows will need a Per App or Per User License.

   Scenario	Toolkit component
   Sample Audit Process	1.Developer Compliance Center (Canvas App) 2.Flow – Compliance detail request 3.Business Process Flow for Auditing resources
   Archive unused apps	4.App Archive and Clean Up – Start Approval and Check Approval (Flows) 5.App Archive and Clean Up Admin View (Model Driven App)
   Act based on certain connector usage	6.Find and add admins as owners for apps that leverage certain connectors (Flow) 7.Find and disable flows that leverage certain connectors (Flow)

    

3. Center of Excellence – Nurture Components
   An essential part of establishing a CoE is nurturing your makers and an internal community. You will want to share best practices and templates and onboard new makers – the assets part of this solution, like the Welcome Email and Template Catalog can help develop a strategy for this motion.
   The Nurture Components solution contains assets relevant to everyone in the organisation.
   The Nurture Components provide a layer on top of the Core Components, it is required to install the Core Components prior to using the Nurture Components.
   License Requirement: Anyone in CoE community will need a Per App or Per User License.

   Scenario	Toolkit component
   Onboard new makers, provide training and share best practices	1.Admin | Welcome Email (Flow) 2.Template Catalog (Canvas App) 3.Admin | Newsletter with Product Updates (Flow) 4.Training in a day Management and Registration (Canvas Apps) 5.Training in a day Feedback Reminder, Registration Confirmation and Reminder (Flow)
   Encourage Adoption	6.App Catalog (Canvas App)

These scenarios align with real needs we observed throughout our customer interactions, and are intended to act as a starting point that should be extended.

DLP Strategy

Today, if a DLP policy is created by an admin, there is no good way of determining what apps will be affected ‘out-of-box’ in the Power Apps admin center. Although policies should definitely be put in place to increase security, implementing a policy in a tenant with existing resources risks disrupting some existing business processes. The goal of the DLP Editor canvas app is to provide admins a view of what resources will be disabled if a new or updated policy were to be enforced. Using the admin connectors, it is able to read DLP policies and details about each app’s connection references, and outputs a list of apps with connection references in opposing groups. Since the connectors also provide details about the current app owner, the app allows the user to communicate with the owner to determine the best course of action based on the requirements of the policy.

Catalog tenant resources, visualize data in Power BI

The admin centers provide a view of Power Apps and Power Automate resources in each environment to admins, but it does not have all the data listed in a tenant-wide view and there are some additional details that might be useful to display together. To make visualizing data in the tenant easier, multiple components are included to help enable such a tenant-wide view.

First, a Flow called “Admin | Sync Template” uses the admin connectors to read all data in all environments, including Power Apps, Power Automate, Connectors, Connection References and Makers. This data is stored in CDS entities that match the schema of the objects returned from the admin connectors, basically copying the data from the API. Once the data is synced, there is a master list that can be easily visualized in the provided Power BI Dashboard. Although it might be the same data that was accessible before in the admin center simply rearranged, there are many meaningful insights that can be made from this kind of command over the data in Power BI. For example, it will show the most active makers and region of makers, or most used connectors across the tenant. This detail can be useful for IT to understand which areas need more support, which might not have been easily surfaced otherwise.

Another component syncs audit log data into CDS, giving the ability to aggregate session count and monthly active users (MAU) totals for apps. This can help identify which apps need more attention; Higher usage can indicate which apps are most used and therefore more prone to affecting a potentially high critical business processes. It’s not only important to understand what apps need support, but also to keep track of how Power Apps and Power Automate resources have been increasing productivity by replacing an old or creating a new business processes.

The Power BI Dashboard is now also available via app source.

App Audit example process

In the example provided, fostering communication between the center of excellence and power users is mutually beneficial. If makers provide more information about the app, those in charge of security and support can be aware of the requirements, risks and business justification for the app. In turn, the app maker can expect a better support model for development and maintenance.

In the starter kit, the PowerApps App entity has additional fields that represent requirements from the center of excellence, which the app maker is responsible for filling out. There is a Flow that reads all the records in the PowerApps App entity, and sends a notification to the app maker if any app is not compliant with the company’s agreement policy. The threshold for determining if an app is compliant is that the maker has provided business justification requirements and the app has been published in the past 60 days. If the app is not compliant, the maker must go to the Developer Compliance Center canvas app and follow the instructions to stay compliant.

Once the business requirements are submitted and ready for review, an admin can review the details in a Power Platform Admin View model driven app. There is a view that filters out all the apps that are compliant and need final validation from an admin. On each app, there is a business process flow that is designed to guide the audit process from reviewing the details to assigning it to the app catalog.

This audit process example in the starter kit only covers PowerApps apps–can you think of a good audit process for validating Flows or Environments?

App Catalog

There are many reasons why an app catalog might be used, but the most common reason is for discoverability. Normally if an app is shared broadly, it might be difficult for some end users to find the correct environment, or dig up the share link that was initially sent out. The canvas app provides a template for displaying apps that have been marked true for ‘In App Catalog’ and ‘Featured In App Catalog’, and have a category, which are all metadata fields in the CDS PowerApps App entity. In this scenario, apps can be featured in the app catalog once they have passed the audit process. This means that not only is the app more discoverable for end users, but also ensures that the app that has been shared broadly is being monitored by some authoritative body. One example of how you can extend this canvas app is to add the ability for users to rate, comment and favorite the apps.

Welcome Email to new makers

If a new maker is detected during the Sync Template Flow, another Flow is kicked off to send a welcome email to that new maker. This scenario is important from a nurturing perspective, because it provides a proactive level of support that will foster desired practices. For example, the welcome email has the option of sharing a list of learning resources and the company’s internal Yammer channel for peer support. This makes the development experience better for the makers because they have a larger channel to seek help from, which can grow more independently and can be self sufficient. This email provides the best medium to introduce other first-time material to makers, such as links to data compliance policies and things of that nature (this is an example of a potential extension).

The solution can and should be customized in the ways that fit each unique organization. Some companies might prefer a more strict auditing process, where they decide to delete resources if not compliant. Others might prefer an even more relaxed approach that what is there. These details are unique to each organization and might not always be common. Understanding what those processes that dictate the way the Power Platform is governed is the overall requirement of the center of excellence, these tools are just means to this end.

In Part 2 of this blog post we share strategies on how to get started with adopting the CoE Starter Kit.

Download

Read the documentation to learn about all the components in the solution, the installation instructions for the solution and additional information on developing a Center of Excellence. This template is intended to represent an example tool set with extendable components, which should be customized to meet each organization’s requirements.

Directly download the solution pack at aka.ms/CoEStarterKitDownload.
View the full documentation and setup instructions at https://docs.microsoft.com/power-platform/guidance/coe/starter-kit

Disclaimer

Although the underlying features and components used to build the Center of Excellence (CoE) Starter Kit (such as Common Data Service, admin APIs, and connectors) are fully supported, the kit itself represents sample implementations of these features. Our customers and community can use and customize these features to implement admin and governance capabilities in their organizations.

If you face issues with:

• Using the kit: Report your issue here: aka.ms/coe-starter-kit-issues. (Microsoft Support won’t help you with issues related to this kit, but they will help with related, underlying platform and feature issues.)
• The core features in Power Platform: Use your standard channel to contact Support.

The post Introducing the PowerApps Center of Excellence Starter Kit appeared first on Microsoft Power Platform Blog.

Each script below allows you to retrieve a particular filtered set of PowerApps and Flows and outputs a list of matching records in a .CSV file. Each record represents the feature used (e.g. custom connector) and includes details such as the App ID, owner of the app, etc.

Install the modules

To use the PowerApps and Flow cmdlets, follow the installation instructions on the PowerApps cmdlet documentation page.

Run the scripts

Once the modules are installed, follow these instructions to run the scripts provided below. If you receive a security warning, you may need to unblock running the downloaded script, see this article for more details.

1. Download the desired script.
2. Run PowerShell as an administrator and make sure you’re in the same directory as the script.
3. Run the script by typing out the name

   .\findFlowsWithHttpAction.ps1

4. Each of these scripts have optional parameters to specify the Environment (EnvironmentName) or the output file path name (Path).

   .\findFlowsWithHttpAction.ps1 -EnvironmentName 820d6103-3f73-4107-a1b2-3449a98f5049 -Path ./myFlowsWithHttp.csv

Scripts

The scripts are designed to output a list of records in a .CSV file that will be created in the same directory as the script, unless specified otherwise. Each record represents a feature, so if there are multiple features in a single PowerApp or Flow, there will be multiple records (e.g., a Flow that uses a custom connector and Http action will have two records for the same Flow).

PowerApps with Custom Connectors

Lists connections to custom connectors being used in a PowerApp.

Download

PowerApps with Premium Connectors

Lists connections to premium connectors being used in a PowerApp.

Download

PowerApps with on Prem Connectors using Data Gateway

Lists connections to an On Premise gateway being used in a PowerApp.

Download

PowerApps used as SharePoint custom forms

List PowerApps that are used as custom forms in the SharePoint List experience.

Download

Flows with HTTP Actions

Lists Flows that use the HTTP request action.

Download

Flows with Custom Connectors

Lists connections to custom connectors being used in a Flow.

Download

Check back as we plan to add to this list!

Updates

• 2019-03-12: Added 3 scripts: one to discover SharePoint custom form PowerApps, one for managing trial environments, and one for managing trial licences (contributed by Steve Jeffery)

The post PowerShell scripts to discover and manage specific features in the Power Platform appeared first on Microsoft Power Platform Blog.