Previously, you needed to install and configure an on-premises data gateway to trigger your desktop flows from the cloud. We are excited to announce that Power Automate Desktop can now connect your machine directly to the cloud and run desktop flows without the use of a gateway. We have also introduced new machine-management capabilities to help scale your robotic process automation (RPA). Try them out starting today, available in preview.

How does it work?

By connecting your desktop to the cloud, you can realize the full potential of your robotic process automation (RPA), choosing when to automatically run your automation, even without needing to be present. Today, the on-premises data gateway creates this bridge by establishing an enterprise-grade secure connection between your device and Power Automate.

We also listened to your feedback and added more monitoring experiences to better understand how your infrastructure is used, optimize it by levering the new management features or scale it using machine groups.

Aiming to keep the set-up experience as streamlined as possible, there are no changes to the installation process. Once Power Automate has been installed, signing into the application will automatically register the machine with the cloud.

When your desktop is successfully registered, a new Machine is created and you can view and edit the settings of this machine directly from Power Automate Desktop. This will create a secure connection between the machine and the cloud environment.

Admins can also control who can register machines and how they can interact with them using a granular set of built-in permissions and roles.

Building off of the monitoring experiences we shipped last December, the Desktop flow queues page is transitioning to become a more robust machine management experience. Here, you can find the list of all the machines that you have created or to which you have access. You can also continue to access the run queues for your existing gateways, if need be.

Scale your automation with machine groups

Just as you could pool your gateways into a gateway cluster, you can easily organize your machines into a Machine group. This concept is important to help scale automations within a business.

It’s now simple to manage access to your machines and machine groups using the familiar share experience. Machines and machine groups also benefit from the sophisticated load distribution and run queue system that we shipped last December.

To connect to your machine, you will continue to use the existing desktop flows connector. You just need to specify your connection option (directly to machine or using an on-premises data gateway) then select your machine from the list and sign in as usual.

More information for better monitoring

Users can also benefit from the additional machine information from within the Desktop flow runs page, where you can now filter your desktop flow runs to those from a specific machine or machine group.

Click here to get started: Manage your flows | Microsoft Power Automate.

Additional documentation is available here:  Manage machines

Hoping that you will find the above updates useful, please feel free to provide your questions and feedback in the Power Automate Community. If you want to learn more about Power Automate Desktop, get started with the below resources:

• Learn more about Power Automate Desktop in our documentation
• Sign up for a free Power Automate trial
• Download Power Automate Desktop

The post Connect directly to machines and new machine management for desktop flows appeared first on Microsoft Power Platform Blog.

• Gateway Member Status, State and Remove Operation
• Gateway Cluster Status
• Remove Gateway Clusters
• Gateway Cluster Settings

Now all features (including these new ones) which gateway admins have access to are available for tenant admins also.

Gateway Member Status, State and Remove Operation

We have added a lot of new management features for gateway members. You can access these features by clicking on the “Details”.

You now see the details in a tabular format which is easier to read.

• Gateway Name- The name of the gateway member.
• Device name – The physical device on which the gateway is installed
• Version – This is the gateway software version installed on the machine
• Status- You can check the status of gateway members by clicking on the icon.
• State – You can enable or disable a gateway member using this toggle. You may want to disable a gateway member when the gateway would be offline for updates, etc.

Gateway Cluster Status 

You can now see the gateway cluster status, so clicking on .

Remove Gateway Cluster  

You can now remove gateway cluster both standard and personal modes from the admin center now.

Gateway Cluster Settings 

In the past only gateway admins could see Gateway Cluster Settings now Tenant admins can also see these settings.

We have also recently introduced Load balancing settings on the admin portal. We have now classified settings as general and ones which applicable to specific to services like Power BI.

Please continue to send us feedback for what new capabilities you’d like to see in the future.

The post New Gateway management features on the Power Platform Admin Center appeared first on Microsoft Power Platform Blog.

Download Latest On-premises Data Gateway

Here is a feature that we would like to highlight with this month’s  release:

Change Gateway Recovery Key

The recovery key provided by gateway admins during installation of gateways cannot be changed today This key is used to create the symmetric key which in turn is used for encrypting credentials in data sources/connections using that gateway.

With the November release of Data Gateways, you will now be able to rotate this key.  More information about recovery keys, detailed description on how to perform this change and associated limitations can be found in the data gateway docs.

Here are some of the features that were released with the October release of gateway:

• Load balance based on CPU and Memory throttling
• Performance Monitoring Improvements
• Improved Gateway Documentation

Load balance based on CPU and Memory throttling

Gateway admins can, now, throttle resources of each gateway member to make sure either a gateway member or the entire gateway cluster isn’t overloaded causing system failures.

If a gateway cluster with load balancing enabled receives a request from one of the cloud services (like Power BI), it randomly selects a gateway member. If this member is already at or over the throttling limit set for CPU or memory, another member within the cluster is selected. If all members within the cluster are in the same state, the request would fail.

To enable this feature, gateway admins would update the following settings in the Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.dll.config file available in the Program Files\On-premises data gateway\ folder.

• CPUUtilizationPercentageThreshold – This configuration allows gateway admins to set a throttling limit for CPU. The permissible range for this configuration is between 0 to 100. A value of 0, which is the default, would indicate that this configuration is disabled.
• MemoryUtilizationPercentageThreshold – This configuration allows gateway admins to set a throttling limit for memory. The permissible range for this configuration is between 0 to 100. A value of 0, which is the default, would indicate that this configuration is disabled.
• ResourceUtilizationAggregateionPeriodInMinutes – This configuration is the time in minutes for which CPU and memory system counters of the gateway machine would be aggregated to be compared against the respective threshold limits set using configurations mentioned above. The default value for this configuration in 5.

Performance Monitoring Improvements

Based on your feedback,  we have now included an additional file called the Query start Report containing the Query start time and also the actual query text. You no longer need to turn on additional logging to obtain the query text. The Query start report contains the following attributes:

In addition to the new report file, we have also updated the Performance monitoring template  in this release. The new version includes not only the new Query text and start time but also new visuals to improve the overall troubleshooting  experience.

Improved Gateway Documentation

We have heard your feedback and re-written the gateway documentation. We split them it into content that’s specific to Power BI and general content that applies to all services that the gateway supports. To provide feedback on each article, or the overall gateway docs experience, scroll to the bottom of any article.

Please continue to send us feedback for what new capabilities you’d like to see in the future.

The post On-premises data gateway November 2019 update is now available appeared first on Microsoft Power Platform Blog.

Manage gateways by Region

In the past,  only gateways within your tenant’s default region were visible on the admin center. Now we are also making management of gateways in other regions possible.

Select the region dropdown to see the list of gateway regions. When you select one of the regions, you will see the list of gateways installed in that region. You can now manage users or view gateway members for these gateways.

Additionally, we have also updated our new PowerShell cmdlets in Public Preview to include region specific cmdlets. Do provide any feedback so we can continue to improve these.

Share Gateways

Earlier this year, we enabled managing administrators for gateways from the power platform admin center. You can now share gateways as Admin,   “Can Use” or “Can use + share”.

Note:  “Can Use” or “Can use + share” applies only to Power Apps and Power Automate.

Get Help

If you run into issues while managing gateways on the Power Platform Admin center, you could create a support ticket with Microsoft. While doing so, please include the session id shown below. This would help us troubleshoot the issue faster.

Please continue to send us feedback for what new capabilities you’d like to see in the future.

The post On-premises Data Gateway management by Region appeared first on Microsoft Power Platform Blog.

Note: These cmdlets require PowerShell core 6.2.2 or higher

The cmdlets are available on PowerShell Gallery and can be installed in an elevated PowerShell session:

Install-Module -Name DataGateway

The entre list of cmdlets can be found using the following cmd.

Get-Command -Module DataGateway*

Examples and descriptions are included in the cmdlets and you can access them using:

get-help <cmdlet-name>

We also plan to make an online version of the documentation available for these cmdlets soon.

Gateway Management:

Following are PowerShell cmdlets which can be used for administering gateway clusters and members:

• Get-DataGatewayCluster: Lists all gateway clusters for the current user or a specific one based on the passed parameters
• Get-DataGatewayClusterStatus: Returns the cluster status along with additional properties
• Get-DataGatewayInstaller: Lists users authorized to install and register gateways within the tenant
• Get-DataGatewayTenantPolicy: Returns the gateway installation and registration policy for the tenant.
• Add-DataGatewayClusterUser: Add a specific user to a gateway cluster for specific permissions
• Set-DataGatewayCluster: Set properties of an existing gateway cluster
• Set-DataGatewayInstaller: Modify list of users who can install and register new gateways on the tenant
• Set-DataGatewayTenantPolicy: Set the gateway installation and registration policy for the tenant
• Remove-DataGatewayCluster: Remove a gateway cluster
• Remove-DataGatewayClusterMember: Remove a gateway member from the corresponding gateway cluster
• Remove-DataGatewayClusterUser: Remove a user from a gateway cluster

In addition, wherever applicable, these cmdlets can be run under a tenant level scope. This would of course require the user to be a global Office 365 admin or a Power BI service admin. For instance, the following cmdlet would return all gateways within your tenant.

Get-DataGatewayCluster -Scope Organization

Similarly, running the same command without the scope parameter or with the “Individual” scope would return the list of gateways, you are administrator of.

Get-DataGatewayCluster -Scope Individual

Another new cmdlet, which allows tenant admins to restrict installation of on-premises data gateway ins standard mode is as follows. The tenant admin can then allow specific users to install using the Set-DataGatewayInstaller cmdlet.

Set-DataGatewayTenantPolicy -ResourceGatewayInstallPolicy Restricted

Only the following cmdlets support the scope of Organization:

• Add-DataGatewayClusterUser
• Remove-DataGatewayClusterUser

We will be enriching these cmdlets with additional management operations in the upcoming releases.

Note: You have the option of using Azure Data Studio with the PowerShell extension to work with these cmdlets:

Please continue to send us feedback for what new capabilities you’d like to see in the future.

The post On-premises data gateway management via PowerShell (Public Preview) appeared first on Microsoft Power Platform Blog.

Download Latest On-premises Data Gateway

This month’s release includes a .Net Framework Update.

.Net Framework Update

We are using .NET 4.7.2 framework for gateways July 2019 version or higher hence some of the operating systems we used to support may no longer be supported. More information about OS which will be supported here.

With the July 2019 version of the gateway, if you don’t have the .NET 4.7.2 framework or higher but are using a supported OS version, the gateway install will prompt the .Net framework install. Choosing not to install the .Net framework will fail the installation of the gateway.

Please continue to send us feedback for what new capabilities you’d like to see in the future.

The post On-premises data gateway July 2019 update is now available appeared first on Microsoft Power Platform Blog.

Here are some of the things that we would like to highlight with this month’s release:

• Gateway Performance Monitoring (Public Preview)
• May version of the mashup engine

Gateway Performance Monitoring (Public Preview)

We are excited to release this new feature for monitoring gateway usage and performance. Traditionally, for monitoring performance, gateway admins have had to depend on manually monitoring performance counters through the Windows Performance Monitor tool. This feature is an out of the box feature which includes additional logging regarding queries and system counters along with a Gateway Performance PBI template file to visualize these. This would insights into gateway usage and allow troubleshooting slow performing queries.

Note: this feature is currently available only for the On-premises data gateway in the standard mode and not for the personal mode.

When you turn this feature on, 3 new log files would be created:

• Query Execution Report – Contains detailed query execution information.
• Query Execution Aggregation Report – Contains query information aggregated to a time interval. The default value is 5 minutes but can be adjusted as described below.
• System Counter Aggregation Report – Contains system counter values aggregated to a time interval. The default value is 5 minutes but can be adjusted as described below.

To enable this feature, please make the following changes to the Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.dll.config file in the “\Program Files\On-premises data gateway” folder

• QueryExecutionReportOn – This attribute when updated to “True” enables additional logging for queries executed using the gateway and would create both the Query Execution and the Query Execution Aggregation Report files.

<setting name="QueryExecutionReportOn" serializeAs="String">

<value>True</value>

</setting>

• Set SystemCounterReportOn – This attribute when updated to “True” enables additional logging for memory and CPU system counters and would create the System Counter Aggregation Report

<setting name="SystemCounterReportOn" serializeAs="String">

<value>True</value>

</setting>

Other values in the config file related to this feature which may be of interest to you would be:

• ReportFilePath – Determines the path where the 3 new log files get stored using the attribute. This is by default the “\Users\PBIEgwService\AppData\Local\Microsoft\On-premises data gateway\Report” or “Windows\ServiceProfiles\PBIEgwService\AppData\Local\Microsoft\On-premises data gateway\Report” depending on the OS version. The PBIEgwService may need to be replaced with the service account running the data gateway.
• ReportFileCount – Determines the number of log files of each kind to be retained. The default value is 10.
• ReportFileSizeInBytes – Determines the size of the file to be maintained. The default value is 104857600.
• QuerExecutionAggregationTimeInMinutes – Determines the number of minutes for which the query execution information would be aggregated. The default value is 5.
• SystemCounterAggregationTimeInMinutes – Determines the number of minutes for which the system counters would be aggregated. The default value is 5.

Once you have made the above changes to the config file, do not forget to restart the gateway for these config values to take effect. You will now start seeing the report files getting generated in the ReportFilePath.

Note: Please wait for 5 minutes until the aggregate files get created in the ReportFilePath folder.

Now, to visualize this data download the Gateway Performance PBI template file and open it using Power BI desktop. You should see a pop-up for the Folder path. Please make sure this is the same as the value in ReportFilePath.

Click on Load and the template file would now start loading the data from your log files. All visuals should then be populated using the data in the reports. You can save this file as a PBIX and publish it to your service for automatic refreshes. Additionally, you could also customize this template file to suit your needs. Do provide any feedback on this new feature and any additional visuals which you would want us to add to the template.

May version of the mashup engine

This month’s Gateway update also includes an updated version of the Mashup Engine.

Last month, we also made available along with the latest gateway download, the previous 4 monthly gateway updates. This will make rolling back to an older version easier in case of issues/exceptions. This will also aid with troubleshooting.

 Please note that this upcoming changes may impact you:

We would be using .NET 4.7.2 framework for gateways July 2019 version or higher hence some of the operating systems we used to support may no longer be supported. More information about OS which will be supported here.

Please continue to send us feedback for what new capabilities you’d like to see in the future.

The post On-premises data gateway May 2019 update is now available appeared first on Microsoft Power Platform Blog.

We are now excited to add 2 new capabilities to this feature:

• Visibility into On-premises data gateway (Personal mode)
• Manage installers for On-premises data gateways

Note:  On-premises data gateway (Personal mode)  can be used only with Power BI

Visibility into On-premises data gateway (Personal mode)

With the last release global admins had visibility into all On-premises data gateways within their enterprise. Now they will also be able to see all personal gateways.

Once you log in as a tenant admin, click on the Data Gateway option in the left panel.  By default, this will load the On-premises data gateways.

Click on the filter in the top right corner and select “On-premises data gateway (Personal mode)” to see all the personal gateways in your enterprise. These gateways will have their owners called out as administrators.

Clicking on “Open gateways” would show the machine running the personal gateway and its version

Clicking on the “Manage administrators” would show the owner of the personal gateway. This cannot be changed due to the security scope of personal gateways.

Note: On-premises data gateways in non-default environments are currently not included.

Manage installers for On-premises data gateways

This feature would help you as a global admin manage who can install On-premises data gateway within your enterprise. Click on the “Manage Gateway installers”

By default, this option would be turned off allowing anyone in your organization to install but you can turn on this feature on by using the toggle below.

You can then add users who can install gateways by typing the name or email in the text box and clicking on Add.

Similarly, you can also remove users who have install permissions. To do that, you would click on the icon next to the installer in question. Here, the tenant admin is removing Anna who we added in the last step. Clicking on Confirm below would remove Anna from the list of gateway installers.

Note: This does not impact gateways which are already installed. This feature only allows or restricts users from installing new gateways going forward.

If a person who doesn’t have access to install gateways tries to install one, they would get the following error once they provide their credentials during gateway registration.

Known issues

We are actively working on fixing the following issue and will update the blog once we do:

• Groups aren’t supported at this point. Only individual users can be added as gateway admins or installer. Existing groups which are gateway admins will not be displayed .

As always, we would love to hear more from you, learn from your experience and improve based on your feedback. Thank You!

The post Enhancements to On-premises data gateway management in the Power Platform Admin center appeared first on Microsoft Power Platform Blog.

———————————

Today, I want to talk about something that opens up the flow beyond the scope of connecting cloud services and being able to help running tasks on your local computer. Very often I forget to lock my computer before leaving the desk only to remember it later in a meeting, but by this time the damage would already be done, such as a prank e-mail to the team from my mailbox. So, we are going to attempt a solution to this problem using the following capabilities:

• Gateways
• File System connector
• Basic PowerShell scripting

Approach

Gateways provide a way for on-premise services to connect to the cloud. Use cases involve connecting a on-premise SQL database to a flow for use in internal systems where customers don’t want their SQL data in the cloud.

In addition, flow buttons are an elegant way to trigger a flow with a single click. (We could also integrate a hardware button from home that could do the same with a HTTP request.)

With the above said, the solution involves the following:

A PowerShell script that is running in your local computer. The script is watching a directory for a file event and locks the computer on it:
 

$watcher = New-Object System.IO.FileSystemWatcher

$watcher.Path = "C:\Lock"

$watcher.Filter = "*.*"

$watcher.IncludeSubdirectories = $true

$watcher.EnableRaisingEvents = $true 

$action = {

    . rundll32.exe user32.dll,LockWorkStation

}   

### DECIDE WHICH EVENTS SHOULD BE WATCHED

Register-ObjectEvent $watcher "Created" -Action $action

Register-ObjectEvent $watcher "Changed" -Action $action

Register-ObjectEvent $watcher "Deleted" -Action $action

Register-ObjectEvent $watcher "Renamed" -Action $action

while ($true) {sleep 5}

Install the Data Gateway configure it with Flow by following the instructions.

Once the installation is complete and the Data gateway is configured, add a “File System” connection by navigating to the “Gear” icon -> Connections -> Create Connection -> File System and select the “Connect via on-premise data gateway”. You should be able to see your data gateway in the drop down list. Complete the connection creation.

The flow in itself is very simple as it just needs to create a file on the click of a button. Most of the heavy lifting is done by the PowerShell script.

Since I added a button trigger, I can use it from my Flow app on my phone to lock my computer on demand.

Now at the click of a button I can lock my workstation and no more prank e-mails. We are also automating our internal deployment with a combination of this technique and approvals to allow a smooth automated deployment workflow using Flow.

Summary

Once completing the seemingly innocent flow, we can extend this to do a variety of tasks by simply defining a protocol over the FS and the PowerShell script and automate many workflows that would require an on-premise workstation to be in picture.

As part of walking through the Flow, we learn the following capabilities from Flow:

• Data gateways
• File System connector
• Flow buttons for mobile

Go forth and automate!

The post Flow of the Week: Local code execution appeared first on Microsoft Power Platform Blog.