Earlier this year, we introduced VNet connectivity for Power BI Datasets and we are now excited to announce the public preview of VNet connectivity for Power Platform Dataflows. The new VNet connectivity feature enables secure outbound connectivity from Power Platform dataflows to data sources within your Azure VNet.

Virtual network (VNet) data gateways (Microsoft-managed) eliminate the overhead of installing, updating, and monitoring on-premises data gateways for connecting to data sources associated to a VNet. They, however, still follow the familiar process of managing security as with an on-premises data gateway.

During public preview, Power platform dataflow users will have access to this feature for free.
For more information on supported data sources please see Supported data sources for Power Platform dataflows.
Creating VNet Gateways, is a 3-step process:

Once you create a VNet gateway, you can manage it on Power platform admin center where you could manage administrators as you do for On-premises data gateway.

You can then use a VNet data gateways in Power Query Online while authoring a dataflow just like you would use On-premises data gateway:

More information on VNet data gateways, limitations, FAQs see What is a virtual network (VNet) data gateway (Preview) | Microsoft Docs.

Do try this feature and continue to send us feedback on how we can improve it but also what new capabilities you’d like to see in the future.

The post Announcing VNet Connectivity for Power Platform Dataflows (Public Preview) appeared first on Microsoft Power Platform Blog.

• Gateway Member Status, State and Remove Operation
• Gateway Cluster Status
• Remove Gateway Clusters
• Gateway Cluster Settings

Now all features (including these new ones) which gateway admins have access to are available for tenant admins also.

Gateway Member Status, State and Remove Operation

We have added a lot of new management features for gateway members. You can access these features by clicking on the “Details”.

You now see the details in a tabular format which is easier to read.

• Gateway Name- The name of the gateway member.
• Device name – The physical device on which the gateway is installed
• Version – This is the gateway software version installed on the machine
• Status- You can check the status of gateway members by clicking on the icon.
• State – You can enable or disable a gateway member using this toggle. You may want to disable a gateway member when the gateway would be offline for updates, etc.

Gateway Cluster Status 

You can now see the gateway cluster status, so clicking on .

Remove Gateway Cluster  

You can now remove gateway cluster both standard and personal modes from the admin center now.

Gateway Cluster Settings 

In the past only gateway admins could see Gateway Cluster Settings now Tenant admins can also see these settings.

We have also recently introduced Load balancing settings on the admin portal. We have now classified settings as general and ones which applicable to specific to services like Power BI.

Please continue to send us feedback for what new capabilities you’d like to see in the future.

The post New Gateway management features on the Power Platform Admin Center appeared first on Microsoft Power Platform Blog.

• Gateway Member Status, State and Remove Operation
• Gateway Cluster Status
• Remove Gateway Clusters
• Gateway Cluster Settings

Now all features to which gateway admins have access are also available for tenant admins (including these new ones.)

Gateway Member Status, State and Remove Operation

We have added a lot of new management features for gateway members. You can access these features by clicking on “Details”.

You now see the details in a tabular format which is easier to read.

• Gateway Name- The name of the gateway member.
• Device name – The physical device on which the gateway is installed
• Version – This is the gateway software version installed on the machine
• Status- You can check the status of gateway members by clicking on the icon.
• State – You can enable or disable a gateway member using this toggle. You may want to disable a gateway member when the gateway would be offline for updates, etc.

Gateway Cluster Status 

You can now see the gateway cluster status, so clicking on .

Remove Gateway Cluster  

You can now remove gateway cluster both standard and personal modes from the admin center now.

Gateway Cluster Settings 

In the past only gateway admins could see Gateway Cluster Settings now Tenant admins can also see these settings.

We have also recently introduced Load balancing settings on the admin portal. We have now classified settings as general and ones which applicable to specific to services like Power BI.

Please continue to send us feedback for what new capabilities you’d like to see in the future.

The post New Gateway management features on Power Platform Admin Center appeared first on Microsoft Power Platform Blog.

Download Latest On-premises Data Gateway

Here is a feature that we would like to highlight with this month’s  release:

Change Gateway Recovery Key

The recovery key provided by gateway admins during installation of gateways cannot be changed today This key is used to create the symmetric key which in turn is used for encrypting credentials in data sources/connections using that gateway.

With the November release of Data Gateways, you will now be able to rotate this key.  More information about recovery keys, detailed description on how to perform this change and associated limitations can be found in the data gateway docs.

Here are some of the features that were released with the October release of gateway:

• Load balance based on CPU and Memory throttling
• Performance Monitoring Improvements
• Improved Gateway Documentation

Load balance based on CPU and Memory throttling

Gateway admins can, now, throttle resources of each gateway member to make sure either a gateway member or the entire gateway cluster isn’t overloaded causing system failures.

If a gateway cluster with load balancing enabled receives a request from one of the cloud services (like Power BI), it randomly selects a gateway member. If this member is already at or over the throttling limit set for CPU or memory, another member within the cluster is selected. If all members within the cluster are in the same state, the request would fail.

To enable this feature, gateway admins would update the following settings in the Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.dll.config file available in the Program Files\On-premises data gateway\ folder.

• CPUUtilizationPercentageThreshold – This configuration allows gateway admins to set a throttling limit for CPU. The permissible range for this configuration is between 0 to 100. A value of 0, which is the default, would indicate that this configuration is disabled.
• MemoryUtilizationPercentageThreshold – This configuration allows gateway admins to set a throttling limit for memory. The permissible range for this configuration is between 0 to 100. A value of 0, which is the default, would indicate that this configuration is disabled.
• ResourceUtilizationAggregateionPeriodInMinutes – This configuration is the time in minutes for which CPU and memory system counters of the gateway machine would be aggregated to be compared against the respective threshold limits set using configurations mentioned above. The default value for this configuration in 5.

Performance Monitoring Improvements

Based on your feedback,  we have now included an additional file called the Query start Report containing the Query start time and also the actual query text. You no longer need to turn on additional logging to obtain the query text. The Query start report contains the following attributes:

In addition to the new report file, we have also updated the Performance monitoring template  in this release. The new version includes not only the new Query text and start time but also new visuals to improve the overall troubleshooting  experience.

Improved Gateway Documentation

We have heard your feedback and re-written the gateway documentation. We split them it into content that’s specific to Power BI and general content that applies to all services that the gateway supports. To provide feedback on each article, or the overall gateway docs experience, scroll to the bottom of any article.

Please continue to send us feedback for what new capabilities you’d like to see in the future.

The post On-premises data gateway November 2019 update is now available appeared first on Microsoft Power Platform Blog.

Download Latest On-premises Data Gateway

Here is a feature that we would like to highlight with this month’s  release:

Change Gateway Recovery Key

The recovery key provided by gateway admins during installation of gateways cannot be changed today This key is used to create the symmetric key which in turn is used for encrypting credentials in data sources/connections using that gateway.

With the November release of Data Gateways, you will now be able to rotate this key.  More information about recovery keys, detailed description on how to perform this change and associated limitations can be found in the data gateway docs.

Here are some of the features that were released with the October release of gateway:

• Load balance based on CPU and Memory throttling
• Performance Monitoring Improvements
• Improved Gateway Documentation

Load balance based on CPU and Memory throttling

Gateway admins can, now, throttle resources of each gateway member to make sure either a gateway member or the entire gateway cluster isn’t overloaded causing system failures.

If a gateway cluster with load balancing enabled receives a request from one of the cloud services (like Power BI), it randomly selects a gateway member. If this member is already at or over the throttling limit set for CPU or memory, another member within the cluster is selected. If all members within the cluster are in the same state, the request would fail.

To enable this feature, gateway admins would update the following settings in the Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.dll.config file available in the Program Files\On-premises data gateway\ folder.

• CPUUtilizationPercentageThreshold – This configuration allows gateway admins to set a throttling limit for CPU. The permissible range for this configuration is between 0 to 100. A value of 0, which is the default, would indicate that this configuration is disabled.
• MemoryUtilizationPercentageThreshold – This configuration allows gateway admins to set a throttling limit for memory. The permissible range for this configuration is between 0 to 100. A value of 0, which is the default, would indicate that this configuration is disabled.
• ResourceUtilizationAggregateionPeriodInMinutes – This configuration is the time in minutes for which CPU and memory system counters of the gateway machine would be aggregated to be compared against the respective threshold limits set using configurations mentioned above. The default value for this configuration in 5.

Performance Monitoring Improvements

Based on your feedback,  we have now included an additional file called the Query start Report containing the Query start time and also the actual query text. You no longer need to turn on additional logging to obtain the query text. The Query start report contains the following attributes:

In addition to the new report file, we have also updated the Performance monitoring template  in this release. The new version includes not only the new Query text and start time but also new visuals to improve the overall troubleshooting  experience.

Improved Gateway Documentation

We have heard your feedback and re-written the gateway documentation. We split them it into content that’s specific to Power BI and general content that applies to all services that the gateway supports. To provide feedback on each article, or the overall gateway docs experience, scroll to the bottom of any article.

Please continue to send us feedback for what new capabilities you’d like to see in the future.

The post On-premises data gateway November 2019 update is now available appeared first on Microsoft Power Platform Blog.

• Manage Gateways by Regions
• Share Gateways
• Get Help

Manage gateways by Region

In the past,  only gateways within your tenant’s default region were visible on the admin center. Now we are also making management of gateways in other regions possible.

Select the region dropdown to see the list of gateway regions. When you select one of the regions, you will see the list of gateways installed in that region. You can now manage users or view gateway members for these gateways.

Additionally, we have also updated our new PowerShell cmdlets in Public Preview to include region specific cmdlets. Do provide any feedback so we can continue to improve these.

Share Gateways

Earlier this year, we enabled managing administrators for gateways from the power platform admin center. You can now share gateways as Admin,   “Can Use” or “Can use + share”.

Note:  “Can Use” or “Can use + share” applies only to PowerApps and Microsoft Flow.

Get Help

If you run into issues while managing gateways on the Power Platform Admin center, you could create a support ticket with Microsoft. While doing so, please include the session id shown below. This would help us troubleshoot the issue faster.

Please continue to send us feedback for what new capabilities you’d like to see in the future.

The post On-premises Data Gateway management by Region appeared first on Microsoft Power Platform Blog.

Manage gateways by Region

In the past,  only gateways within your tenant’s default region were visible on the admin center. Now we are also making management of gateways in other regions possible.

Select the region dropdown to see the list of gateway regions. When you select one of the regions, you will see the list of gateways installed in that region. You can now manage users or view gateway members for these gateways.

Additionally, we have also updated our new PowerShell cmdlets in Public Preview to include region specific cmdlets. Do provide any feedback so we can continue to improve these.

Share Gateways

Earlier this year, we enabled managing administrators for gateways from the power platform admin center. You can now share gateways as Admin,   “Can Use” or “Can use + share”.

Note:  “Can Use” or “Can use + share” applies only to Power Apps and Power Automate.

Get Help

If you run into issues while managing gateways on the Power Platform Admin center, you could create a support ticket with Microsoft. While doing so, please include the session id shown below. This would help us troubleshoot the issue faster.

Please continue to send us feedback for what new capabilities you’d like to see in the future.

The post On-premises Data Gateway management by Region appeared first on Microsoft Power Platform Blog.

Note: These cmdlets require PowerShell core 6.2.2 or higher

The cmdlets are available on PowerShell Gallery and can be installed in an elevated PowerShell session:

Install-Module -Name DataGateway

The entre list of cmdlets can be found using the following cmd.

Get-Command -Module DataGateway*

Examples and descriptions are included in the cmdlets and you can access them using:

get-help <cmdlet-name>

We also plan to make an online version of the documentation available for these cmdlets soon.

Gateway Management:

Following are PowerShell cmdlets which can be used for administering gateway clusters and members:

• Get-DataGatewayCluster: Lists all gateway clusters for the current user or a specific one based on the passed parameters
• Get-DataGatewayClusterStatus: Returns the cluster status along with additional properties
• Get-DataGatewayInstaller: Lists users authorized to install and register gateways within the tenant
• Get-DataGatewayTenantPolicy: Returns the gateway installation and registration policy for the tenant.
• Add-DataGatewayClusterUser: Add a specific user to a gateway cluster for specific permissions
• Set-DataGatewayCluster: Set properties of an existing gateway cluster
• Set-DataGatewayInstaller: Modify list of users who can install and register new gateways on the tenant
• Set-DataGatewayTenantPolicy: Set the gateway installation and registration policy for the tenant
• Remove-DataGatewayCluster: Remove a gateway cluster
• Remove-DataGatewayClusterMember: Remove a gateway member from the corresponding gateway cluster
• Remove-DataGatewayClusterUser: Remove a user from a gateway cluster

In addition, wherever applicable, these cmdlets can be run under a tenant level scope. This would of course require the user to be a global Office 365 admin or a Power BI service admin. For instance, the following cmdlet would return all gateways within your tenant.

Get-DataGatewayCluster -Scope Organization

Similarly, running the same command without the scope parameter or with the “Individual” scope would return the list of gateways, you are administrator of.

Get-DataGatewayCluster -Scope Individual

Another new cmdlet, which allows tenant admins to restrict installation of on-premises data gateway ins standard mode is as follows. The tenant admin can then allow specific users to install using the Set-DataGatewayInstaller cmdlet.

Set-DataGatewayTenantPolicy -ResourceGatewayInstallPolicy Restricted

Only the following cmdlets support the scope of Organization:

• Add-DataGatewayClusterUser
• Remove-DataGatewayClusterUser

We will be enriching these cmdlets with additional management operations in the upcoming releases.

Note: You have the option of using Azure Data Studio with the PowerShell extension to work with these cmdlets:

Please continue to send us feedback for what new capabilities you’d like to see in the future.

The post On-premises data gateway management via PowerShell (Public Preview) appeared first on Microsoft Power Platform Blog.

The cmdlets are available on PowerShell Gallery and can be installed in an elevated PowerShell session:

Install-Module -Name DataGateway

The entre list of cmdlets can be found using the following cmd

Get-Command -Module DataGateway*

Examples and descriptions are included in the cmdlets and you can access them using:

get-help <cmdlet-name>

We also plan to make an online version of the documentation available for these cmdlets soon.

Gateway Management:

Following are PowerShell cmdlets which can be used for administering gateway clusters and members:

• Get-DataGatewayCluster: Lists all gateway clusters for the current user or a specific one based on the passed parameters
• Get-DataGatewayClusterStatus: Returns the cluster status along with additional properties
• Get-DataGatewayInstaller: Lists users authorized to install and register gateways within the tenant
• Get-DataGatewayTenantPolicy: Returns the gateway installation and registration policy for the tenant.
• Add-DataGatewayClusterUser: Add a specific user to a gateway cluster for specific permissions
• Set-DataGatewayCluster: Set properties of an existing gateway cluster
• Set-DataGatewayInstaller: Modify list of users who can install and register new gateways on the tenant
• Set-DataGatewayTenantPolicy: Set the gateway installation and registration policy for the tenant
• Remove-DataGatewayCluster: Remove a gateway cluster
• Remove-DataGatewayClusterMember: Remove a gateway member from the corresponding gateway cluster
• Remove-DataGatewayClusterUser: Remove a user from a gateway cluster

In addition, wherever applicable, these cmdlets can be run under a tenant level scope. This would of course require the user to be a global Office 365 admin or a Power BI service admin. For instance, the following cmdlet would return all gateways within your tenant.

Get-DataGatewayCluster -Scope Organization

Similarly, running the same command without the scope parameter or with the “Individual” scope would return the list of gateways, you are administrator of.

Get-DataGatewayCluster -Scope Individual

Another new cmdlet, which allows tenant admins to restrict installation of on-premises data gateway ins standard mode is as follows. The tenant admin can then allow specific users to install using the Set-DataGatewayInstaller cmdlet.

Set-DataGatewayTenantPolicy -ResourceGatewayInstallPolicy Restricted

Only the following cmdlets support the scope of Organization:

• Add-DataGatewayClusterUser
• Remove-DataGatewayClusterUser

We will be enriching these cmdlets with additional management operations in the upcoming releases.

Note: You have the option of using Azure Data Studio with the PowerShell extension to work with these cmdlets.

Please continue to send us feedback for what new capabilities you’d like to see in the future.

The post On-premises data gateway management via PowerShell (Public Preview) appeared first on Microsoft Power Platform Blog.

As a gateway admin,  you will now be able to 

• See all data gateways(standard and personal mode) that you are admin of
• Manage gateway administrators
• Search on gateway cluster names and contact info

Note: Please refresh the page after logging in if you don’t see this latest feature.

As a  Office 365 Global admin or a Power BI service administrator, you will be able to see all gateways you manage in addition to the all gateways in your enterprise. You can switch between these two views using this toggle in the top right corner of the page:

The Manage On-premises data gateway installers for standard mode operation is restricted to only Office 365 Global admins and Power BI service administrators

Note: As part of this feature, we have enabled only sharing of gateways as admins. Sharing for “Can Use” and “Can use + Share” will be made available as part of a future release.

Do try out feature and provide your feedback. As always we would love to hear from you, learn from your experience and improve based on your feedback.

Thank you!

The post Gateway Management On Power Platform Admin Center appeared first on Microsoft Power Platform Blog.