There’s a new version and a new task for our release pipelines that use the Azure-hosted agents. These changes have been introduced recently to support the new MSAL authentication libraries for the LCS service connection used to upload and deploy the deployable packages.
The current service connections use Azure Active Directory (Azure AD) Authentication Library (ADAL), and support for ADAL will end in June 2022.
This means that if we don’t update the Asset Upload and Asset Deployment to their new versions (1.* and 2.* respectively) the release pipelines could stop working after 30th June 2022.
If you receive the LCS email notifications for your projects you already know this: all Tier 1 virtual machines from Microsoft’s subscription will be gone as early as 1 December!
This is what the emails say:
As communicated previously, Microsoft is removing the use of Remote Desktop Protocol (RDP) to access environments managed by Microsoft. As RDP access is required for development, going forward customers will be required to develop using a Cloud Hosted Environment or download a local “Virtual Hard Disk” (VHD) within Lifecycle Services. Cloud Hosted Environments will allow customers to manage the compute, size, and cost of these environments. This infrastructure change will ensure that customers decouple development tools from any running environment.
In addition, effective November 1, Tier 1 environments will not be included in the purchase of Dynamics 365 Finance, Dynamics 365 Supply Chain Management, Dynamics 365 Project Operations, or Dynamics 365 Commerce apps. The ability to purchase additional Add-On tier 1 environments will also be removed at this time. Beginning December 1, Remote Desktop Protocol (RDP) access for the existing Tier 1 Developer environments, managed by Microsoft, will be removed and decommissioned. Customers will need to preserve or move data within these environments by this date. See the FAQ below with links to existing documentation.
Microsoft will continue to invest in development tools and processes to allow customers to extend the rich capabilities available within Dynamics 365. Learn about one of these key investments, which allows for build automation that uses Microsoft-hosted agents and Azure Pipelines. This approach helps customers avoid the setup, maintenance, and cost of deploying build virtual machines (VMs). It also reuses the existing setup of build agents to run other .Net build automation.
Azure credits will be provided for qualifying customers to use for deploying Tier 1’s using Cloud Hosted Environments. Complete this survey to submit your request.
Sincerely, it’s been a bit of a surprise. We had already been informed of the RDP removal as the email says, and the removal of build VMs has been a rumor for, at least, 2 years. But this is pretty drastic and with such short notice! December is less than two months away!
But wait… instead of speculating, Evaldas Landauskas has asked Microsoft and it looks like the virtual machines won’t be immediately deleted on the 1st but progressively:
Tonight we’ve got a new email from LCS with detailed and updated dates. So finally the dates have been pushed a bit and this is the schedule:
November 1, 2020: no more Tier 1 add-on purchases or deployments. Empty slots will be removed.
December 1, 2020: RDP access will be removed.
January 30, 2021: notices will be sent regarding deallocation and deletion of Tier 1 VMs.
What to do now?
That depends on which use you’re making of that VM and if you have add-on Tier 1 environments. And another thing to ask will probably be the cost of replacing that VM.
I only use it as a build server
If you only have one Tier 1 VM and use it as the build server you have two options:
You will need a VM if you’re running tests or DB sync as a part of your build process. This is the only way. Regarding costs: you could deploy a B8MS VM with 2 128GB Premium SSD disks for around 280€ (330$) per month. You could even try with a B4MS for about 160€/month (190$).
If you don’t need that, or want to have a CI build to just compile the code you can just set up the Azure-hosted builds. And if you need extra agents they’re cheaper than any build VM
I use it as a dev VM
If you’re using add-on Microsoft managed VMs for development you need to deploy a new VM in your (or your customer’s) subscription.
Concerned about the extra cost? Don’t be, if you deploy a DS12 V2 VM, with 3 128GB Premium SSD disks, and use it for 8 hours a day, and 20 days per month, you’ll pay around 120€ (140$) per month.
In both cases and if you read the email you’ll see that Microsoft will give out Azure credits in exchange for these VMs, but how many credits is not known yet. I hope this eases the transition but I’m sure there’ll be plenty of complaining 😂
After waiting for it for a long time it’s here! If any of your customers has self-service sandbox environments you’ve been doing this by hand. We’ve been on self-service for over a year and a half with a customer, since the private preview, and we’ve REALLY missed this feature in Azure DevOps.
The new LCS DB API endpoint to create a database export has been published! With it we now have a way of automating and scheduling a database refresh from your Dynamics 365 FnO production environment to a developer or Tier 1 VM.
You can learn more about the LCS DB REST API by reading these posts I wrote some time ago. You might want to read them because I’m skipping some steps which are already explained there:
And remember: this is currently in private preview. If you want to join the preview you first need to be part of the Dynamics 365 Insider Program where you can join the “Dynamics 365 for Finance and Operations Insider Community“. Once invited to the Yammer organization you can ask to join the “Self-Service Database Movement / DataALM” group where you’ll get the information to add yourself to the preview and enable it on LCS.
Basically, automation. Right now the API only allows the refresh from one Microsoft Dynamics 365 for Finance and Operations environment to another, so the idea is having fresh data from production in our UAT environments daily. I don’t know which new operations the API will support in the future but another idea could be adding the DB export operation (creating a bacpac) to the pipeline and having a copy of prod ready to be restored in a Dev environment.
Since last October we’ve been able to try the preview of Microsoft Dynamics 365 for Finance and Operations Database Movement API which allows us to list and download DB backups and start DB refreshes using a REST API.
If you want to join the preview you first need to be part of the MSDyn365FO Insider Program where you can join the “Dynamics 365 for Finance and Operations Insider Community“. Once invited to the Yammer organization you can ask to join the “Self-Service Database Movement / DataALM” group where you’ll get the information to add yourself to the preview and enable it on LCS.
Right now Microsoft Dynamics 365 for Finance and Operations has an old style monolithic architecture, even it’s now in Azure’s cloud, what we really have is a single (or multiple for Tier 2+ environments) VM that runs everything: the AOS/IIS, Azure SQL Server, the Batch service, MR, etc. Exactly the same as AX 2009/2012.
This is going to change in the coming months with the self-service deployments. We’ll move from the monolithic architecture to microservices that will run all the needed components with the help of Azure’s Service Fabric. MSDyn365FO will be on a real SAAS model.
Before starting let me clarify that all these changes will only apply to Microsoft-managed Tier 2+ environments: sandbox and production environments. The build environment (until it’s made obsolete) and the cloud-hosted environments on the customer or partner subscription will still be single VMs.
When you deploy a new environment it will start deploying without waiting for Microsoft to do it (it’s self-service!). Additionally, thanks to the new microservices architecture, it will be ready to use in under 30 minutes compared to 6-8 hours of regular environments. The first time feels like…
We still need to fill out the subscription estimator for licensing purposes and for MS to estimate the size of the production environment. The self-service environments can be escalated more flexibly and quickly.
No RDP access
The access to the VM desktop has been removed because… well, I guess it’s because there’s no VM anymore. All the operations that could need us to access the RDP can be done from LCS.
No SQL Server access
Yes, no RDP access means no RDP access to the SQL box either. We still have access to the Azure SQL DB, we just need to ask for it from LCS and it’s granted in seconds:
Additionally you must whitelist your IP (or the one you’ll access SQL from) from the Maintain – Enable access button on LCS to be able to connect to the Azure SQL Server. The access to the DB and the firewall rule will be enabled for 8 hours.
As usual, there’s no access to the production DB.
One deployable package to rule them all
If you’ve recently tried to deploy a deployable package (DP) without all the packages the environment has (basically generating the DP for a single model/package from Visual Studio) you must’ve noticed the warning about the difference in the packages from the DP and the environment.
With the self-service deployments you must include all models/packages AND!!ISVs in one single deployable package.
First, we can start the deployment to production without the 5 hours in notice we need to schedule now. We still can schedule the deployment but we can also start it instantly.
Next, the way the production environment is updated changes a bit from what we’re used to. With the new deployments we will update the sandbox environment as we do now, once it’s done we’ll select a sandbox environment to be promoted into production. This is probably another benefit of the architecture changes.
In the future the deployment downtime will also be reduced to zero for the service updates as long as you’re on the latest update. This won’t be available for custom DPs.
How do I get this?
At the moment this is only available for some new customers. Current customers will be migrated during the coming months, MS will contact the customers to schedule a maintenance window to apply the changes.
We got into the private deployment preview program almost a year ago with one of my customers. The customer is now live with the self-service environments and everything has been fine so far.
But the beginning was a bit hard. Some of the functionalities were still not available at the moment, like DB refresh or… package deployment. Yes, we needed to ask MS to deploy our DPs each time. We couldn’t even put the environments in maintenance mode! In the first months of 2019 a lot of functionality was added to LCS and in June we finally got the production self-service update functionality available. The help we’ve gotten from Microsoft’s product team has been very valuable and they have unlocked some issues that were stopping the progress of the project.