I’m sorry for my English-speaking readers because, maybe, this post will be a bit useless for you as all the content I’ll talk about is in Spanish. But it’s always good to know!
In the last few days I’ve taken part in a community event, the 365 Saturday online, and I’ve also started a podcast. I want to talk a bit about this.
Dynamics Power Spain Online 2020
This has been my fourth participation as a speaker in the last three years and as usual I’ve presented a session with Juanan. This time we’ve talked about using Azure DevOps with Microsoft Dynamics 365 for Finance and Operations.
It’s a topic I write about a lot, but we really think there’s still many people using it in a wrong way or just using the source control part. And that’s bad!
You can watch our session below, and as I said before, it’s only in Spanish and I think there’s no subtitles from Youtube.
There’s more sessions from other Axazure colleagues, again in Spanish:
Many Axazure sessions as you can see. That’s what happens when you promote sharing with the community! You can see the other sessions in 365 Saturday Madrid‘s Youtube channel, some are in English!
Xpp.dev, the podcast!
Yes! Juan Antonio Tomás and myself have started a podcast in 2020! Why? Each time there’s a new feature for MSDyn365FO, an announcement, a preview, whatever we spend some time talking about it and what could we do with it. So we thought “Why don’t we record this?”. And now we have a podcast which you can listen, even though it’s also in Spanish…
There’s another reason to do this: the Finance and Operations technical community in Spain. It makes me terribly jealous to see the strength of Dynamics 365 CD/Power Platform, .NET, Azure and other technical communities. We don’t have this for AX and that’s what we want!
We would like to have a bigger technical community! This is how we’ll try to encourage other people, sharing what’s coming for FnO. We of course accept collaborations, and if anybody wants to be interviewed or participate we’re totally open!
The idea of community we have is something really simple and I think it’s something we’ve learnt from Antonio and the essence of El Rincón Dynamics. A free and collaborative place where anybody can learn and share and connect. It’s very easy, right?
There might even be people that doesn’t fully understand why do we share what we know freely, instead of keeping everything to us, because that decreases our personal value, we’re sharing our secrets. And this way of thinking is so, so, so much wrong! I might share what I know, but I also have over 10 years’ experience behind me, the mix of these two things is what my value is.
I’ve written this post after Mötz Jensen asked to in a long and really interesting Twitter discussion on branching and version control in Dynamics 365 for Finance and Operations. This is Denis Trunin‘s tweet that initiated it all:
Just go and read all the replies, there’s some of the most interesting content I’ve read on Twitter in months.
When deciding which branching strategy to use, you have to think what will suit your team and follow the KISS principle. Keep your branching strategy as simple as you can! You don’t want to spend most of your day managing the code instead of coding, right? Or trying to fix bad merges…
Also keep in mind that your strategy might not be the same if you’re working on a a customer implementation project or developing an ISV.
Main – Release
This one of the most simple strategies. You start your work on the Main branch, all developers working with it. You keep working only with Main until Go-live.
When the project is going live branch Main and create a Release branch. This new branch will be the one you’ll use to promote code to the production environment. Development of new features and bug fixes is done on the Main branch.
When a development is done, or a bug fixed we will merge the changeset (or changesets) to the Release branch. Yes, we will be doing cherry picking. I know it has bad reputation, but we depend on the customer validating the changes…
On our projects we have at least 2 Tier 2+ environments. We use one for testing and validation to which we deploy the DP created from the Main branch. The other Tier 2+ environment is the one we use for user testing and deploy to production. This second environment will be updated with the DP from the Release branch.
Dev – Main – Release
This is something we’ve been doing lately trying to emulate Git branches for development. We’re using a Dev branch for every developer. We work on our Dev branch, we do all the check-ins we want to do during a development and when it’s done we merge all the changesets or all the branch in a single changeset to Main. Finally we Forward Reverse Integrate Main into our Dev branch to get the changes from other developers.
Yes, it does involve a bit more of merging on the Dev – Main part. But the idea behind this strategy is having a clean list of single changesets in our Main branch for each development. Why? Because… cherry picking…
We will work with Dev and Main until Go Live, when we’ll branch the Main branch and create the Release one. The Tier 2+ environments will be serviced in the same manner as with the Main – Release strategy.
As I said the idea is having a clean list of changesets to move developments from Main to Release and solve all merging conflicts in the Dev branches. Each developer is responsible of his branch and resolving conflicts there.
We’ve been working for some months with this strategy and the results are OK and we’re not having issues regarding too many management. In the future we’ll try with Git, but Juanan will explain that part!
First of all: train yourself and your team. Remember, using a VCS is mandatory, this is part of your job now. Find somebody that can help even if he/she is outside the AX world. The problems of software development are more or less the same regardless of the language we use.
Don’t keep pending changesets to be merged forever. The amount of merge conflicts that will appear is directly proportional to the time the changeset has been waiting to be merged.
Remember to Keep it simple, stupid (or Keep it stupid simple), KISS. Don’t follow blindly what a guy on the Internet is telling you because he might have different needs in his projects than you.
So this is how we do branching at Axazure. Are there better ways of doing it? Sure! Can this be improved? I have no doubts about it. But this works for us, which is the important thing.
Behold #XppGroupies! The day we’ve been waiting for has come! The Azure hosted builds are in public preview with PU35!! We can now stop asking Joris when will this be available, because it already is! Check the docs!
I’ve been able to write this because, thanks to Antonio Gilabert, we’ve been testing this at Axazure for a few months with access to the private preview. And of course thanks to Joris for inviting us to the preview!
What does this mean? We no longer need a VM to run the build pipelines! Nah, we still need! If you’re running tests or synchronizing the DB as a part of your build pipeline you still need the VM. But we can move CI builds to the Azure hosted agent!
Remember this is a public preview. If you want to join the preview you first need to be part of the Dynamics 365 Insider Program where you can join the “Dynamics 365 for Finance and Operations Insider Community“. Once invited you should see a new LCS project called PEAP Assets, and inside its Asset Library you’ll find the nugets in the Nuget package section.
The new LCS DB API endpoint to create a database export has been published! With it we now have a way of automating and scheduling a database refresh from your Dynamics 365 FnO production environment to a developer or Tier 1 VM.
You can learn more about the LCS DB REST API reading these posts I wrote some time ago. You might want to read them because I’m skipping some steps which are already explained there:
And remember: this is currently in private preview. If you want to join the preview you first need to be part of the Dynamics 365 Insider Program where you can join the “Dynamics 365 for Finance and Operations Insider Community“. Once invited to the Yammer organization you can ask to join the “Self-Service Database Movement / DataALM” group where you’ll get the information to add yourself to the preview and enable it on LCS.
A Key Vault is a service that allows us to safely store certificates or secrets and later use them in our applications and services. And like many other Azure services it has a cost but it’s really low and, for a normal use, you will be billed like a cent or none a month. Don’t be stingy with security!
Basically, automation. Right now the API only allows the refresh from one Microsoft Dynamics 365 for Finance and Operations environment to another, so the idea is having fresh data from production in our UAT environments daily. I don’t know which new operations the API will support in the future but another idea could be adding the DB export operation (creating a bacpac) to the pipeline and having a copy of prod ready to be restored in a Dev environment.
The possibility of doing real CI/CD is one of my favorite MSDyn365FO things, going from “What’s source control?” to “Mandatory source control or die” has been a blessing. I’ll never get tired of saying this.
I want to start this second part with a little rant. As I said in the first part, those who have been working with AX for several years were used to not using version-control systems. MSDyn365FO has taken us to uncharted territory, so it is not uncommon for different teams to work in different ways, depending on their experience and what they’ve found in the path. There’s an obvious interest factor here, each team will need to invest some time to discover what’s better for them regarding code, branching and methodologies. Many times this will be based on experimentation and test-error, and with the pace of some projects this turns out bad. And here’s where I’ve been missing some guidance from Microsoft (but maybe I’ve just not found it).
Regardless of this rant, the journey and all I’ve learnt has been, and I think will be, pretty fun 😉
The truth is that I’d love a FastTrack session about this and, I think, it doesn’t exist. EDIT: it looks like I did definitely overlooked it and there is a FastTrack session called Developer ALM which talks a bit about all this. Thanks to Dag Calafell (twitter) for pointing this out!
In the first part we learnt that the Main folder is created when deploying the Build VM. The usual is that in an implementation project all development will be done on that branch until the Go Live, and just before that a new dev branch will be created. The code tree will look like this:
From this moment on, the development VMs need to be mapped to this new development branch. This will allow us to keep developing on the Dev branch and decided when the changes are promoted to the Main one.
This branching strategy is really simple and will keep us mostly worries-free. In my previous job, we went on with a 3 branches strategy, Main, Test and Dev, merging from Dev to Test and from Test to Main. A terrible mistake. Having to mantain 2 sets of changesets is harder and with version ugrades, dozens of pending changeset waiting to be merged and an ISV partner taht sometimes would not help much, everything was kind of funny (“funny”). But I learnt a lot!
Anyway, just some advice: try to avoid having pending changesets to be merged for long. The amount of merge conflicts that will appear is directly proportional to the time the changeset has been waiting to be merged.
At this point, I cannot emphasize enough what I mean by normal. As I say, I wrote all of this based on my experience. It’s obviously not the same working for an ISV than for an implementation partner. An ISV has different needs, it has to mantain different code versions to support all their customers and they don’t need to work in a Dev-Main manner. They could have one (or more) branch for each version. However, since the end of overlayering this is not necessary :). More ideas about this can be found in the article linked at the beggining of this post.
This build definition has all the default steps active. We can disable (or remove) all the steps we’re not going to use. For example, the testing steps can be removed if we have no unit testing. Or the DB sync and report deployment too.
We can also create new build definitions from scratch, however it’s easier to clone the default one and modify it to other branches or needs.
Since 8.1 all the X++ hotfixes are gone, the updates are applied in a deployable package (binaries!). This implies that the Metadatada folder will only contain our custom packages and models, no standard packages anymore. Up until 8.0, having a build definition compiling and generating a DP only with our models was a good idea. In this way we could have a deployable package ready in less time than having to compile standard packages with hotfixes plus ours. Should we need to apply a hotfix we’d just queue the default build pointing to the Main root, otherwise we’d just generate our packages. Using this strategy, we reduced the DP generation time from 1h15m to 9m in one of our customer’s project.
But that was in the past, and all this is outdated information. Right now I hope everybody is as close to 8.1 as possible because One Version is coming in April!
Another useful option is having a build definition that will only compile the code:
It may look a bit useless until you enable the continuous integration option:
Right after every developer’s check-in a build will be queued, and the code compiled. In case there’s a compilation error we’ll be notified about it. Of course, we all build the solutions before checking them in. Right?
And because we all know that “Slow and steady wins the race” but at some point during a project, that’s not possible this kind of build definition can help us out. Especially when merging code conflicts from a dev branch to Main. This will allow us to be 100% sure when creating a DP for release to production that it’ll work. I can tell you that having to do a release to prod in a hurry and seeing the Main build failing is not nice.
Somebody with far more experience and knowledge than me can think, wait but this can also be done with…
What we accomplish with a gated check-in is that the build agent will launch an automated compilation BEFORE checking-in the code. If it fails, the changeset is not made until the errors are fixed and checked-in again.
This option might seem perfect for the merge check-ins to the Main branch. I’ve found some issues trying to use it, for example:
If multiple merge & check-ins from the same development are done and the first fails but the second doesn’t, you’ll still have pending merges to be done.
Issues with error notifications and pending code on dev VMs.
If many check-ins are made you’ll end up with lots of queued builds (and we only have one available agent per DevOps project).
I’m sure this probably has a solution, but I haven’t found it. And I think the CI option is working perfectly to us to validate code. As I’ve already said, all of this is product of trial-error, we’ve learnt to use this while working with it.
I guess the biggest conclusion is that with MSDyn365FO we must use DevOps. It’s mandatory, there’s no other option. If there’s anyone out there not doing it, do it. Now. Review how you work and let’s forget and don’t look back at how we used to work with AX, technically speaking MSDyn365FO is a different product.
Truth is that MSDyn365FO has taken developers to a more classic approach of software projects, like .NET or Java. But we’re still special. An ERP project has a lot of peculiarities, and not having to create a product from scratch, having a base that makes us follow a path, limits us in some aspects, and the usage of certain techniques or methodologies.
I hope these two posts about Azure DevOps can help somebody. And if anyone with more experience or better ideas wants to recommend anything, comments are open!
One of the major changes we got with Dynamics 365 has been the mandatory use of a source control system. In older versions we had MorphX VCS for AX 2009 and the option to use TFS in AX 2009 and AX 2012 (and there’s training available about this on El rincón Dynamics, in Spanish), but it wasn’t mandatory. Actually, always from my experience, I think most of projects used no source control other than comments in the code.
Azure DevOps in MsDyn365FO
In Microsoft Dynamics 365 for Finance and Operations the source control tool Azure DevOps offers, is not just a source control tool but aTHE tool that will be our One Ring for our projects (I hope that not for binding us in darkness). From project management to the functional team, everybody can be involved in using Azure DevOps to manage the project and team.
BPM synchronization and task creation, team planning, source control, automated builds and releases, are some of the tools it offers. All these changes will need some learning from the team, but in the short-term all of this will help the team to better manage the project.
As I said it looks like the technical team is the most affected for the addition on source control to Visual Studio, but it’s the most benefited too…
The first thing we need to do when starting a new implementation project, is linking LCS to the DevOps project we’ll be using. Everything is really well documented.
Once done we’ll have to deploy the build server. This is usually done in the dev box on Microsoft’s subscription. When this VM gets deployed the basic source tree will be created in the DevOps project:
With the source tree now available, we can map the development machines and start working. The Main folder you see in the image is a regular folder, but we can convert it into a branch if we need it.
In the image above, you can see the icon for Main changes when it’s converted in a branch. Branches allow us to perform some actions that aren’t available to folders. Some differences can be seen in the context menu:
For instance, branches can display the hierarchy of all the project branches (in this case it’s only Main and Dev so it’s quite simple :P).
Properties dialogs are different too. The folder one:
And the branch one, where we can see the different relationships between the other branches created from Main:
This might be not that interesting or useful, but one of the things converting a folder into a branch is seeing where has a changeset been merge into. We’ll see this in part 2.
I strongly recommend moving the Projects folder out of the Main branch into the root of the project, at the same level as BuildProcessTemplates and Trunk. If you don’t, and end up working in Main and Dev branches, Visual Studio’s solutions and projects will still be checked-in in the Main branch. It will spare you of small heart attacks when you receive the build email with the changeset summary, thinking something went into production 🙂