Tag

xpp

Browsing

We’re finally getting a throttling functionality for OData integrations!

It’s one of the most common requirements from a customer: the need to integrate Dynamics 365 with other systems. With the (back in the day) new AX7 we got a new way of integrating using the OData REST endpoint and the exposed Data Entities.

But integrations using the OData endpoints have low performance and for high-volume integrations it’s better to use the Data management package REST API. A (not so) high volume usage of the OData REST API will translate into performance issues.

The throttling functionality is in preview starting version 10.0.13 which is currently in PEAP. It will be enforced starting April 2021. You can join the Data Management, Data Entities, OData and Integrations Yammer group for more info. Remember you need to join the Insider Program for Dynamics 365 first to be able to access the Yammer group.

If you want to learn more about OData and throttling you can check these resources:

A short one! Some time ago I explained how to add a multi selection lookup to a SysOperation dialog and in this post I’ll explain how to add a Menu Item as a Function button to the SysOperation dialog.

Before the SysOperation Framework was introduced in AX2012, we used the RunBase Framework, and maybe doing these things looked easier/quickier with RunBase because all the logic was in a single class. But in the end what we need to do is practically the same but we have to do it in the UIBuilder class.

Let me show you and explain all the code. I’ll only show the DataContract and UIBuilder classes as they’re the only important ones in this case.

Compiler warnings. Warnings. They’re not errors, only warnings. You can just overlook and forget them, right? Well, I hope you aren’t.

“But even the standard code done by Microsoft throws warnings!”, you could say. And that’s true, but that’s not your code, it’s Microsoft’s. If a functionality you’re using breaks because they didn’t care about their warnings, you can open a support request and it’s Microsoft’s job to fix it. If your code breaks some functionality because you didn’t care about a warning, it’s your job to fix it, and your customer will want it as fast as you’d want Microsoft to fix their error.

That’s why we should be warned about warnings (badum tss).

First 2020 post! Happy new year! Yes, I know it’s already past mid January…

When you add a field to a SysOperation Framework Data Contract the lookup that the framework creates (if the EDT has a lookup) is a simple, single select lookup. Let’s see how to create a multi select lookup in MSDyn365FO.

The SysOperation Framework and MVC

But first of all a bit of an introduction! The SysOperation Framework was introduced in Dynamics AX 2012 to replace the RunBase Framework, and is used to create processes that will be run by the batch server. The RunBase classes are still around in Dynamics 365 for Finance and Operations. Some standard processes still use it while others use it to later call a SysOperations Framework class.

Some time ago I had to create an interface between MSDyn365FO and a web service that returned data as XML. I decided to use X++’s XML classes (XmlDocument,  XmlNodeList, XmlElement, etc…) to parse the XML and get the data. These classes are terrible. You get the job done but in an ugly way. There’s a better method to quickly parse XML or JSON in MSDyn365FO.

Feature management has been around in Microsoft Dynamics 365 for Finance and Operations for some time now. Before that features were enabled through flighting running a SQL query on dev and UAT boxes (and the DSE team would do it on production).

Now we have a nice workspace showing all the available features and flighting is still around too. The main difference between flighting and features is that flighting is enabled to a selected group of customers, like a preview of a feature.

Unless you’ve been working for an ISV there’s a high percentage of probabilities that you’ve never cared about Dynamics Best Practices (BP), or maybe you have. I haven’t worked for an ISV myself but back when I started working with AX I was handed the development BP document and I’ve tried to follow most of them when writing code.

But BPs could be ignored and not implemented without any issue. This is why Microsoft will publish…

Application Checker

Application Checker is a tool that will change that. It will force some rules that our code will have to meet, otherwise the code won’t compile (and maybe won’t even deploy to the environments).

We got an advance of it during last MBAS session “X++ programming with quality” by Dave Froslie and Peter Villadsen. Unfortunately the session wasn’t recorded.

Application Checker: enforcing better coding practices? 3

Application Checker: enforcing better coding practices? 4

App checker is built on BaseX, an XML analysis tool, and powers Socratex which Microsoft uses to track code quality. I don’t know if Socratex will be publicly released and I don’t remember if this was clarified during the session.

The set of rules can be found in Application Checker’s GitHub project and it’s still WIP. I think there’s loooots of things to decide before this goes GA, and I’m a bit worried and afraid of some of the rules 😛

Rule types

There’s different types of rules, some will become errors and other warnings. For example:

ExtensionsWithoutPrefix.xq: this rule will throw an error avoiding your code to compile. It checks if the extension class has a name ending with _Extension and an attribute ExtensionOf. If it has it must have a prefix. E.g.: if we extend the class CustPostInvoice it can’t be named CustPostInvoice_Extension, it needs a prefix like CustPostInvoiceAAS_Extension.

Application Checker: enforcing better coding practices? 5

SelectForUpdateAbsent.xq: this rule will throw a warning. When there’s a forUpdate clause in a select statement and no doUpdate, update, delete, doDelete or write is called later it will let us know.

Application Checker: enforcing better coding practices? 6

As of today, there’s 21 rules in the GitHub project. You can contribute to the project, and you could enforce your own rules without sending them to the project on your dev boxes, just add them to the local rules folder. I’d create a rule that makes the space after an if/while/for/switch mandatory and throws an error otherwise, but that’s only a bit of my OCD when writing/reading code.

Try it on your code

We can already use Application Checker on our development environments since PU26, I think. We just need to install JRE and BaseX in the dev box and select the check when doing a full build.

Application Checker: enforcing better coding practices? 7

Some examples

ComplexityIndentationCombined.xq

This query checks the (wait for it…) cyclomatic complexity of the methods. I’ll try to explain it… Cyclomatic complexity is a metric for software quality, and is the number of independent paths in the code. Depending on the number of ifs, whiles, sitches, etc… the code can have different outcomes through different paths, that’s what complexity calculates.

Taking this as an example, a dumb one but ignore it, just look at the amount of different paths that could happen:

In App checker the error appears when the complexity is over 30. I’ve used Lizard code complexity analyzer to calculate the complexity of the method below and I’m getting a 49.

The rule also checks for the indentation depth, failing if it’s greater than 2. In the end the purpose of both rules is to try to cut up long/large methods, which will also help in enabling more extension points in different places of our logic, like Microsoft did with Data Provider classes for reports.

Application Checker: enforcing better coding practices? 8

BalancedTtsStatement.xq

This one gives me mixed feelings. The rule checks that the ttsbegin and the ttscommit of a method are in the same scope. So the following is not possible:

Application Checker: enforcing better coding practices? 9

Imagine you’ve developed an integration with an external application that writes data to an intermediate table in MSDyn365FO and you process all pending data sequentially. You don’t want to throw an error if something goes wrong because you need the process to continue with the following record, so you ttsabort the wrong line, store the error and continue. If this is not possible… how should we do this? Create a batch that creates a task for each line to process?

Plus, the standard models have plenty of ttscommit inside if statements.

RecursiveMethods.xq

Application Checker: enforcing better coding practices? 10

This rule will block the use of recursion on static methods. I don’t get why. Application checker should be a way to better coding practices, not forbidding some patterns. If somebody gets a recursive method to prod and the exit condition isn’t met… hello testing?

Some final thoughts

Will this force developers to code better? I don’t think so, but that’s probably not Application checker’s purpose. For centuries humans have found ways to bypass rules, laws and all kinds of restrictions and this won’t be an exception.

Will it help? Hell yes! But the best way to ensure code quality is promoting the best practices in your team, through internal trainings or code reviews. And even then if someone doesn’t care about clean code will keep on writing terrible code, which might work but won’t be beautiful at all.

Finally, I’m not sure about some rules, like avoiding recursion on static methods or the tts thing. We’ll just have to wait and see which rules make it to the final release and how will Application checker be finally implemented in the MSDyn365FO application lifecycle by blocking (or not) the deployments of code which doesn’t pass all the checks or if it will be included into the build process.

In Microsoft Dynamics 365 for Finance and Operations we can execute the CRUD operations from code in two different ways, record-per-record or set-based.

Microsoft’s recommendation is to always use set-based operations, if possible, as you can check on the Implementation Best Practices for Dynamics 365: Performance best practices for a successful Dynamics 365 Finance and Operations implementation session from last June’s Business Applications Summit.

Why?

Set-based Vs. Record-per-record

When we run a query in MSDyn365FO we’re using its data access layer which will later be translated into real SQL. We can see the differences using xRecord’s getSQLStatement with generateonly on the query (and forceliterals to show the parameter’s values) to get the SQL query. For example if we run the following code:

Slow set-based operations? 11

We’ll get this SQL statement:

 

We can see all the fields are being selected, and the where clause contains the account number we selected (plus DataAreaId and Partition).

When a while select is run on MSDyn365FO a select SQL statement is executed on SQL Server for each loop of the while. The same happens if an update or delete is executed inside the loop. This is know as record-per-record operation.

Imagine you need to update all the customers with the customer group 10 to update their note. We could do this with a while select, like this:

Slow set-based operations? 12

This would make as many calls as customers from the group 10 existed to SQL Server, one for each loop. Or we could use set-based operations:

Slow set-based operations? 13

This will execute a single SQL statement on SQL Server that will update all the customers with the customer group 10 instead of a query for each customer:

There’s three set-based operations in MSDyn365FO, update_recordset to update records, insert_recordset to create records and delete_from to delete the records. Plus we can make massive inserts using RecordSortedList and RecordInsertList.

Running this methods instead of while selects should obviously be faster as it’s only executing a single SQL query. But…

Why could my set-based operations be running slow?

There’s some well-documented scenarios in which set-based operations fall back to record-per-record operations as we can see in the following table:

DELETE_FROM UPDATE_RECORDSET INSERT_RECORDSET ARRAY_INSERT Use … to override
Non-SQL tables Yes Yes Yes Yes Not applicable
Delete actions Yes No No No skipDeleteActions
Database log enabled Yes Yes Yes No skipDatabaseLog
Overridden method Yes Yes Yes Yes skipDataMethods
Alerts set up for table Yes Yes Yes No skipEvents
ValidTimeStateFieldType property not equal to None on a table Yes Yes Yes Yes Not applicable

In the example, if the update method of the CustTable is overriden (which it is) the operation from the update_recordset piece will be run like a while select that updates each record.

In the case of the update_recordset this can be solved calling the skipDataMethods method before running the update:

Slow set-based operations? 14

This will avoid calling the update method (or insert in case of the insert_recordset), more or less like calling doUpdate in a loop. The rest of the methods can be overriden with the corresponding method on the last column.

So, for bulk updates I’d always use set-based operations and enable this on data entities too with the EnableSetBasedSqlOperations property.

And now another but is coming.

Should I always use set-based operations when updating large sets of data?

Well it depends on which data you’re working with. There’s a wonderful blog post from Denis Trunin called “Blocking in D365FO(and why you shouldn’t always follow MS recommendations)” that shows a perfect example where set-based operations would be counterproductive.

As always, developing an ERP is quite sensitive, and similar scenarios can have different solutions. Analyze the requirements and decide which one to use.

Do NOT follow this link or you will be banned from the site!