Figure 1. Maestro Studio Architectural Layers and Components
We are often asked: "why embark on a tough journey to build tools to help Azure Marketplace publishers of Azure Applications when Microsoft provides so much documentation, training and many other resources?" or "why fix something that ain't broken?"
The simple answer lies in the following ancient Chinese proverb (possibly originally from work by Xunzi):
Tell me and I forget,
teach me and I may remember,
involve me and I learn.
-Xunzi
By the end of this post, it will become clear why we built Maestro Studio Ensemble and why it is critically needed.
Azure Marketplace Background
The Azure Marketplace is a central location where many types of offers can be published by both first-party publishers (i.e. Microsoft or one of its sponsored ISVs) or third-party publishers (mainly Microsoft partners). One of the key types of offers is Azure Applications. Azure Applications have two flavors (or "plans"): Solution Templates and Managed Applications.
Solution Templates
Solution Templates provide a mechanism where a software product or solution can be packaged using a deployment template (known as an Azure Resource Manager, or ARM, template -- basically a JSON file describing the resources to be deployed), and UI definition (known as a createUiDefinition and represented by a JSON file called createUiDefinition.json) that provides a visual interface to help collect input needed for customizing the resources deployed by the deployment template.
The Solution Template is packaged in a .zip file that contains the key artifacts (a minimum of two files: mainTemplate.json -- the ARM Template -- and createUiDefinition.json, but can also include any binaries, scripts, or other ARM Templates that are referenced and used by the "main" ARM Template). A Solution Template represents the basic unit of deployment for Azure Applications, deploying the solution to a Resource Group selected by the end user. In Azure Marketplace lingo, a Solution Template is non-transactable, meaning it cannot have a pricing model attached to it -- the pricing is that of resource(s) deployed by its ARM Template(s).
Managed Applications
This is where a Managed Application comes in. It is transactable (or was until May 2024 -- see sidebar below), so it can have a pricing model, and it also includes the same resources as a Solution Template and then some. Its deployment model is slightly different though. Instead of the straight-forward deployment of Solution Templates to a Resource Group a user picks, a Managed Application deploys a Managed Application resource to that Resource Group but hides the deployed resources in another managed Resource Group.
As of May 2024, the pricing model of Azure Managed Applications changed so that the monthly pricing defined on Managed App plans can no longer be used to account for software IP licensing. Here's an excerpt from the Azure documentation that explains the change:
"Pricing for your Azure App using the per-month price and metered billing must only account for the management fee (i.e., may not be used for IP/software costs, Azure infrastructure, or add-ons). Use the underlying Virtual Machine or Container Offer to transact IP/software costs."
For more details, check out the following documentation page: https://learn.microsoft.com/en-us/partner-center/marketplace/azure-app-managed#define-pricing
The Managed Application resource acts as the portal to the managed Resource Group. A Managed Application publisher can elect to restrict end user access to the managed Resource Group to be be simply view-only to avoid accidental damage to its resources (for example, accidental deletion of unused or forgotten resources when performing regular housekeeping of subscription resources -- a common cause of disruption in many cases). A Managed Application is also very useful for ISVs who want to have easy access to deployed resources in order to support them when there are production issues, or to perform regular maintenance. Managed Applications are very powerful and offer many useful features that ISVs can benefit from.
So, Where's the Problem?
All of this seems really good and amazing. And it is, at least on the surface.
While creating Azure Applications requires a fair amount of administrative work (for example, providing marketing information about the product and customizing it to fit the format and structure required by Azure Marketplace offers and plans), we have found -- based both on our first-hand experience while working for Microsoft and by asking Azure Marketplace publishers -- that creating and managing the technical artifacts of an Azure Application offer is a tedious, frustrating, and time-consuming process. A significant number of publishers rarely update their published Azure Applications due to the amount of work involved, and those that do, usually rely on a System Integrator (SI) to do the work for them.
System Integrators to the Rescue
System Integrators (and sometimes Managed Solution Providers, or MSPs) are a great vehicle to get Azure Applications published in a consistent and compliant manner. They know the ins and the outs of the process, they know the rules, and they have gone through the process many, many times. However, this is not always a viable solution for everyone. It involves a process, which for some publishers that want to publish often, can throttle progress.
While using SIs creates a dependency on a third-party, that is actually not a bad thing and is not that different from any software contract work. SIs play an important role and are probably the key reason there is a good number of Azure Applications published today.
However, it does not appear that they can scale fast enough to accommodate the droves of Microsoft partners with Azure Marketplace publisher accounts that wish to sell their solutions on the Azure Marketplace. This is mainly due to having little/insufficient automation and relying mainly on throwing bodies at the problem.
The Bottleneck
The key problem is that it is very tough to own the work done by an SI and build in-house expertise when desired. ISVs that invest in that in-house expertise deserve a medal because it is a tough, risky investment; it can have a significant ROI if they can manage to keep the in-house resources, but that is not always the case. The work involved in building ARM Templates is a necessity and is typically part of the charters of most ISV's DevOps teams. While not as cool as the popular cloud DevOps Infrastructure as Code (IaC) tools like HashiCorp's Terraform, DevOps teams still put up with it and have gotten accustomed to it. So, the in-house ARM Template expertise is usually there.
The same cannot be said about the UI definition though. Not only is it not as well documented and supported as ARM Templates, there are very few samples, and many of them are either overly simplistic (and impractical, unless you're building a Hello, World! UI presentation) or plagued with many bugs -- a clear consequence of the unavailability of tools supporting the UI definition mechanism. Many samples on the Microsoft Azure GitHub repos have not been updated in years, or, to make matters worse, have had bugs or have been broken for many years. As a result, it is no surprise that most Azure Applications published to the Azure Marketplace have a simple, basic UI. This is not always because the publisher wants to create a simple UI, but rather because it is so much harder and time-consuming to build a truly interactive and useful User Experience (UX) -- a process usually marred with frustrating trial-and-error science experiments and requiring a healthy dose of patience!
For years, we've spent (and watched our colleagues at Microsoft spend) days crafting CreateUiDefinition and ARM Templates to help partners and customers. The level of patience needed is so exhausting that we've always thought to ourselves: "surely, there must be a better way to do this..." Yes, there now is!
Azure Application Tooling & Resources
Microsoft provides a lot of useful resources that range from documentation, GitHub samples, and training workshops and hands-on labs. Azure Marketplace publishers who wish to publish an Azure Application get to benefit from several useful tools to help them on their journey to get their software solution published:
- A sandbox environment where the UI definition (createUiDefinition) can be tested separately from the deployment (ARM Template).
- A validation tool (PowerShell scripts), called arm-ttk, that validates the UI definition JSON file and the ARM Template.
- Popular IDEs, like Visual Studio Code or Visual Studio, support editing JSON files, especially when they follow a JSON schema, in which case you get to benefit from auto-completion and popup suggestions while editing the JSON document.
- Documentation, GitHub samples, and hands-on labs, and other training materials and regularly scheduled workshops.
While the current tools and resources are good and sufficient for small projects, they are not always adequate for medium to large, real-life projects.
Let's expand a bit on each of those items so we can start to understand where some of the problems actually lie.
The CreateUiDefinition (CUID) Sandbox
The CreateUiDefinition sandbox is a good tool for quickly validating UI definition JSON files to test their different scenarios and verify that the output they produce is as expected.
However, the sandbox is disconnected from the ARM Templates and does not provide an end-to-end experience to validate that the combined UI definition plus ARM Template work correctly. This is by design. But testing a createUiDefinition.json file without its corresponding mainTemplate.json file means that you lose the all-too-important pre-flight validation that can detect issues in the ARM Template based on input from the CUID.
The CUID sandbox only validates the JSON schema of the createUiDefinition. It does not validate the embedded expressions inside the JSON.
What are CUID Expressions?
A CUID can have embedded expressions that help turn the User Interface (UI) into a User Experience (UX).
Publishers use expressions to call built-in functions (think of them as Excel functions you'd use in formulas) that can perform certain operations (like logical or math operations) or allow different UI widgets to interact with each other.
This allows a publisher to build a powerful UX that can capture user input without turning the UI into a data-entry form.
Using expressions to validate input and filter out invalid scenarios helps minimize the potential of failure when a customer tries to deploy a publisher's solution from the Azure Marketplace.
Expressions are probably the most powerful feature of CUID, but they are also the feature that is the most difficult to master or to use effectively and is really under-utilized compared to its potential.
The arm-ttk Validation Tool
arm-ttk (ttk = test tool-kit) is a tool consisting of a bunch of PowerShell scripts and modules that implement test cases that the Azure Marketplace has compiled and share as part of the Azure Marketplace documentation. Much of those test cases are the result of lessons learned from the most common causes of failures for Azure Applications published on the Azure Marketplace. The tool aims to minimize issues in production by catching them early on in the process. Most good DevOps teams integrate the arm-ttk tool as part of their linting phase of verification of changes made to ARM or CUID Templates stored in source control. This helps keep the source JSON files sanitized and consistent. arm-ttk helps add a level of standardization.
Arm-ttk seems to take a hybrid approach in its implementation where it de-serializes the JSON into a model, but also uses regular expressions to detect certain patterns specified in the ttk test cases. This hybrid approach introduces a lot of false positives, especially in ARM and CUID Templates that are a bit more advanced.
The tool is very useful for publishing to the Azure Marketplace and is used in the automation that the Azure Marketplace certification team employs to quickly validate the technical artifacts before publishing to Azure Marketplace staging for preview.
One caveat though is that arm-ttk seems to be backed by a small team, probably maintaining it on a part-time basis. This can cause unnecessary throttling when blocking issues or bugs in the toolset are encountered as it might take a while before they are fixed. Fortunately, in such cases an exception can be requested (and usually granted) from the Azure Marketplace certification team to allow the offer publishing to move forward.
IDEs and JSON Editors
The main drawback of the IDEs and text editors today is that they rely on the JSON schemas to add productivity.
For ARM Templates, the experience is very productive because ARM Templates have JSON schemas that are always well-maintained and regularly updated. The tooling and IDE extensions provide a really good experience.
Unfortunately, this is very lacking when it comes to CreateUiDefinition templates. The CUID schemas are stale and have not been updated in over 3 years (at the time of writing of this blog post). They reflect an old structure that has changed more than just a bit over those years. This means that relying on the schemas to provide IDE productivity (like auto-completion) becomes an incomplete and limited experience instead. Besides, that experience is only limited to validating the UI controls described by the JSON, but not the expressions embedded inside the JSON. This means you can write a perfectly valid CUID but with awfully broken and invalid expressions. This is not what you'd really expect in IDEs.
Documentation, GitHub Samples, Training Workshops
The ARM Template and CUID documentation is really good, especially in the case of ARM Template. This is due to their broad use.
As for CUID Templates, due to their limited use (compared to ARM Templates), they don't seem to get the same amount of time and attention. Similar to the case of outdated JSON schemas, the CUID documentation has become a bit outdated too. The documentation of the UI elements is missing a lot of newer properties and even newer UI controls, and the documentation of the functions is also missing some newer functions (at least 8 new ones). This limits the ability of publishers to utilize the latest and greatest and to fully utilize the platform.
Training workshops are very useful and are well run. They help publishers get started and provide useful guidance.
The Solution
The first step towards a viable solution is the realization that the publisher is someone who does not have enough time to learn all the ins and outs of some JSON variant (like that of ARM or CUID Templates) and cannot, for practical reasons, invest the amount of time needed to master creating such artifacts in a way that utilizes their powerful features and harnesses them to present a great, interactive solution that can be presented with confidence and pride to Azure Marketplace customers.
Maestro Studio Ensemble
Figure 2. Maestro Studio Ensemble Design Environment
With Maestro Studio Ensemble, our aim is to channel that power and those features into a single tool so that publishers have a single pane of glass that they can use in building, previewing, validating and publishing the technical artifacts of Azure Applications. We strongly believe this will help them become more productive and be able to build their technical artifacts much, much faster than with what is available today.
To achieve this, we made Ensemble a turnkey solution that comes with built-in, boiler-plate quickstart templates that address beginner as well as advanced, real-world scenarios:
- This ensures that publishers don't start from scratch.
- The minute the application is launched, a template can be selected and previewed to see it exactly as it would look when published to the Azure Marketplace.
- Existing publishers can import their technical artifacts package, edit it and leverage the Visual Designer and Visual Expression Designer.
- New publishers can take a built-in template and publish it to the Azure Marketplace with minimal effort and most importantly, with minimal knowledge of the inner workings and mundane details of CUID.
Maestro Studio Ensemble allows you to build your UI that represents your product; it allows you to publish Azure Applications today at the speed of tomorrow!
Conclusion
As mentioned at the beginning of this post, only by involving the publishers in a productive manner that fully respects their limited available time, can publishers be more productive and creative and publish their solutions as fast as they need to.
This post barely scratches the surface when it comes to Maestro Studio Ensemble's capabilities and features. The goal of this post is to provide the background and the context upon which we will build in future blog posts and how-to tutorials that dig deeper into each and every feature of Maestro Studio Ensemble.
In the mean time, we recommend that you check out the links below to take a look at the features and to request a demo so you can see the application in action.
Resources: