Defining Your Enterprise Monitoring Strategy: Close the Gaps with SCOM 2019 and Azure Monitor
Sameer Mhaisekar is a Sr. Technical Consultant at Green House Data and the founder of Experts Live India. Follow him on LinkedIn or Twitter and check out his blog AnalyticOps Insights.
When using Microsoft technologies in your enterprise IT stack, you have a few native options for systems monitoring and alerts. Two recent product developments — folding Operations Management Suite (OMS) functionality into Azure Monitor, as well as the release of the new SCOM 2019 — have reignited the debate to determine whether Azure Monitor can entirely replace the long standing, good-old SCOM (System Center Operations Manager).
In a way, I feel this comparison is a bit unfair, like comparing apples with oranges. Ultimately the two products can work together and overlap in order to eliminate monitoring gaps in your environment. So which monitoring solution would work the best for your enterprise? Let’s try to figure out!
Could new SCOM 2019 features be used instead of Azure Monitor?
SCOM 2019 includes new cloud-based capabilities and integration with Azure, so you can use it to monitor your cloud resources and even plug into Log Analytics or Power BI. It also makes monitoring specific applications easier. For example, if you have an SQL server installed on a virtual machine (VM), either hosted on premise or in the cloud, you may not be able to monitor some specific metrics related to SQL with Azure Monitor. A Management Pack (MP) for SCOM could be installed that would provide those metrics out of the box. In general, SCOM is more customizable to custom applications or unique use cases.
On the flip side, Azure Monitor is also capable of monitoring on premise workloads, even if it is more geared towards Azure-native workloads and resources. It just generally takes some more thought and planning by manually creating monitors and scenarios.
Either tool can dump monitoring data in Log Analytics for use with other Azure features like Automation. So how should you decide which is a better fit for your enterprise IT environment? To help us analyze, let us break this down for three scenarios:
- Your infrastructure is entirely on premise,
- Your infrastructure is entirely on cloud and
- Your infrastructure is hybrid – spread across on premise AND cloud.
Your Infrastructure is entirely on premise
In this scenario, your infrastructure is entirely on premise. All your servers, applications, web sites are running in your data center. The answer here is pretty simple – SCOM all the way! Here’s why:
- SCOM is locally installed on your data center – SCOM is perfect for monitoring on premise workloads, may it be the operating system, databases, clusters, file shares, web applications, or even physical devices.
- You can get very granular and customized with SCOM – Management Packs (MPs) for SCOM are great. If you’re not sure what you’re supposed to monitor, just import the relevant Management Pack(s) and it literally does all the work for you. It discovers and monitors everything it is designed to see within your infrastructure and alerts you on it, often from a custom dashboard. SCOM can not only alert, but also diagnose to help resolve these alerts by itself using its powerful Diagnostics and Recovery features.
- Third party solutions – If you’re using products from third party vendors outside of Microsoft, there are often vendors (either the creator or independent software developers) that provide their own Management Packs to effectively monitor products through SCOM. These MPs may be free or paid.
- Custom Management Pack authoring – Now what about your custom applications? You can author your own management packs using various tools like VSAE (Visual Studio Authoring Extension) or the authoring console — albeit with some training and experience.
- Community solutions – Finally, a not-strictly-technical point that is still worth considering is since SCOM has been around for a very long time, there is an ecosystem around this product. Many companies provide solutions that greatly enhance your SCOM experience, and fill out the gaps that SCOM suffers out-of-the-box. And there are tons of (free) community solutions and learning material available that will help you make the most of your SCOM!
- You may already have a SCOM investment — if you’re already monitoring with SCOM, the update process is relatively simple and you don’t have to throw away all of your custom monitoring dashboards and MPs.
The SCOM 2019 Monitoring tab in action.
Now, of course it’s not ALL good; SCOM has some limitations too:
- SCOM is often too chatty – The Management Pack framework for SCOM is very powerful, but sometimes too detailed at the same time. SCOM is notoriously famous for being too noisy. As soon as you import a Management Pack, it is immediately active and starts throwing out all kinds of alerts that you may not be concerned with.
- SCOM requires frequent tuning – Due to the fact it creates a lot of noise, you have to tune your SCOM very carefully and often constantly over the years. This introduces administrative overhead.
- SCOM has a sharp learning curve – There’s no other way to say it – SCOM requires practice! It can be very counter-intuitive sometimes, and needs some time to get used to. That is why you need to invest into an experienced team of SCOM admins to handle your infrastructure well. And as you step into custom Management Pack authoring, it has even sharper learning curve and requires a deep understanding of the product you’re trying to monitor.
- SCOM needs to be maintained – Besides the tuning of alerts themselves, SCOM is an application like any other that resides completely within your data center, meaning it is entirely your responsibility to install it, manage it, update it, troubleshoot it and keep it running efficiently. If you’re not very experienced with SCOM, this can throw you in an endless loop of troubleshooting.
- SCOM is dependent on other products – SCOM generally requires help from the other technologies or products in System Center suite for end-to-end lifecycle or automation (eg. SCORCH or Powershell). This is not necessarily a flaw but just the way it’s supposed to be – running as part of the suite.
Your infrastructure is entirely cloud-based
Now, here the choice gets a little tricky. Do you buy SCOM and install it on your servers as IaaS or go with Azure Monitor? You can of course use your SCOM on cloud servers just as if you were using it on premise. However, Azure Monitor provides some great value for IaaS and I would generally recommend it over SCOM for cloud-first or cloud-only environments. Here’s why:
- Little to no maintenance – Since Azure Monitor is a SaaS solution, you are not responsible for maintaining it. The biggest advantage of that is you can just start on-boarding the servers for monitoring, without having to worry about the scalability or design changes in the capacity of the monitoring app itself. Which, after all, is the main reason you’re running everything in the cloud!
- Easy to set up – Since Azure Monitor requires no “installation”, it can be up and running in a matter of minutes. It is now integrated with Azure Portal itself so it is even easier to manage from a single pane of glass.
- No upfront cost – If you’re set with an Azure Portal, you can spin up your monitoring solution with no upfront cost to pay. You only pay for what you use. This also lightens the initial burden of budget just to get up and about your business.
- Easy to manage – Azure Monitor is usually friendlier to use and gets the job done quicker than SCOM. And since it basically resides in the same place (the cloud) as your other Azure resources, it can retrieve better insights on those IaaS and PaaS resources than SCOM.
- Pretty dashboards – With integration with PowerBI and the power of HTML5, you can produce some really neat dashboards and views in Azure Monitor that present monitoring data visually for quicker interpretation.
- Querying capabilities – Using the powerful IntelliSense-assisted query builder, you can query the data you’ve collected to the exact point you need. This query can also later be used to create custom alerts or dashboards.
- Alert processing integrations and automation – Using Azure Monitor you can easily forward your alert or query output to another systems for further processing with various options like Webhooks, ITSM, emails, triggering an Azure function, or an Automation Runbook.
Virtual Machines being monitored within Azure Monitor
Now, some limitations of Azure Monitor:
- It is still evolving – Compared to SCOM, Azure Monitor is still very young and evolving. While constant improvements are great to have, they also mean that Monitor simply still lacks some features and flexibility that a mature tool like SCOM provides.
- It requires you to know what you want – Since Azure Monitor does not run on the Management Pack framework, you are responsible for creating your alerts — which means you need to know what kind of alerts you want and how to set it up.
- It requires query knowledge – It is fundamental that you learn the Data Query language to maintain and manage your monitoring effectively with Azure Monitor. SCOM can still allow you to monitor primarily using the GUI, but Azure Monitor does not.
- It can get expensive – Alerts you configure within Azure Monitor are an ARM resources themselves and hence come with an attached cost for each alert generated. Alerting with Azure Monitor can therefore be more expensive than using SCOM, especially you have a large number of alerts in your subscription. However, it is not a fixed and upfront cost and is distributed over years so generally Monitor does not put a great burden on the budget.
Your infrastructure is hybrid – spread across on premise AND cloud
A hybrid cloud scenario in which you are still using legacy on premise data center resources while also expanding into cloud IaaS, PaaS, and SaaS is the most interesting scenario. It is also one that provides the biggest challenges, but also the best value from your monitoring solution. You guessed it right – use the best of both monitoring solutions!
When you integrate both Azure Monitor and SCOM, something greater comes out. They complement each other very well and also bridge the gaps of each other. So you get the depth and customization of monitoring with SCOM and the precision and cloud integration of Azure Monitor.
Better yet, they both use the same agent, called the MMA (Microsoft Monitoring Agent). This is especially helpful when migrating to the cloud or communicating between cloud servers and on premise resources, as either Monitor or SCOM can discover which machines are talking to which other infrastructure components and analyze that data so you can better configure networking, security, and other settings.
Log Analytics can receive data from both Systems Center 2019 and Azure Monitor for other uses.
Why use both SCOM and Azure Monitor?
- You’re now getting more useful data rather than spam. Azure Monitor’s querying capability plays an important role here. You collect the data from Azure resources as well as on premise servers, and only extract the data you need for alerts that are meaningful to you.
- Azure Monitor provides a single pane of glass for alerts and ways to manage them across your infrastructure, so it reduces the administrative overhead considerably.
- With all the data going into Azure Monitor, you can actually shut off a lot of workloads you don’t need from SCOM which means better performance and less underlying resources used to run SCOM monitoring.
- SCOM monitors what it monitors best (on premise infrastructure) while Azure Monitor Monitors what it monitors best (cloud resources).
- You can reduce the dependency on only one monitoring solution and run these two in parallel for resiliency — i.e. if Monitor happens to miss a critical alert, SCOM will likely pick it up.
- You can leverage PowerBI to visualize the data from both SCOM and Monitor within your Azure subscription.
- With the recent release of SCOM 2019, which includes new capabilities and better visibility into cloud resources, this integration has become even better!
- It is much more cost-effective considering the returns it provides in value in a long run, both in terms of avoiding downtime due to performance or configuration problems and when considering the administrative overhead involved in configuring an efficient monitoring workflow across a complex hybrid environment.
So there you have it — two monitoring tools from Microsoft for each of the primary hosting scenarios. Hopefully that gives you more context and helps you make a decision to proceed with the best way of monitoring for your specific environment.