How we review monitoring tools

Sources we work from

Every review starts with the vendor’s own material: the current documentation, the installation and sizing guides, the pricing and licensing pages, and the release notes for at least the last two major versions. We read the changelog closely, because that is where upgrade pain, deprecations and schema migrations get disclosed before anyone outside the project notices them.

Where a vendor offers a public demo instance or a time-limited free trial, we work through it to check that the interface, discovery flow and alerting rules behave the way the documentation says — that gap, between what a product claims and what its own UI actually does, is where a lot of reviews go soft. Beyond the vendor, we read widely reported admin experience: release threads, GitHub issue trackers, community forums and public case studies, weighted toward reports from people running the product at a scale close to our typical reader’s. For numbers beyond what public sources cover — thousands of hosts, dozens of remote sites — we rely on the vendor’s own sizing guides and say so explicitly. A figure we cannot trace to a named source does not get printed.

The six criteria

Every review answers the same questions, in whatever order suits the product:

• Time to first useful dashboard. From an empty VM to a view someone would actually keep open. Hours, not marketing minutes.
• Database and storage growth. How history is stored, how it is pruned, and what that means for disk after a quarter.
• Poller and server sizing. Where a single server stops coping, and how the product scales out with proxies, satellites or extra pollers.
• Template and check quality. Whether stock checks for Linux, Windows and common network gear are sensible defaults or need rewriting.
• Alert noise controls. Dependencies, flapping detection, escalation delays and maintenance windows — and whether they work as documented.
• Upgrade pain. What the last two major upgrades required: schema migrations, config changes, downtime, rollback options.

Licensing and cost sit alongside these. For open-source tools we look at what the license allows and what the company behind it sells. For commercial tools we describe the pricing model (per sensor, per device, per node) and only quote figures we can source, always with a pointer to the vendor’s current price page.

Three products, Angry IP Scanner, Wireshark and LizardSystems Network Scanner, are not monitoring systems, so several of these criteria do not apply to them. We review them from a monitoring admin’s seat instead: how well they help seed or audit a monitoring inventory, how easily they prove whether poller traffic reaches a device, and what they leave for the monitor to do. For a desktop tool we also note how recently the vendor released an update and which operating systems it officially supports.

The readout: OK, WARN, CRIT

Every product carries a three-line readout borrowed from monitoring itself. OK is the strongest point the documentation and the admin record support. WARN is the operational cost you should plan for. CRIT is the issue most likely to rule the product out for a particular team. They are editorial judgements, not live data and not a score. A CRIT line does not mean a product is bad; Nagios Core’s lack of built-in graphing is a dealbreaker for some teams and irrelevant to others.

How comparison tables are ordered

Category tables list products in the order we most often recommend them for that job, based on the criteria above. Order is never sold, and no vendor pays us for links or placement. Each table includes at least six products, including at least one commercial option, so readers can see where paying for software starts to make sense.

Updates

We revisit a review when a major version ships, when licensing changes, or when a reader reports an error. The date under each headline is the last time the page was checked against the current release, not merely edited for typos.

What we will not do

• Host, mirror or repackage any software.
• Accept payment for placement, a better readout or a pre-publication look at a review.
• Invent prices, free tiers or partnerships that do not exist.
• Publish user ratings or testimonials we cannot verify.

Questions about the method are welcome at editor@scannethub.biz.