← All field notes

Black Lantern Labs Team  ·  Monitoring & triage

Monitoring Is Easy. Triage Is the Job.

Continuous collection is the easy part of "watching" an environment. Logs arrive. Scanners chirp. Dashboards stay green until they are not. The work that actually reduces risk is slower and less photogenic: deciding which signals matter in this stack, which can wait, and which need a human who already knows the layout.

Collection is not coverage

Many programs confuse volume with vigilance. More feeds feel like more safety. In practice, untriaged volume trains people to ignore alerts. A useful watch has a defined surface, a reason each source is collected, and a habit of asking whether today's noise changed anything that matters.

Automation belongs in the repetitive middle: pulling inventories, noticing new exposure, lining up candidate CVEs against known software. Judgment belongs with an operator: is this reachable, is it authentic, is it urgent, who owns the fix.

Environment-weighted vulnerability triage

New vulnerabilities show up constantly. Most are irrelevant to a given environment. Treating every headline as a fire drill burns the same people you need when a real path appears.

Triage that earns trust answers boring questions first:

  • Do we run the affected component, and which version?
  • Is it exposed to a realistic attacker starting point?
  • What data or control sits behind it?
  • What is the smallest change that closes the path?

A short prioritized list beats a guilt PDF. Teams that patch by scoreboard instead of path often leave the dangerous medium open while they chase a critical on a box nobody can reach.

Context when something looks wrong

When an odd DNS change, login pattern, or service appears, the first hour often decides whether the event is a footnote or an incident. The expensive failure mode is bringing in help that spends that hour learning the network diagram.

Craft here means retained context: what "normal" looks like for this org, which systems are crown jewels, which vendors sit in the path, and who can authorize a disruptive change. Monitoring without context produces tickets. Monitoring with context produces decisions.

What a useful periodic narrative contains

Whether the audience is engineering leads or operators, a useful summary is concrete:

  1. Watched - which surfaces and checks were in play
  2. Found - what changed or looked wrong, with enough detail to verify
  3. Did - what was escalated, fixed, accepted, or still open

Skip vanity uptime poetry. Skip CVE tourism. If the reader cannot tell what improved, the narrative failed.

A common category error

Technical monitoring and security program ownership are different jobs. Providers who blur "we watch systems" with "we own the written program, risk register, and leadership reporting" create confusion for buyers and for the IT partners already in the room. Keep the jobs distinct even when one organization eventually does both under separate scopes.

Bottom line

Watching an environment is mostly triage: weighted attention, honest severity, and humans who know the place. Collection without judgment is just a louder inbox.

← All field notes