Want to reduce "alarm fatigue" caused by false positives? Then quit alarming the wrong stuff. Here’s how to decide what needs to be alarmed:
If it requires immediate human attention, it's an alarm. Otherwise, it's a log entry.
This is one of the most common mistakes I see in the field when I’m working with clients on streamlining their operations and reducing downtime. There is a strong (super-strong!) tendency to say, “I’m not sure, so I’d better make this an alarm.”
In many organizations, there’s also a fear factor: “If I decide not to alarm this, and then we have a critical system outage that could have been prevented, I’ll get blamed for not enabling this alarm.”
The cure for this is twofold: (1) alarm-or-log decisions should be made by a team, so no one person has the weight of the company’s downtime on their shoulders, and (2) the manager over the alarm-or-log team needs to review and approve the team’s plan and take full ownership of it. The manager needs to have the kind of personal integrity it takes to say, “I’m responsible.”
SIDENOTE: In the Navy I was taught, “You can delegate authority, but you can never delegate responsibility. You are responsible for everything your team does.” In the civilian world I’ve seen a lot of bad managers rise in power by blaming a subordinate, and then “fixing” the problem by firing and replacing the scapegoat.
This environment of fear results in everything being alarmed, and the result is that nothing is effectively alarmed.
THE WRONG SOLUTION TO ALARM FATIGUE
The wrong solution to alarm fatigue is excessive automation. Listen to me: there is no automated detection system in a complex network that can evaluate every combination of events. The speed and scale of automated monitoring systems is essential in large networks, but they augment the human agents, they don’t replace them.
#CallMeIfYouNeedMe #FIFONetworks +1 206-465-2422
Cybersecurity - Networks - Wireless – Telecom – VoIP