For those that deal with attack trees/graphs in #infosec, how much weight do you put into probability? To me, human probability is an arbitrary and unnecessary metric.

Am I wrong to think that human probability (e.g., skill set, demographic, likelihood of successful compromise) is far too random to be considered in a tabletop attack tree variable?

I'm trying to determine why this is a common thing.

@pidvicious That is the thing about being up against human adversaries, they are inherantly unpredictable.
@pidvicious Sometimes a person will do an unlikely thing because it is unlikely, and therefore more likely to work.
@pidvicious It is common to include threat actor likelihood of initiation and capability in risk calculations as it is a necessary component of determining probability of success, which combined with impact gives you risk.
@pidvicious "but wait" you might say, "you just said people are inherantly unpredictable!" Yes, individuals are, but populations are more predictable.
@pidvicious As long as it is taken as a generalization, and understood that it is impossible to account for unknown/unknowns or "black swan" events in risk models, then they are more useful than not.
@pidvicious This is my concern with the folks that are advocating for quantitative risk analysis over qualitative. There is no doubt that truly quantitative risk analysis would be far superior, if it actually works. But all of the quantitative risk models I've seen so far combine qualitative likelihood in with quantitative impact, and pass it off as a quantitative score ($x money over $y years), which gives it more gravitas. But risk scores should be treated as estimates of general behavior, not actual predictions, as I have not seen any model yet which provides accurate quantitative likelihood.
@pidvicious Beyond human behavior, I also believe it is an NP-hard problem to determine the attack graph itself, and most of them I've seen are overly simplistic.
@pidvicious The challenge as I see it is the large number of ways for an attacker to change their context, and each context is the start of a new attack graph. This is more commonly called "pivoting" and the number of potential combinations can be very large, even for fairly simple architectures.

@cbyrd01 Thanks for the feedback, Christopher. You are right that it shouldn't be abandoned completely as a metric and made some good points regarding its purpose.

I guess what throws me off is putting a numerical likelihood value next to a skill set (e.g., a teenage hacker has 50% chance of accomplishing X). And maybe that's statistically correct, but I'd need to see documentation supporting that number before I assigned it any risk value.

Cheers!

@pidvicious One thing that can help significantly is calibrated probability assessment. It can help you have more confidence and consistency in predictions like you mention. I have seen it in action, and the training really does help with that. If you haven't seen it before, here's a link to a good background article on Wikipedia. https://en.wikipedia.org/wiki/Calibrated_probability_assessment
Calibrated probability assessment - Wikipedia

@pidvicious That said, even with calibration it is still a qualitative assessment. It is useful for qualitiative risk assessment, but IMHO its a mistake to treat it as quantitiatve.