Mellow Root

Trusting is hard when risk adds up: why we can't be trusted

When working with security you usually need to restrict systems and people. Maybe you need an MDM (Mobile Device Manager) that controls which applications may or may not be installed, or maybe you need to restrict access to your cloud environment, virtual machines, and databases.

Some of the controls you implement will prevent people from doing what they want, or restrict them from working like they used to. People get frustrated and say things like "I feel like you don't trust me," and it's a valid concern because you don't fully trust them, do you?

I'll try to explain why people (including you and me) can't be trusted, and present a mental model one can use to get a better intuition about risks.

Good intentions are not enough

Even if you're the most loyal person whose good intentions I fully believe in, it won't prevent you from making a mistake or being misled. What if I grant you access to all systems only for your computer to be hacked? What if someone tricks you with phishing or social engineering? You still have your good intentions intact, but that didn't help the system or organization from being breached.

Good intentions today, but not tomorrow

Despite your good intentions today, they might be different in the future. What if your mother is dying of cancer and the treatment is ruining you both financially and emotionally, and then an external party gives you a great offer if you leak some information to them? What if you get fired for reasons you disagree with? What if schizophrenia or some other mental illness shows up and wreaks havoc?

It's easy for you to think "I would never do $BAD_THING, I'm a good boy!" but it's harder to believe that about everyone else. That takes me to my last point.

Risk adds up

Risk is usually defined as risk = likelihood * impact, here I'll only talk about the likelihood.

Let's make up some likelihood numbers for some of the scenarios mentioned above and see how it plays out:

Adding up risks example chart
Adding risks together like this is not realistically accurate, but serves as an example

These scenarios might seem unlikely to most people. You might consider yourself an emotionally and financially stable person that won't fall for any phishing shenanigans. But remember that most people think that about themselves and yet these things happen all the time.

If I trust each of my colleagues to not be breached with a 98.8% certainty, there is still a 1.2% chance they will. Then multiply that by the number of colleagues I have. If I have 75 colleagues I'm almost certain someone will have a bad time.

# average likelihood * number of colleagues = total likelihood 
1.2% * 75 = 90%

Adding risks together like this is very simplified and somewhat inaccurate, but it still works as a mental model.


You might be a security risk even if you have good intentions because bad things can happen unintentionally. Your situation might change and cause your good intentions to change with it.

Even if you think it's very unlikely that risks related to you plays out, you can probably understand it's not unlikely that they will for someone. An organization need to take the combined risk into consideration, not only the specific individual.

#mental-model #rant #risk #security