Thursday, January 12, 2017




Let's look at the simple(ish) case of driving. There's an extraordinary amount of trust required. When you're behind the wheel, you're not only trusting every single engineer who designed every component of your vehicle, but of every surrounding vehicle as well. You're trusting the quality assurance processes of each manufacturer. You're trusting the assembly line technician. You're trusting the mechanics. The part manufacturers. Not to mention the surrounding infrastructure (who would want a road to collapse when they are driving on it).

You also need to trust the drivers around you. You trust that the person driving next to you on the highway isn't going to randomly jerk the wheel into you and cause an accident. You trust that they aren't drunk or otherwise impaired. That they are also driving in the shared best interest of safety.

Yet we all know that blind trust in any of those items is bad. Driving schools teach "Defensive Driving" techniques, to prepare you if someone does drive poorly around you. You take your car to mechanics to be inspected so you can verify that the parts that the engineer said were going to last, are going to be safe and last.

Each and every one of these "Trusts" are leaky, meaning that we all know of instances that each and every one of them have failed (remember the Ford Pinto?). Yet it's far more useful to us to trust than not to, even though we all know that trust isn't perfect.

As a society, we've built mechanisms to balance the benefits of reducing the necessary trust against the costs that the measures take to implement.

An Aviation Lesson

Compare driving a car to flying a plane. The regulations are far stiffer for the pilot in pretty much all cases. You probably know that it's far harder to get a pilot's license than a driver's license. A basic pilot's license takes 40-80 hours of training plus a comprehensive exam, plus 6 hours of training every 2 years. And there are many more licenses beyond that.


Another great example is in the case of air traffic control. Airspace (in the USA at least) is divided into 7 "classes", with 3 of them being "controlled" (you're not allowed to enter without talking to someone first).

Away from metropolitan areas, you can fly in uncontrolled airspace without talking on a radio. You're flying all by yourself using "Visual Flight Rules" (you need to see where you're going). You trust that other pilots are looking for you, are following rules as well as established conventions. In areas with more traffic, the rules increase.


Trust Is Always Required

No matter what situation you're in, there will always be some amount of required trust. Take AWS for example. You're trusting that Amazon really doesn't have the keys to the server they built for you. You're trusting that they are going to give you the uptime they promise (or in Amazon's case, that they imply). You're trusting that the machine is what they say it is. You're trusting them for a whole lot more than that in reality, but there's always trust required.



Could you prevent the possibility in the first place? Not in the generic case. Someone effectively needs access to the root key (either directly, or through another machine). You can lock your engineers down or you can give them total freedom. Or, you could do the smart thing and give them enough freedom so they can do their jobs effectively, and implement "verification" systems to detect breach of trust (even for "honorable" reasons).

This is what many companies do. Trust, but verify.

Perfectly Secure Applications Are Unusable

Security is always a trade-off against usability. The only way to make a perfectly secure system is to not make the system in the first place. The trick isn't making the system perfectly secure, the trick is balancing the security requirements with the costs (monetary, time and usability) to implement those requirements.

It's important to note that I'm not talking about security requirements that are about diligence in construction, such as SQL Injection, XSS, etc. Those costs are literally nothing compared to the wrong way, if you do it right from the beginning. So the investment in education and training on those types of attacks is vital and should never be traded away. There's no reason to go out to sea in a boat full of holes in the hull.




After all, you're going to trust a janitor in a supermarket a lot more than you're going to trust a janitor in a nuclear power plant. Both require trust, but must you tailor the limitations and mitigations to the problem at hand.