Discussion about this post

User's avatar
Overcoming Bias Commenter's avatar

Thanks. I think I understand. To summarize, the crucial difference seems to be that security design has to contend with adversarial inputs while normal program design has to deal only with neutral inputs.

Expand full comment
Overcoming Bias Commenter's avatar

Security is like certain sorts of correctness, and unlike most other features, in that it's a universal property rather than an existential one.

"There should be a way for a user to export their data to CSV" is an existential requirement, that can be fulfilled in an existing system by adding and integrating a module.

"There should not be any sequence of inputs that crashes the program" is a universal requirement (or negated existential, if you prefer), and code nearly anywhere in the system could potentially violate it. These kinds of properties get harder to check or to add as the system gets bigger.

In a distributed system, security properties for each component tend to be of the form "No [communication from other nodes] can cause me to do [undesirable or restricted behavior] unless they [prove it's authorized]." Showing that the program can't be convinced to do the thing through unorthodox means (e.g. buffer overruns) requires wide-ranging scrutiny and possible changes to the program itself. Providing appropriate and verifiable proofs of authorization requires accommodation from the network protocol (so everyone agrees on how to send the proofs) and from all other distributed nodes (so they actually send the necessary and sufficient authorization proofs), so it's a very costly change at the level of the network even if authorization can be a self-contained module in each network component.

In the case of the cell carriers, they formulated the wrong security property when they created the system: something like "Only actual cell carriers can connect to the SS7 network," instead of "Don't release information about a phone to anyone but the carrier it belongs to." This requires a change to the protocol (difficult or impossible to do piecemeal) and to all the different pieces of software that constitute the network.

Expand full comment
5 more comments...

No posts