The Fewest Doors

In March 2024, a software engineer named Andres Freund noticed something odd on the Linux machines he was working with. Logging in over SSH was using a surprising amount of CPU, and a debugging tool was throwing strange errors. He timed it. Before a recent update, a login attempt took 0.299 seconds. After it, 0.807 seconds.

Half a second. Most people would never have noticed. Freund did, because he knew his system intimately, and he pulled on the thread. At the end of it he found a backdoor, planted with years of patience in a small compression library called xz, which was on its way into the heart of millions of servers. It was caught before it reached the stable versions of the big Linux distributions. Had it got there, someone would have held a key to a large part of the internet.

If you’ve read Mechanical Sympathy, you’ll recognise what saved the day: an engineer with a feel for how his machine should behave. Hold on to that. It turns out to be one of the deepest security principles there is.

This piece is long, because the subject is wide. It’s written for people who build software, and it moves in seven steps:

1. The ground you stand on: why your code is the smallest part of your security.
2. The principles: the ideas that have held since 1975.
3. The materials: memory, languages and the hardware itself.
4. Other people’s code: the supply chain, and the story of xz.
5. Cryptography, and what security really costs.
6. The smallest castles: embedded devices.
7. A method: how to decide, and how to build.

Part I. The ground you stand on

Your code is the thinnest layer

When we picture our application, we picture the code we wrote. But that code sits on a tall stack of things we didn’t write and mostly can’t see.

Every layer in that stack can undermine every layer above it. A flaw in the processor can leak your secrets no matter how carefully you wrote your code. A compromised library runs with all your program’s powers. A tampered build server can put something into your release that was never in your repository.

Security people have a name for the set of everything that must work correctly for a system to be secure: the trusted computing base, or TCB. The single most useful goal in secure design is simple to say and hard to do: make the trusted computing base as small as possible. Every line, library, service and person you have to trust is a door someone could come through.

Ken Thompson’s warning

In 1983, Ken Thompson, one of the creators of Unix, accepted the Turing Award with a short lecture, published the following year as Reflections on Trusting Trust. In it, he described an attack that still unsettles people forty years later.

He imagined modifying a C compiler so that, whenever it compiled the Unix login program, it secretly added a password that would let him log in as anyone. Then he went one step further. He taught the compiler to recognise when it was compiling itself, and to insert both tricks into the new compiler. Once that poisoned compiler was installed, he could delete every trace of the attack from the source code. The source would be clean. The binaries would still carry the backdoor, forever, reproducing itself every time the compiler was rebuilt.

His conclusion was blunt: you can’t trust code you did not totally create yourself, and no amount of reading source code will save you from a dishonest tool beneath it.

Nobody can create everything themselves. So the real lesson isn’t despair. It’s this: trust is a chain, and you should know where yours begins. We’ll come back to how engineers fight back.

Is the safest system no system at all?

You might have the intuition that the most secure system is the one that doesn’t exist, and the next most secure is one that isn’t connected to anything. Security people agree. Gene Spafford, one of the field’s elders, said as far back as 1989 that the only truly secure system is one that is switched off, cast in a block of concrete and sealed in a lead-lined room — and even then he had his doubts.

That’s a joke with a serious shape. Picture a line running from no system to general-purpose computer on the internet. Every step to the right adds usefulness, and every step adds doors.

But notice that even the left-hand end leaks.

• Air gaps can be crossed. The Stuxnet worm, discovered in 2010, reached the isolated computers of an Iranian uranium-enrichment plant, very probably on removable drives such as USB sticks. It used several previously unknown Windows flaws, took control of the industrial controllers spinning the centrifuges, changed their speeds to damage them — and meanwhile reported normal readings to the operators. An air gap is a high wall, not a sealed box. People carry things over walls.
• Single-purpose devices grow extra doors. In 2015, researchers Charlie Miller and Chris Valasek showed they could reach a Jeep Cherokee over its cellular connection, get into its entertainment system, and from there reach more vital parts of the car. The fix was a recall of 1.4 million vehicles. The radio was a door into the vehicle.

Some places solve this with physics rather than software. Power stations and other critical industrial sites sometimes use data diodes: hardware that lets data travel in only one direction, typically an optical transmitter on one side and a receiver on the other, with no path back. Monitoring data can flow out; nothing can flow in. No bug in a program can make a beam of light go backwards.

So your intuition is right, and it generalises into the first real principle of this piece:

Every feature, every connection, every library, every stored byte and every person with access is a door. Security begins with having fewer of them.

Part II. The principles

Eight ideas from 1975

In 1975, Jerome Saltzer and Michael Schroeder, at MIT, published a paper called The Protection of Information in Computer Systems. In it they listed eight design principles. Half a century of technology has passed, and almost nothing about them has dated. Here they are, translated into today’s language:

1. Economy of mechanism. Keep the design small and simple. You can only check what you can understand.
2. Fail-safe defaults. Base access on explicit permission, not on the absence of a ban. When in doubt, the answer is no. A new user, a new endpoint, a new device: closed until opened.
3. Complete mediation. Check every access, every time. Don’t let a permission checked once at the front door be assumed for every room afterwards.
4. Open design. Security must not depend on the design being secret. Assume the attacker has your source code.
5. Separation of privilege. Where it matters, require two keys. A release signed by two people. A transfer approved twice.
6. Least privilege. Give every program and person only the powers they need for the job, and only for as long as they need them.
7. Least common mechanism. Minimise the machinery shared between users, because shared machinery is a channel between them.
8. Psychological acceptability. Make the secure way the easy way. If protection gets in the way, people will route around it.

The last one is the most neglected. A great many real-world failures happen not because the right lock didn’t exist, but because it was inconvenient to use. A system that is secure only when people behave perfectly isn’t secure.

Two ways to build

Tony Hoare, the British computer scientist who gave us Quicksort, said in his own Turing Award lecture in 1980 that there are two ways of constructing a software design. One is to make it so simple that there are obviously no deficiencies. The other is to make it so complicated that there are no obvious deficiencies. The first, he added, is far more difficult.

That sentence is, quietly, the whole of security engineering.

What ten years of qmail taught

In the late 1990s, Daniel J. Bernstein wrote a mail server called qmail, at a time when mail servers were among the most attacked software on earth. Ten years later, he looked back. In that decade only four bugs had been found in qmail 1.0 releases — none of them a security hole — while over a million internet mail servers were running qmail or its descendants.

His reflections, Some thoughts on security after ten years of qmail 1.0, are among the wisest pages ever written on the subject. He boiled his approach down to three moves:

• Eliminate bugs. Security holes are just bugs that someone can exploit. So change your programming environment to make whole kinds of bugs hard to write. Don’t just be careful; arrange things so that carefulness is less needed.
• Eliminate code. Bugs come per line. Fewer lines per feature means fewer holes.
• Eliminate trusted code. Put most of the program in a “prison” where, whatever bugs it has, it can’t break the security rules. Then the small part that must be trusted becomes small enough to check properly.

And he made a pointed remark about one of the 1975 principles: minimising privilege is not the same as minimising the amount of trusted code. Dropping privileges is good, but it can give false comfort. The real question is always: how much code would have to be wrong for this to be broken?

He also said something every engineer should hear: make it secure first, then work on making it faster.

Separate the dangerous part

Here’s what eliminating trusted code looks like in a program you’ve almost certainly used.

OpenSSH, the tool that lets you log in to remote machines, has to do two very different things. It has to talk to strangers on the internet, which is dangerous. And it has to do a few things only a powerful, privileged program can do, like switching to your user account after you log in. Early on, both lived in one process, running with full power. A bug in the networking code meant full control of the machine.

In 2003, Niels Provos and colleagues split it in two:

A small, privileged monitor does only the handful of things that need power. A larger, unprivileged worker does all the talking to the network, locked in an empty directory with no rights at all. When the worker needs something privileged, it sends a message down a narrow pipe, and the monitor checks every request against a strict list of what’s allowed at that stage. Anything unexpected, and the monitor shuts everything down.

In the paper’s analysis, about two-thirds of the code — counting the libraries — ended up running without privilege. The change touched only around two percent of the existing lines, and logins were no slower. Several earlier OpenSSH bugs, the authors argued, would have been contained.

Look at the shape: two boxes and a pipe of messages. Remember, change, send, used as a security tool. Once you see this pattern you find it everywhere: browsers putting each website in its own sandboxed process, phones giving each app its own locked room, OpenBSD’s pledge, which lets a program promise up front the few kinds of thing it will ever do, and kills it on the spot if it breaks that promise.

Treat input as a hostile language

One more principle, and it explains an astonishing number of disasters.

Every program that receives data — a request, a file, a message — is running a little interpreter for whatever language that data is written in. If that interpreter is sloppy, the attacker gets to program your program.

The Log4Shell vulnerability of December 2021 was the perfect example. Log4j, a hugely popular Java logging library, would look inside the text it was asked to log for special ${...} expressions, and act on them. One of those expressions told it to fetch and run code from another server. So an attacker could type a carefully crafted string into a username field, or a browser header, anything that might end up in a log, and the server would obediently download and run the attacker’s code. It scored 10 out of 10 for severity. Writing a line to a log file, the most innocent thing a program does, had become a way in.

A group of researchers working under the banner of language-theoretic security (LangSec) put the lesson into slogans worth taping above your desk:

• Full recognition before processing. Check that input is entirely valid, against a precise grammar, before you act on any of it.
• Don’t give input more power than it needs. The simpler the input language, the harder it is to abuse. Data should never be able to become code.

Part III. The materials

Memory safety: the 70% problem

If you write C or C++ — and anyone heading towards low-latency trading systems or embedded firmware will write a great deal of it — there is a number you should know.

Microsoft has reported that around 70% of the security vulnerabilities it assigns each year are memory-safety bugs, and Google’s Chromium team found the same share among its serious security bugs: reading or writing outside the memory a program meant to touch, using memory after it has been freed, and their many cousins. In C and C++, the language lets you make these mistakes, and attackers have spent decades turning them into weapons.

The most encouraging result in recent years came from Android. Rather than rewrite its old C and C++ code, Google began writing new code in memory-safe languages such as Rust, Kotlin and Java. Over about six years, memory-safety bugs fell from 76% of Android’s vulnerabilities in 2019 to 24% in 2024, and the number found each year fell from 223 to under 50.

The reason it worked is beautiful. Google found that vulnerabilities are concentrated in new and recently changed code. Old code that has been running for years has had its bugs shaken out; it matures and gets safer with time. So you don’t need to rewrite the world. You only need to stop adding new unsafe code. The dangerous part of the system shrinks naturally, year by year.

This is Bernstein’s eliminate bugs at industrial scale: don’t ask programmers to be perfect; choose materials in which whole classes of mistakes cannot be made. For C and C++ code that must remain, the same spirit applies: strict coding standards such as MISRA C and CERT C, bounds-checked abstractions, compiler hardening, sanitizers in testing, and fuzzing — throwing millions of random and malformed inputs at your parsers until something breaks, before an attacker does it for you.

The hardware is not neutral

In Mechanical Sympathy, we met the cache: the processor’s little desk drawer of recently used memory, which makes everything fast. In January 2018, the world learned that the drawer can also be a spy.

Spectre and Meltdown exploited a trick processors use to go faster: guessing which way the program will go next and running ahead speculatively. When the guess is wrong, the processor throws the results away. But it doesn’t tidy the cache. And by carefully timing how long memory reads take — a fast read means the data is in the drawer — an attacker could recover secrets the program was never allowed to see, even data belonging to the operating system itself. The fixes slowed many machines noticeably.

A few years earlier, researchers had shown that hammering one row of memory chips rapidly enough can flip bits in neighbouring rows — Rowhammer — and that this physical glitch could be turned into a way to gain control of a machine.

The lesson for software people is humbling. The very tricks that make our machines fast are side doors, and the layer at the bottom of Figure 1 is not a solid floor.

Building a chain of trust from the ground up

So how does a device ever start from a trustworthy state? With a chain of trust that begins in something that cannot be changed.

When the device powers on, the first code to run is a small piece of read-only memory built into the chip itself, impossible to alter. Inside the chip, in that ROM or in one-time fuses, sits a fingerprint of the manufacturer’s public key. The boot ROM checks the bootloader’s digital signature against that fingerprint and refuses to run it if it doesn’t match. The bootloader checks the operating system the same way, and so on up the chain. If any link fails, the device stops.

This is exactly Thompson’s lesson, turned into engineering: you can’t avoid trusting something, so make the thing you trust as small, simple and unchangeable as possible, and verify everything else from there.

Proof, and capabilities

Two ideas point towards a future where some of this becomes far less fragile.

The first is formal verification. In 2009, a team in Australia published a mathematical proof that their operating-system kernel, seL4, about 8,700 lines of C, does exactly what its specification says: no crashes, no unsafe memory operations, no behaviour outside the design. It took years of work. And the proof itself honestly lists its assumptions: that the compiler, the bits of assembly code and the hardware behave correctly. Even a proof rests on a trusted base. But seL4 shows how far you can push it when the trusted part is small enough. Economy of mechanism is what makes proof possible.

The second is capability hardware. Researchers at Cambridge and SRI International, working with Arm, built CHERI, a way of extending processors so that every pointer carries its own bounds and permissions, checked by the hardware on every memory access. A pointer to a 16-byte buffer simply cannot be used to reach byte 17. Arm built an experimental chip, Morello, with UK government backing. A smaller variant, CHERIoT, is aimed at exactly the embedded devices we’ll get to shortly. If it spreads, whole families of the 70% problem stop being possible even in C.

Part IV. Other people’s code

The story of xz

Now let’s go back to Andres Freund and his half-second, because what he found is the clearest picture we have of the modern threat.

xz is a small compression library. It’s the kind of thing nobody thinks about, which is precisely why it was chosen. Like many such projects, it was maintained by essentially one volunteer, Lasse Collin, unpaid, in his spare time.

In late 2021, a new contributor calling themselves “Jia Tan” sent a first small, helpful patch. More followed. They were good patches. In 2022, a handful of other accounts — names that, it later turned out, appeared almost nowhere else on the internet — began writing to the project’s mailing list complaining that progress was too slow, and that the maintainer should hand over to someone with more time. Collin replied honestly that he had limited energy, that it was an unpaid hobby project, and that Jia Tan had been helping and might take a bigger role. Over the following months, Jia Tan became a co-maintainer, with the power to make releases.

In February 2024, they struck. The attack was hidden with real skill:

• The payload was committed to the repository in plain sight, but disguised as binary test files: deliberately corrupt compressed data of the kind a compression library needs for its tests, and which nobody reviews byte by byte.
• The trigger, a small modified build script, appeared only in the release tarball, the downloadable package that Linux distributions actually built from. It wasn’t in the repository people read.
• During the build, that script quietly extracted the hidden code and stitched it into the library, but only on 64-bit x86 Linux, only when building Debian or Red Hat-style packages, only with the expected compiler and linker — staying silent everywhere else.
• At runtime, it used an obscure feature of the dynamic linker, introduced in an earlier “performance” patch, to swap out a cryptographic function used by the SSH server. Anyone holding the attacker’s private key could then run commands on the machine.

It got into the development versions of Debian and Fedora — the ones that eventually become stable releases. And then a careful engineer noticed his logins were half a second slow.

The xz story is worth sitting with, because almost every lesson of this piece is inside it:

• The attack targeted a person, not a program: a tired volunteer, pressured by fake voices.
• It hid in what nobody looks at: binary test files, release tarballs, build scripts.
• It reached its target through a dependency of a dependency. Many distributions had patched their SSH server to link with system libraries that in turn pulled in xz. OpenSSH itself never asked for it.
• It was defeated by mechanical sympathy: someone who knew what normal looked like.

It keeps happening

xz is not alone. It’s the most elegant example of a pattern:

• event-stream, 2018. A popular JavaScript package changed hands when a stranger offered to take over its maintenance. The new maintainer added a dependency that hid code aimed at stealing from one particular Bitcoin wallet app.
• SolarWinds, 2020. Attackers got into the build servers of a company that makes IT management software. A small piece of malware watched for the product’s build to start and swapped in a poisoned source file at the last moment, so the backdoor was compiled into an official, signed update. Up to 18,000 customers installed it; the attackers then chose a few hundred to exploit, including government agencies.
• Heartbleed, 2014. A bug in OpenSSL, the library securing a large share of the web’s encrypted traffic, let anyone read chunks of a server’s memory. It emerged afterwards that OpenSSL had one full-time developer and received around $2,000 a year in donations — enough, it was said, for the electricity bill.

There’s a well-known cartoon of all modern digital infrastructure as a towering stack of blocks, held up at the bottom by one small block labelled as a project some random person in Nebraska has been thanklessly maintaining since 2003. Every engineer laughs at it, and then goes a bit quiet.

What engineers can do about it

You can’t read every line of every dependency. But you can do a great deal.

• Have fewer dependencies. This is death cleaning for your package.json. Every dependency is a stranger you’ve given the keys to, along with all of their dependencies. Before adding one, ask whether thirty lines of your own code would do.
• Pin and lock. Use lockfiles and exact versions, ideally with cryptographic hashes, so that what you build today is exactly what you reviewed yesterday, and a new release can’t sneak in on its own.
• Update deliberately. Not never, and not automatically the moment something new appears, but on purpose, reading what changed. The xz backdoor did little real harm partly because it hadn’t yet reached the slower, more conservative stable releases.
• Review what you actually consume. xz showed that the release tarball and the repository can differ. Build from the source you’ve read.
• Reproducible builds. Arrange your builds so that anyone, building the same source, gets a bit-for-bit identical result. Then a poisoned build server, SolarWinds-style, gets caught the moment someone independent rebuilds and compares.
• Provenance and signing. The SLSA framework sets out levels of build integrity, from simply recording how a package was built, up to builds on hardened, isolated platforms whose signed records even insiders can’t easily forge. Sigstore makes signing practical by tying short-lived signatures to an identity and recording every signature in a public, append-only transparency log that anyone can audit.
• Know what’s inside. Keep a software bill of materials (SBOM) — a list of every component in what you ship. When the next Log4Shell lands, you’ll know in minutes whether you’re affected, not weeks.
• Fund the foundations. If your company depends on a volunteer project, the cheapest security measure you’ll ever buy may be paying its maintainer, so that they’re not tired and alone when a friendly stranger offers to help.

And Thompson’s compiler? In 2005, David A. Wheeler published an answer called diverse double-compiling. Compile the compiler’s source with the suspect compiler, and separately with a completely different, independently trusted compiler; then use each result to compile the source again. If the two final outputs match bit for bit, the suspect compiler is honest to its source. You still have to trust something, but it no longer has to be the thing under suspicion.

Part V. Cryptography, and what security really costs

A few rules of the craft

Cryptography is the mathematics of keeping messages secret and proving who sent them. In Remember, Change, Send, every computer system was made of remembering, changing and sending. Cryptography protects the first and the last: data at rest in storage, and data in motion across networks.

Its oldest rule dates from 1883. The Dutch linguist Auguste Kerckhoffs wrote that a cipher should stay secure even if everything about it, except the key, falls into enemy hands. Claude Shannon put it more bluntly in 1949: the enemy knows the system. It’s Saltzer and Schroeder’s open design again. Security that depends on a secret algorithm is security that’s waiting to expire.

A few more rules, learned the hard way:

• Don’t invent your own. Not your own cipher, and ideally not even your own way of combining standard ones. Use well-reviewed, high-level libraries designed to be hard to misuse, which choose safe defaults for you.
• Beware of time. In 1996, Paul Kocher showed that how long a cryptographic operation takes can reveal the key. Good libraries compute in constant time. Your own code comparing secrets — passwords, tokens — should too.
• Keys are the real problem. Encryption turns a large secret (your data) into a small one (the key). It doesn’t make the secret go away. Where keys live, who can reach them, how they’re rotated, and what happens when one is lost: that’s where most of the actual engineering is.
• Know where the message is readable. Encryption in transit protects data on the wire, but the server can read it. Encryption at rest protects it on disk, but the running service can read it. End-to-end encryption, as in Signal, means only the people at the ends can read it, not even the company running the service. Each is a different promise. Be clear which one you’re making.

Is secure software more expensive?

You’d expect the answer to be yes, because of all the computation that encryption needs. Here’s the surprise: the computation is the cheap part.

In 2010, Adam Langley at Google wrote up what happened when Gmail switched every connection to encrypted HTTPS. Encryption accounted for less than 1% of the servers’ CPU load, less than 10 kilobytes of memory per connection, and less than 2% extra network traffic. Google added no extra machines and no special hardware. Modern processors have instructions built specifically to make common encryption fast. His summary: TLS is no longer computationally expensive.

So the costs are real, but they live somewhere else. They sit below the waterline:

• Engineering time. Threat modelling, careful review, fuzzing, separating privileges, keeping dependencies tidy. Good work takes longer.
• Key management and recovery. With true end-to-end encryption, the company can’t reset your password and recover your data. That’s the point, and it’s also a real cost to users who lose their keys.
• Features you don’t get. A server that can’t read your data can’t search it, analyse it or show you clever things about it. Privacy and convenience pull against each other, though Ann Cavoukian’s Privacy by Design principles insist on looking for designs where you get both.
• Friction. Every check that slows an attacker also slows a legitimate user a little. Good design (psychological acceptability) is about making that friction tiny.
• The long tail. Patching for years. The UK now requires connected-device makers to publish how long they’ll provide security updates. Supporting a product safely for five or ten years costs money.
• Hardware, for embedded. A secure element to hold keys, more flash memory to hold two copies of the firmware so updates can safely roll back, a slightly bigger chip. Pennies per device, multiplied by millions.
• Speed, sometimes. The fixes for Spectre and Meltdown made some workloads noticeably slower. Sometimes safety costs performance, and Bernstein’s advice applies: secure first, then fast.

And then there’s the deeper reason the bill so often goes unpaid. Ross Anderson, the Cambridge professor who did as much as anyone to make security an engineering discipline, showed with Tyler Moore that many security failures are really economic failures. In a famous phrase, the danger is that the person guarding a system is not the person who suffers when it fails. A device maker saves a few pence by shipping a default password; the cost lands later, on someone else, when the device joins a botnet.

That’s why laws are now moving the cost back onto the people who can prevent the harm. But it’s also why the honest answer to is secure software more expensive? is: yes, at the start — and dramatically cheaper over the life of the system. Insecurity isn’t free. It’s a debt, collected later, with interest, often from someone else.

A note on the future

One more shadow on the horizon: quantum computers. A large enough one would break most of the public-key cryptography we use today. None exists yet. But an attacker could record encrypted traffic now and decrypt it later — harvest now, decrypt later — so long-lived secrets are already at risk. In August 2024, the US standards body NIST published its first three post-quantum standards (FIPS 203, 204 and 205). The engineering lesson is called crypto agility: design systems so that the algorithm can be changed without rebuilding everything around it.

Part VI. The smallest castles

Embedded devices — the thermostat, the camera, the router, the insulin pump, the car — are where all of this gets most concrete, because they’re often deployed by the million, rarely updated, and connected to everything.

In 2016, a piece of malware called Mirai enslaved huge numbers of cheap internet cameras and video recorders. How? It carried a list of about sixty factory-default usernames and passwords, and simply tried them. On many devices the credentials for remote access were hard-coded into the firmware, with no way for owners to change them. The resulting army of cameras attacked a major internet infrastructure company in October 2016, and with it took a long list of famous websites offline for hours.

Since 29 April 2024, it has been illegal in the UK to sell consumer connectable products with universal default passwords. The same law, the Product Security and Telecommunications Infrastructure regime, requires makers to publish a way to report security problems and to state the minimum period for which they’ll provide security updates. The EU’s Cyber Resilience Act goes further: since September 2026 manufacturers have had to report actively exploited vulnerabilities, and from December 2027 products with digital elements must be secure by design and by default to be sold in the EU at all.

For engineers building embedded systems, secure by default looks like this:

• No shared secrets. Every device gets unique credentials, or the user sets them at first use.
• Secure boot, anchored in the chip (Figure 4), so only signed firmware can run.
• Signed, safe updates. Updates verified before installation, with two firmware slots so a failed update rolls back rather than bricking the device. The update mechanism is both your most important defence and your most valuable target, so guard its keys like the crown jewels.
• Close the workshop doors. Lock debug interfaces such as JTAG and SWD before shipping. Turn off every service the device doesn’t need: no Telnet, no forgotten web admin page.
• Compartments. Use the processor’s memory protection to keep the network stack, the cryptography and the application in separate rooms, so a bug in one doesn’t own the others.
• Memory-safe where possible. Rust is increasingly practical on microcontrollers; where C remains, follow a strict standard and fuzz every parser.
• Do one thing. A thermostat doesn’t need a shell. A single-purpose device is the right-hand end of Figure 2’s spectrum only if you let it grow features it doesn’t need.

That last point is your intuition again. The “dumb” device, with only the features it was built for, is the more secure one. Not because simplicity is a virtue in itself, but because every feature you didn’t add is a door that doesn’t exist.

Part VII. A method

Decide first, while you’re calm

The Apollo engineers in The Computer That Knew What to Drop survived their alarm because they had decided, long before the crisis, what mattered most. Security starts the same way.

Adam Shostack, who spent years teaching threat modelling at Microsoft and beyond, reduced it to four questions anyone can ask, with no special jargon:

1. What are we working on? Draw it: the boxes, the arrows, the data, the people.
2. What can go wrong? In plain words. Who would want to misuse this, and how?
3. What are we going to do about it? For each answer: remove it, reduce it, accept it, or transfer it.
4. Did we do a good job? Look back, honestly, and do it again next time.

It’s the same priority table from the Apollo piece, and the same tiered inventory from The Keyring and the Drawer, applied to a system instead of a household: find the crown jewels, find the master keys, and spend your care where it matters most.

Secure by default

In 2023, America’s cybersecurity agency CISA, together with partners in many other countries, published principles for Secure by Design software. The heart of it is a shift in who carries the burden: manufacturers, not customers, should own their customers’ security outcomes. In practice that means shipping products that are safe out of the box:

• Multi-factor authentication on by default, and not a paid extra.
• No default passwords.
• Logging available without an upgrade.
• Whole classes of vulnerability eliminated at the root — by choosing memory-safe languages, or using frameworks that make injection impossible — rather than patched one bug at a time.

Privacy by design asks the same of data: privacy as the default setting, built in from the start, so a user who does nothing is protected. Under UK and EU data protection law it isn’t optional; data protection by design and by default is a legal duty.

And at the scale of a whole organisation, Google’s BeyondCorp work, published in 2014, popularised zero trust: there is no trusted inside. Instead of a firewall around a privileged internal network, every request to every application must prove who and what it comes from, wherever it comes from. It’s Saltzer and Schroeder’s complete mediation, applied to a company.

The fewest doors, as a checklist

If you take one page from this piece to your desk, take this one:

• Know what you protect. Ask the four questions before you build, and again before you ship.
• Collect less. Data you don’t store can’t leak. Delete on a schedule.
• Build less. Fewer features, fewer services, fewer open ports. The safest feature is the one you didn’t build.
• Trust less code. Shrink the trusted base. Separate the part that talks to strangers from the part that holds power, and join them with a narrow channel of checked messages.
• Parse strictly. Treat every input as a hostile language. Validate completely before acting.
• Choose safe materials. Memory-safe languages for new code; strict standards, sanitizers and fuzzing for the C that remains.
• Depend on less, and know what you depend on. Fewer dependencies, pinned and hashed, reproducibly built, signed, listed in an SBOM, and updated on purpose.
• Default to closed. Deny unless allowed. No default passwords. MFA on. Secure out of the box.
• Use cryptography, don’t invent it. Standard, high-level libraries; constant time; serious key management; agility for the future.
• Anchor trust in hardware. Secure boot, signed updates, keys in secure elements, debug ports locked.
• Plan for the bad day. Updates, rollbacks, incident runbooks and a way for people to report problems, all decided while you’re calm.
• Make the safe way the easy way. If security is hard to use, people will go around it.
• Know your system. Learn what normal looks like, so you notice when it isn’t.

---

Bruce Schneier, who has written about this longer and more clearly than almost anyone, has a line he’s repeated for a quarter of a century: security is a process, not a product. It’s never finished, any more than a house is ever finally clean. It’s housekeeping. Small, careful, regular work, done by people who care about the place.

And now the two ideas that this long walk kept coming back to.

The first is that minimalism and security are the same discipline. Every principle in this journal — fewer things, chosen well, kept close and understood — turns out to be what the best security engineers have been saying since 1975. Economy of mechanism is minimalism. Least privilege is essentialism. Collecting less is death cleaning. The smallest trusted base is the smallest, best-understood house. Hoare’s “obviously no deficiencies” is only reachable by people willing to build something simple.

The second is that the best defence is attention. Not a product, not a tool, but an engineer who understands their system so well that half a second out of place makes them stop and look. Mechanical sympathy isn’t only how you make software fast. It’s how, in the spring of 2024, a large part of the internet was kept safe.

Build small. Know your machine. Leave the fewest doors.

Sources

• Jerome Saltzer and Michael Schroeder, The Protection of Information in Computer Systems (1975); summary at Wikipedia
• Ken Thompson, Reflections on Trusting Trust (Turing lecture 1983, published 1984); discussed by Adrian Colyer
• C. A. R. Hoare, The Emperor’s Old Clothes (1980 Turing lecture); discussed by Adrian Colyer
• Daniel J. Bernstein, Some thoughts on security after ten years of qmail 1.0 (2007); discussed by Adrian Colyer
• Niels Provos, Markus Friedl and Peter Honeyman, Preventing Privilege Escalation (USENIX Security 2003)
• OpenBSD pledge(2) manual page
• LANGSEC explained in a few slogans
• Gene Spafford, Quotable Spaf
• On Stuxnet: CSO Online; on the Jeep recall: Computerworld
• CISA, The Urgent Need for Memory Safety in Software Products; Google’s Android results via The Hacker News
• CERT/CC, Spectre and Meltdown vulnerability note
• Cloudflare, Anchoring trust: a hardware secure boot story
• Gerwin Klein et al., seL4: Formal Verification of an OS Kernel (SOSP 2009)
• Capability Hardware Enhanced RISC Instructions (CHERI), Wikipedia
• Russ Cox, Timeline of the xz open source attack and The xz attack shell script; Andres Freund’s original report on oss-security
• npm, Details about the event-stream incident (2018)
• SecurityWeek, SUNSPOT malware used to insert backdoor into SolarWinds product
• On Log4Shell: noze; on OpenSSL after Heartbleed: Core Infrastructure Initiative, Wikipedia
• SLSA build levels; Reproducible Builds; Sigstore
• David A. Wheeler, diverse double-compiling, discussed by Bruce Schneier
• Kerckhoffs’s principle, Wikipedia
• Adam Langley, Overclocking SSL (2010)
• Ross Anderson and Tyler Moore, The Economics of Information Security (2006); Ross Anderson, Security Engineering, 3rd edition (2020)
• On post-quantum standards: Encryption Consulting
• Brian Krebs, Hacked cameras, DVRs powered today’s massive internet outage (2016)
• techUK, The PSTI Act for consumer IoT, explained; Finite State, Cyber Resilience Act timeline
• Adam Shostack, The Four Question Framework
• CISA Secure by Design principles, summarised by Secureframe; Ann Cavoukian, Privacy by Design: the 7 foundational principles
• Rory Ward and Betsy Beyer, BeyondCorp: A New Approach to Enterprise Security (2014)
• Bruce Schneier, Secrets and Lies (2000), reviewed at schneier.com