Dry Runs, Fixtures and Self-Tests: Testing a Script Before It Touches Anything

Why this matters

Writing Idempotent Deployment Scripts explained what makes a script safe to run twice. This chapter is the other half: how to be sure a script is safe to run once, on real files, before it has ever run there. The tools are old and plain: a preview mode, a copy to practise on, and a check at the end. Together they let you change a live website with the same calm you would have changing a scratch file.

What you'll learn

The six safeguards in a real one-time installer, how to build a fixture and what to test on it, and why a script that stops and writes nothing is a success, not a failure.

The example: an installer with six safeguards

This site's changes are made by small one-time Node.js scripts. The header of one of them states its own safety rules: it is a dry run by default and writes only with --apply; it backs up every file it touches; it reloads the changed code in a fresh process and runs a self-test; and on any failure it restores every file. Here are the safeguards one by one.

1. Dry run by default. The script writes only if you explicitly ask:

const APPLY = process.argv.includes('--apply');

Run it without the flag and it prints what it would do and stops. You can run the preview ten times at no risk, and read the plan before you approve it.

2. Every edit must match exactly once. Each change is described as a file, a label, a pattern to find, and a replacement, and the pattern must match exactly once. Zero matches means the file is not what the script expected. Two matches means the edit would land in the wrong place or twice. Either way the script aborts and writes nothing. An aborted run costs you a minute. A wrong edit can cost you an afternoon.

3. A backup before every write. Whatever happens next, the original is saved. A script with a bug still leaves a clean way back.

4. A way to point it at a copy. The script accepts a --root= argument, so the same code can run against a copy of the project instead of the real one:

const rootArg = process.argv.find(a => a.startsWith('--root='));
const ROOT = rootArg ? path.resolve(rootArg.slice(7)) : process.cwd();

This is what makes fixtures possible, and it costs two lines.

5. A self-test in a fresh process. After writing, the script starts a new program that loads the changed files from scratch and checks them. A fresh process matters: your running script may still hold the old version in memory, so only a new one sees what is really on disk.

6. Restore everything on any failure. If the self-test fails, the script puts every file back. All the edits happen or none do, so you are never left with half a change.

Build a fixture

A fixture is a small copy of the files your script will touch, made only for testing. Make a scratch folder with the few files the script reads, in the shape the real project has. Then run the script against it with --root=. On a fixture, a mistake is free and the result is repeatable.

Test more than the happy path. For a script like this, the useful cases are:

1. The dry run changes nothing. Compare the files before and after.
2. Apply works, and the self-test reports success.
3. A second run does nothing. This is the idempotence check from the previous chapter.
4. A missing input aborts cleanly, with the file left as it was.
5. An anchor that appears twice aborts.
6. Windows line endings survive. A script that rewrites a CRLF file with LF endings would show up as a change on every line.
7. A deliberately broken self-test triggers the restore. Break the check on purpose and confirm the original comes back. A restore you have never seen working is only a hope.

A real lesson: never anchor on a file you have not read

The log for this tutor project records that the first attempt at one change aborted without writing, because it looked for text that the real file did not contain. That is the safeguards working: the exactly-once rule stopped it, nothing was damaged, and the fix was to read the actual file before writing the pattern. The rule that came out of it is short and worth keeping: read the file before you write the script that edits it.

Why stopping is a good result

New programmers often feel that a script which aborts has failed. For a script that edits live files, it is the opposite. Stopping with a clear message and untouched files means every check did its job. The failure to fear is the quiet one, where a script runs happily and does the wrong thing.

Try it yourself

Write a small script that adds one line to a text file, with all six safeguards: a dry run unless --apply is given, a check that the target text appears exactly once, a backup, a --root= option, a check at the end that reads the file back, and a restore if the check fails. Then run the seven fixture tests above against a copy. If you can make each of the failing cases fail on purpose and see the file come back unchanged, the script is ready.

What's next

Settings are the other thing scripts must not hard-code. Read Config Is Not Code. To see a change like this travel from your branch to a live site, read From git push to a Running Container on Cloud Run.

Continue learning

Mark this article as read

← Back to Guides