Series

What we run

A series about operating the infrastructure underneath everything else — the migrations, the failures, and the checks that turned out to be pointed at the wrong thing.

2 posts

Back to all posts

Infrastructure

Special vdev: 113 TB hanging on two files

The special vdev of a 113 TB pool sat on two qcow2 files — both on the same consumer SSD, with 11% of its endurance left. Moving it meant operating on a live mirror with no room to be wrong. And it surfaced two checks that had been pointed at the wrong thing the whole time.

Patrick Lehmann

12 August 20269 min read

Engineering

357 Seconds per Push, and the Tests Were Not the Problem

Our pre-push gate took almost six minutes. The container read the repository through a bind mount, and that alone accounted for 324 of the 357 seconds.

Patrick Lehmann

27 August 20266 min read