Booting my Raspberry Pi 4 from a USB device

USB boot has been possible since the Raspberry Pi 2 Model B (v1.2), but it has only become really worthwhile with the Pi 4. Here is some information, tips, tricks and opinions explaining why and how.