Back to overview
Container filesystem 'file not found' bug
Sep 13 at 01:59pm EDT
Affected services
CPU functions
GPU functions
Resolved
Sep 13 at 01:59pm EDT
At 12:57 UTC a container filesystem change rolled out across workers which caused spurious 'file not found' (ENOENT) errors against the container's rootfs filesystem.
We identified the bad commit and rolled back to prevent new workers from picking up the bug.
We then worked to roll over all running production workers which were affected by the bad change and scale up new workers.
All affected workers have been quarantined and thus new containers will start on new, healthy workers.
We apologize for the disruption.
Affected services
CPU functions
GPU functions