OS-level issues on Linux, macOS, Windows; drivers, permissions, shells.
When a Linux server is slow, the order of the first commands matters more than the tools. A ten-minute triage routine that tells CPU, memory, disk and network pressure apart, what each column actually means, and the mistakes that send people down the wrong path.
When a server is slow and the dashboard is vague, a dozen standard commands narrow the problem to CPU, memory, disk, network or a single process in about five minutes. This is the order we run them in and how we read each output.
Every night API latency tripled for an hour and the 1 Gbit uplink of one server sat at 100%. We traced it from interface counters to a host, a socket and finally a backup job with no rate limit, then shaped it with tc and fixed the schedule.
Laptops and servers are where most attacks land. A practical, layered endpoint security programme for a small company: inventory, patching, hardening baselines, EDR, disk encryption, application control and what to do when a device is lost.
Celery workers stopped picking up thumbnail jobs one by one, with no errors in the logs. A Popen.wait() on a chatty child filled the 64 KiB pipe buffer and both processes waited on each other. We fixed the deadlock, added timeouts, and learned why timeout alone is not enough with shell=True.
We swapped Docker for rootless Podman on a small RHEL host and the stack failed twice: rootlessport refused port 80, then PostgreSQL could not write its bind-mounted data directory. Both come from running without root: unprivileged ports, user namespaces and SELinux labels.
A ledger worker went down with No space left on device while du could account for only 31 GB of a 98 GB volume. A log file deleted by logrotate was still open in the JVM and kept growing. We freed it without a restart and fixed the rotation.
After a Tuesday dnf update, image uploads on our Rocky Linux PHP servers returned 500s. A repository update moved ImageMagick to a new library version, and our hand-built imagick extension still linked to the old one. How we found it in dnf history, rolled back, pinned versions and made updates boring again.
An engineer with nine keys in ssh-agent could not log in to a new server: the connection closed with Too many authentication failures before the right key was tried. How OpenSSH offers keys, why MaxAuthTries cuts it off, and a per-host config that fixes it and is safer.
Many runbooks still say to run shred on sensitive files before deleting them. On SSDs, copy-on-write filesystems and cloud volumes, overwriting in place often leaves the old data elsewhere. Why, and the methods that work: encryption, crypto-erase and device sanitize commands.
Uploads and deploys failed with No space left on device while df showed 30% free. The filesystem had run out of inodes: a session directory held 9.6 million tiny files. How we found it with df -i, cleaned it without locking the box, and stopped it coming back.
A kernel CVE lands and someone asks whether we are affected. Using CVE-2025-21756 in vsock as the worked example, this is our playbook: find which hosts are exposed, mitigate where the module is unused, livepatch where possible, and schedule reboots without drama.
Why does deleting a file in a later Dockerfile step not shrink the image? Why is the first write to a large file inside a container slow? A hands-on explanation of image layers and overlayfs, built with mount commands you can run yourself, and what it means for images and databases.
Our nightly report export ran perfectly when we typed it, and failed every night as a systemd service with node: command not found. Services get no login shell, no .bashrc and a working directory of /. Here is how we made the unit self-contained.
A five-minute cron export started taking six minutes as data grew. Two runs then wrote the same CSV at once, and a partner rejected a file with mangled rows. We serialised the job with flock, bounded it with timeout, and published the file with an atomic rename.
Requests on a reporting server took ten seconds every morning while CPU sat mostly idle. iowait at 60% pointed at storage; iostat and pidstat pointed at logrotate compressing a 40 GB log on the same volume as the database. How we found it and moved the work off the hot path.
During an incident you rarely need a dashboard; you need answers from a log file in under a minute. The handful of grep, awk, sort, uniq, cut and jq patterns we use most, each run against the same sample nginx and JSON logs, with the output and the mistakes to avoid.
A worker image built fine but every container exited at once with "no such file or directory" for a file that clearly existed. A Windows checkout had turned the shebang into /bin/bash with a carriage return. We fixed it with .gitattributes and a CI check.
Our deploy script printed a pg_dump authentication error, then happily ran migrations and switched traffic to a half-migrated release. Bash does not stop on errors by default, and a pipe hid the failure even where we checked. Here is the strict-mode setup and its traps.
Every engineer who downloaded our internal deploy helper got a "damaged" dialog. The app was fine; a CI step modified the bundle after signing and Gatekeeper checked the quarantined copy. We now sign, notarize and staple in the right order.
A queue worker with Restart=always was found dead in the morning with result start-limit-hit. systemd had stopped restarting it after five crashes in ten seconds while Redis was briefly down. How to read the failure, tune StartLimit and RestartSec, and stop flapping services from staying dead.
Our thumbnail service hit Errno 24 at about 1,000 concurrent connections even though ulimit -n showed 65535. systemd services never read limits.conf. We set LimitNOFILE in a drop-in and fixed a client leak that was eating descriptors.
Analysts had an explicit rwx ACL on a shared reports directory and still got Permission denied. A routine chmod 750 had quietly lowered the ACL mask, and files from the ETL job were born with a read-only mask. getfacl showed it all in one word: effective.
Our single-host production server stopped accepting writes at 03:12 with no space left on device. Docker had quietly filled the disk with a 31 GB container log, old images and build cache. Here is how we found each piece and stopped it from coming back.
After a routine deploy our Node.js API failed to start with EADDRINUSE on port 3000. Everyone blamed TIME_WAIT. The real cause was the old node process surviving the restart, because systemd only killed the npm wrapper above it. How we proved it with ss and fixed the unit.