Performance & Scaling
Bug Report
Spark daily 1TB batch dies on one task: skewed join keys, AQE limits and salting
Our nightly Spark job over 1TB of events finished 1,999 of 2,000 join tasks in minutes, then one task ran for an hour and the executor was killed for exceeding memory. The cause was null and hot join keys that adaptive skew handling did not split. Here is the diagnosis and the fix.