FastAPI QueuePool limit reached: leaked SQLAlchemy sessions in dependencies

Our invoicing API, a FastAPI service in front of PostgreSQL, started paging us two or three times a day. Latency jumped to exactly 30 seconds for a large share of requests, then the requests failed with 500s. After ten to twenty minutes it recovered without anyone touching it. The traceback was the same every time:

sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30.00 (Background on this error at: https://sqlalche.me/e/20/3o7r)

Traffic during the incidents was ordinary, around 40 requests per second per instance. The database was healthy: CPU under 20%, no slow queries, no lock waits.

Reproducing it

We could not reproduce it with a quick local load test at first, because a short test ends before the problem builds up. A longer run did it. With four Uvicorn workers locally and a steady 30 requests per second for ten minutes against a mix of endpoints, the error appeared after about seven minutes.

What we ruled out

• Pool too small for load. 15 connections per worker is plenty for 40 requests per second when queries take 5 ms. Each connection should be busy well under 10% of the time.
• Slow queries holding connections. pg_stat_statements showed nothing slower than 50 ms at p99.
• Network or database restarts. No disconnect errors, and pool_pre_ping was already on.

The first real clue came from PostgreSQL itself during an incident:

SELECT state, left(query, 60) AS last_query, count(*)
FROM pg_stat_activity
WHERE datname = 'invoicing'
GROUP BY 1, 2
ORDER BY 3 DESC;

        state        |              last_query                     | count
---------------------+---------------------------------------------+-------
 idle in transaction | SELECT users.id, users.email, users.org_id  |    52
 idle in transaction | UPDATE invoices SET pdf_status=%(pdf_sta... |     6
 idle                | SELECT 1                                    |     2
 active              | SELECT state, left(query, 60) AS last_qu... |     1

Fifty-eight connections sat "idle in transaction": a transaction was open, the client was doing nothing with it, and it was not returned to the pool. Almost all had last run the user lookup from our authentication dependency.

Root cause

We added SQLAlchemy pool event listeners that record a stack trace on every checkout, then dumped the ones still checked out after 60 seconds. Two call sites appeared.

The auth dependency opened its own session and never closed it. It had been copied from an authentication example early in the project:

def get_current_user(token: str = Depends(oauth2_scheme)) -> User:
    db = SessionLocal()
    payload = decode_token(token)
    user = db.scalar(select(User).where(User.id == payload["sub"]))
    if user is None:
        raise HTTPException(status_code=401)
    return user

The query autobegins a transaction and checks out a connection. Nothing calls db.close(). When the function returns, the session is unreachable, but a SQLAlchemy Session and its SessionTransaction reference each other, so CPython's reference counting cannot free them. Only the cyclic garbage collector can, and the oldest generation is collected rarely. Until then the connection stays checked out.

A background task reused the request's session after it was closed. The PDF endpoint passed the request-scoped db to BackgroundTasks. Since FastAPI 0.106.0, the exit code of a yield dependency runs before background tasks. So the session was already closed when the task ran. A closed Session is still usable: the task's query checked out a fresh connection, and nobody ever closed it again.

Both paths leaked one connection per call until a full GC pass returned them, with a warning in the logs about the garbage collector cleaning up a connection that was never checked in. That is why the service "healed" itself.

What is happening under the hood

SQLAlchemy's QueuePool keeps up to pool_size connections (default 5), can open max_overflow extra ones (default 10), and makes a caller wait up to pool_timeout seconds (default 30) when all are checked out. The TimeoutError means 15 connections were checked out for 30 seconds straight.

A Session checks out a connection lazily on its first query, starts a transaction (autobegin), and keeps that connection until commit(), rollback() or close(). Returning from a function does not release it. The connection is returned only when someone ends the transaction or the session is garbage collected.

FastAPI dependencies with yield exist for exactly this: code before yield runs before the endpoint, code after yield runs when the request is done. The dependency system also caches a dependency within one request, so get_current_user and the endpoint can share one session if both depend on get_db.

Environment

Minimal reproduction

A self-contained app that leaks the same way. Run it against any PostgreSQL:

# app.py
import gc
from fastapi import BackgroundTasks, Depends, FastAPI
from sqlalchemy import create_engine, text
from sqlalchemy.orm import Session, sessionmaker

engine = create_engine(
    "postgresql+psycopg://app:app@localhost:5432/demo",
    pool_size=2, max_overflow=0, pool_timeout=3,
)
SessionLocal = sessionmaker(engine)
app = FastAPI()


def get_db():
    db = SessionLocal()
    try:
        yield db
    finally:
        db.close()


def leaky_user():
    db = SessionLocal()  # never closed
    return db.execute(text("SELECT 1")).scalar_one()


def touch_later(db: Session):
    db.execute(text("SELECT 1"))  # session was closed; this checks out a new connection


@app.get("/leak")
def leak(user=Depends(leaky_user)):
    return {"user": user, "pool": engine.pool.status()}


@app.get("/leak-bg")
def leak_bg(tasks: BackgroundTasks, db: Session = Depends(get_db)):
    tasks.add_task(touch_later, db)
    return {"ok": True}


@app.get("/gc")
def collect():
    gc.collect()
    return {"pool": engine.pool.status()}

uvicorn app:app --port 8000 &
for i in 1 2 3; do curl -s localhost:8000/leak; echo; done

{"user":1,"pool":"Pool size: 2  Connections in pool: 0 Current Overflow: 0 Current Checked out connections: 1"}
{"user":1,"pool":"Pool size: 2  Connections in pool: 0 Current Overflow: 0 Current Checked out connections: 2"}
Internal Server Error

The third request waits three seconds and fails with the same QueuePool limit error. Calling /gc returns the connections. /leak-bg exhausts the pool the same way.

Who is affected

Any FastAPI or Starlette app that creates SessionLocal() outside a yield dependency or a with block, passes request-scoped sessions to background tasks, or creates sessions in helpers, middleware or WebSocket handlers without closing them. Small pools and sync handlers make it appear sooner; large pools only delay it.