Linux Permission denied for a group member with an ACL grant: the ACL mask

Our analytics team shares /srv/reports on a reporting server. Access is granted with a POSIX ACL to the analysts group, so the directory can stay owned by the ETL service account. On a Monday, analysts could still read reports but could no longer save anything:

analyst1@reports-01:~$ touch /srv/reports/2025-01/summary.csv
touch: cannot touch '/srv/reports/2025-01/summary.csv': Permission denied

Editing existing files failed the same way. Nothing in the ACL setup had been changed by the team that owns it, and getfacl at first glance still showed the grant:

analyst1@reports-01:~$ getfacl /srv/reports/2025-01
# file: srv/reports/2025-01
# owner: etl
# group: etl
user::rwx
group::r-x
group:analysts:rwx              #effective:r-x
mask::r-x
other::---
default:user::rwx
default:group::r-x
default:group:analysts:rwx
default:mask::rwx
default:other::---

The analysts entry says rwx, but the comment next to it says the effective permission is r-x.

Reproducing it

sudo mkdir -p /tmp/acl-demo && sudo chown root:root /tmp/acl-demo
sudo setfacl -m g:analysts:rwx /tmp/acl-demo
getfacl -c /tmp/acl-demo | grep -E 'analysts|mask'   # rwx, mask rwx
sudo chmod 750 /tmp/acl-demo
getfacl -c /tmp/acl-demo | grep -E 'analysts|mask'   # rwx, but effective r-x

chmod 750 did not touch the analysts entry, yet it took away their write access.

What we ruled out

• Group membership. id analyst1 listed analysts, and the user had logged in again since being added, so the session carried the group.
• Traversal of parent directories. namei -l /srv/reports/2025-01 showed execute permission for the user on every component.
• AppArmor or another LSM. journalctl -k | grep -i denied showed nothing for these paths, and the server runs no SELinux.
• Filesystem mounted without ACL support. ext4 enables ACLs by default, and the entries were clearly being stored and partly honoured (reads worked).

Root cause

Two things lowered the ACL mask, which caps every named user and group entry.

1. A hardening script, run across all servers the previous Friday, executed chmod -R 750 /srv/reports to remove access for "others". On a file or directory with an extended ACL, the group bits of chmod set the mask, not the owning group entry. 750 set the mask to r-x, which capped the analysts entry at r-x.
2. Separately, report files written by the ETL job were created with mode 0640, because the application explicitly asked for 0640. With a default ACL on the directory, a new file's mask is the default mask limited by the group bits of the create mode. So every new report got mask::r--, and analysts could never edit them, even before the chmod.

The first problem broke directories; the second had been silently making new files read-only for weeks, which nobody noticed because analysts mostly read.

What is happening under the hood

A POSIX ACL with any named user or group entries has a mask entry. The kernel's access check for a process that matches a named user, the owning group, or a named group entry uses the entry's permissions ANDed with the mask. The owner (user::) and other:: are not affected by the mask.

To stay compatible with programs that only know about mode bits, the group bits shown by ls -l and set by chmod map to the mask when an extended ACL exists. That is why ls -l shows drwxr-x---+ here: the r-x in the group position is the mask, and the trailing + tells you there is an ACL. Running chmod g-w or chmod 750 is therefore a way of lowering the ceiling for every named entry at once.

For new files, the default ACL of the parent directory is copied to the file, and then the permission bits requested by the creating process are applied (the umask is ignored when a default ACL exists): the requested group bits become the upper bound of the new mask. A default mask of rwx cannot help if the program asks for 0640.

setfacl -m recalculates the mask automatically to cover all entries unless you pass -n. That is why the original setup looked right.

Environment

Minimal reproduction

sudo groupadd -f analysts
sudo mkdir -p /srv/acl-repro
sudo setfacl -m g:analysts:rwx -m d:g:analysts:rwx /srv/acl-repro

# New file created with mode 0640 by a program
sudo install -m 0640 /dev/null /srv/acl-repro/report.csv
getfacl -c /srv/acl-repro/report.csv
# group:analysts:rwx   #effective:r--
# mask::r--

Who is affected

Anyone combining ACLs with tools that think in mode bits: hardening scripts, configuration management that enforces mode: on directories, rsync -p or cp -p copying in files with restrictive modes, and services with conservative umasks. The owner and "others" are unaffected, so the problem only shows up for the people the ACL was meant to help.

The fix

First, restore the mask on the existing tree. Re-applying the named entries makes setfacl recalculate the mask; using X gives execute only to directories (and files that are already executable):

sudo setfacl -R -m g:analysts:rwX /srv/reports
sudo setfacl -R -d -m g:analysts:rwX /srv/reports   # applies to directories only

We could have set the mask directly with setfacl -R -m m::rwX, but re-stating the grant is clearer in a runbook and survives future edits.

Second, change the hardening script so it removes "other" access without touching the group bits:

- chmod -R 750 /srv/reports
+ chmod -R o-rwx /srv/reports

chmod o-rwx only changes the other class, so the mask is left alone.

Third, make the ETL service create group-writable files, so the mask on new files allows the analysts' rw. The application had an explicit 0640 mode for output files, which we changed to 0660 in its configuration. That was the real fix: when the parent directory has a default ACL, the kernel ignores the process umask, so only the mode the program requests limits the new mask. We still set a group-friendly umask in the unit for the few paths without a default ACL, and set the setgid bit on directories so new files and directories inherit the group:

# /etc/systemd/system/etl-reports.service.d/umask.conf
[Service]
UMask=0007

sudo find /srv/reports -type d -exec chmod g+s {} +
sudo systemctl daemon-reload && sudo systemctl restart etl-reports

Alternatives and trade-offs

• Make analysts the owning group and drop the ACL. Simpler when one group needs access; we have two groups with different rights, so we kept ACLs.
• A cron job that re-applies ACLs. Hides the underlying cause and leaves windows where access is broken.
• setfacl -n prevents mask recalculation. Useful only when you deliberately want a lower mask; it is not what you want for grants.

How to test the fix

# As root: inspect a directory and a freshly created file
getfacl -c /srv/reports/2025-01 | grep -E 'analysts|mask'
sudo systemctl start etl-reports-run.service    # produce one new report
getfacl -c "$(ls -t /srv/reports/2025-01/*.csv | head -1)" | grep -E 'analysts|mask'
# group:analysts:rw-  (no effective comment means the mask does not reduce it)

# As an analyst
sudo -u analyst1 touch /srv/reports/2025-01/test-write && echo write-ok
sudo -u analyst1 rm /srv/reports/2025-01/test-write

When the mask allows everything an entry grants, getfacl prints no effective comment at all, which is the quickest visual check.

Confirming in production

After the fix, we searched the whole tree for any entry still reduced by its mask:

sudo getfacl -R -p /srv/reports 2>/dev/null | grep -B8 'effective:' | grep '^# file:'

The output was empty. Analysts confirmed they could save and edit reports, and the next three ETL runs produced files with an rw- effective permission for the group.

Regression guard

• The hardening script was changed fleet-wide to use o-rwx rather than absolute numeric modes on paths that may carry ACLs, and it now skips any path listed in a small exclusions file.
• Our Ansible role for the reports server manages ACLs with the ansible.posix.acl module, so a configuration run puts both entries and default entries back if anything changes them.
• A daily check runs the getfacl search above and alerts if it prints anything.

Security implications

The mask is a useful safety valve: lowering it cuts every named grant at once, which is a fast way to make a share read-only during an incident without editing individual entries. That cuts both ways. Tools that change mode bits can silently weaken or, in the opposite direction, re-widen ACL grants. A chmod 770 on an ACL'd directory raises the mask and can re-enable named entries someone thought were capped. When auditing access on Linux, ls -l is not enough; look for the + and read the ACL. We also kept "other" at no access throughout and resisted the temptation to "fix" the problem with chmod 777, which would have exposed financial reports to every local account, including service users.

Prevention checklist

• Read getfacl output for effective comments before assuming an ACL grant works.
• Never use absolute numeric chmod on ACL'd paths; change only the class you mean.
• Give services that write to shared paths a group-writable file mode, and a matching umask.
• Use setgid directories plus default ACLs for shared trees.
• Manage ACLs with configuration management so drift is corrected and visible.

References

• acl(5) manual page, access check algorithm and mask: https://man7.org/linux/man-pages/man5/acl.5.html
• setfacl(1) manual page: https://man7.org/linux/man-pages/man1/setfacl.1.html
• getfacl(1) manual page: https://man7.org/linux/man-pages/man1/getfacl.1.html
• systemd.exec(5), UMask=: https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html
• Ansible ansible.posix.acl module: https://docs.ansible.com/ansible/latest/collections/ansible/posix/acl_module.html