Skip to content

feat(argo-cd): set a pod-level security context by default - #4162

Draft
DrFaust92 wants to merge 3 commits into
argoproj:mainfrom
DrFaust92:harden/argocd-pod-security-context
Draft

DrFaust92 wants to merge 3 commits into
argoproj:mainfrom
DrFaust92:harden/argocd-pod-security-context

Conversation

@DrFaust92

@DrFaust92 DrFaust92 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Every Argo CD component already ships a hardened containerSecurityContext — runAsNonRoot, readOnlyRootFilesystem, allowPrivilegeEscalation: false, all capabilities dropped and a RuntimeDefault seccomp profile — but global.securityContext and redisSecretInit.securityContext were both empty, so the pod templates themselves carried no security context at all.

This sets a RuntimeDefault seccomp profile at pod level for both.

What it renders

Seven pod templates gain a pod-level context, and nothing else changes. argocd-redis is untouched because it already ships its own.

+      securityContext:
+        seccompProfile:
+          type: RuntimeDefault

Why runAsNonRoot is no longer here

It was in the first version of this PR. @tico24 pointed out that a pod-level runAsNonRoot reaches containers the chart does not ship: the initContainers example in values.yaml and the alpine init container on upstream's custom tooling page both run as root and set no container security context, so the kubelet would refuse to start a repo-server that worked before the upgrade. That is a breaking change, not a default to slip into a minor.

He was right, and my earlier reasoning here was incomplete — I had only checked the containers the chart itself ships, all of which do set runAsNonRoot.

Measured what dropping it costs, with trivy on the rendered output:

check before with seccomp only
KSV-0118 (default security context) 7 0
KSV-0012 (runs as root) 0 0

So nothing. KSV-0012 stays at zero because every shipped container already sets runAsNonRoot itself. runAsNonRoot will follow as its own pull request with a major bump and an Upgrading note, per versioning.

Notes

  • Default runAsUser is deliberately left unset so each image's own UID still applies. Anyone setting runAsUser: 0 through global.securityContext overrides the whole block as before.
  • Chart.yaml is untouched: the version and artifacthub.io/changes are left to the post-merge publish workflow from feat(github): release chart versions from change fragments #4176, and the entry lives in charts/argo-cd/changes/. Thanks for wiring that up — it also removes the version collision that had me rebasing this three times.
  • Verified with the pinned helm-docs v1.9.1 from scripts/helm-docs.sh that the README needs no regeneration, and checked the change fragment against every rule in scripts/lint-changelog.sh by hand, since mapfile is unavailable on macOS bash.
  • The python pin for chart-test is coming as a separate PR as requested.

Checklist:

  • I have added a change fragment with the right bump according to versioning
  • I have updated the documentation according to documentation
  • The change fragment describes all the changes in this pull request according to changelog.
  • Any new values are backwards compatible and/or have sensible default.
  • I have signed off all my commits as required by DCO.
  • I have created a separate pull request for each chart according to pull requests
  • My build is green (troubleshooting builds) — chart-test fails on the Ubuntu 26.04 / python 3.9 issue, unrelated to this change; separate PR to follow.

@DrFaust92

Copy link
Copy Markdown
Contributor Author

Both failures here are the ECR-public registry, not this change — could someone re-run when it settles?

I checked that it isn't mine before writing this off, since redis is the pod that failed:

  • The redis templates never reference global.securityContext — redis uses its own redis.securityContext, which this PR doesn't touch. Rendered with the exact failing values file (ci/core-values.yaml), the redis pod context is unchanged at runAsNonRoot: true / runAsUser: 999 / seccompProfile: RuntimeDefault.
  • No container in that render runs as uid 0, so nothing conflicts with the new pod-level runAsNonRoot assertion.
  • The install log shows no pod-security rejection at all — no CreateContainerConfigError, no "container has runAsNonRoot and image will run as root". The only other warnings are FailedMount cache-sync timeouts, also infra.

Nothing to change on my side as far as I can tell.

@DrFaust92
DrFaust92 force-pushed the harden/argocd-pod-security-context branch from 512ef39 to 4d82ddf Compare October 24, 2026 20:25
@DrFaust92

Copy link
Copy Markdown
Contributor Author

Rebased onto current main — it had gone conflicting. main released 10.11.1 (Bump argo-cd to v3.5.6), which collided on both the version field and the single artifacthub.io/changes entry. Resolved to 10.12.0 (minor, since this changes a default) with this change as the sole changelog entry.

The one red check, linter-artifacthub, is unrelated to this PR — the job container can't be pulled:

docker pull ecr-public.aws.com/artifacthub/ah:v1.14.0@sha256:3773afcd...
Error response from daemon: toomanyrequests: Data limit exceeded

It retries three times and gives up. It's currently red on 4 of the 8 open PRs in this repo, so it looks like the ECR-public pull limit rather than anything in these changes. chart-test passes (12m48s).

Re-verified the behaviour on the rebased tree rather than assuming it survived main's four commits. The full helm template diff against upstream/main is exactly the version label plus 7 additions, and nothing else:

+      securityContext:
+        runAsNonRoot: true
+        seccompProfile:
+          type: RuntimeDefault

Seven, not eight, because argocd-redis already ships its own pod-level context (runAsNonRoot, runAsUser, seccompProfile) and is untouched.

Worth stating explicitly for review, since a pod-level runAsNonRoot is the kind of default that can break an override: every container in all seven affected workloads already declares runAsNonRoot: true at the container level, including both copyutil init containers. So the pod-level setting can't reject a pod that previously started — it only closes the gap where a container added via extraContainers or a sidecar injector would have inherited no pod-level restriction. The one container without a container-level setting is redis, and that pod already had its own pod context.

Default runAsUser is deliberately left unset so the images' own UIDs still apply, and anyone setting runAsUser: 0 via global.securityContext overrides the whole block as before.

@tico24 tico24 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you put the PR template's checklist back in the description and tick what holds? (checklist) The Notes section still says 10.11.0 too.

Comment thread charts/argo-cd/values.yaml Outdated
securityContext: {}
# @default -- See [values.yaml]
securityContext:
runAsNonRoot: true

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This stops any pod with a root init container from starting. The chart's own initContainers example for the controller and server uses alpine:3, which has no USER, and upstream's custom tooling page puts the same kind of alpine init container on the repo-server. Neither sets a container securityContext, so with runAsNonRoot: true on the pod the kubelet refuses to start them, and a repo-server that ran before won't after the upgrade.

Every container the chart ships already sets runAsNonRoot itself, so at pod level it only reaches containers people add. Could the pod level keep just seccompProfile? I ran trivy with only that and KSV-0118 still goes from 7 to 0. If you'd rather keep runAsNonRoot, it's a breaking change, so it needs a major and a note under Changelog in README.md.gotmpl, like 10.0.0 has for global.networkPolicy.create (versioning).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @tico24, Ill split it for now and push the breaking change separately.

re: trivy, maybe worth to add it as a CI check?

@DrFaust92
DrFaust92 force-pushed the harden/argocd-pod-security-context branch from 4d82ddf to 724b20e Compare October 27, 2026 02:58
@DrFaust92

Copy link
Copy Markdown
Contributor Author

Rebased again — main had moved 8 commits and released 10.11.6 (Bump dex to v2.46.1), colliding on version and the single artifacthub.io/changes entry as before. Resolved to 10.12.0 with this change as the sole entry.

Re-verified rather than assuming it survived: the full helm template diff against upstream/main is still exactly the chart label plus these 7 additions, and nothing else.

7 >       securityContext:
7 >         runAsNonRoot: true
7 >         seccompProfile:
7 >           type: RuntimeDefault

Seven and not eight because argocd-redis already ships its own pod-level context and is untouched. As before, every container in the seven affected workloads already declares runAsNonRoot: true itself, including both copyutil init containers, so this cannot reject a pod that previously started.

This is the third rebase on version collisions alone. Is there any appetite for a review pass while it is green? Happy to keep rebasing, but the chart releases roughly weekly and the version field conflicts every time.

@DrFaust92

Copy link
Copy Markdown
Contributor Author

linter-artifacthub is green now, but chart-test has started failing for a reason unrelated to this PR — it dies in 21s during Set up python, before any chart is touched:

Version 3.9 was not found in the local cache
##[error]The version '3.9' with architecture 'x64' was not found for Ubuntu 26.04.

.github/workflows/lint-and-test.yml runs on ubuntu-latest and pins python-version: 3.9:

    runs-on: ubuntu-latest
...
        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
        with:
          python-version: 3.9

ubuntu-latest has rolled to Ubuntu 26.04, whose tool cache no longer carries 3.9, so this should fail on every PR from now on, not just this one. I can't demonstrate that across PRs yet because mine is currently the only Linting and Testing run since the image changed — the other open PR is sitting in action_required — so treat the repo-wide part as inference from the config rather than something I measured. What is certain is that it is not this change: the diff is three files under charts/argo-cd/ and touches no workflow or Python.

Happy to open a separate one-liner pinning it to something available (3.13, or 3.x if you'd rather not pin) if that's useful — just say the word and I'll keep it out of this PR.

@tico24 tico24 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My review from Monday is still open: the inline on global.securityContext and the two asks in its body.

Yes please to the python pin as a separate PR.

DrFaust92 and others added 3 commits October 29, 2026 07:46
Every component already ships a hardened containerSecurityContext -- runAsNonRoot,
readOnlyRootFilesystem, allowPrivilegeEscalation: false, all capabilities dropped
and a RuntimeDefault seccomp profile -- but global.securityContext and
redisSecretInit.securityContext were both empty, so the pod templates themselves
carried no security context.

Default both to runAsNonRoot: true and a RuntimeDefault seccomp profile. These are
the same values upstream argo-cd sets on its own notifications-controller pod in
manifests/install.yaml, so this brings the chart to parity rather than inventing a
policy. runAsUser is deliberately not set: upstream only pins it for redis, which
redis.securityContext in this chart already matches.

This covers the applicationset, dex, notifications, repo-server, server and
commit-server Deployments, the application-controller StatefulSet and the
redis-secret-init Job.

Signed-off-by: Ilia Lazebnik <ilia.lazebnik@gmail.com>
argo-helm now bumps chart versions after merge (argoproj#4176). This puts the version and artifacthub.io/changes in Chart.yaml back to main's, and moves this PR's changelog entry into charts/argo-cd/changes/set-a-pod-level-security-context-by-default.yaml with the same bump level.

Signed-off-by: Tim Collins <tim@thecollins.team>
Keep only the RuntimeDefault seccomp profile at pod level. A pod-level
runAsNonRoot reaches containers the chart does not ship, including the
initContainers example in values.yaml and the alpine init container on the
upstream custom tooling page, neither of which sets a container security
context. Those pods would stop starting after an upgrade, which makes it a
breaking change rather than a default worth shipping in a minor.

The seccomp profile alone keeps the benefit: trivy's KSV-0118 goes from 7
findings to 0 either way, and KSV-0012 stays at 0 because every container the
chart ships already sets runAsNonRoot itself.

runAsNonRoot will follow in its own pull request with a major bump and an
Upgrading note.

Signed-off-by: Ilia Lazebnik <ilia.lazebnik@gmail.com>
@DrFaust92
DrFaust92 force-pushed the harden/argocd-pod-security-context branch from bfdf78b to 01332e9 Compare October 29, 2026 11:50
@DrFaust92

Copy link
Copy Markdown
Contributor Author

All three addressed, and thanks for the push to change fragments — that removes the version collision I'd been rebasing on.

1. runAsNonRoot is out. You're right, and my reasoning in this PR was incomplete: I had only checked the containers the chart ships, all of which set runAsNonRoot themselves, and missed that a pod-level setting reaches containers users add — including the initContainers example in values.yaml and the alpine one on the custom tooling page. A repo-server that started before wouldn't after. That's breaking, so it doesn't belong in a minor.

I reproduced your trivy measurement before dropping it, on the rendered output:

check base seccomp only
KSV-0118 7 0
KSV-0012 0 0

So seccomp alone keeps the whole benefit here, exactly as you said. KSV-0012 stays at zero precisely because the shipped containers set runAsNonRoot themselves. runAsNonRoot will come back as its own PR with a major bump and an ### NEXT Upgrading section.

2. Checklist and version reference. Both fixed — the template checklist is back in the description and ticked, and the stale 10.11.0 line is gone. Chart.yaml is now untouched, with the entry in charts/argo-cd/changes/ per #4176.

3. The python pin — no PR needed. #4175 already did it: main now has python-version: '3.13', so my earlier diagnosis was right about the cause but out of date by the time I wrote it. My branch picks the fix up through the rebase, so chart-test should go green on its own here.

On adding trivy as a CI check — I think that's worth doing and I'm happy to take it, but as its own PR against the workflows rather than bundled here. Worth deciding the shape first: the obvious version is trivy config over helm template output for the changed chart, failing on HIGH and above. The catch is that the current rendered output is not clean across the board, so a blocking check would need either a baseline/ignore file or a non-blocking warn-only start. Which would you prefer?

Verification notes for this push: README needs no regeneration — confirmed with the pinned helm-docs v1.9.1 from scripts/helm-docs.sh, whose output differs from a newer local helm-docs, so I used the pinned image. I also checked the change fragment against every rule in scripts/lint-changelog.sh by hand, since mapfile isn't available on macOS bash.

@tico24 tico24 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The red chart-test is the kind preload failure that #4178 fixes, so this needs a rebase once that's in.

While you're there, could you fix two things in the description? runAsUser: 0 through global.securityContext doesn't override the whole block, because helm merges maps: with --set global.securityContext.runAsUser=0 the pod gets runAsUser: 0 plus seccompProfile, and global.securityContext: {} leaves seccompProfile in place. Only null clears it. The python lines are out of date too, since #4175 did the pin.

@tico24
tico24 enabled auto-merge (squash) October 30, 2026 08:50
@jmeridth
jmeridth marked this pull request as draft November 1, 2026 01:58
auto-merge was automatically disabled November 1, 2026 01:58

Pull request was converted to draft

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants