OpenShift SCC Compatibility: Common Issues and Solutions

Supporting your customers
1

Introduction

OpenShift enforces stricter security policies than standard Kubernetes distributions through Security Context Constraints (SCCs). While your workload may run seamlessly in vanilla Kubernetes, it might encounter issues in OpenShift due to these enhanced security measures.

Understanding Security Context Constraints

Default SCC: restricted-v2

When not explicitly specified, most workloads in OpenShift clusters will have the default restricted-v2 SCC applied. This is the most restrictive SCC and serves as OpenShift’s security baseline.

Key restrictions of restricted-v2:

  • Runs containers with arbitrarily assigned user IDs
  • Prevents running as root or specific user IDs
  • Restricts access to host resources
  • Limits filesystem permissions

More info from OpenShift documentation

Testing with Replicated Compatibility Matrix

Use Replicated Compatibility Matrix to quickly verify your application’s compatibility with OpenShift:

# create an OpenShift cluster
replicated cluster create --distribution openshift --instance-type r1.large --disk 50 --version 4.19.0-okd

# access the cluster
replicated cluster shell <cluster-id>

# deploy your application
helm install OR kubectl kots install <your-app>

Common Issues and Solutions

Issue 1: Workload Failed to Schedule

Symptom: You see errors like:

Warning  FailedCreate      2m37s (x17 over 5m21s)  statefulset-controller  create Pod <release>-rabbitmq-0 in StatefulSet <release>-rabbitmq failed error: pods "<release>-rabbi
tmq-0" is forbidden: unable to validate against any security context constraint: [provider "anyuid": Forbidden: not usable by user or serviceaccount, provider restricted-v2: .
pec.securityContext.fsGroup: Invalid value: [[]int64{1001}: 1001 is not an allowed group, provider restricted-v2: .containers[0].runAsUser: Invalid value: 1001: must be in the
 ranges: [1000850000, 1000859999], provider "restricted": Forbidden: not usable by user or serviceaccount, provider "nonroot-v2": Forbidden: not usable by user or serviceaccou
nt, provider "nonroot": Forbidden: not usable by user or serviceaccount, provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount, provider "hostmount-anyu
id-v2": Forbidden: not usable by user or serviceaccount, provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount, provider "hostnetwork-v2"
: Forbidden: not usable by user or serviceaccount, provider "hostnetwork": Forbidden: not usable by user or serviceaccount, provider "hostaccess": Forbidden: not usable by use
r or serviceaccount, provider "insights-runtime-extractor-scc": Forbidden: not usable by user or serviceaccount, provider "hostpath-provisioner": Forbidden: not usable by user
 or serviceaccount, provider "node-exporter": Forbidden: not usable by user or serviceaccount, provider "privileged": Forbidden: not usable by user or serviceaccount]

Root Cause: Your securityContext contains values that violate the restricted-v2 SCC, specifically:

  • runAsUser is outside the allowed UID range
  • fsGroup is not in the permitted group range

Solution:

  • Always include securityContext values in your chart templates, remove hardcoded user/group IDs when OpenShift is detected.
  • Let OpenShift assign UIDs from its allowed range automatically
  • Use conditional logic in Helm chart template to detect OpenShift and provide sane defaults that align with restricted-v2. E.g. remove runAsUser, runAsGroup and fsGroup when OpenShift is detected

We can use this template to detect OpenShift

{{- if .Capabilities.APIVersions.Has "security.openshift.io/v1" }}

If KOTS template is used, we can also use

repl{{ eq Distribution "openshift" }}

Issue 2: Permission Denied Errors

Symptom: Pods crash with permission denied errors in logs

E.g.

nginx: [emerg] mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)

Common Causes:

  • Container process writes to directories owned by root (e.g., /var/cache)
  • Container process listens on privileged ports (< 1024)
  • Init containers running chown to update volume permissions
  • Overlapping mounts causing permission conflicts (e.g., ConfigMap mounted over PVC path)

Solutions:

  1. Use OpenShift-compatible images:

    • Instead of docker.io/nginx, use nginxinc/nginx-unprivileged
    • Look for images that support arbitrary user IDs

  2. Modify your Dockerfile: Follow OpenShift guidelines - directories and files written by processes must be owned by the root group and be read/writable by that group:

E.g.

RUN chgrp -R 0 /some/directory && \
chmod -R g=u /some/directory

More info on image guidelines in OpenShift.

Troubleshooting Checklist

When encountering issues:

Check which SCC is being applied: oc describe pod <pod-name> | grep scc
Verify user/group IDs are within allowed ranges
Ensure directories have proper group permissions (root group)
Confirm no processes require privileged ports
Review volume mount permissions and ownership
Check for OpenShift-compatible base images
Test with appropriate SCC if restricted-v2 is too restrictive. E.g. oc adm policy add-scc-to-user anyuid -z <service-account> -n <namespace>