mirror of
https://github.com/MichaelCade/90DaysOfDevOps.git
synced 2025-01-05 13:07:58 +07:00
711 lines
43 KiB
Markdown
711 lines
43 KiB
Markdown
# Compliance and Vulnerability Scanning provided by Red Hat OpenShift Operators
|
||
|
||
Many components make up the security posture of a well-designed and architected platform. For a Red Hat OpenShift platform, this will include implementing the AAA model we covered on [Day 61](/2023/day61.md), container security, certificates to validate access to the platform and between the workloads that run and communicate on and with the platform, data encryption, vulnerability scanning, compliance auditing and remediation, as just a shortlisted example.
|
||
|
||
In this post, I'm going to focus on just two areas of security inside of Red Hat OpenShift; Compliance and Vulnerability Scanning.
|
||
|
||
# Red Hat OpenShift Compliance Operator
|
||
|
||
## Compliance Overview
|
||
|
||
In the context of Kubernetes, compliance refers to the adherence of OpenShift deployments to various industry standards, regulations, and best practices. Compliance is essential for organizations operating in regulated industries like healthcare, finance, or government sectors, as well as for those who prioritize security and privacy.
|
||
|
||
OpenShift, as a Kubernetes-based container orchestration platform, provides several features and tools to help organizations achieve and maintain compliance. These features include:
|
||
|
||
- Security and access control: OpenShift provides robust security features like Role-Based Access Control (RBAC), network policies, and Security Context Constraints (SCCs) to manage access to cluster resources and ensure the secure deployment and operation of applications.
|
||
|
||
- Auditing and monitoring: OpenShift's built-in auditing and monitoring capabilities make it easy to track user activities, resource usage, and system events. This information is crucial for detecting and responding to security incidents, meeting regulatory requirements, and troubleshooting issues.
|
||
|
||
- Image and container security: OpenShift's integrated container registry, image signing, and image scanning features help ensure the integrity and security of container images. Additionally, OpenShift enforces security best practices through resource constraints and pod security policies.
|
||
|
||
- Encrypted communication: OpenShift supports TLS/SSL for secure communication between cluster components, as well as between the cluster and external clients. This helps protect sensitive data in transit.
|
||
|
||
- Compliance Operator: Red Hat OpenShift provides the Compliance Operator, with an OpenShift-native tool that helps organizations evaluate and enforce compliance policies. The Compliance Operator leverages the OpenSCAP framework and can be configured to meet specific regulatory requirements or security standards, such as NIST, PCI-DSS, HIPAA, or GDPR.
|
||
|
||
Achieving and maintaining compliance in OpenShift involves configuring the platform according to industry standards, regulations, and best practices, continuously monitoring and auditing the environment, and promptly addressing any identified issues. By utilising the capabilities and resources offered by OpenShift, organisations have the opportunity to establish secure and regulation-compliant ecosystems for their applications and workloads, ensuring optimal performance and adherence to industry standards.
|
||
|
||
In this post, we'll be focusing on the Compliance Operator, which empowers the platform administrators by allowing them to define the desired compliance state for their cluster. It offers a comprehensive overview of discrepancies between the current and target states, as well as actionable insights to address these gaps effectively.
|
||
|
||
The Compliance Operator provides the ability to describe the required compliance state of a cluster and report overviews of gaps and ways to remediate them. The Compliance Operator assesses compliance of both the Kubernetes API resources of OpenShift Container Platform, as well as the nodes running the cluster. The Compliance Operator uses OpenSCAP, a NIST-certified tool, to scan and enforce security policies provided by the content. You can view details of the out-of-the-box compliance profiles that are [provided here](https://docs.openshift.com/container-platform/4.12/security/compliance_operator/compliance-operator-supported-profiles.html).
|
||
|
||
### Installing the Compliance Operator
|
||
|
||
1. We need to create a specially managed OpenShift namespace. Save the below content as ```openshift-compliance-namespace.yaml``` and apply using the ```oc apply -f openshift-compliance-namespace.yaml``` command.
|
||
|
||
````yaml
|
||
apiVersion: v1
|
||
kind: Namespace
|
||
metadata:
|
||
labels:
|
||
openshift.io/cluster-monitoring: "true"
|
||
pod-security.kubernetes.io/enforce: privileged
|
||
name: openshift-compliance
|
||
````
|
||
|
||
2. Now we need to define an ```OperatorGroup``` for the Compliance operator. Save the below content as ```openshift-compliance-operator-group.yaml``` and apply using the ```oc apply -f openshift-compliance-operator-group.yaml``` command.
|
||
|
||
````yaml
|
||
apiVersion: operators.coreos.com/v1
|
||
kind: OperatorGroup
|
||
metadata:
|
||
name: compliance-operator
|
||
namespace: openshift-compliance
|
||
spec:
|
||
targetNamespaces:
|
||
- openshift-compliance
|
||
````
|
||
|
||
3. Define and apply a subscription to the Operator. Save the below content as ```openshift-compliance-operator-subscription.yaml``` and apply using the ```oc apply -f openshift-compliance-operator-subscription.yaml``` command.
|
||
|
||
````yaml
|
||
apiVersion: operators.coreos.com/v1alpha1
|
||
kind: Subscription
|
||
metadata:
|
||
name: compliance-operator-sub
|
||
namespace: openshift-compliance
|
||
spec:
|
||
channel: "release-0.1"
|
||
installPlanApproval: Automatic
|
||
name: compliance-operator
|
||
source: redhat-operators
|
||
sourceNamespace: openshift-marketplace
|
||
````
|
||
|
||
4. You can verify the Operator installation with the below command;
|
||
|
||
````sh
|
||
$ oc get csv -n openshift-compliance
|
||
$ oc get deploy -n openshift-compliance
|
||
````
|
||
### Reviewing the profiles and the rules
|
||
|
||
We can see the installed profiles by running the below command, each profile has the product name that it applies to added as a prefix to the profile’s name. ocp4-e8 applies the Essential 8 benchmark to the OpenShift Container Platform product, while rhcos4-e8 applies the Essential 8 benchmark to the Red Hat Enterprise Linux CoreOS (RHCOS) product.
|
||
|
||
````sh
|
||
$ oc get -n openshift-compliance profiles.compliance
|
||
NAME AGE
|
||
ocp4-cis 107m
|
||
ocp4-cis-node 107m
|
||
ocp4-e8 107m
|
||
ocp4-high 107m
|
||
ocp4-high-node 107m
|
||
ocp4-moderate 107m
|
||
ocp4-moderate-node 107m
|
||
ocp4-nerc-cip 107m
|
||
ocp4-nerc-cip-node 107m
|
||
ocp4-pci-dss 107m
|
||
ocp4-pci-dss-node 107m
|
||
rhcos4-e8 107m
|
||
rhcos4-high 107m
|
||
rhcos4-moderate 107m
|
||
rhcos4-nerc-cip 107m
|
||
````
|
||
|
||
We can view the details of a profile by running:
|
||
|
||
````sh
|
||
$ oc get -n openshift-compliance -oyaml profiles.compliance ocp4-cis
|
||
````
|
||
|
||
````yaml
|
||
apiVersion: compliance.openshift.io/v1alpha1
|
||
description: This profile defines a baseline that aligns to the Center for Internet
|
||
Security® Red Hat OpenShift Container Platform 4 Benchmark™, V1.1. This profile
|
||
includes Center for Internet Security® Red Hat OpenShift Container Platform 4 CIS
|
||
Benchmarks™ content. Note that this part of the profile is meant to run on the Platform
|
||
that Red Hat OpenShift Container Platform 4 runs on top of. This profile is applicable
|
||
to OpenShift versions 4.6 and greater.
|
||
id: xccdf_org.ssgproject.content_profile_cis
|
||
kind: Profile
|
||
metadata:
|
||
annotations:
|
||
compliance.openshift.io/image-digest: pb-ocp477wpm
|
||
compliance.openshift.io/product: redhat_openshift_container_platform_4.1
|
||
compliance.openshift.io/product-type: Platform
|
||
creationTimestamp: "2023-03-31T09:09:52Z"
|
||
generation: 1
|
||
labels:
|
||
compliance.openshift.io/profile-bundle: ocp4
|
||
name: ocp4-cis
|
||
namespace: openshift-compliance
|
||
ownerReferences:
|
||
- apiVersion: compliance.openshift.io/v1alpha1
|
||
blockOwnerDeletion: true
|
||
controller: true
|
||
kind: ProfileBundle
|
||
name: ocp4
|
||
uid: 19c2e4a5-094f-416a-a06b-eb0598e39618
|
||
resourceVersion: "12971302"
|
||
uid: 6dc3cca4-5649-43ae-8c46-614f82fd6744
|
||
rules:
|
||
- ocp4-accounts-restrict-service-account-tokens
|
||
- ocp4-accounts-unique-service-account
|
||
- ocp4-api-server-admission-control-plugin-alwaysadmit
|
||
- ocp4-api-server-admission-control-plugin-alwayspullimages
|
||
- ocp4-api-server-admission-control-plugin-namespacelifecycle
|
||
- ocp4-api-server-admission-control-plugin-noderestriction
|
||
- ocp4-api-server-admission-control-plugin-scc
|
||
- ocp4-api-server-admission-control-plugin-securitycontextdeny
|
||
- ocp4-api-server-admission-control-plugin-service-account
|
||
- ocp4-api-server-anonymous-auth
|
||
- ocp4-api-server-api-priority-flowschema-catch-all
|
||
- ocp4-api-server-api-priority-gate-enabled
|
||
- ocp4-api-server-audit-log-maxbackup
|
||
- ocp4-api-server-audit-log-maxsize
|
||
- ocp4-api-server-audit-log-path
|
||
- ocp4-api-server-auth-mode-no-aa
|
||
- ocp4-api-server-auth-mode-node
|
||
- ocp4-api-server-auth-mode-rbac
|
||
- ocp4-api-server-basic-auth
|
||
- ocp4-api-server-bind-address
|
||
...
|
||
- ocp4-scc-limit-privileged-containers
|
||
- ocp4-scc-limit-process-id-namespace
|
||
- ocp4-scc-limit-root-containers
|
||
- ocp4-scheduler-no-bind-address
|
||
- ocp4-secrets-consider-external-storage
|
||
- ocp4-secrets-no-environment-variables
|
||
- ocp4-version-detect-in-hypershift
|
||
- ocp4-version-detect-in-ocp
|
||
title: CIS Red Hat OpenShift Container Platform 4 Benchmark
|
||
````
|
||
|
||
To read the descriptions of one of the rules that form the policy:
|
||
|
||
````sh
|
||
$ oc get -n openshift-compliance rules ocp4-api-server-anonymous-auth -oyaml
|
||
````
|
||
|
||
````yaml
|
||
apiVersion: compliance.openshift.io/v1alpha1
|
||
checkType: Platform
|
||
description: |-
|
||
By default, anonymous access to the OpenShift API is enabled, but at the same time, all requests must be authorized. If no authentication mechanism is used, the request is assigned the system:anonymous virtual user and the system:unauthenticated virtual group. This allows the authorization layer to determin which requests, if any, is an anonymous user authorized to make. To verify the authorization rules for anonymous requests run the following:
|
||
|
||
$ oc describe clusterrolebindings
|
||
|
||
and inspect the bidnings of the system:anonymous virtual user and the system:unauthenticated virtual group. To test that an anonymous request is authorized to access the readyz endpoint, run:
|
||
|
||
$ oc get --as="system:anonymous" --raw='/readyz?verbose'
|
||
|
||
In contrast, a request to list all projects should not be authorized:
|
||
|
||
$ oc get --as="system:anonymous" projects
|
||
id: xccdf_org.ssgproject.content_rule_api_server_anonymous_auth
|
||
instructions: |-
|
||
Run the following command to view the authorization rules for anonymous requests:
|
||
$ oc describe clusterrolebindings
|
||
Make sure that there exists at least one clusterrolebinding that binds
|
||
either the system:unauthenticated group or the system:anonymous
|
||
user.
|
||
To test that an anonymous request is authorized to access the readyz
|
||
endpoint, run:
|
||
$ oc get --as="system:anonymous" --raw='/readyz?verbose'
|
||
In contrast, a request to list all projects should not be authorized:
|
||
$ oc get --as="system:anonymous" projects
|
||
kind: Rule
|
||
metadata:
|
||
annotations:
|
||
compliance.openshift.io/image-digest: pb-ocp477wpm
|
||
compliance.openshift.io/rule: api-server-anonymous-auth
|
||
control.compliance.openshift.io/CIS-OCP: 1.2.1
|
||
control.compliance.openshift.io/NERC-CIP: CIP-003-8 R6;CIP-004-6 R3;CIP-007-3
|
||
R6.1
|
||
control.compliance.openshift.io/NIST-800-53: CM-6;CM-6(1)
|
||
control.compliance.openshift.io/PCI-DSS: Req-2.2
|
||
policies.open-cluster-management.io/controls: 1.2.1,CIP-003-8 R6,CIP-004-6 R3,CIP-007-3
|
||
R6.1,CM-6,CM-6(1),Req-2.2
|
||
policies.open-cluster-management.io/standards: CIS-OCP,NERC-CIP,NIST-800-53,PCI-DSS
|
||
creationTimestamp: "2023-03-31T09:09:53Z"
|
||
generation: 1
|
||
labels:
|
||
compliance.openshift.io/profile-bundle: ocp4
|
||
name: ocp4-api-server-anonymous-auth
|
||
namespace: openshift-compliance
|
||
ownerReferences:
|
||
- apiVersion: compliance.openshift.io/v1alpha1
|
||
blockOwnerDeletion: true
|
||
controller: true
|
||
kind: ProfileBundle
|
||
name: ocp4
|
||
uid: 19c2e4a5-094f-416a-a06b-eb0598e39618
|
||
resourceVersion: "12971338"
|
||
uid: 12db5786-4ff6-4e80-90e0-2b370541f6e1
|
||
rationale: When enabled, requests that are not rejected by other configured authentication
|
||
methods are treated as anonymous requests. These requests are then served by the
|
||
API server. If you are using RBAC authorization, it is generally considered reasonable
|
||
to allow anonymous access to the API Server for health checks and discovery purposes,
|
||
and hence this recommendation is not scored. However, you should consider whether
|
||
anonymous discovery is an acceptable risk for your purposes.
|
||
severity: medium
|
||
title: Ensure that anonymous requests to the API Server are authorized
|
||
````
|
||
|
||
### Running a Scan
|
||
|
||
Now that we have the operator installed, we have two key configurations:
|
||
|
||
- ScanSettings - This is the Schema for the scansettings API, and therefore you will provide configurations for running the scan, such as where it stores data, configurations for the container that is run on the platform to perform the scan, which components are subject to be scanned, and schedules to run the scan.
|
||
- ScanSettingsBinding - This is the Schema for the scansettingbindings API. This is used to bind the ScanSettings configuration to the compliance profiles you want to run against your chosen components.
|
||
|
||
There is a default ScanSettings called ```default``` supplied when you install the Compliance Operator, which you can run against your system.
|
||
|
||
````sh
|
||
$ oc describe scansettings default -n openshift-compliance
|
||
|
||
Name: default
|
||
Namespace: openshift-compliance
|
||
Labels: <none>
|
||
Annotations: <none>
|
||
API Version: compliance.openshift.io/v1alpha1
|
||
Kind: ScanSetting
|
||
Max Retry On Timeout: 3
|
||
Metadata:
|
||
Creation Timestamp: 2023-03-31T09:09:33Z
|
||
Generation: 1
|
||
Resource Version: 12971055
|
||
UID: 78945c1e-c323-40d8-87d9-c571275d58e3
|
||
Raw Result Storage:
|
||
Node Selector:
|
||
node-role.kubernetes.io/master:
|
||
Pv Access Modes:
|
||
ReadWriteOnce
|
||
Rotation: 3
|
||
Size: 1Gi
|
||
Tolerations:
|
||
Effect: NoSchedule
|
||
Key: node-role.kubernetes.io/master
|
||
Operator: Exists
|
||
Effect: NoExecute
|
||
Key: node.kubernetes.io/not-ready
|
||
Operator: Exists
|
||
Toleration Seconds: 300
|
||
Effect: NoExecute
|
||
Key: node.kubernetes.io/unreachable
|
||
Operator: Exists
|
||
Toleration Seconds: 300
|
||
Effect: NoSchedule
|
||
Key: node.kubernetes.io/memory-pressure
|
||
Operator: Exists
|
||
Roles:
|
||
master
|
||
worker
|
||
Scan Tolerations:
|
||
Operator: Exists
|
||
Schedule: 0 1 * * *
|
||
Show Not Applicable: false
|
||
Strict Node Scan: true
|
||
Timeout: 30m
|
||
Events: <none>
|
||
````
|
||
You can read more details about the above [configuration here](https://docs.openshift.com/container-platform/4.12/security/compliance_operator/compliance-scans.html#running-compliance-scans_compliance-operator-scans), however, most of the lines are self-explanatory.
|
||
|
||
There is a second provided ScanSettings called ```default-auto-apply```, which follows the same premise as the above, however` will auto-remediate any findings as part of the scan.
|
||
|
||
Now we need to create a ScanSettingBinding to the CIS benchmark profiles. Create a YAML file using the below content called ````compliance-scansettingbinding.yaml```` and apply with ````oc apply -f compliance-scansettingbinding.yaml````.
|
||
|
||
````yaml
|
||
apiVersion: compliance.openshift.io/v1alpha1
|
||
kind: ScanSettingBinding
|
||
metadata:
|
||
name: cis-compliance
|
||
namespace: openshift-compliance
|
||
profiles:
|
||
- name: ocp4-cis-node
|
||
kind: Profile
|
||
apiGroup: compliance.openshift.io/v1alpha1
|
||
- name: ocp4-cis
|
||
kind: Profile
|
||
apiGroup: compliance.openshift.io/v1alpha1
|
||
settingsRef:
|
||
name: default
|
||
kind: ScanSetting
|
||
apiGroup: compliance.openshift.io/v1alpha1
|
||
````
|
||
|
||
At this stage of the workflow, the Compliance Operator reconciles the ScanSettingBinding object, taking into account both the ```Binding``` and ```Bound``` settings. As a result, it generates a ComplianceSuite object along with the corresponding ComplianceScan objects to streamline the compliance evaluation process.
|
||
|
||
|
||
````sh
|
||
$ oc get compliancescan -n openshift-compliance
|
||
NAME PHASE RESULT
|
||
ocp4-cis AGGREGATING NOT-AVAILABLE
|
||
ocp4-cis-node-master AGGREGATING NOT-AVAILABLE
|
||
ocp4-cis-node-worker AGGREGATING NOT-AVAILABLE
|
||
````
|
||
The scans will now work through the scanning phases and finish on the ```DONE``` phase once completed. At this point, you are probably going to find that the result is ```NON-COMPLIANT```. Now we can review the scan results and apply any necessary remediations to bring the cluster into compliance with the profile.
|
||
|
||
````sh
|
||
$ oc get compliancescan -n openshift-compliance
|
||
NAME PHASE RESULT
|
||
ocp4-cis DONE NON-COMPLIANT
|
||
ocp4-cis-node-master DONE NON-COMPLIANT
|
||
ocp4-cis-node-worker DONE NON-COMPLIANT
|
||
````
|
||
|
||
Let's look at the first scan in more detail, as we also want to capture the label attached, so we can get the full results to view.
|
||
|
||
````sh
|
||
$ oc get compliancescan -n openshift-compliance ocp4-cis -o yaml
|
||
````
|
||
|
||
````yaml
|
||
oc get compliancescan -n openshift-compliance ocp4-cis -o yaml
|
||
apiVersion: compliance.openshift.io/v1alpha1
|
||
kind: ComplianceScan
|
||
metadata:
|
||
creationTimestamp: "2023-03-31T10:49:41Z"
|
||
finalizers:
|
||
- scan.finalizers.compliance.openshift.io
|
||
generation: 1
|
||
labels:
|
||
compliance.openshift.io/suite: cis-compliance
|
||
name: ocp4-cis
|
||
namespace: openshift-compliance
|
||
ownerReferences:
|
||
- apiVersion: compliance.openshift.io/v1alpha1
|
||
blockOwnerDeletion: true
|
||
controller: true
|
||
kind: ComplianceSuite
|
||
name: cis-compliance
|
||
uid: c40db43f-0635-4731-b537-5ad3fc08cc06
|
||
resourceVersion: "13035410"
|
||
uid: 9d5b10b8-f67c-44f3-b13d-0765cb037091
|
||
spec:
|
||
content: ssg-ocp4-ds.xml
|
||
contentImage: registry.redhat.io/compliance/openshift-compliance-content-rhel8@sha256:c4bf5b2b20ff538adbc430b7ee993fbd7c291203a9810534005148304e3b169b
|
||
maxRetryOnTimeout: 3
|
||
profile: xccdf_org.ssgproject.content_profile_cis
|
||
rawResultStorage:
|
||
nodeSelector:
|
||
node-role.kubernetes.io/master: ""
|
||
pvAccessModes:
|
||
- ReadWriteOnce
|
||
rotation: 3
|
||
size: 1Gi
|
||
tolerations:
|
||
- effect: NoSchedule
|
||
key: node-role.kubernetes.io/master
|
||
operator: Exists
|
||
- effect: NoExecute
|
||
key: node.kubernetes.io/not-ready
|
||
operator: Exists
|
||
tolerationSeconds: 300
|
||
- effect: NoExecute
|
||
key: node.kubernetes.io/unreachable
|
||
operator: Exists
|
||
tolerationSeconds: 300
|
||
- effect: NoSchedule
|
||
key: node.kubernetes.io/memory-pressure
|
||
operator: Exists
|
||
scanTolerations:
|
||
- operator: Exists
|
||
scanType: Platform
|
||
showNotApplicable: false
|
||
strictNodeScan: true
|
||
timeout: 30m
|
||
status:
|
||
conditions:
|
||
- lastTransitionTime: "2023-03-31T10:51:20Z"
|
||
message: Compliance scan run is done running the scans
|
||
reason: NotRunning
|
||
status: "False"
|
||
type: Processing
|
||
- lastTransitionTime: "2023-03-31T10:51:20Z"
|
||
message: Compliance scan run is done and has results
|
||
reason: Done
|
||
status: "True"
|
||
type: Ready
|
||
phase: DONE
|
||
remainingRetries: 3
|
||
result: NON-COMPLIANT
|
||
resultsStorage:
|
||
name: ocp4-cis
|
||
namespace: openshift-compliance
|
||
warnings: |-
|
||
could not fetch /apis/flowcontrol.apiserver.k8s.io/v1alpha1/flowschemas/catch-all: the server could not find the requested resource
|
||
could not fetch /apis/logging.openshift.io/v1/namespaces/openshift-logging/clusterlogforwarders/instance: the server could not find the requested resource
|
||
could not fetch /apis/apps/v1/namespaces/openshift-sdn/daemonsets/sdn: daemonsets.apps "sdn" not found
|
||
could not fetch "/api/v1/nodes/NODE_NAME/proxy/configz": the server could not find the requested resource
|
||
could not fetch "/api/v1/nodes/NODE_NAME/proxy/configz": the server could not find the requested resource
|
||
could not fetch "/api/v1/nodes/NODE_NAME/proxy/configz": the server could not find the requested resource
|
||
could not fetch "/api/v1/nodes/NODE_NAME/proxy/configz": the server could not find the requested resource
|
||
could not fetch "/api/v1/nodes/NODE_NAME/proxy/configz": the server could not find the requested resource
|
||
could not fetch "/api/v1/nodes/NODE_NAME/proxy/configz": the server could not find the requested resource
|
||
could not fetch "/api/v1/nodes/NODE_NAME/proxy/configz": the server could not find the requested resource
|
||
could not fetch "/api/v1/nodes/NODE_NAME/proxy/configz": the server could not find the requested resource
|
||
could not fetch "/api/v1/nodes/NODE_NAME/proxy/configz": the server could not find the requested resource
|
||
could not fetch "/api/v1/nodes/NODE_NAME/proxy/configz": the server could not find the requested resource
|
||
could not fetch "/api/v1/nodes/NODE_NAME/proxy/configz": the server could not find the requested resource
|
||
could not fetch "/api/v1/nodes/NODE_NAME/proxy/configz": the server could not find the requested resource
|
||
could not fetch "/api/v1/nodes/NODE_NAME/proxy/configz": the server could not find the requested resource
|
||
could not fetch "/api/v1/nodes/NODE_NAME/proxy/configz": the server could not find the requested resource
|
||
could not fetch /apis/hypershift.openshift.io/v1beta1/namespaces/clusters/hostedclusters/None: the server could not find the requested resource
|
||
Kubelet configs for 90days-ocp-72ptq-master-1 are not consistent with role master, Diff: [{"op":"replace","path":"/address","value":"192.168.200.183"}] of KubeletConfigs for master role will not be saved.
|
||
Kubelet configs for 90days-ocp-72ptq-worker-x7v4j are not consistent with role worker, Diff: [{"op":"replace","path":"/address","value":"192.168.200.194"}] of KubeletConfigs for worker role will not be saved.
|
||
Kubelet configs for 90days-ocp-72ptq-master-2 are not consistent with role master, Diff: [{"op":"add","path":"/address","value":"192.168.200.181"}] of KubeletConfigs for master role will not be saved.
|
||
Kubelet configs for 90days-ocp-72ptq-master-0 are not consistent with role control-plane, Diff: [{"op":"replace","path":"/address","value":"192.168.200.185"}] of KubeletConfigs for control-plane role will not be saved.
|
||
Kubelet configs for 90days-ocp-72ptq-master-2 are not consistent with role control-plane, Diff: [{"op":"add","path":"/address","value":"192.168.200.181"}] of KubeletConfigs for control-plane role will not be saved.
|
||
Kubelet configs for 90days-ocp-72ptq-worker-5cgp8 are not consistent with role worker, Diff: [{"op":"add","path":"/address","value":"192.168.200.187"}] of KubeletConfigs for worker role will not be saved.
|
||
````
|
||
|
||
In the output, we can see the warnings from the results straight away. To view the full results we can run the command below using the label from the above output. Alternatively, I can list all failed results across the conducted scans by running the command:
|
||
|
||
````sh
|
||
$ oc get compliancecheckresults -n openshift-compliance -l 'compliance.openshift.io/check-status=FAIL'
|
||
NAME STATUS SEVERITY
|
||
ocp4-cis-api-server-encryption-provider-cipher FAIL medium
|
||
ocp4-cis-api-server-encryption-provider-config FAIL medium
|
||
ocp4-cis-audit-log-forwarding-enabled FAIL medium
|
||
ocp4-cis-configure-network-policies-namespaces FAIL high
|
||
ocp4-cis-kubeadmin-removed FAIL medium
|
||
ocp4-cis-node-master-kubelet-enable-protect-kernel-defaults FAIL medium
|
||
ocp4-cis-node-master-kubelet-enable-protect-kernel-sysctl FAIL medium
|
||
ocp4-cis-node-worker-kubelet-enable-protect-kernel-defaults FAIL medium
|
||
ocp4-cis-node-worker-kubelet-enable-protect-kernel-sysctl FAIL medium
|
||
````
|
||
|
||
We can look at the individual ```ComplianceRemediation``` in more detail with the command;
|
||
|
||
````sh
|
||
oc get compliancecheckresults -n openshift-compliance ocp4-cis-audit-log-forwarding-enabled -o yaml
|
||
````
|
||
|
||
Which provides us the following output:
|
||
|
||
````yaml
|
||
apiVersion: compliance.openshift.io/v1alpha1
|
||
description: |-
|
||
Ensure that Audit Log Forwarding Is Enabled
|
||
OpenShift audit works at the API server level, logging all requests coming to the server. Audit is on by default and the best practice is to ship audit logs off the cluster for retention. The cluster-logging-operator is able to do this with the
|
||
|
||
ClusterLogForwarders
|
||
|
||
resource. The forementioned resource can be configured to logs to different third party systems. For more information on this, please reference the official documentation: https://docs.openshift.com/container-platform/4.6/logging/cluster-logging-external.html
|
||
id: xccdf_org.ssgproject.content_rule_audit_log_forwarding_enabled
|
||
instructions: |-
|
||
Run the following command:
|
||
oc get clusterlogforwarders instance -n openshift-logging -ojson | jq -r '.spec.pipelines[].inputRefs | contains(["audit"])'
|
||
The output should return true.
|
||
kind: ComplianceCheckResult
|
||
metadata:
|
||
annotations:
|
||
compliance.openshift.io/rule: audit-log-forwarding-enabled
|
||
creationTimestamp: "2023-03-31T10:51:02Z"
|
||
generation: 1
|
||
labels:
|
||
compliance.openshift.io/check-severity: medium
|
||
compliance.openshift.io/check-status: FAIL
|
||
compliance.openshift.io/scan-name: ocp4-cis
|
||
compliance.openshift.io/suite: cis-compliance
|
||
name: ocp4-cis-audit-log-forwarding-enabled
|
||
namespace: openshift-compliance
|
||
ownerReferences:
|
||
- apiVersion: compliance.openshift.io/v1alpha1
|
||
blockOwnerDeletion: true
|
||
controller: true
|
||
kind: ComplianceScan
|
||
name: ocp4-cis
|
||
uid: 9d5b10b8-f67c-44f3-b13d-0765cb037091
|
||
resourceVersion: "13034914"
|
||
uid: c5b7342d-af35-4944-86bd-8a4f907acccc
|
||
rationale: Retaining logs ensures the ability to go back in time to investigate or
|
||
correlate any events. Offloading audit logs from the cluster ensures that an attacker
|
||
that has access to the cluster will not be able to tamper with the logs because
|
||
of the logs being stored off-site.
|
||
severity: medium
|
||
status: FAIL
|
||
````
|
||
|
||
This particular remediation has no automatic remediation, but you can see [another example here](https://docs.openshift.com/container-platform/4.12/security/compliance_operator/compliance-operator-remediation.html#compliance-review_compliance-remediation), whereby a ```MachineConfig``` resource will be applied by the remediation. Applying remediation can take different forms, depending on what the remediation is. Therefore I won't dive into this detail today, I think we've covered enough to understand how to run and view compliance checks against our platform. You can read more about [remediations here](https://docs.openshift.com/container-platform/4.12/security/compliance_operator/compliance-operator-remediation.html#compliance-applying_compliance-remediation). The final point I wanted to highlight on this topic, is the ```oc-compliance``` plugin, which extends the functionality of the ```oc``` CLI tool, you can few the [details and how to use it here](https://docs.openshift.com/container-platform/4.12/security/compliance_operator/oc-compliance-plug-in-using.html).
|
||
|
||
# Red Hat Quay Container Security Operator
|
||
|
||
## Vulnerability Scanning Overview
|
||
|
||
Vulnerability scanning in Red Hat OpenShift refers to the process of inspecting container images for known security issues, such as outdated software packages, misconfigurations, or exposed sensitive information. The goal of vulnerability scanning is to identify and remediate potential security risks in container images before they are deployed to the OpenShift cluster, thus enhancing the overall security posture of the platform.
|
||
|
||
OpenShift provides several features and tools to facilitate vulnerability scanning:
|
||
|
||
- Integrated container registry: OpenShift includes a built-in container registry to store and manage container images. The integrated registry allows for a more streamlined and secure process when scanning images for vulnerabilities, as it eliminates the need to rely on external registries.
|
||
|
||
- ImageStreams: An ImageStream in OpenShift is an abstraction that represents a series of related container images, typically different versions of the same application. ImageStreams simplify the process of tracking and deploying container images, making it easier to apply vulnerability scanning and remediation across multiple versions of an application.
|
||
|
||
- Image signing and trust: OpenShift supports container image signing and the enforcement of signature-based trust policies. This feature ensures that only trusted and verified images can be deployed to the cluster, helping to prevent the deployment of images with known vulnerabilities.
|
||
|
||
- Third-party integrations: OpenShift can be easily integrated with external vulnerability scanning tools and platforms, such as Aqua Security, Sysdig, or Twistlock. These tools can be configured to automatically scan container images stored in the OpenShift registry and provide detailed reports on identified vulnerabilities and suggested remediation steps.
|
||
|
||
- OpenShift Operators: OpenShift supports the use of Operators, which are automated software extensions that manage applications and their components. Operators can be used to deploy and manage vulnerability scanning tools within the OpenShift cluster, ensuring a consistent and automated scanning process. Red Hat provides the ```Red Hat Quay Container Security Operator```, however, you can also implement third-party scanners such as [Trivy](https://github.com/aquasecurity/trivy) from [Aqua Security](https://aquasec.com/).
|
||
|
||
By leveraging these features and tools, Red Hat OpenShift enables organizations to perform comprehensive vulnerability scanning on container images, reducing the risk of security breaches and enhancing the overall security of the platform.
|
||
|
||
Focusing on the ```Red Hat Quay Container Security Operator```, this provides the following:
|
||
|
||
- Watches containers associated with pods on all or specified namespaces
|
||
|
||
- Queries the container registry where the containers came from for vulnerability information, provided an image’s registry is running image scanning (such as Quay.io or a Red Hat Quay registry with Clair scanning)
|
||
|
||
- Exposes vulnerabilities via the ImageManifestVuln object in the Kubernetes API
|
||
|
||
## Installing the Red Hat Quay Container Security Operator
|
||
|
||
This time I'm going to provide instructions on how to perform these steps in the Red Hat Console interface, rather than CLI.
|
||
|
||
1. In the administrator view within the console, navigate to Operators > OperatorHub and search for "Red Hat Quay Container Security Operator".
|
||
2. Select the tile, and click to install.
|
||
3. Confirm the settings
|
||
- All namespaces and automatic approval strategy are selected, by default.
|
||
4. Select Install. The Container Security Operator appears after a few moments on the Installed Operators screen.
|
||
|
||
When you now browse to the homepage of the Red Hat OpenShit Console, you will see an "Image vulnerability" component on the status tile, which you can select to see high-level information about your estate.
|
||
|
||
|
||
|
||
You can click on the circle graph or the namespaces to see more details which takes you to the navigation page of Adminstration (1) > Image Vulnerabilities (2).
|
||
|
||
Now you can see a list of the vulnerabilities and you can change the project view for all projects or a specific one to curate this list.
|
||
|
||
|
||
|
||
Clicking the image name (3) will show further manifest details via the OpenShift console, including which pods are affected. Or you can click the Manifest details on the right-hand side (4) which will take you to the Quay Security Scanner report hosted on [Quay.io](https://Quay.io).
|
||
|
||
Below shows the Image manifest details including each vulnerability found with that image, and links to appropriate documentation, such as CVE information held on [access.redhat.com](https://access.redhat.com
|
||
)
|
||
|
||
|
||
Below is the ```affected pods``` tab view on the Image manifests page.
|
||
|
||
|
||
|
||
And finally the Quay Security Scanner page for one of the images shown in the report. This was for my pacman application, you can see the Quay.io report yourself [here](https://quay.io/repository/ifont/pacman-nodejs-app/manifest/sha256:196ae9a1a33a2d32046a46739779ca273667f1d4f231f8a721e8064c3509405e?tab=vulnerabilities)(free sign up account required).
|
||
|
||
|
||
|
||
Of course we can also see information in the command line too, to see all vulnerabilities found, use the command:
|
||
|
||
```sh
|
||
oc get vuln --all-namespaces
|
||
NAMESPACE NAME AGE
|
||
openshift-apiserver-operator sha256.01974e4c0e0d112e09bee8fe2625d565d3d62fa42013b38d7ce43d2d40f6057a 20h
|
||
openshift-apiserver sha256.13640b919950fc648219c528ee7ed30262bae856566fbd6c4cb5e15ffd457d6f 20h
|
||
openshift-apiserver sha256.8829aefa24dd606d2fe3ff86b97858c07acedae5f5eb3f044c20395762e7c02b 20h
|
||
openshift-authentication-operator sha256.31b617cec5c22e187cc22da606fc6998ea3529b1b6e8d80d1799c3dc9705997e 20h
|
||
openshift-authentication sha256.41e06255fc823c0082a74466b69ccfb672947b7075ea43a10e729c5f39314d00 20h
|
||
openshift-cloud-controller-manager-operator sha256.a7856b6371fc4a7ade8a678daca149db6c6a55ee7137d9e308721d2d3bebf364 20h
|
||
openshift-cloud-credential-operator sha256.1986315effe0f3ee415e86df3a87765268ed1da405c7a297c278e1d7030286a4 20h
|
||
...
|
||
openshift-vsphere-infra sha256.ddf81e535cf7a6b2775f3db690ec1e6eaa1c7427a0f9b98ce120d8ad06520440 20h
|
||
test-app sha256.196ae9a1a33a2d32046a46739779ca273667f1d4f231f8a721e8064c3509405e 20h
|
||
```
|
||
|
||
You can inspect the details of a specific vulnerability by running the command:
|
||
|
||
```sh
|
||
oc describe vuln -n {namespace} {sha ID of vuln}
|
||
|
||
# Example
|
||
|
||
oc describe vuln -n openshift-apiserver-operator sha256.01974e4c0e0d112e09bee8fe2625d565d3d62fa42013b38d7ce43d2d40f6057a
|
||
```
|
||
|
||
Example output:
|
||
|
||
```yaml
|
||
Name: sha256.01974e4c0e0d112e09bee8fe2625d565d3d62fa42013b38d7ce43d2d40f6057a
|
||
Namespace: openshift-apiserver-operator
|
||
Labels: openshift-apiserver-operator/openshift-apiserver-operator-7bd84bd596-pgpct=true
|
||
Annotations: <none>
|
||
API Version: secscan.quay.redhat.com/v1alpha1
|
||
Kind: ImageManifestVuln
|
||
Metadata:
|
||
Creation Timestamp: 2023-03-30T19:01:40Z
|
||
Generation: 17
|
||
Resource Version: 13206497
|
||
UID: dbec02e5-e4c6-412f-b561-757237844d43
|
||
Spec:
|
||
Features:
|
||
Name: pip
|
||
Version: 9.0.3
|
||
Vulnerabilities:
|
||
Description: Pip 21.1 updates its dependency 'urllib3' to v1.26.4 due to security issues.
|
||
Metadata: {"UpdatedBy": "pyupio", "RepoName": "pypi", "RepoLink": "https://pypi.org/simple", "DistroName": "", "DistroVersion": "", "NVD": {"CVSSv3": {"Vectors": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:L/A:N", "Score": 6.5}}}
|
||
Name: pyup.io-40291 (CVE-2021-28363)
|
||
Namespace Name: pyupio
|
||
Severity: Medium
|
||
Description: A flaw was found in python-pip in the way it handled Unicode separators in git references. A remote attacker could possibly use this issue to install a different revision on a repository. The highest threat from this vulnerability is to data integrity. This is fixed in python-pip version 21.1.
|
||
Metadata: {"UpdatedBy": "pyupio", "RepoName": "pypi", "RepoLink": "https://pypi.org/simple", "DistroName": "", "DistroVersion": "", "NVD": {"CVSSv3": {"Vectors": "CVSS:3.1/AV:N/AC:L/PR:L/UI:R/S:U/C:N/I:H/A:N", "Score": 5.7}}}
|
||
Name: pyup.io-42559 (CVE-2021-3572)
|
||
Namespace Name: pyupio
|
||
Severity: Medium
|
||
Description: Pip before 19.2 allows Directory Traversal when a URL is given in an install command, because a Content-Disposition header can have ../ in a filename, as demonstrated by overwriting the /root/.ssh/authorized_keys file. This occurs in _download_http_url in _internal/download.py.
|
||
Metadata: {"UpdatedBy": "pyupio", "RepoName": "pypi", "RepoLink": "https://pypi.org/simple", "DistroName": "", "DistroVersion": "", "NVD": {"CVSSv3": {"Vectors": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:H/A:N", "Score": 7.5}}}
|
||
Name: pyup.io-38765 (CVE-2019-20916)
|
||
Namespace Name: pyupio
|
||
Severity: High
|
||
Name: openssl-libs
|
||
Version: 1:1.1.1k-7.el8_6
|
||
Vulnerabilities:
|
||
Description: OpenSSL is a toolkit that implements the Secure Sockets Layer (SSL) and Transport Layer Security (TLS) protocols, as well as a full-strength general-purpose cryptography library.
|
||
|
||
Security Fix(es):
|
||
|
||
* openssl: X.400 address type confusion in X.509 GeneralName (CVE-2023-0286)
|
||
|
||
For more details about the security issue(s), including the impact, a CVSS score, acknowledgments, and other related information, refer to the CVE page(s) listed in the References section.
|
||
Fixedby: 1:1.1.1k-8.el8_6
|
||
Link: https://access.redhat.com/errata/RHSA-2023:1441 https://access.redhat.com/security/cve/CVE-2023-0286
|
||
Metadata: {"UpdatedBy": "RHEL8-rhel-8.6-eus", "RepoName": "cpe:/o:redhat:rhel_eus:8.6::baseos", "RepoLink": null, "DistroName": "Red Hat Enterprise Linux Server", "DistroVersion": "8", "NVD": {"CVSSv3": {"Vectors": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:N/A:H", "Score": 7.4}}}
|
||
Name: RHSA-2023:1441: openssl security update (Important)
|
||
Namespace Name: RHEL8-rhel-8.6-eus
|
||
Severity: High
|
||
Name: urllib3
|
||
Version: 1.24.2
|
||
Vulnerabilities:
|
||
Description: Urllib3 1.26.5 includes a fix for CVE-2021-33503: When provided with a URL containing many @ characters in the authority component, the authority regular expression exhibits catastrophic backtracking, causing a denial of service if a URL were passed as a parameter or redirected to via an HTTP redirect.
|
||
Metadata: {"UpdatedBy": "pyupio", "RepoName": "pypi", "RepoLink": "https://pypi.org/simple", "DistroName": "", "DistroVersion": "", "NVD": {"CVSSv3": {"Vectors": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H", "Score": 7.5}}}
|
||
Name: pyup.io-43975 (CVE-2021-33503)
|
||
Namespace Name: pyupio
|
||
Severity: High
|
||
Description: urllib3 before 1.25.9 allows CRLF injection if the attacker controls the HTTP request method, as demonstrated by inserting CR and LF control characters in the first argument of putrequest(). NOTE: this is similar to CVE-2020-26116.
|
||
Metadata: {"UpdatedBy": "pyupio", "RepoName": "pypi", "RepoLink": "https://pypi.org/simple", "DistroName": "", "DistroVersion": "", "NVD": {"CVSSv3": {"Vectors": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:C/C:L/I:L/A:N", "Score": 7.2}}}
|
||
Name: pyup.io-38834 (CVE-2020-26137)
|
||
Namespace Name: pyupio
|
||
Severity: High
|
||
Name: setuptools
|
||
Version: 39.2.0
|
||
Vulnerabilities:
|
||
Description: Python Packaging Authority (PyPA) setuptools before 65.5.1 allows remote attackers to cause a denial of service via HTML in a crafted package or custom PackageIndex page. There is a Regular Expression Denial of Service (ReDoS) in package_index.py.
|
||
Metadata: {"UpdatedBy": "pyupio", "RepoName": "pypi", "RepoLink": "https://pypi.org/simple", "DistroName": "", "DistroVersion": "", "NVD": {"CVSSv3": {"Vectors": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:N/I:N/A:H", "Score": 5.9}}}
|
||
Name: pyup.io-52495 (CVE-2022-40897)
|
||
Namespace Name: pyupio
|
||
Severity: Medium
|
||
Image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256
|
||
Manifest: sha256:01974e4c0e0d112e09bee8fe2625d565d3d62fa42013b38d7ce43d2d40f6057a
|
||
Status:
|
||
Affected Pods:
|
||
openshift-apiserver-operator/openshift-apiserver-operator-7bd84bd596-pgpct:
|
||
cri-o://dd4fcb700a95c041d19bf7829d3e07516ccf2a36522027f920d76ed0aa57f84c
|
||
Fixable Count: 1
|
||
High Count: 4
|
||
Highest Severity: High
|
||
Last Update: 2023-03-31 15:31:32.853602342 +0000 UTC
|
||
Medium Count: 3
|
||
Events: <none>
|
||
```
|
||
|
||
# Summary
|
||
|
||
Whilst for this 2023 edition focusing on DevSecOps, we could have purely spent time focusing on Security and Compliance for Red Hat OpenShift in-depth, I wanted to start at a higher level, understanding why you would choose an enterprise Kubernetes offering, and what features will enhance your cloud-native platform. Hopefully, this has given you a solid understanding of this offering, as well as being able to understand the basics of how to run and operate it. Another area we only touched upon briefly was application deployment, instead focusing on the security posture of deploying workloads, rather than the methods of building and running the applications themselves. This topic of application build, deployment and management requires a whole section on its own.
|
||
|
||
I urge you to spend time reading through the official documentation for Red Hat OpenShift, it's quite comprehensive with the information you need to fully get to grips and operate the platform.
|
||
|
||
## Resources
|
||
|
||
- Red Hat OpenShift Documentation
|
||
- [OpenShift Container Platform security and compliance](https://docs.openshift.com/container-platform/4.12/security/index.html)
|
||
- [Understanding container security](https://docs.openshift.com/container-platform/4.12/security/container_security/security-understanding.html#security-understanding)
|
||
- [Troubleshooting the Compliance Operator](https://docs.openshift.com/container-platform/4.12/security/compliance_operator/compliance-operator-troubleshooting.html)
|
||
- [Running the Red Hat Quay Container Security Operator](https://docs.openshift.com/container-platform/4.12/security/pod-vulnerability-scan.html)
|
||
- [Securing container content](https://docs.openshift.com/container-platform/4.12/security/container_security/security-container-content.html)
|
||
- [Red Hat OpenShift security guide (ebook)](https://www.redhat.com/en/resources/openshift-security-guide-ebook)
|
||
- YouTube - [ CVE and CVSS explained | Security Detail | Presented by Red Hat](https://www.youtube.com/watch?v=oSyEGkX6sX0) |