mirrors/90DaysOfDevOps
1
0
Fork 0
mirror of https://github.com/MichaelCade/90DaysOfDevOps.git synced 2025-02-24 13:48:02 +07:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'MichaelCade:main' into main

Browse Source
This commit is contained in:
Sourav Kumar 2023-01-22 12:55:05 +05:30 committed by GitHub
commit 4bd2698f81
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 382 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
2023.md
View File

@ -62,11 +62,11 @@ Or contact us via Twitter, my handle is [@MichaelCade1](https://twitter.com/Mich
- [✔️] 🐧 17 > [Fuzzing Advanced](2023/day17.md)
- [✔️] 🐧 18 > [DAST](2023/day18.md)
- [✔️] 🐧 19 > [IAST](2023/day19.md)
- [] 🐧 20 > [](2023/day20.md)
- [✔️] 🐧 20 > [Practical Lab on IAST and DAST](2023/day20.md)
### Continuous Delivery & Deployment
- [] 🌐 21 > [](2023/day21.md)
- [✔️] 🌐 21 > [Continuous Image Repository Scan](2023/day21.md)
- [] 🌐 22 > [](2023/day22.md)
- [] 🌐 23 > [](2023/day23.md)
- [] 🌐 24 > [](2023/day24.md)

157
2023/day20.md
View File

@ -1,10 +1,153 @@
# IAST and DAST in conjunction - lab time
After learning what IAST and DAST are it's time to get our hands dirty and perform an exercise in which we use these processes to find vulnerabilities in real applications.
**NOTE:** There are no open-source IAST implementations, so we will have to use a commerical solution.
Don't worry, there is a free-tier, so you will be able to follow the lab without paying anything.
This lab is based on this [repo](https://github.com/rstatsinger/contrast-java-webgoat-docker).
It contains a vulnerable Java application to be tested and exploited, Docker and Docker Compose for easy setup and [Contrast Community Edition](https://www.contrastsecurity.com/contrast-community-edition?utm_campaign=ContrastCommunityEdition&utm_source=GitHub&utm_medium=WebGoatLab) for IAST solution.
## Prerequisites
- [Docker](https://www.docker.com/products/docker-desktop/)
- [Docker Compose](https://docs.docker.com/compose/)
- Contrast CE account. Sign up for free [here](https://www.contrastsecurity.com/contrast-community-edition?utm_campaign=ContrastCommunityEdition&utm_source=GitHub&utm_medium=WebGoatLab).
**NOTE:** The authors of this article and of the 90 Days of DevOps program are in way associated or affilited with Contrast Security.
We are using this commercial solution, because there is not an open-source one, and because this one has a free-tier that does not require paying or providing a credit card.
1. As there are no open-source IAST implementation will use a commercial one with some free licenses. For this purpose, you will need 2 componenets:
IAST solution from here - https://github.com/rstatsinger/contrast-java-webgoat-docker . You need docker and docker-compose installed in mac or linux enviroment (this lab is tested on Mint). Please follow the README to create account in Contrast.
2. For running the IAST there are few ways to do it- manually via a DAST scanner, ...
- Easiest way to do it is to use ZAP proxy. For this purpose install ZAP from here - https://www.zaproxy.org/download/
- Install zap-cli - https://github.com/Grunny/zap-cli
- Run ZAP proxy (from installed location, in Mint it is by default in /opt/zaproxy)
- Set env variables for ZAP_API_KEY and ZAP_PORT
- Run several commands with zap cli. For example: zap-cli quick-scan -s all --ajax-spider -r http://127.0.0.1:8080/WebGoat/login.mvc . You should see some results in contrast UI.
IAST solution from here - <https://github.com/rstatsinger/contrast-java-webgoat-docker>. You need docker and docker-compose installed in mac or linux enviroment (this lab is tested on Mint). Please follow the README to create account in Contrast.
## Getting started
To start, clone the [repository](https://github.com/rstatsinger/contrast-java-webgoat-docker).
Get your credentials from Contrast Security.
Click on your name in the top-right corner -> `Organization Settings` -> `Agent`.
Get the values for `Agent Username`, `Agent Service Key` and `API Key`.
Replace these values in the `.env.template` file in the newly cloned repository.
**NOTE:** These values are secret.
Do not commit them to Git.
It's best to put the `.env.template` under `.gitignore` so that you don't commit these values by mistake.
## Running the vulnerable application
To run the vulnerable application, run:
```sh
./run.sh
```
or
```sh
docker compose up
```
Once ready, the application UI will be accessible on <http://localhost:8080/WebGoat>.
## Do some damage
Now that we have a vulnerable application let's try to exploit it.
1. Install ZAP Proxy from [here](https://www.zaproxy.org/download/)
An easy way to do that is via a DAST scanner.
One such scanner is [ZAP Proxy](https://www.zaproxy.org/).
It is a free and open-source web app scanner.
2. Install `zap-cli` from [here](https://github.com/Grunny/zap-cli)
Next, install `zap-cli`.
`zap-cli` is an open-source CLI for ZAP Proxy.
3. Run ZAP proxy
Run ZAP Proxy from its installed location.
In Linux Mint it is by default in `/opt/zaproxy`.
In MacOS it is in `Applications`.
4. Set env variables for `ZAP_API_KEY` and `ZAP_PORT`
Get these values from ZAP Proxy.
Go to `Options...` -> `API` to get the API Key.
Go to `Options...` -> `Network` -> `Local Servers/Proxies` to configure and obtain the port.
5. Run several commands with `zap-cli`
For example:
```sh
zap-cli quick-scan -s all --ajax-spider -r http://127.0.0.1:8080/WebGoat/login.mvc
```
Alternatively, you can follow the instructions in the [repo](https://github.com/rstatsinger/contrast-java-webgoat-docker/blob/master/Lab-WebGoat.pdf)
to cause some damage to the vulnerable application.
6. Observe findings in Constrast
Either way, if you go to the **Vulnerabilities** tab for your application in Contrast you should be able to see that Contrast detected the vulnerabilities
and is warning you to take some action.
## Bonus: Image Scanning
We saw how an IAST solution helped us detect attacks by observing the behaviour of the application.
Let's see whether we could have done something to prevent these attacks in the first place.
The vulnerable application we used for this demo was packages as a container.
Let's scan this container via the `grype` scanner we learned about in Days [14](day14.md) and [15](day15.md) and see the results.
```sh
$ grype contrast-java-webgoat-docker-webgoat
✔ Vulnerability DB [no update available]
✔ Loaded image
✔ Parsed image
✔ Cataloged packages [316 packages]
✔ Scanned image [374 vulnerabilities]
NAME INSTALLED FIXED-IN TYPE VULNERABILITY SEVERITY
apt 1.8.2.3 deb CVE-2011-3374 Negligible
axis 1.4 java-archive GHSA-55w9-c3g2-4rrh Medium
axis 1.4 java-archive GHSA-96jq-75wh-2658 Medium
bash 5.0-4 deb CVE-2019-18276 Negligible
bash 5.0-4 (won't fix) deb CVE-2022-3715 High
bsdutils 1:2.33.1-0.1 deb CVE-2022-0563 Negligible
bsdutils 1:2.33.1-0.1 (won't fix) deb CVE-2021-37600 Low
commons-beanutils 1.8.3 java-archive CVE-2014-0114 High
commons-beanutils 1.8.3 java-archive CVE-2019-10086 High
commons-beanutils 1.8.3 1.9.2 java-archive GHSA-p66x-2cv9-qq3v High
commons-beanutils 1.8.3 1.9.4 java-archive GHSA-6phf-73q6-gh87 High
commons-collections 3.2.1 java-archive CVE-2015-6420 High
commons-collections 3.2.1 3.2.2 java-archive GHSA-6hgm-866r-3cjv High
commons-collections 3.2.1 3.2.2 java-archive GHSA-fjq5-5j5f-mvxh Critical
commons-fileupload 1.3.1 java-archive CVE-2016-1000031 Critical
commons-fileupload 1.3.1 java-archive CVE-2016-3092 High
commons-fileupload 1.3.1 1.3.2 java-archive GHSA-fvm3-cfvj-gxqq High
commons-fileupload 1.3.1 1.3.3 java-archive GHSA-7x9j-7223-rg5m Critical
commons-io 2.4 java-archive CVE-2021-29425 Medium
commons-io 2.4 2.7 java-archive GHSA-gwrp-pvrq-jmwv Medium
coreutils 8.30-3 deb CVE-2017-18018 Negligible
coreutils 8.30-3 (won't fix) deb CVE-2016-2781 Low
curl 7.64.0-4+deb10u3 deb CVE-2021-22922 Negligible
curl 7.64.0-4+deb10u3 deb CVE-2021-22923 Negligible
<truncated>
```
As we can see this image is full with vulnerabilities.
If we dive into each one we will see we have vulnerabilities like RCE (Remote Code Execution), SQL Injection, XML External Entity Vulnerability, etc.
## Week Summary
IAST and DAST are important methods that can help us find vulnerabilities in our application via monitoring its behaviour.
This is done once the application is already deployed.
Container Image Scanning can help us find vulnerabilities in our application based on the library that are present inside the container.
Image Scanning and IAST/DAST are not mutually-exclusive.
They both have their place in a Secure SDLC and can help us find different problems before the attackers do.

230
2023/day21.md
View File

@ -0,0 +1,230 @@
# Continuous Image Repository Scan
In [Day 14](day14.md), we learned what container image scanning is and why it's important.
We also learned about tools like Grype and Trivy that help us scan our container images.
However, in modern SDLCs, a DevSecOps engineer would rarely scan container images by hand, e.g., they would not be running Grype and Trivy locally and looking at every single vulnerability.
Instead, they would have the image scanning configured as part of the CI/CD pipeline.
This way, they would be sure that all the images that are being built by the pipelines are also scanned by the image scanner.
These results could then be sent by another system, where the DevSecOps engineers could look at them and take some action depending on the result.
A sample CI/CD pipeline could look like this:
0. _Developer pushes code_
1. Lint the code
2. Build the code
3. Test the code
4. Build the artifacts (container images, helm charts, etc.)
5. Scan the artifacts
6. (Optional) Send the scan results somewhere
7. (Optional) Verify the scan results and fail the pipeline if the verification fails
8. Push the artifacts to a repository
A failure in the scan or verify steps (steps 6 and 7) would mean that our container will not be pushed to our repository, and we cannot use the code we submitted.
Today, we are going to take a look at how we can set up such a pipeline and what would be a sensible configuration for one.
## Setting up a CI/CD pipeline with Grype
Let's take a look at the [Grype](https://github.com/anchore/grype) scanner.
Grype is an open-source scanner maintained by the company [Anchore](https://anchore.com/).
### Scanning an image with Grype
Scanning a container image with Grype is as simple as running:
```shell
grype <IMAGE>
```
For example, if we want to scan the `ubuntu:20.04` image, we can run:
```shell
$ grype ubuntu:20.04
✔ Vulnerability DB [no update available]
✔ Pulled image
✔ Loaded image
✔ Parsed image
✔ Cataloged packages [92 packages]
✔ Scanned image [19 vulnerabilities]
NAME INSTALLED FIXED-IN TYPE VULNERABILITY SEVERITY
coreutils 8.30-3ubuntu2 deb CVE-2016-2781 Low
gpgv 2.2.19-3ubuntu2.2 deb CVE-2022-3219 Low
libc-bin 2.31-0ubuntu9.9 deb CVE-2016-20013 Negligible
libc6 2.31-0ubuntu9.9 deb CVE-2016-20013 Negligible
libncurses6 6.2-0ubuntu2 deb CVE-2021-39537 Negligible
libncurses6 6.2-0ubuntu2 deb CVE-2022-29458 Negligible
libncursesw6 6.2-0ubuntu2 deb CVE-2021-39537 Negligible
libncursesw6 6.2-0ubuntu2 deb CVE-2022-29458 Negligible
libpcre3 2:8.39-12ubuntu0.1 deb CVE-2017-11164 Negligible
libsystemd0 245.4-4ubuntu3.19 deb CVE-2022-3821 Medium
libtinfo6 6.2-0ubuntu2 deb CVE-2021-39537 Negligible
libtinfo6 6.2-0ubuntu2 deb CVE-2022-29458 Negligible
libudev1 245.4-4ubuntu3.19 deb CVE-2022-3821 Medium
login 1:4.8.1-1ubuntu5.20.04.4 deb CVE-2013-4235 Low
ncurses-base 6.2-0ubuntu2 deb CVE-2021-39537 Negligible
ncurses-base 6.2-0ubuntu2 deb CVE-2022-29458 Negligible
ncurses-bin 6.2-0ubuntu2 deb CVE-2021-39537 Negligible
ncurses-bin 6.2-0ubuntu2 deb CVE-2022-29458 Negligible
passwd 1:4.8.1-1ubuntu5.20.04.4 deb CVE-2013-4235 Low
```
Of course, you already know that because we did it on [Day 14](day14.md).
However, this command will only output the vulnerabilities and exit with a success code.
So if this were in a CI/CD pipeline, the pipeline would be successful even if we have many vulnerabilities.
The person running the pipeline would have to open it, see the logs and manually determine whether the results are OK.
This is tedious and error prone.
Let's see how we can enforce some rules for the results that come out of the scan.
### Enforcing rules for the scanned images
As we already established, just scanning the image does not do much except for giving us visibility into the number of vulnerabilities we have inside the image.
But what if we want to enforce a set of rules for our container images?
For example, a good rule would be "an image should not have critical vulnerabilities" or "an image should not have vulnerabilities with available fixes."
Fortunately for us, this is also something that Grype supports out of the box.
We can use the `--fail-on <SEVERITY>` flag to tell Grype to exit with a non-zero exit code if, during the scan, it found vulnerabilities with a severity higher or equal to the one we specified.
This will fail our pipeline, and the engineer would have to look at the results and fix something in order to make it pass.
Let's tried it out.
We are going to use the `springio/petclinic:latest` image, which we already found has many vulnerabilities.
You can go back to [Day 14](day14.md) or scan it yourself to see how much exactly.
We want to fail the pipeline if the image has `CRITICAL` vulnerabilities.
We are going to run the can like this:
```shell
$ grype springio/petclinic:latest --fail-on critical
✔ Vulnerability DB [no update available]
✔ Loaded image
✔ Parsed image
✔ Cataloged packages [212 packages]
✔ Scanned image [168 vulnerabilities]
NAME INSTALLED FIXED-IN TYPE VULNERABILITY SEVERITY
spring-core 5.3.6 java-archive CVE-2016-1000027 Critical
spring-core 5.3.6 java-archive CVE-2022-22965 Critical
...
1 error occurred:
* discovered vulnerabilities at or above the severity threshold
$ echo $?
1
```
We see two things here:
- apart from the results, Grype also outputted an error that is telling us that this scan violated the rule we had defined (no CRITICAL vulnerabilities)
- Grype exited with exit code 1, which indicates failure.
If this were a CI pipeline, it would have failed.
When this happens, we will be blocked from merging our code and pushing our container to the registry.
This means that we need to take some action to fix the failure so that we can finish our task and push our change.
Let's see what our options are.
### Fixing the pipeline
Once we encounter a vulnerability that is preventing us from publishing our container, we have a few ways we can go depending on the vulnerability.
#### 1. The vulnerability has a fix
The best-case scenario is when this vulnerability is already fixed in a newer version of the library we depend on.
One such vulnerability is this one:
```text
NAME INSTALLED FIXED-IN TYPE VULNERABILITY SEVERITY
snakeyaml 1.27 1.31 java-archive GHSA-3mc7-4q67-w48m High
```
This is a `High` severity vulnerability.
It's coming from the Java package `snakeyaml`, version `1.27`.
Grype is telling us that this vulnerability is fixed in version `1.31` of the same library.
In this case, we can just upgrade the version of this library in our `pom.xml` or `build.gradle` file,
test our code to make sure nothing breaks with the new version,
and submit the code again.
This will build a new version of our container, re-scan it, and hopefully, this time, the vulnerability will not come up, and our scan will be successful.
### 2. The vulnerability does not have a fix, but it's not dangerous
Sometimes a vulnerability we encounter will not have a fix available.
These are so-called zero-day vulnerabilities that are disclosed before a fix is available.
We can see two of those in the initial scan results:
```text
NAME INSTALLED FIXED-IN TYPE VULNERABILITY SEVERITY
spring-core 5.3.6 java-archive CVE-2016-1000027 Critical
spring-core 5.3.6 java-archive CVE-2022-22965 Critical
```
When we encounter such a vulnerability, we need to evaluate how severe it is and calculate the risk of releasing our software with that vulnerability in it.
We can determine that the vulnerability does not constitute any danger to our software and its consumers.
One such case might be when a vulnerability requires physical access to the servers to be exploited.
If we are sure that our physical servers are secure enough and an attacker cannot get access to them, we can safely ignore this vulnerability.
In this case, we can tell Grype to ignore this vulnerability and not fail the scan because of it.
We can do this via the `grype.yaml` configuration file, where we can list vulnerabilities we want to ignore:
```yaml
ignore:
# This is the full set of supported rule fields:
- vulnerability: CVE-2016-1000027
fix-state: unknown
package:
name: spring-core
version: 5.3.6
type: java-archive
# We can list as many of these as we want
- vulnerability: CVE-2022-22965
# Or list whole packages which we want to ignore
- package:
type: gem
```
Putting this in our configuration file and re-running the scan will make our pipeline green.
However, it is crucial that we keep track of this file and not ignore vulnerabilities that have a fix.
For example, when a fix for this vulnerability is released, it's best we upgrade our dependency and remove this vulnerability from our application.
That way, we will ensure that our application is as secure as possible and there are no vulnerabilities that can turn out to be more severe than we initially thought.
### 3. Vulnerability does not have a fix, and IT IS dangerous
The worst-case scenario is if we encounter a vulnerability that does not have a fix, and it is indeed dangerous, and there is a possibility to be exploited.
In that case, there is no right move.
The best thing we can do is sit down with our security team and come up with an action plan.
We might decide it's best to do nothing while the vulnerability is fixed.
We might decide to manually patch some stuff so that we remove at least some part of the danger.
It really depends on the situation.
Sometimes, a zero-day vulnerability is already in your application that is deployed.
In that case, freezing deploys won't help because your app is already vulnerable.
That was the case with the Log4Shell vulnerability that was discovered in late 2021 but has been present in Log4j since 2013.
Luckily, there was a fix available within hours, but next time we might not be this lucky.
## Summary
As we already learned in [Day 14](day14.md), scanning your container images for vulnerabilities is important as it can give you valuable insights about
the security posture of your images.
Today we learned that it's even better to have it as part of your CI/CD pipeline and to enforce some basic rules about what vulnerabilities you have inside your images.
Finally, we discussed the steps we can take when we find a vulnerability.
Tomorrow we are going to take a look at container registries that enable this scanning out of the box and also at scanning other types of artifacts.
See you on [Day 22](day22.md).