index.md 19.3 KB
Newer Older
1 2 3 4
---
type: reference, howto
---

5
# Dynamic Application Security Testing (DAST) **(ULTIMATE)**
6

7
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/4348)
8 9
in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.4.

10 11 12 13 14
NOTE: **4 of the top 6 attacks were application based.**
Download our whitepaper,
["A Seismic Shift in Application Security"](https://about.gitlab.com/resources/whitepaper-seismic-shift-application-security/)
to learn how to protect your organization.

15 16 17 18 19 20 21 22 23 24 25
Running [static checks](../sast/index.md) on your code is the first step to detect
vulnerabilities that can put the security of your code at risk. Yet, once
deployed, your application is exposed to a new category of possible attacks,
such as cross-site scripting or broken authentication flaws. This is where
Dynamic Application Security Testing (DAST) comes into place.

## Overview

If you are using [GitLab CI/CD](../../../ci/README.md), you can analyze your running web application(s)
for known vulnerabilities using Dynamic Application Security Testing (DAST).

26
You can take advantage of DAST by either [including the CI job](#configuration) in
27 28 29 30 31 32 33
your existing `.gitlab-ci.yml` file or by implicitly using
[Auto DAST](../../../topics/autodevops/index.md#auto-dast-ultimate)
that is provided by [Auto DevOps](../../../topics/autodevops/index.md).

GitLab checks the DAST report, compares the found vulnerabilities between the source and target
branches, and shows the information right on the merge request.

34 35 36 37
NOTE: **Note:**
This comparison logic uses only the latest pipeline executed for the target branch's base commit.
Running the pipeline on any other commit has no effect on the merge request.

38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62
![DAST Widget](img/dast_all.png)

By clicking on one of the detected linked vulnerabilities, you will be able to
see the details and the URL(s) affected.

![DAST Widget Clicked](img/dast_single.png)

[Dynamic Application Security Testing (DAST)](https://en.wikipedia.org/wiki/Dynamic_Application_Security_Testing)
is using the popular open source tool [OWASP ZAProxy](https://github.com/zaproxy/zaproxy)
to perform an analysis on your running web application.

By default, DAST executes [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan) and will perform passive scanning only. It will not actively attack your application.

However, DAST can be [configured](#full-scan)
to also perform a so-called "active scan". That is, attack your application and produce a more extensive security report.
It can be very useful combined with [Review Apps](../../../ci/review_apps/index.md).

## Use cases

It helps you automatically find security vulnerabilities in your running web
applications while you are developing and testing your applications.

## Requirements

To run a DAST job, you need GitLab Runner with the
P
Philippe Lafoucrière 已提交
63
[`docker` executor](https://docs.gitlab.com/runner/executors/docker.html).
64

65
## Configuration
66

67 68
For GitLab 11.9 and later, to enable DAST, you must
[include](../../../ci/yaml/README.md#includetemplate) the
69
[`DAST.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/DAST.gitlab-ci.yml)
70 71 72
that's provided as a part of your GitLab installation.
For GitLab versions earlier than 11.9, you can copy and use the job as defined
in that template.
73

74
Add the following to your `.gitlab-ci.yml` file:
75 76 77

```yaml
include:
78
  - template: DAST.gitlab-ci.yml
79 80 81 82 83

variables:
  DAST_WEBSITE: https://example.com
```

84 85
There are two ways to define the URL to be scanned by DAST:

86 87 88 89
1. Set the `DAST_WEBSITE` [variable](../../../ci/yaml/README.md#variables).

1. Add it in an `environment_url.txt` file at the root of your project.
    This is great for testing in dynamic environments. In order to run DAST against
90
    an app that is dynamically created during a GitLab CI pipeline, have the app
91 92 93 94
    persist its domain in an `environment_url.txt` file, and DAST will
    automatically parse that file to find its scan target.
    You can see an [example](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml)
    of this in our Auto DevOps CI YML.
95

96
If both values are set, the `DAST_WEBSITE` value will take precedence.
S
Seth Berger 已提交
97

98 99 100
The included template will create a `dast` job in your CI/CD pipeline and scan
your project's source code for possible vulnerabilities.

101
The results will be saved as a
102 103 104 105 106 107
[DAST report artifact](../../../ci/yaml/README.md#artifactsreportsdast-ultimate)
that you can later download and analyze. Due to implementation limitations we
always take the latest DAST artifact available. Behind the scenes, the
[GitLab DAST Docker image](https://gitlab.com/gitlab-org/security-products/dast)
is used to run the tests on the specified URL and scan it for possible vulnerabilities.

108 109 110 111
By default, the DAST template will use the latest major version of the DAST Docker image. Using the `DAST_VERSION` variable,
you can choose to automatically update DAST with new features and fixes by pinning to a major version (e.g. 1), only update fixes by pinning to a minor version (e.g. 1.6) or prevent all updates by pinning to a specific version (e.g. 1.6.4).
Find the latest DAST versions on the [Releases](https://gitlab.com/gitlab-org/security-products/dast/-/releases) page.

112
### Authenticated scan
113 114 115 116 117

It's also possible to authenticate the user before performing the DAST checks:

```yaml
include:
118
  - template: DAST.gitlab-ci.yml
119 120 121 122 123 124 125 126 127 128 129

variables:
  DAST_WEBSITE: https://example.com
  DAST_AUTH_URL: https://example.com/sign-in
  DAST_USERNAME: john.doe@example.com
  DAST_PASSWORD: john-doe-password
  DAST_USERNAME_FIELD: session[user] # the name of username field at the sign-in HTML form
  DAST_PASSWORD_FIELD: session[password] # the name of password field at the sign-in HTML form
  DAST_AUTH_EXCLUDE_URLS: http://example.com/sign-out,http://example.com/sign-out-2 # optional, URLs to skip during the authenticated scan; comma-separated, no spaces in between
```

130
The results will be saved as a
131 132 133 134
[DAST report artifact](../../../ci/yaml/README.md#artifactsreportsdast-ultimate)
that you can later download and analyze.
Due to implementation limitations, we always take the latest DAST artifact available.

135
### Full scan
136 137 138 139 140 141

DAST can be configured to perform [ZAP Full Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Full-Scan), which
includes both passive and active scanning against the same target website:

```yaml
include:
142
  - template: DAST.gitlab-ci.yml
143 144 145 146 147

variables:
  DAST_FULL_SCAN_ENABLED: "true"
```

148 149 150 151 152 153 154 155 156 157
#### Domain validation

The DAST job can be run anywhere, which means you can accidentally hit live web servers
and potentially damage them. You could even take down your production environment.
For that reason, you should use domain validation.

Domain validation is not required by default. It can be required by setting the [environment variable](#available-variables) `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` to true.

```yaml
include:
158
  - template: DAST.gitlab-ci.yml
159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236

variables:
  DAST_FULL_SCAN_ENABLED: "true"
  DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED: "true"
```

Since ZAP full scan actively attacks the target application, DAST sends a ping to the target (normally defined in `DAST_WEBSITE` or `environment_url.txt`) beforehand.

If `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` is false or unset, the scan will _proceed_ unless the response to the ping
includes a `Gitlab-DAST-Permission` header with a value of `deny`.

If `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` is true, the scan will _exit_ unless the response to the ping
includes a `Gitlab-DAST-Permission` header with a value of `allow`.

Here are some examples of adding the `Gitlab-DAST-Permission` header to a response in Rails, Django, and Node (with Express).

##### Ruby on Rails

Here's how you would add a [custom header in Ruby on Rails](https://guides.rubyonrails.org/action_controller_overview.html#setting-custom-headers):

```ruby
class DastWebsiteTargetController < ActionController::Base
  def dast_website_target
    response.headers['Gitlab-DAST-Permission'] = 'allow'

    head :ok
  end
end
```

##### Django

Here's how you would add a [custom header in Django](https://docs.djangoproject.com/en/2.2/ref/request-response/#setting-header-fields):

```python
class DastWebsiteTargetView(View):
    def head(self, *args, **kwargs):
      response = HttpResponse()
      response['Gitlab-Dast-Permission'] = 'allow'

      return response
```

##### Node (with Express)

Here's how you would add a [custom header in Node (with Express)](http://expressjs.com/en/5x/api.html#res.append):

```javascript
app.get('/dast-website-target', function(req, res) {
  res.append('Gitlab-DAST-Permission', 'allow')
  res.send('Respond to DAST ping')
})
```

##### Domain validation header via a proxy

It's also possible to add the `Gitlab-DAST-Permission` header via a proxy.

###### NGINX

The following config allows NGINX to act as a reverse proxy and add the `Gitlab-DAST-Permission` [header](http://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header):

```
# default.conf
server {
    listen 80;
    server_name localhost;

    location / {
        proxy_pass http://test-application;
        add_header Gitlab-DAST-Permission allow;
    }
}
```

###### Apache

Apache can also be used as a [reverse proxy](https://httpd.apache.org/docs/2.4/mod/mod_proxy.html)
237
to add the `Gitlab-DAST-Permission` [header](https://httpd.apache.org/docs/current/mod/mod_headers.html).
238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256

To do so, add the following lines to `httpd.conf`:

```
# httpd.conf
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule proxy_http_module modules/mod_proxy_http.so

<VirtualHost *:80>
  ProxyPass "/" "http://test-application.com/"
  ProxyPassReverse "/" "http://test-application.com/"
  Header set Gitlab-DAST-Permission "allow"
</VirtualHost>
```

[This snippet](https://gitlab.com/gitlab-org/security-products/dast/snippets/1894732) contains a complete `httpd.conf` file
configured to act as a remote proxy and add the `Gitlab-DAST-Permission` header.

257
### Customizing the DAST settings
258

Q
Quantum Operations 已提交
259
The DAST settings can be changed through environment variables by using the
260
[`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`.
261
These variables are documented in [available variables](#available-variables).
262 263 264 265 266

For example:

```yaml
include:
267
  - template: DAST.gitlab-ci.yml
268 269 270 271 272 273 274 275 276

variables:
  DAST_WEBSITE: https://example.com
  DAST_TARGET_AVAILABILITY_TIMEOUT: 120
```

Because the template is [evaluated before](../../../ci/yaml/README.md#include) the pipeline
configuration, the last mention of the variable will take precedence.

277
### Overriding the DAST template
278 279 280 281 282 283 284

If you want to override the job definition (for example, change properties like
`variables` or `dependencies`), you need to declare a `dast` job after the
template inclusion and specify any additional keys under it. For example:

```yaml
include:
285
  - template: DAST.gitlab-ci.yml
286 287 288 289 290 291 292 293 294 295 296 297

dast:
  stage: dast # IMPORTANT: don't forget to add this
  variables:
    DAST_WEBSITE: https://example.com
    CI_DEBUG_TRACE: "true"
```

As the DAST job belongs to a separate `dast` stage that runs after all
[default stages](../../../ci/yaml/README.md#stages),
don't forget to add `stage: dast` when you override the template job definition.

298
### Available variables
299 300 301 302 303 304 305 306 307 308 309 310 311 312

DAST can be [configured](#customizing-the-dast-settings) using environment variables.

| Environment variable        | Required   | Description                                                                    |
|-----------------------------| ----------|--------------------------------------------------------------------------------|
| `DAST_WEBSITE`  | yes | The URL of the website to scan. |
| `DAST_AUTH_URL` | no | The authentication URL of the website to scan. |
| `DAST_USERNAME` | no | The username to authenticate to in the website. |
| `DAST_PASSWORD` | no | The password to authenticate to in the website. |
| `DAST_USERNAME_FIELD` | no | The name of username field at the sign-in HTML form. |
| `DAST_PASSWORD_FIELD` | no | The name of password field at the sign-in HTML form. |
| `DAST_AUTH_EXCLUDE_URLS` | no | The URLs to skip during the authenticated scan; comma-separated, no spaces in between. |
| `DAST_TARGET_AVAILABILITY_TIMEOUT` | no | Time limit in seconds to wait for target availability. Scan is attempted nevertheless if it runs out. Integer. Defaults to `60`. |
| `DAST_FULL_SCAN_ENABLED` | no | Switches the tool to execute [ZAP Full Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Full-Scan) instead of [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan). Boolean. `true`, `True`, or `1` are considered as true value, otherwise false. Defaults to `false`. |
313
| `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` | no | Requires [domain validation](#domain-validation) when running DAST full scans. Boolean. `true`, `True`, or `1` are considered as true value, otherwise false. Defaults to `false`. |
314
| `DAST_AUTO_UPDATE_ADDONS` | no | Set to `false` to pin the versions of ZAProxy add-ons to those provided with the DAST image. Defaults to `true`. |
315

316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357
### DAST command-line options

Not all DAST configuration is available via environment variables. To find out all possible options, run the following configuration.
Available command-line options will be printed to the job log:

```yaml
include:
  template: DAST.gitlab-ci.yml

dast:
  script:
    - /analyze --help
```

You must then overwrite the `script` command to pass in the appropriate argument. For example, AJAX spidering can be enabled by using `-j`, as shown in the following configuration:

```yaml
include:
  template: DAST.gitlab-ci.yml

dast:
  script:
    - export DAST_WEBSITE=${DAST_WEBSITE:-$(cat environment_url.txt)}
    - /analyze -j -t $DAST_WEBSITE
```

### Custom ZAProxy configuration

The ZAProxy server contains many [useful configurable values](https://gitlab.com/gitlab-org/gitlab/issues/36437#note_245801885).
Many key/values for `-config` remain undocumented, but there is an untested list of [possible keys](https://gitlab.com/gitlab-org/gitlab/issues/36437#note_244981023).
Note that these options are not supported by DAST, and may break the DAST scan when used. An example of how to rewrite the Authorization header value with `TOKEN` follows:

```yaml
include:
  template: DAST.gitlab-ci.yml

dast:
  script:
    - export DAST_WEBSITE=${DAST_WEBSITE:-$(cat environment_url.txt)}
    - /analyze -z"-config replacer.full_list\(0\).description=auth -config replacer.full_list\(0\).enabled=true -config replacer.full_list\(0\).matchtype=REQ_HEADER -config replacer.full_list\(0\).matchstr=Authorization -config replacer.full_list\(0\).regex=false -config replacer.full_list\(0\).replacement=TOKEN" -t $DAST_WEBSITE
```

358 359 360 361 362
### Cloning the project's repository

The DAST job does not require the project's repository to be present when running, so by default
[`GIT_STRATEGY`](../../../ci/yaml/README.md#git-strategy) is set to `none`.

363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387
## Running DAST in an offline air-gapped installation

DAST can be executed on an offline air-gapped GitLab Ultimate installation using the following process:

1. Host the DAST image `registry.gitlab.com/gitlab-org/security-products/dast:latest` in your local
   Docker container registry.
1. Add the following configuration to your `.gitlab-ci.yml` file. You must replace `image` to refer
   to the DAST Docker image hosted on your local Docker container registry:

   ```yaml
   include:
     - template: DAST.gitlab-ci.yml

   dast:
     image: registry.example.com/namespace/dast:latest
     script:
        - export DAST_WEBSITE=${DAST_WEBSITE:-$(cat environment_url.txt)}
        - /analyze -t $DAST_WEBSITE --auto-update-addons false -z"-silent"
   ```

The option `--auto-update-addons false` instructs ZAP not to update add-ons.

The option `-z` passes the quoted `-silent` parameter to ZAP. The `-silent` parameter ensures ZAP
does not make any unsolicited requests including checking for updates.

388 389 390 391 392
## Reports

The DAST job can emit various reports.

### JSON
393 394

CAUTION: **Caution:**
395 396
The JSON report artifacts are not a public API of DAST and their format is expected to change in the future.

397
The DAST tool always emits a JSON report report file called `gl-dast-report.json` and sample reports can be found in the [DAST repository](https://gitlab.com/gitlab-org/security-products/dast/-/tree/master/test/end-to-end/expect).
398 399

There are two formats of data in the JSON report that are used side by side: the proprietary ZAP format which will be eventually deprecated, and a "common" format which will be the default in the future.
400

401
### Other formats
402

403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422
Reports can also be generated in Markdown, HTML, and XML.

Reports can be published as artifacts using the following configuration:

```yaml
include:
  template: DAST.gitlab-ci.yml

dast:
  script:
    - export DAST_WEBSITE=${DAST_WEBSITE:-$(cat environment_url.txt)}
    - /analyze -r report.html -w report.md -x report.xml -t $DAST_WEBSITE
    - cp /zap/wrk/report.{html,md,xml} "$PWD"
  artifacts:
    paths:
      - report.html
      - report.md
      - report.xml
      - gl-dast-report.json
```
423

424 425 426
## Security Dashboard

The Security Dashboard is a good place to get an overview of all the security
427
vulnerabilities in your groups, projects and pipelines. Read more about the
428 429
[Security Dashboard](../security_dashboard/index.md).

430 431 432 433 434 435 436 437 438 439 440 441 442 443
## Bleeding-edge vulnerability definitions

ZAProxy first creates rules in the `alpha` class. After a testing period with the community, they are promoted to `beta`. DAST uses `beta` definitions by default. To request `alpha` definitions, use `-a` as shown in the following configuration:

```yaml
include:
  template: DAST.gitlab-ci.yml

dast:
  script:
    - export DAST_WEBSITE=${DAST_WEBSITE:-$(cat environment_url.txt)}
    - /analyze -a -t $DAST_WEBSITE
```

444 445 446 447
## Interacting with the vulnerabilities

Once a vulnerability is found, you can interact with it. Read more on how to
[interact with the vulnerabilities](../index.md#interacting-with-the-vulnerabilities).
448 449 450 451 452

## Vulnerabilities database update

For more information about the vulnerabilities database update, check the
[maintenance table](../index.md#maintenance-and-update-of-the-vulnerabilities-database).
453 454 455 456 457 458 459 460 461 462 463 464

<!-- ## Troubleshooting

Include any troubleshooting steps that you can foresee. If you know beforehand what issues
one might have when setting this up, or when something is changed, or on upgrading, it's
important to describe those, too. Think of things that may go wrong and include them here.
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.

Each scenario can be a third-level heading, e.g. `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484

## Troubleshooting

### Running out of memory

By default, ZAProxy, which DAST relies on, is allocated memory that sums to 25%
of the total memory on the host.
Since it keeps most of its information in memory during a scan,
it is possible for DAST to run out of memory while scanning large applications.
This results in the following error:

```
[zap.out] java.lang.OutOfMemoryError: Java heap space
```

Fortunately, it is straightforward to increase the amount of memory available
for DAST by overwriting the `script` key in the DAST template:

```yaml
include:
485
  - template: DAST.gitlab-ci.yml
486 487 488 489 490 491 492 493 494

dast:
  script:
    - export DAST_WEBSITE=${DAST_WEBSITE:-$(cat environment_url.txt)}
    - /analyze -t $DAST_WEBSITE -z"-Xmx3072m"
```

Here, DAST is being allocated 3072 MB.
Change the number after `-Xmx` to the required memory amount.