* Add binary encoding logic into ActionDispatch::Request::Utils

Moving the logic to set binary encoding into ActionDispatch::Request::Utils
will allow us to encode from GET and POST in ActionDispatch::Request.

* Refactor binary encoding logic

- Move binary encoding calls into GET, POST and path_parameters
- Remove binary encoding from ActionDispatch::Http::Request
- This way, we only raise an invalid encoding exception if the controller is not requesting
parameters in binary encoding

* Check if encoding is valid in ActionDispatch::Request#POST and raise BadRequest if invalid

* Fix multipart_params_test that has binary-encoded params containing invalid UTF-8 characters

* Address PR comments

* Pass action and controller to Request::Utils.set_binary_encoding

[Rafael Mendonça França + Adrianna Chang]

Allow `assert_recognizes` routing assertions to work on mounted root routes.

308 status code introduced in https://tools.ietf.org/html/rfc7538
preserves the request method unlike 301 status code which would convert
POST requests to GET.

This allows applications to safely upgrade to Rails 6.1 without
breaking tokens while the deploy is still being rolled out.

Prior to this commit, when multiple cookie domains were specified, the
first domain that was a substring of the request host was chosen.  This
allowed, for example, the "example.com" domain to be chosen when the
request host was "example.com.au" or even "myexample.com".

This commit ensures a domain is chosen only if it is equal to or is a
superdomain of the request host.

Fixes #37760.

When using an external build process (webpack, grunt) it's helpful for
rails to be able to serve those assets. Brotli has better compression
than gzip and should eventually replace it for static assets.

When using an external build process (webpack, grunt) it's helpful for
rails to be able to serve those assets. Brotli has better compression
than gzip and will eventually replace it for static assets.

This makes it safe to replace `reject { |k, v| v.nil? }` with `compact`,
even when the receiver is an `ActionController::Parameters`.

This matches Hash's behaviour for those methods.

Depends on newer Rack API.

This reverts commit fbf1d82e.

Rack decided to tolerate proxies which choose to attach ports to
X-Forwarded-For IPs by stripping the port:
https://github.com/rack/rack/pull/1251. Attaching a port is rare in the
wild but some proxies (notably Microsoft Azure's App Service) do it.

Without this patch, remote_ip will ignore X-Forwarded-For IPs with ports
attached and the return value is less likely to be useful.

Rails should do the same thing. The stripping logic is already available
in Rack::Request::Helpers, so change the X-Forwarded-For retrieval
method from ActionDispatch::Request#x_forwarded_for (which returns the
raw header) to #forwarded_for, which returns a stripped array of IP
addresses, or nil. There may be other benefits hiding in Rack's
implementation.

We can't call ips_from with an array (and legislating for that inside
ips_from doesn't appeal), so refactor out the bit we need to apply in
both cases (verifying the IP is acceptable to IPAddr and that it's not a
range) to a separate method called #sanitize_ips which reduces an array of
maybe-ips to an array of acceptable ones.

- We used the `fixture_path` before `file_fixture_path` was a thing,
  but now that we have the latter we should use it.

  `fixture_path` is solely used by Active Record so it seems wrong
  to be using that in ActionPack.

Motivation is twofold:

  * We are gradually removing `require_dependency` from the framework.

  * Let `helper` work if `config.add_autoload_paths_to_load_path` is
    disabled.
Co-authored-by: NJean Boussier <jean.boussier@gmail.com>

From #38142

[Rafael Mendonça França + Nick Soracco]

`url_for` will now use "https://" as the default protocol when
`Rails.application.config.force_ssl` is set to true.

Action Mailer already behaves this way, effectively.  This commit
extends that behavior application-wide.

Closes #23543.

Base64 strict-encoded CSRF tokens are not inherently websafe, which makes
them difficult to deal with. For example, the common practice of sending
the CSRF token to a browser in a client-readable cookie does not work properly
out of the box: the value has to be url-encoded and decoded to survive transport.

Now, we generate Base64 urlsafe-encoded CSRF tokens, which are inherently safe
to transport.  Validation accepts both urlsafe tokens, and strict-encoded tokens
for backwards compatibility.

In a distributed configuration like rolling update, users may observe
both old and new instances during deployment. Users may be served by a
new instance and then by an old instance.

That means when the server changes `cookies_serializer` from `:marshal`
to `:hybrid` or the server changes `use_authenticated_cookie_encryption`
from `false` to `true`, users may lose their sessions if they access the
server during deployment.

We added fallbacks to downgrade the cookie format when necessary during
deployment, ensuring compatibility on both old and new instances.

The `ActionDispatch::Session::MemcacheStore` is still vulnerable
given it requires the gem dalli to be updated as well.

CVE-2019-16782

Enabling `SameSite` cookie protection is an addition to CSRF protection,
where cookies won't be sent by browsers in cross-site POST requests when set to `:lax`.

`:strict` disables cookies being sent in cross-site GET or POST requests.

Passing `:none` disables this protection and is the same as previous versions albeit a `; SameSite=None` is appended to the cookie.

See upgrade instructions in config/initializers/new_framework_defaults_6_1.rb.

More info [here](https://tools.ietf.org/html/draft-west-first-party-cookies-07)

_NB: Technically already possible as Rack supports SameSite protection, this is to ensure it's applied to all cookies_

= This feature existed back in 2012 https://github.com/rails/rails/commit/5e7d6bba79393de0279917f93b82f3b7b176f4b5
  but got reverted with the incentive that there was a better approach.
  After discussions, we agreed that it's a useful feature for apps
  that have a really large set of routes.
Co-authored-by: NYehuda Katz <wycats@gmail.com>

in initializer block to the selenium driver
for non-headless browsers

* refactored browser options initialization.
* improved method names in AD::SystemTesting::Browser
* improved AD::SystemTest driver tests

This change will allow subscribers to the notification to report on anything related to the request that they might need

- `respond_to any` doesn't allow to specify a content type and
  the content type in the response will be based on the request
  format.

  ```ruby
    def my_action
      respond_to do |format|
        format.html { render(html: 'hello') }
        format.any { render(json: { foo: 'bar'}) }
      end
    end

    get('my_action.csv')
    # Before this patch, content type was `text/csv'
    # Ather this patch, content type is correctly set to whateve we did in the `format.any` block
  ```

  If the client specify the type of data he wants but the server
  doesn't know how to handle it and return plain text (or whatever)
  I don't think it make sense to falsey claim that we are returning
  a `text/csv` a response where in fact we are returning something else.

  Fix #37345

When a test method name includes a slash (e.g. `test "signup on the
/signup page"`) the screenshot is generated in the nested directory on
systems that use slash as a directory separator (e.g. a screenshot
called `signup_page.png` is generated within `failures_signup_on_the_`).

Nesting screenshots causes an issue with `tmp:clear` rake task:

```
== Removing old logs and tempfiles ==
rails aborted!
Errno::EISDIR: Is a directory @ apply2files - tmp/screenshots/failures_signup_on_the_
/var/lib/gems/2.5.0/gems/railties-5.2.3/lib/rails/tasks/tmp.rake:41:in `block (3 levels) in <top (required)>'
/var/lib/gems/2.5.0/gems/railties-5.2.3/lib/rails/commands/rake/rake_command.rb:23:in `block in perform'
...
Tasks: TOP => tmp:clear => tmp:screenshots:clear
```

While the error could be prevented by changing `tmp:clear` task, there's
no reason to generate deep directory structures for tests using slashes.

To prevent a similar problem on Windows, we'll also "sanitize"
backslashes.

Replacing the problamatic characters with dashes seems to be a safe
workaround, although dash is very arbitrary choice in this case.
Co-Authored-By: NLouis-Michel Couture <louim_1@hotmail.com>

Updated the setter to clear the value in the `@remote_ip` instance
variable before setting the header that the value is derived from in the
getter.

Allow setting a different log level per request.