* Add binary encoding logic into ActionDispatch::Request::Utils

Moving the logic to set binary encoding into ActionDispatch::Request::Utils
will allow us to encode from GET and POST in ActionDispatch::Request.

* Refactor binary encoding logic

- Move binary encoding calls into GET, POST and path_parameters
- Remove binary encoding from ActionDispatch::Http::Request
- This way, we only raise an invalid encoding exception if the controller is not requesting
parameters in binary encoding

* Check if encoding is valid in ActionDispatch::Request#POST and raise BadRequest if invalid

* Fix multipart_params_test that has binary-encoded params containing invalid UTF-8 characters

* Address PR comments

* Pass action and controller to Request::Utils.set_binary_encoding

[Rafael Mendonça França + Adrianna Chang]

https://github.com/rails/rails/commit/895134bcff294b05d6f6ebb101f60e3083ec5461#diff-7789e18ec0a61c6845755a257e6eb34fR3
https://github.com/rails/rails/commit/ee9e308f686ac1c7b0714671a1dcf1d96de87d70#diff-5e9551294914b338d923032fa904c6beR103
https://github.com/rails/rails/commit/5f63c771f7ce751fefd000470bf4ccab421988d7#diff-600d5368b55e46ed961abb4295977ac3R506
https://github.com/rails/rails/commit/48b6bacbc534d4be3fa89bc19ea83c357a20e598#diff-279ac5c088a3ee7e9f954bbc10d1b773R246
https://github.com/rails/rails/commit/49adb7f4c61de8cbb02c4a207dbd56093d99e795#diff-0cbe7171fdd7821fd5836896849141c0R111
https://github.com/rails/rails/commit/9bfe89e68ee61d9c8ed71ac98c783a7e47bdb608#diff-9c1c95b001e3680ba8a473cf23508b68R8
https://github.com/rails/rails/commit/a0f18e60900fc45eb3524ab3cdfe57be430d6016#diff-bef0ae2c16dacf77bfd2ba099af8a330R47
https://github.com/rails/rails/commit/8b4d344815655027d9f7584c0a59271dce8f1d5a#diff-09660f59aee7ec893ad609315e8843c5R18
https://github.com/rails/rails/commit/49d1b5a98dc50ca59bbfb817f90eb5c2c2645d30#diff-531e71a9d5fdc8181f38ce4416553694R107
https://github.com/rails/rails/commit/b674f04756e2e23fc18308319e4de8ff562575f4#diff-7521c0bb452244663b689e77658e63e3R98
https://github.com/rails/rails/commit/a9012af6883284a35bc5e6166507d429fe10482d#diff-60bdcf1e1954ec56d41fc4c7bd8d3e39R512
https://github.com/rails/rails/commit/6380aee182a6acb4f5062c5aa377d290e696e4b7#diff-5158c234d5e19bb1cffadcbc088400f1R262
https://github.com/rails/rails/commit/c7820d8124c854760a4d288334f185de2fb99446#diff-f4614e7ba8081c0c9e20fe64b6374228R60
https://github.com/rails/rails/commit/8df7ed3b88fc9f19446d7207a745a331893b81cd#diff-0495ed68f82d1bbde593ab5491eab24aR169
https://github.com/rails/rails/commit/6f94260ac67755fbc4d89e7dd9ba6be04abfa9e7#diff-7ecb6672f3fd37dfb79c4539395d3857R28
https://github.com/rails/rails/commit/03e44f93001db97953917e0a100c627e189e2be6#diff-e9234c4c6668852448059c0a35840bcaR185

Calling self in an action of a controller generates an endless stream of
characters, including the request object and all instances variables.
This can be frustrating when using a debugger in a controller and
accidentally calling `self` generates output for a couple of seconds.

This shortens inspect to only show the class name.

    MyController.new.inspect # => "#<MyController:0x00000000005028>"

Calling request in an action of a controller generates an endless stream of
characters, including the Rack app and middlewares.
This can be frustrating when using a debugger in a controller and
accidentally calling `request` generates output for a couple of seconds.

Inspect on ActionDispatch::Request is shortened to the most relevant
attributes and uses the same format as used for request in the logs:

    "#<ActionDispatch::Request POST "https://example.com/path/of/some/uri?q=1" for 1.2.3.4>"

Previously the same class, ActionView::TestCase::TestController, was
used to build a controller for every ActionView::TestCase class.

This caused issues when helpers/helper methods were set directly on the
controller (which from one test we seem to want to support).

This commit solves this by creating a new controller class for every
test case, which gives the controller a unique set of helpers to work
with.
Co-authored-by: NJohn Crepezzi <seejohnrun@github.com>

Bring back comment (mislaid in commit a5b2fff6) that explains why HTTP Basic Auth check uses `&`; it is useful for learners and mitigates `&` being accidentally replaced with `&&` one day.

Add documentation of Proc usage for SameSite property to configuring.md

Address PR comments
Co-authored-by: NAdrianna Chang <adrianna.chang@shopify.com>

Fixes #39937

Allow `assert_recognizes` routing assertions to work on mounted root routes.

* Reduce allocations in transition_table

* Use predicate symbol? instead of is_a?

`Pattern#from_sting` was introduced in bb207ea7 to support creating
patterns from strings in tests (removing a conditional in the initialize
method that had been there only for the sake of those tests).

`Pattern#build` was introduced in 947ebe9a, and was similarly used only
in tests.

My understanding is that [`Mapping#path`][path] is the only place where
we initialize a `Path::Pattern` in library code.

Since these two methods appear to be used only in tests, this
commit moves them out of the class and into a test helper.

My reasoning for doing this is that I am doing some performance work
that may involve a modification to how `Path::Pattern`s get initialized,
and I will have more confidence in that work if I know for sure that
these methods are test helpers that can be modified freely.

[path]: https://github.com/rails/rails/blob/master/actionpack/lib/action_dispatch/routing/mapper.rb#L192-L194

If request format is JSON, keys are converted to Strings, so we have to convert them to Symbols.

Before this commit we initialized all Symbols with the default regexp,
then later on reassigned any symbols descending from stars with either
their regexp from `@requirements` or the default greedy regexp.

With this commit we initialize all Symbols descending from Stars with
the greedy regexp at parse time. This allows us to get rid of the star
branch in path/pattern, since any regexps from `@requirements` will
already have been set in the symbol branch of this code.

This is essentially an alternate version of #38901. Getting rid of the
extra branch makes some performance work I am doing a bit easier, plus
it saves us a few method calls. Also the constant saves us from creating
the same regexp multiple times, and it is nice to give that regexp a
name.

The signature in Rack https://github.com/rack/rack/blob/master/lib/rack/session/abstract/id.rb#L415 expects a response as the second argument

Awhile back tenderlove and I worked to improve performance of
integration tests and remove controller tests. The second never
happened, we ended up soft deprecating them but never did that
completely.

Now that we're revisting that work we need a way to override these two
methods in tests so that we can convert tests in Rails to be integration
tests instead of controller tests. Splitting these two concerns into two
methods allows us to overwrite them to work for our needs while
refactoring the test harness code.

These methods are private because they should not be used by an
application or gems, they will be removed when the refactoring has been
completed.
Co-authored-by: NAaron Patterson <tenderlove@ruby-lang.org>

Along the same lines as #38901, #39903, and #39914, this commit removes
one pass through the journey ast in an attempt to improve application
boot time.

Before this commit, `Mapper#path` would iterate through the ast to alter
the regex for custom routes. With this commit we move the regex
alterations to initialize, where we were already iterating through the
ast.

The Benchmark for this change is similar to what we have been seeing in
the other PRs.

```
Warming up --------------------------------------
               after    13.121k i/100ms
              before     7.416k i/100ms
Calculating -------------------------------------
               after    128.469k (± 3.1%) i/s -    642.929k in 5.009391s
              before     76.561k (± 1.8%) i/s -    385.632k in 5.038677s

Comparison:
               after:   128469.4 i/s
              before:    76560.8 i/s - 1.68x  (± 0.00) slower

Calculating -------------------------------------
               after   160.000  memsize (     0.000  retained)
                         3.000  objects (     0.000  retained)
                         0.000  strings (     0.000  retained)
              before   360.000  memsize (     0.000  retained)
                         6.000  objects (     0.000  retained)
                         0.000  strings (     0.000  retained)

Comparison:
               after:        160 allocated
              before:        360 allocated - 2.25x more
```

Along the same lines as https://github.com/rails/rails/pull/38901,
this commit makes a small performance enhancement by iterating through
the ast once instead of twice when initializing a Mapping.

This stemmed from work to improve the boot time of an application with
3500+ routes. This patch only shaves off about 70ms from our boot time,
and about 25000 allocations, but every little bit counts!

Benchmark
---

```rb

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"

  git_source(:github) { |repo| "https://github.com/#{repo}.git" }

  # Activate the gem you are reporting the issue against.
  gem "rails", path: "/Users/daniel/Desktop/oss/rails/rails"
  gem "benchmark-ips"
  gem "benchmark-memory", require: "benchmark/memory"
end

require "action_controller/railtie"

class TestApp < Rails::Application
  config.root = __dir__
  config.hosts << "example.org"
  config.session_store :cookie_store, key: "cookie_store_key"
  secrets.secret_key_base = "secret_key_base"

  config.logger = Logger.new($stdout)
  Rails.logger = config.logger

  routes.draw do
    get("/*wildcard", to: "controller#index")
  end
end
```

With the benchmarking code inserted into Mapper#initialize:

```rb
after = -> {
  path_params = []
  wildcard_options = {}
  ast.each do |node|
    if node.symbol?
      path_params << node.to_sym
    elsif node.star? && formatted != false
      # Add a constraint for wildcard route to make it non-greedy and match the
      # optional format part of the route by default.
      wildcard_options[node.name.to_sym] ||= /.+?/
    end
  end

  wildcard_options.merge(options)
}

before = -> {
  path_params = ast.find_all(&:symbol?).map(&:to_sym)

  add_wildcard_options(options, formatted, ast)
}

puts "IPS"

Benchmark.ips do |x|
  x.report("before") { before.call }
  x.report("after") { after.call }
  x.compare!
end

puts "MEMORY"

Benchmark.memory do |x|
  x.report("before") { before.call }
  x.report("after") { after.call }
  x.compare!
end
```

The results are:

```
IPS
Warming up --------------------------------------
              before    14.352k i/100ms
               after    30.852k i/100ms
Calculating -------------------------------------
              before    135.675k (± 3.7%) i/s -    688.896k in   5.084368s
               after    288.126k (± 3.3%) i/s -      1.450M in   5.038072s

Comparison:
               after:   288126.4 i/s
              before:   135675.1 i/s - 2.12x  (± 0.00) slower

MEMORY
Calculating -------------------------------------
              before   360.000  memsize (     0.000  retained)
                         7.000  objects (     0.000  retained)
                         0.000  strings (     0.000  retained)
               after   200.000  memsize (     0.000  retained)
                         4.000  objects (     0.000  retained)
                         0.000  strings (     0.000  retained)

comparison:
               after:        200 allocated
              before:        360 allocated - 1.80x more  end
```

Co-authored-by: NEugene Kenny <elkenny@gmail.com>

Update wording for cookies, adding information about the http request/response header. Also added the bytes to the cookie size limit

308 status code introduced in https://tools.ietf.org/html/rfc7538
preserves the request method unlike 301 status code which would convert
POST requests to GET.

The controller test case is trying to encode json parameters in the same
way it encodes query parameters. However we need to encode json
parameters as json.

This was broken in #39534 because `generate_extras` is encoding
everything into query parameters. But json parameters are in the post
body and need to be treated differently than the query parameters.

Followup to #39651

Aaron Patterson <tenderlove@ruby-lang.org>

[CVE-2020-8185]

- In #39534, `Routing::RouteSet#generate_extras` (which get called)
  by ActionController::TestCase now go through `path_for` which strips
  out all key in the options hash that have the same name as the ones
  defined in `RESERVED_OPTIONS`.

Follow up to c07dff72.

Actually it is not the cop's fault, but we mistakenly use `^`, `$`, and
`\Z` in much places, the cop doesn't correct those conservatively.

I've checked all those usage and replaced all safe ones.

This allows applications to safely upgrade to Rails 6.1 without
breaking tokens while the deploy is still being rolled out.

Prior to this commit, when multiple cookie domains were specified, the
first domain that was a substring of the request host was chosen.  This
allowed, for example, the "example.com" domain to be chosen when the
request host was "example.com.au" or even "myexample.com".

This commit ensures a domain is chosen only if it is equal to or is a
superdomain of the request host.

Fixes #37760.

This PR converts the route params array into an object and moves the
error generation code into its own object as well. This is a refactoring
of internal code to make it easier to change and simplier for internals
to interface with. We have two new objects:

1) `RouteWithParams`

Previously `#generate` returned an array of parameterized parts for a
route. We have now turned this into an object with a `path` and a
`params methods` to be able to access the parts we need.

2) `MissingRoute`

`#generate` was also responsible for constructing the message for the
error. We've moved this code into the new `MissingRoute` object so it
does that work instead.

This change makes these methods reusable throughout the code and easier
for future refactoring we plan to do.
Co-authored-by: NAaron Patterson <aaron.patterson@gmail.com>

This reverts commit ea791e53.

reason: these ivars are modified within other synchronize blocks,
        and so need to be blocked from other threads for ensuring therad safety.
Co-authored-by: NRyuta Kamizono <kamipo@gmail.com>