Description
-----------

Upon `EC_GROUP_new_from_ecparameters()` check if the parameters match any
of the built-in curves. If that is the case, return a new
`EC_GROUP_new_by_curve_name()` object instead of the explicit parameters
`EC_GROUP`.

This affects all users of `EC_GROUP_new_from_ecparameters()`:
- direct calls to `EC_GROUP_new_from_ecparameters()`
- direct calls to `EC_GROUP_new_from_ecpkparameters()` with an explicit
  parameters argument
- ASN.1 parsing of explicit parameters keys (as it eventually
  ends up calling `EC_GROUP_new_from_ecpkparameters()`)

A parsed explicit parameter key will still be marked with the
`OPENSSL_EC_EXPLICIT_CURVE` ASN.1 flag on load, so, unless
programmatically forced otherwise, if the key is eventually serialized
the output will still be encoded with explicit parameters, even if
internally it is treated as a named curve `EC_GROUP`.

Before this change, creating any `EC_GROUP` object using
`EC_GROUP_new_from_ecparameters()`, yielded an object associated with
the default generic `EC_METHOD`, but this was never guaranteed in the
documentation.
After this commit, users of the library that intentionally want to
create an `EC_GROUP` object using a specific `EC_METHOD` can still
explicitly call `EC_GROUP_new(foo_method)` and then manually set the
curve parameters using `EC_GROUP_set_*()`.

Motivation
----------

This has obvious performance benefits for the built-in curves with
specialized `EC_METHOD`s and subtle but important security benefits:
- the specialized methods have better security hardening than the
  generic implementations
- optional fields in the parameter encoding, like the `cofactor`, cannot
  be leveraged by an attacker to force execution of the less secure
  code-paths for single point scalar multiplication
- in general, this leads to reducing the attack surface

Check the manuscript at https://arxiv.org/abs/1909.01785 for an in depth
analysis of the issues related to this commit.

It should be noted that `libssl` does not allow to negotiate explicit
parameters (as per RFC 8422), so it is not directly affected by the
consequences of using explicit parameters that this commit fixes.
On the other hand, we detected external applications and users in the
wild that use explicit parameters by default (and sometimes using 0 as
the cofactor value, which is technically not a valid value per the
specification, but is tolerated by parsers for wider compatibility given
that the field is optional).
These external users of `libcrypto` are exposed to these vulnerabilities
and their security will benefit from this commit.

Related commits
---------------

While this commit is beneficial for users using built-in curves and
explicit parameters encoding for serialized keys, commit
b783beeadf6b80bc431e6f3230b5d5585c87ef87 (and its equivalents for the
1.0.2, 1.1.0 and 1.1.1 stable branches) fixes the consequences of the
invalid cofactor values more in general also for other curves
(CVE-2019-1547).

The following list covers commits in `master` that are related to the
vulnerabilities presented in the manuscript motivating this commit:

- d2baf88c43 [crypto/rsa] Set the constant-time flag in multi-prime RSA too
- 311e903d84 [crypto/asn1] Fix multiple SCA vulnerabilities during RSA key validation.
- b783beeadf [crypto/ec] for ECC parameters with NULL or zero cofactor, compute it
- 724339ff44 Fix SCA vulnerability when using PVK and MSBLOB key formats

Note that the PRs that contributed the listed commits also include other
commits providing related testing and documentation, in addition to
links to PRs and commits backporting the fixes to the 1.0.2, 1.1.0 and
1.1.1 branches.

This commit includes a partial backport of
https://github.com/openssl/openssl/pull/8555
(commit 8402cd5f75f8c2f60d8bd39775b24b03dd8b3b38)
for which the main author is Shane Lontis.

Responsible Disclosure
----------------------

This and the other issues presented in https://arxiv.org/abs/1909.01785
were reported by Cesar Pereida García, Sohaib ul Hassan, Nicola Tuveri,
Iaroslav Gridin, Alejandro Cabrera Aldaya and Billy Bob Brumley from the
NISEC group at Tampere University, FINLAND.

The OpenSSL Security Team evaluated the security risk for this
vulnerability as low, and encouraged to propose fixes using public Pull
Requests.

_______________________________________________________________________________
Co-authored-by: NShane Lontis <shane.lontis@oracle.com>

(Backport from https://github.com/openssl/openssl/pull/9808)
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9809)

This amends the entry added in a6186f39
with the relevant CVE.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9800)

Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NTomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: NNicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/9781)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9738)

Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9734)

(cherry picked from commit 46a9cc9451213039fd53f62733b2ccd04e853bb2)

Improve handling of low entropy at start up from /dev/urandom by waiting for
a read(2) call on /dev/random to succeed.  Once one such call has succeeded,
a shared memory segment is created and persisted as an indicator to other
processes that /dev/urandom is properly seeded.

This does not fully prevent against attacks weakening the entropy source.
An attacker who has control of the machine early in its boot sequence
could create the shared memory segment preventing detection of low entropy
conditions.  However, this is no worse than the current situation.

An attacker would also be capable of removing the shared memory segment
and causing seeding to reoccur resulting in a denial of service attack.
This is partially mitigated by keeping the shared memory alive for the
duration of the process's existence.  Thus, an attacker would not only need
to have called call shmctl(2) with the IPC_RMID command but the system
must subsequently enter a state where no instances of libcrypto exist in
any process.  Even one long running process will prevent this attack.

The System V shared memory calls used here go back at least as far as
Linux kernel 2.0.  Linux kernels 4.8 and later, don't have a reliable way
to detect that /dev/urandom has been properly seeded, so a failure is raised
for this case (i.e. the getentropy(2) call has already failed).
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9595)

[manual merge]

The macro TLS_MD_MASTER_SECRET_CONST is supposed to hold the ascii string
"extended master secret". On EBCDIC machines it actually contained the
value "extecded master secret"
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9430)

(cherry picked from commit c1a3f16f735057b45df1803d58f40e4e17b233e5)

CLA: trivial
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NShane Lontis <shane.lontis@oracle.com>
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9295)

Mingw config targets assumed that resulting programs and libraries are
installed in a Unix-like environment and the default installation
prefix was therefore set to '/usr/local'.

However, mingw programs are installed in a Windows environment, and
the installation directories should therefore have Windows defaults,
i.e. the same kind of defaults as the VC config targets.

A difficulty is, however, that a "cross compiled" build can't figure
out the system defaults from environment the same way it's done when
building "natively", so we have to fall back to hard coded defaults in
that case.

Tests can still be performed when cross compiled on a non-Windows
platform, since all tests only depend on the source and build
directory, and otherwise relies on normal local paths.

CVE-2019-1552
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9400)

The rand pool support allocates maximal sized buffers -- this is typically
12288 bytes in size.  These pools are allocated in secure memory which is a
scarse resource.  They are also allocated per DRBG of which there are up to two
per thread.

This change allocates 64 byte pools and grows them dynamically if required.
64 is chosen to be sufficiently large so that pools do not normally need to
grow.
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9428)

(cherry picked from commit a6a66e4511eec0f4ecc2943117a42b3723eb2222)

This avoids leaking bit 0 of the private key.

Backport-of: #9363
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/9435)

CLA: trivial
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/9275)

Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9118)

Reviewed-by: NMatt Caswell <matt@openssl.org>

Reviewed-by: NMatt Caswell <matt@openssl.org>

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9017)

Disabled by default

Fixes #8360
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8370)

(cherry picked from commit ac4033d658e4dc210ed4552b88069b57532ba3d7)

Fixes: #8737
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
GH: #8741
(cherry picked from commit 70b0b977f73cd70e17538af3095d18e0cf59132e)

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

This commit adds a dedicated function in `EC_METHOD` to access a modular
field inversion implementation suitable for the specifics of the
implemented curve, featuring SCA countermeasures.

The new pointer is defined as:
`int (*field_inv)(const EC_GROUP*, BIGNUM *r, const BIGNUM *a, BN_CTX*)`
and computes the multiplicative inverse of `a` in the underlying field,
storing the result in `r`.

Three implementations are included, each including specific SCA
countermeasures:
  - `ec_GFp_simple_field_inv()`, featuring SCA hardening through
    blinding.
  - `ec_GFp_mont_field_inv()`, featuring SCA hardening through Fermat's
    Little Theorem (FLT) inversion.
  - `ec_GF2m_simple_field_inv()`, that uses `BN_GF2m_mod_inv()` which
    already features SCA hardening through blinding.

From a security point of view, this also helps addressing a leakage
previously affecting conversions from projective to affine coordinates.

This commit also adds a new error reason code (i.e.,
`EC_R_CANNOT_INVERT`) to improve consistency between the three
implementations as all of them could fail for the same reason but
through different code paths resulting in inconsistent error stack
states.
Co-authored-by: NNicola Tuveri <nic.tuv@gmail.com>

(cherry picked from commit e0033efc30b0f00476bba8f0fa5512be5dc8a3f1)
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NNicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8262)

The original 1.1.1 design was to use SSL_CB_HANDSHAKE_START and
SSL_CB_HANDSHAKE_DONE to signal start/end of a post-handshake message
exchange in TLSv1.3. Unfortunately experience has shown that this confuses
some applications who mistake it for a TLSv1.2 renegotiation. This means
that KeyUpdate messages are not handled properly.

This commit removes the use of SSL_CB_HANDSHAKE_START and
SSL_CB_HANDSHAKE_DONE to signal the start/end of a post-handshake
message exchange. Individual post-handshake messages are still signalled in
the normal way.

This is a potentially breaking change if there are any applications already
written that expect to see these TLSv1.3 events. However, without it,
KeyUpdate is not currently usable for many applications.

Fixes #8069
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8096)

(cherry picked from commit 4af5836b55442f31795eff6c8c81ea7a1b8cf94b)

The commit 5dc40a83c74be579575a512b30d9c1e0364e6a7b forgot
to add a short description to the CHANGES file.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8144)

(cherry picked from commit b2aea0e3d9a15e30ebce8b6da213df4a3f346155)

When computing the end-point shared secret, don't take the
terminating NULL character into account.
Please note that this fix breaks interoperability with older
versions of OpenSSL, which are not fixed.

Fixes #7956
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7957)

(cherry picked from commit 09d62b336d9e2a11b330d45d4f0f3f37cbb0d674)

It turns out that the strictness that was implemented in
EVP_PKEY_asn1_new() (see Github openssl/openssl#6880) was badly placed
for some usages, and that it's better to do this check only when the
method is getting registered.

Fixes #7758
Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7847)

(cherry picked from commit a86003162138031137727147c9b642d99db434b1)

Also adds missing copyright boilerplate to util/mktar.sh
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7696)

(cherry picked from commit b42922ea2f605fd6c42faad1743fb27be5f7f1f3)

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NNicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/7664)

Replace ECDH_KDF_X9_62() with internal ecdh_KDF_X9_63()
Signed-off-by: NAntoine Salon <asalon@vmware.com>
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NNicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/7345)

(cherry picked from commit ffd89124bdfc9e69349492c3f15383bb35520a11)

In pull request #4328 the seeding of the DRBG via RAND_add()/RAND_seed()
was implemented by buffering the data in a random pool where it is
picked up later by the rand_drbg_get_entropy() callback. This buffer
was limited to the size of 4096 bytes.

When a larger input was added via RAND_add() or RAND_seed() to the DRBG,
the reseeding failed, but the error returned by the DRBG was ignored
by the two calling functions, which both don't return an error code.
As a consequence, the data provided by the application was effectively
ignored.

This commit fixes the problem by a more efficient implementation which
does not copy the data in memory and by raising the buffer the size limit
to INT32_MAX (2 gigabytes). This is less than the NIST limit of 2^35 bits
but it was chosen intentionally to avoid platform dependent problems
like integer sizes and/or signed/unsigned conversion.

Additionally, the DRBG is now less permissive on errors: In addition to
pushing a message to the openssl error stack, it enters the error state,
which forces a reinstantiation on next call.

Thanks go to Dr. Falko Strenzke for reporting this issue to the
openssl-security mailing list. After internal discussion the issue
has been categorized as not being security relevant, because the DRBG
reseeds automatically and is fully functional even without additional
randomness provided by the application.

Fixes #7381
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7382)

(cherry picked from commit 3064b55134434a0b2850f07eff57120f35bb269a)

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NBen Kaduk <kaduk@mit.edu>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7167)

Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7160)

Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7019)

Reviewed-by: NBen Kaduk <kaduk@mit.edu>
Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6741)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6949)

Signed-off-by: NPatrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6870)

Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6880)