Basically we use EXFLAG_INVALID for all kinds of out of memory and
all kinds of parse errors in x509v3_cache_extensions.

[extended tests]
Reviewed-by: NTomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10756)

This reverts commit 68436f0a.

The OMC did not vote in favour of backporting this to 1.1.1, so this
change should be reverted.
Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11282)

Description:openssl 1.1.1d used bu libhapverify
Team:OTHERS
Feature or Bugfix:Feature
Binary Source:Yes, it is
PrivateCode(Yes/No):No

Change-Id: I8968f9c0f146b587da17a3e603bd04fb7b4c505b
Reviewed-on: http://mgit-tm.rnd.huawei.com/7842784Tested-by: Npublic jenkins <public_jenkins@notesmail.huawei.com>
Reviewed-by: Nhouyuezhou 00386575 <hou@huawei.com>
Reviewed-by: Nlinyibin 00246405 <linyibin@huawei.com>
Reviewed-by: Nweiping 00548480 <ping.wei@huawei.com>

Reviewed-by: NViktor Dukhovni <viktor@openssl.org>
GH: #10786
(cherry picked from commit b744f915ca8bb37631909728dd2529289bda8438)

Apart from public and internal header files, there is a third type called
local header files, which are located next to source files in the source
directory. Currently, they have different suffixes like

  '*_lcl.h', '*_local.h', or '*_int.h'

This commit changes the different suffixes to '*_local.h' uniformly.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9681)

Currently, there are two different directories which contain internal
header files of libcrypto which are meant to be shared internally:

While header files in 'include/internal' are intended to be shared
between libcrypto and libssl, the files in 'crypto/include/internal'
are intended to be shared inside libcrypto only.

To make things complicated, the include search path is set up in such
a way that the directive #include "internal/file.h" could refer to
a file in either of these two directoroes. This makes it necessary
in some cases to add a '_int.h' suffix to some files to resolve this
ambiguity:

  #include "internal/file.h"      # located in 'include/internal'
  #include "internal/file_int.h"  # located in 'crypto/include/internal'

This commit moves the private crypto headers from

  'crypto/include/internal'  to  'include/crypto'

As a result, the include directives become unambiguous

  #include "internal/file.h"       # located in 'include/internal'
  #include "crypto/file.h"         # located in 'include/crypto'

hence the superfluous '_int.h' suffixes can be stripped.

The files 'store_int.h' and 'store.h' need to be treated specially;
they are joined into a single file.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9681)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9847)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NTomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9639)

(cherry picked from commit c70e2ec33943d3bd46d3d9950f774307feda832b)

Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NTomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9614)

(cherry picked from commit cae665dfa6ccec743a7f39cf80676d7d2d787e56)

Fix a few places where calling ossl_isdigit does the wrong thing on
EBCDIC based systems.
Replaced with ascii_isdigit.
Reviewed-by: NTomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9556)

(cherry picked from commit 48102247ff513d4c57b40b19c1d432f37b9e4b02)

Cosmetic changes to use the X509_STORE_lock/unlock functions.
Renamed some ctx variables to store.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9366)

(cherry picked from commit 7a9abccde7b7a5e36efe42d89246f6cfd4d59f44)

x509 store's objects cache can get corrupted when using dir lookup
method in multithreaded application. Claim x509 store's lock when
accessing objects cache.

CLA: trivial
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NTomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NShane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9326)

(cherry picked from commit a161738a708b5e284a4714edc0c976606ea7cb26)

Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9059)

(cherry picked from commit 9fd6f7d1cd2a3c8e2bc69dcb8bde8406eb6c2623)

The member value.ptr is undefined for those ASN1 types.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9278)

(cherry picked from commit 94f4d58a87eac9c6fe4cb46b998656bd6d6f03a5)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9034)

If using a custom X509_LOOKUP_METHOD then calls to
X509_STORE_CTX_get_by_subject may crash due to an incorrectly initialised
X509_OBJECT being passed to the callback get_by_subject function.

Fixes #8673
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8698)

(cherry picked from commit b926f9deb3dc79d00f0a989370e95867516a3a17)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8347)

If the second PUBKEY is malformed there is use after free.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8122)

(cherry picked from commit 5dc40a83c74be579575a512b30d9c1e0364e6a7b)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8116)

(cherry picked from commit 53649022509129bce8036c8fb4978dbce9432a86)

Additionally avoid undefined behavior with
in-place memcpy in X509_CRL_digest.

Fixes #8099
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8112)

(cherry picked from commit a727627922b8a9ec6628ffaa2054b4b3833d674b)

The check_key_level() function currently fails when the public key
cannot be extracted from the certificate because its algorithm is not
supported.  However, the public key is not needed for the last
certificate in the chain.

This change moves the check for level 0 before the check for a
non-NULL public key.

For background, this is the TPM 1.2 endorsement key certificate.
I.e., this is a real application with millions of certificates issued.
The key is an RSA-2048 key.

The TCG (for a while) specified

     Public Key Algorithm: rsaesOaep

rather than the commonly used

     Public Key Algorithm: rsaEncryption

because the key is an encryption key rather than a signing key.
The X509 certificate parser fails to get the public key.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7906)

Also, some readers of the code find starting the count at 1 for EE
cert confusing (since RFC5280 counts only non-self-issued intermediate
CAs, but we also counted the leaf).  Therefore, never count the EE
cert, and adjust the path length comparison accordinly.  This may
be more clear to the reader.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(cherry picked from commit dc5831da59e9bfad61ba425d886a0b06ac160cd6)

At the bottom of https://tools.ietf.org/html/rfc5280#page-12 and
top of https://tools.ietf.org/html/rfc5280#page-13 (last paragraph
of above https://tools.ietf.org/html/rfc5280#section-3.3), we see:

   This specification covers two classes of certificates: CA
   certificates and end entity certificates.  CA certificates may be
   further divided into three classes: cross-certificates, self-issued
   certificates, and self-signed certificates.  Cross-certificates are
   CA certificates in which the issuer and subject are different
   entities.  Cross-certificates describe a trust relationship between
   the two CAs.  Self-issued certificates are CA certificates in which
   the issuer and subject are the same entity.  Self-issued certificates
   are generated to support changes in policy or operations.  Self-
   signed certificates are self-issued certificates where the digital
   signature may be verified by the public key bound into the
   certificate.  Self-signed certificates are used to convey a public
   key for use to begin certification paths.  End entity certificates
   are issued to subjects that are not authorized to issue certificates.

that the term "self-issued" is only applicable to CAs, not end-entity
certificates.  In https://tools.ietf.org/html/rfc5280#section-4.2.1.9
the description of path length constraints says:

   The pathLenConstraint field is meaningful only if the cA boolean is
   asserted and the key usage extension, if present, asserts the
   keyCertSign bit (Section 4.2.1.3).  In this case, it gives the
   maximum number of non-self-issued intermediate certificates that may
   follow this certificate in a valid certification path.  (Note: The
   last certificate in the certification path is not an intermediate
   certificate, and is not included in this limit.  Usually, the last
   certificate is an end entity certificate, but it can be a CA
   certificate.)

This makes it clear that exclusion of self-issued certificates from
the path length count applies only to some *intermediate* CA
certificates.  A leaf certificate whether it has identical issuer
and subject or whether it is a CA or not is never part of the
intermediate certificate count.  The handling of all leaf certificates
must be the same, in the case of our code to post-increment the
path count by 1, so that we ultimately reach a non-self-issued
intermediate it will be the first one (not zeroth) in the chain
of intermediates.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(cherry picked from commit ed422a2d0196ada0f5c1b6e296f4a4e5ed69577f)

Change all calls to getenv() inside libcrypto to use a new wrapper function
that use secure_getenv() if available and an issetugid then getenv if not.

CPU processor override flags are unchanged.

Extra checks for OPENSSL_issetugid() have been removed in favour of the
safe getenv.
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/7047)

(cherry picked from commit 5c39a55d04ea6e6f734b627a050b9e702788d50d)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7176)

gcc 4.6 (arguably erroneously) warns about our use of 'free' as
the name of a function parameter, when --strict-warnings is enabled:

crypto/x509/x509_meth.c: In function 'X509_LOOKUP_meth_set_free':
crypto/x509/x509_meth.c:61:12: error: declaration of 'free' shadows a global declaration [-Werror=shadow]
cc1: all warnings being treated as errors
make[1]: *** [crypto/x509/x509_meth.o] Error 1

(gcc 4.8 is fine with this code, as are newer compilers.)
Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6991)

In some cases it's about redundant check for return value, in some
cases it's about replacing check for -1 with comparison to 0.
Otherwise compiler might generate redundant check for <-1. [Even
formatting and readability fixes.]
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6860)

Documentation says "at most B<len> bytes will be written", which
formally doesn't prohibit zero. But if zero B<len> was passed, the
call to memcpy was bound to crash.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6860)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6538)

Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6430)

OpenSSL 1.1.0 made the X509_LOOKUP_METHOD structure opaque, so
applications that were previously able to define a custom lookup method
are not able to be ported.

This commit adds getters and setters for each of the current fields of
X509_LOOKUP_METHOD, along with getters and setters on several associated
opaque types (such as X509_LOOKUP and X509_OBJECT).
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6152)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6371)

Only check the CN against DNS name contraints if the
`X509_CHECK_FLAG_NEVER_CHECK_SUBJECT` flag is not set, and either the
certificate has no DNS subject alternative names or the
`X509_CHECK_FLAG_ALWAYS_CHECK_SUBJECT` flag is set.

Add pertinent documentation, and touch up some stale text about
name checks and DANE.
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NTim Hudson <tjh@openssl.org>

If the lengths of both names is 0 then don't attempt to do a memcmp.

Issue reported by Simon Friedberger, Robert Merget and Juraj Somorovsky.
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/6291)

A previous change of this function introduced a fragility when the
destination happens to be the same as the source.  Such alias isn't
recommended, but could still happen, for example in this kind of code:

    X509_NAME *subject = X509_get_issuer_name(x);

    /* ... some code passes ... */

    X509_set_issuer_name(x, subject);

Fixes #4710
Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6280)

Reviewed-by: NRich Salz <rsalz@openssl.org>
GH: #6208

Commit 5b748dea added this function which should have taken a const
argument.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6154)

Fixes #5061
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6139)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6145)