An attack is simple, if the first CMS_recipientInfo is valid but the
second CMS_recipientInfo is chosen ciphertext. If the second
recipientInfo decodes to PKCS #1 v1.5 form plaintext, the correct
encryption key will be replaced by garbage, and the message cannot be
decoded, but if the RSA decryption fails, the correct encryption key is
used and the recipient will not notice the attack.

As a work around for this potential attack the length of the decrypted
key must be equal to the cipher default key length, in case the
certifiate is not given and all recipientInfo are tried out.

The old behaviour can be re-enabled in the CMS code by setting the
CMS_DEBUG_DECRYPT flag.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9777)

(cherry picked from commit 5840ed0cd1e6487d247efbc1a04136a41d7b3a37)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5689)

If a mem allocation failed we would ignore it. This commit fixes it to
always check.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5596)

Since return is inconsistent, I removed unnecessary parentheses and
unified them.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4541)

CLA: trivial
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4515)

unified them.
- return (0); -> return 0;
- return (1); -> return 1;
- return (-1); -> return -1;
Reviewed-by: NStephen Henson <steve@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4500)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NKurt Roeckx <kurt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1264)

Some misc return value checks
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

The PKCS7_dataFinal() function allocates a memory buffer but then fails
to free it on an error condition.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

This was done by the following
        find . -name '*.[ch]' | /tmp/pl
where /tmp/pl is the following three-line script:
        print unless $. == 1 && m@/\* .*\.[ch] \*/@;
        close ARGV if eof; # Close file to reset $.

And then some hand-editing of other files.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Continuing from previous commit ensure our style is consistent for malloc
return checks.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

RT#4081
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

CVE-2015-1790
Reviewed-by: NRich Salz <rsalz@openssl.org>

There are header files in crypto/ that are used by a number of crypto/
submodules.  Move those to crypto/include/internal and adapt the
affected source code and Makefiles.

The header files that got moved are:

crypto/cryptolib.h
crypto/md32_common.h
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

Don't check for NULL before calling OPENSSL_free
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Add OPENSSL_clear_free which merges cleanse and free.
(Names was picked to be similar to BN_clear_free, etc.)
Removed OPENSSL_freeFunc macro.
Fixed the small simple ones that are left:
        CRYPTO_free CRYPTO_free_locked OPENSSL_free_locked
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Don't check for NULL before calling a free routine.  This gets X509_.*free:
    x509_name_ex_free X509_policy_tree_free X509_VERIFY_PARAM_free
    X509_STORE_free X509_STORE_CTX_free X509_PKEY_free
    X509_OBJECT_free_contents X509_LOOKUP_free X509_INFO_free
Reviewed-by: NRichard Levitte <levitte@openssl.org>

EVP_.*free; this gets:
        EVP_CIPHER_CTX_free EVP_PKEY_CTX_free EVP_PKEY_asn1_free
        EVP_PKEY_asn1_set_free EVP_PKEY_free EVP_PKEY_free_it
        EVP_PKEY_meth_free; and also EVP_CIPHER_CTX_cleanup
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

This commit handles BIO_ACCEPT_free BIO_CB_FREE BIO_CONNECT_free
BIO_free BIO_free_all BIO_vfree
Reviewed-by: NMatt Caswell <matt@openssl.org>

Ensure RAND_bytes return value is checked correctly, and that we no longer
use RAND_pseudo_bytes.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Remove old M_ASN1_ macros and replace any occurences with the corresponding
function.

Remove d2i_ASN1_bytes, d2i_ASN1_SET, i2d_ASN1_SET: no longer used internally.
Reviewed-by: NRich Salz <rsalz@openssl.org>

In PKCS#7, the ASN.1 content component is optional.
This typically applies to inner content (detached signatures),
however we must also handle unexpected missing outer content
correctly.

This patch only addresses functions reachable from parsing,
decryption and verification, and functions otherwise associated
with reading potentially untrusted data.

Correcting all low-level API calls requires further work.

CVE-2015-0289

Thanks to Michal Zalewski (Google) for reporting this issue.
Reviewed-by: NSteve Henson <steve@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Keep one #if 0 but rename the symbol to be more descriptive of what
it's doing (you can disable support for old broken Netscape software).
Reviewed-by: NTim Hudson <tjh@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

Fix memory leaks in 'goto err' cases.

continue with symmetric decryption process to avoid leaking timing
information to an attacker.

Thanks to Ivan Nestlerode <inestlerode@us.ibm.com> for discovering
this issue. (CVE-2012-0884)