diff options
author | Pete Bentley <prb@google.com> | 2021-09-23 10:49:18 +0100 |
---|---|---|
committer | Pete Bentley <prb@google.com> | 2021-09-27 19:05:23 +0100 |
commit | 8bb65ff676b006f67cccd75a8f4cd6f8a296409e (patch) | |
tree | 741bef31d3d7f6d72020097ad72fc7b7a1e0ec9c /linux-aarch64/crypto/fipsmodule | |
parent | f54ba3566f9139b088c0328ceccc69b7d5dab09d (diff) | |
download | boringssl-8bb65ff676b006f67cccd75a8f4cd6f8a296409e.tar.gz |
external/boringssl: Sync to 66e61c577d39e757bf491468f651461fa79fd5e1.
This includes the following changes:
https://boringssl.googlesource.com/boringssl/+log/c1571feb5faf5cce844354c63d0f3e842464bea3..66e61c577d39e757bf491468f651461fa79fd5e1
* Allow PKCS7_sign to work for signing kernel modules.
* Speed up constant-time base64 decoding.
* Unwind remnants of ASN1_TFLG_NDEF.
* acvptool: add CS3 support.
* Ignore SIGPIPE in the bssl tool.
* Add FIPS counters for AES-GCM in EVP_AEAD.
* Refresh fuzzer corpus for ECH draft-13.
* Fix the TLS fuzzers for ECH draft-13.
* Clarify that TLS sessions are not application sessions.
* Fix BN_prime_checks_for_validation to align with false-positive rate.
* Add maskHash to RSA_PSS_PARAMS for compat
* Remove ASN1_OP_I2D_* callbacks.
* Don't read it->funcs without checking it->itype.
* Reject missing required fields in i2d functions.
Update-Note: Structures with missing mandatory fields can no longer be
encoded. Note that, apart from the cases already handled by preceding
CLs, tasn_new.c will fill in non-NULL empty objects everywhere. The main
downstream impact I've seen of this particular change is in combination
with other bugs. Consider a caller that does:
* Reject -1 types in ASN1_TYPE and MSTRINGs when encoding.
Update-Note: A default-constructed object with a required ANY or
string-like CHOICE field cannot be encoded until the field is specified.
Note this affects i2d_X509: notBefore and notAfter are string-like
CHOICEs in OpenSSL.
* Correctly handle invalid ASN1_OBJECTs when encoding.
Update-Note: A default-constructed object with a required ASN1_OBJECT
field can no longer be encoded without initializing the ASN1_OBJECT.
Note this affects X509: the signature algorithm is an ASN1_OBJECT. Tests
that try to serialize an X509_new() must fill in all required fields.
(Production code is unlikely to be affected because the output was
unparsable anyway, while tests sometimes wouldn't notice.)
* Check for invalid CHOICE selectors in i2d functions.
Update-Note: An invalid CHOICE object (e.g. GENERAL_NAME) will now fail
when encoded, rather than be silently omitted. In particular, CHOICE
objects are default-initialized by tasn_new.c in an empty -1 state.
Structures containing a required CHOICE field can no longer be encoded
without filling in the CHOICE.
* Fix x509_name_ex_i2d error-handling.
* Correctly propagate errors in i2d functions.
Update-Note: Some error cases which were silently misinterpreted as
missing OPTIONAL elements will now cause encoding to fail.
* acvptool: add hmacDRBG support
* Check for __TRUSTY__ instead of TRUSTY.
* Update comment for ECH draft-13.
* Silence a GCC false positive warning.
* Switch to the new, simpler WHATWG URL formulation.
* Revert "Guard use of sdallocx with BORINGSSL_SDALLOCX"
* Fix calculation of draft-13 ECH confirmation signal.
* Update to draft-ietf-tls-esni-13.
* Reword SSL_get0_ech_name_override documentation.
* Remove SSL_set_verify_result.
* Make most of crypto/x509 opaque.
Update-Note: Patch cl/390055173 into the roll that includes this. This
unexports most of the X.509 structs, aligning with OpenSSL. Use the
accessor APIs instead.
* Remove V_ASN1_APP_CHOOSE.
Update-Note: V_ASN1_APP_CHOOSE is removed. I only found one use, which
has been fixed.
* Rewrite ASN1_PRINTABLE_type and add tests.
* Include SHA512-256 in EVP_get_digestbyname and EVP_MD_do_all.
* NUL is not printable.
Update-Note: ASN1_mbstring_ncopy will no longer allow PrintableString
for strings containing NUL.
* Make RSA_check_key more than 2x as fast.
* Benchmark RSA private key parsing.
* Work around yet another MSVC 2015 SFINAE bug.
* Avoid re-hashing the transcript multiple times.
* Make ssl_parse_extensions a little easier to use.
* Deduplicate our three ServerHello parsers.
* Merge in OpenSSL's X.509 corpus.
* Run X509_print in the certificate fuzzer.
* Fix some error-handling in i2v functions.
* Fix typo.
* OPENSSL_strndup should not return NULL given {NULL, 0}.
* Rewrite name constraints matching with CBS.
* Add some tests for name constraints.
* Fix i2v_GENERAL_NAME to not assume NUL terminated strings
* Do not rely on ASN1_STRING being NUL-terminated.
* Add a CBB_add_zeros helper.
* Linkify RFCs in documentation.
* Refer to RFCs consistently.
* runner: Test session IDs over 32 bytes.
* Process the TLS 1.3 cipher suite in one place.
* Guard use of sdallocx with BORINGSSL_SDALLOCX
* Bump minimum GCC version and note impending VS2015 deprecation.
* Add Span::first() and Span::last().
* Simplify built-in BIOs slightly.
* Fix some error returns from SSL_read and SSL_write.
* Fix negative ENUMERATED values in multi-strings.
* Add a test for ASN1_mbstring_copy and clean up.
* Remove ASN1_TFLG_SET_ORDER.
* Fix ASN1_STRING_print_ex with negative integers.
* Check i2d_ASN1_TYPE's return value in ASN1_STRING_print_ex.
* Document ASN.1 printing functions.
* Move some ASN1 printing functions to crypto/asn1.
* Move a_strex.c back to asn1, split X509_NAME bits out.
* Unwind io_ch abstraction in print functions.
* Implement ASN1_STRING_print_ex_fp, etc., with file BIOs.
* Remove OPENSSL_NO_FP_API ifdefs.
* Move X509_ALGOR to x509.h.
* Unexport BIT_STRING_BITNAME.
* Unexport ub_* constants.
Update-Note: Removed some unnamespaced constants.
* Always use an ASN1_STRING_TABLE global mask of UTF8String.
Update-Note: The global mask for ASN1_STRING_set_by_NID is now always
UTF-8. Callers that want another type should reconsider and, if UTF-8 is
still unsuitable, just pass the actual desired type into
ASN1_mbstring_copy, X509_NAME_ENTRY_set_data, etc
* Document ASN1_mbstring_copy.
* Update ghashv8-armx.pl from upstream.
* Align with upstream on 'close STDOUT' lines.
* Avoid double-expanding variables in CMake.
* Reject years outside 0000-9999 in ASN1_GENERALIZEDTIME_adj.
* Add some tests for time_t to ASN1_TIME conversions.
* Remove ASN1_STRING_FLAG_MSTRING.
Update-Note: ASN1_STRING_FLAG_MSTRING is no longer defined and
X509_time_adj_ex now behaves more predictably. Callers that actually
wanted to lock to a specific type should call ASN1_UTCTIME_adj or
ASN1_GENERALIZEDTIME_adj instead.
* Document another batch of functions.
* Clarify BIO_new_mum_buf's lifetime rules.
* generate_ech.cc: include needed headers
* Don't overread in poly_Rq_mul
* acvp: recognise another style of JSON.
* Revert "Revert "Revert "Disable check that X.509 extensions implies v3."""
Test: atest CtsLibcoreTestCases CtsLibcoreOkHttpTestCases
Change-Id: I4f2228ef815ded0599322186ab7bad49ab1bb5af
Diffstat (limited to 'linux-aarch64/crypto/fipsmodule')
-rw-r--r-- | linux-aarch64/crypto/fipsmodule/ghashv8-armx64.S | 326 |
1 files changed, 325 insertions, 1 deletions
diff --git a/linux-aarch64/crypto/fipsmodule/ghashv8-armx64.S b/linux-aarch64/crypto/fipsmodule/ghashv8-armx64.S index 62e58842..9480a38a 100644 --- a/linux-aarch64/crypto/fipsmodule/ghashv8-armx64.S +++ b/linux-aarch64/crypto/fipsmodule/ghashv8-armx64.S @@ -15,6 +15,7 @@ #endif #include <openssl/arm_arch.h> +#if __ARM_MAX_ARCH__>=7 .text .arch armv8-a+crypto .globl gcm_init_v8 @@ -65,8 +66,48 @@ gcm_init_v8: ext v17.16b,v22.16b,v22.16b,#8 //Karatsuba pre-processing eor v17.16b,v17.16b,v22.16b ext v21.16b,v16.16b,v17.16b,#8 //pack Karatsuba pre-processed - st1 {v21.2d,v22.2d},[x0] //store Htable[1..2] + st1 {v21.2d,v22.2d},[x0],#32 //store Htable[1..2] + //calculate H^3 and H^4 + pmull v0.1q,v20.1d, v22.1d + pmull v5.1q,v22.1d,v22.1d + pmull2 v2.1q,v20.2d, v22.2d + pmull2 v7.1q,v22.2d,v22.2d + pmull v1.1q,v16.1d,v17.1d + pmull v6.1q,v17.1d,v17.1d + ext v16.16b,v0.16b,v2.16b,#8 //Karatsuba post-processing + ext v17.16b,v5.16b,v7.16b,#8 + eor v18.16b,v0.16b,v2.16b + eor v1.16b,v1.16b,v16.16b + eor v4.16b,v5.16b,v7.16b + eor v6.16b,v6.16b,v17.16b + eor v1.16b,v1.16b,v18.16b + pmull v18.1q,v0.1d,v19.1d //1st phase + eor v6.16b,v6.16b,v4.16b + pmull v4.1q,v5.1d,v19.1d + + ins v2.d[0],v1.d[1] + ins v7.d[0],v6.d[1] + ins v1.d[1],v0.d[0] + ins v6.d[1],v5.d[0] + eor v0.16b,v1.16b,v18.16b + eor v5.16b,v6.16b,v4.16b + + ext v18.16b,v0.16b,v0.16b,#8 //2nd phase + ext v4.16b,v5.16b,v5.16b,#8 + pmull v0.1q,v0.1d,v19.1d + pmull v5.1q,v5.1d,v19.1d + eor v18.16b,v18.16b,v2.16b + eor v4.16b,v4.16b,v7.16b + eor v20.16b, v0.16b,v18.16b //H^3 + eor v22.16b,v5.16b,v4.16b //H^4 + + ext v16.16b,v20.16b, v20.16b,#8 //Karatsuba pre-processing + ext v17.16b,v22.16b,v22.16b,#8 + eor v16.16b,v16.16b,v20.16b + eor v17.16b,v17.16b,v22.16b + ext v21.16b,v16.16b,v17.16b,#8 //pack Karatsuba pre-processed + st1 {v20.2d,v21.2d,v22.2d},[x0] //store Htable[3..5] ret .size gcm_init_v8,.-gcm_init_v8 .globl gcm_gmult_v8 @@ -118,6 +159,8 @@ gcm_gmult_v8: .align 4 gcm_ghash_v8: AARCH64_VALID_CALL_TARGET + cmp x3,#64 + b.hs .Lgcm_ghash_v8_4x ld1 {v0.2d},[x0] //load [rotated] Xi //"[rotated]" means that //loaded value would have @@ -244,9 +287,290 @@ gcm_ghash_v8: ret .size gcm_ghash_v8,.-gcm_ghash_v8 +.type gcm_ghash_v8_4x,%function +.align 4 +gcm_ghash_v8_4x: +.Lgcm_ghash_v8_4x: + ld1 {v0.2d},[x0] //load [rotated] Xi + ld1 {v20.2d,v21.2d,v22.2d},[x1],#48 //load twisted H, ..., H^2 + movi v19.16b,#0xe1 + ld1 {v26.2d,v27.2d,v28.2d},[x1] //load twisted H^3, ..., H^4 + shl v19.2d,v19.2d,#57 //compose 0xc2.0 constant + + ld1 {v4.2d,v5.2d,v6.2d,v7.2d},[x2],#64 +#ifndef __ARMEB__ + rev64 v0.16b,v0.16b + rev64 v5.16b,v5.16b + rev64 v6.16b,v6.16b + rev64 v7.16b,v7.16b + rev64 v4.16b,v4.16b +#endif + ext v25.16b,v7.16b,v7.16b,#8 + ext v24.16b,v6.16b,v6.16b,#8 + ext v23.16b,v5.16b,v5.16b,#8 + + pmull v29.1q,v20.1d,v25.1d //H·Ii+3 + eor v7.16b,v7.16b,v25.16b + pmull2 v31.1q,v20.2d,v25.2d + pmull v30.1q,v21.1d,v7.1d + + pmull v16.1q,v22.1d,v24.1d //H^2·Ii+2 + eor v6.16b,v6.16b,v24.16b + pmull2 v24.1q,v22.2d,v24.2d + pmull2 v6.1q,v21.2d,v6.2d + + eor v29.16b,v29.16b,v16.16b + eor v31.16b,v31.16b,v24.16b + eor v30.16b,v30.16b,v6.16b + + pmull v7.1q,v26.1d,v23.1d //H^3·Ii+1 + eor v5.16b,v5.16b,v23.16b + pmull2 v23.1q,v26.2d,v23.2d + pmull v5.1q,v27.1d,v5.1d + + eor v29.16b,v29.16b,v7.16b + eor v31.16b,v31.16b,v23.16b + eor v30.16b,v30.16b,v5.16b + + subs x3,x3,#128 + b.lo .Ltail4x + + b .Loop4x + +.align 4 +.Loop4x: + eor v16.16b,v4.16b,v0.16b + ld1 {v4.2d,v5.2d,v6.2d,v7.2d},[x2],#64 + ext v3.16b,v16.16b,v16.16b,#8 +#ifndef __ARMEB__ + rev64 v5.16b,v5.16b + rev64 v6.16b,v6.16b + rev64 v7.16b,v7.16b + rev64 v4.16b,v4.16b +#endif + + pmull v0.1q,v28.1d,v3.1d //H^4·(Xi+Ii) + eor v16.16b,v16.16b,v3.16b + pmull2 v2.1q,v28.2d,v3.2d + ext v25.16b,v7.16b,v7.16b,#8 + pmull2 v1.1q,v27.2d,v16.2d + + eor v0.16b,v0.16b,v29.16b + eor v2.16b,v2.16b,v31.16b + ext v24.16b,v6.16b,v6.16b,#8 + eor v1.16b,v1.16b,v30.16b + ext v23.16b,v5.16b,v5.16b,#8 + + ext v17.16b,v0.16b,v2.16b,#8 //Karatsuba post-processing + eor v18.16b,v0.16b,v2.16b + pmull v29.1q,v20.1d,v25.1d //H·Ii+3 + eor v7.16b,v7.16b,v25.16b + eor v1.16b,v1.16b,v17.16b + pmull2 v31.1q,v20.2d,v25.2d + eor v1.16b,v1.16b,v18.16b + pmull v30.1q,v21.1d,v7.1d + + pmull v18.1q,v0.1d,v19.1d //1st phase of reduction + ins v2.d[0],v1.d[1] + ins v1.d[1],v0.d[0] + pmull v16.1q,v22.1d,v24.1d //H^2·Ii+2 + eor v6.16b,v6.16b,v24.16b + pmull2 v24.1q,v22.2d,v24.2d + eor v0.16b,v1.16b,v18.16b + pmull2 v6.1q,v21.2d,v6.2d + + eor v29.16b,v29.16b,v16.16b + eor v31.16b,v31.16b,v24.16b + eor v30.16b,v30.16b,v6.16b + + ext v18.16b,v0.16b,v0.16b,#8 //2nd phase of reduction + pmull v0.1q,v0.1d,v19.1d + pmull v7.1q,v26.1d,v23.1d //H^3·Ii+1 + eor v5.16b,v5.16b,v23.16b + eor v18.16b,v18.16b,v2.16b + pmull2 v23.1q,v26.2d,v23.2d + pmull v5.1q,v27.1d,v5.1d + + eor v0.16b,v0.16b,v18.16b + eor v29.16b,v29.16b,v7.16b + eor v31.16b,v31.16b,v23.16b + ext v0.16b,v0.16b,v0.16b,#8 + eor v30.16b,v30.16b,v5.16b + + subs x3,x3,#64 + b.hs .Loop4x + +.Ltail4x: + eor v16.16b,v4.16b,v0.16b + ext v3.16b,v16.16b,v16.16b,#8 + + pmull v0.1q,v28.1d,v3.1d //H^4·(Xi+Ii) + eor v16.16b,v16.16b,v3.16b + pmull2 v2.1q,v28.2d,v3.2d + pmull2 v1.1q,v27.2d,v16.2d + + eor v0.16b,v0.16b,v29.16b + eor v2.16b,v2.16b,v31.16b + eor v1.16b,v1.16b,v30.16b + + adds x3,x3,#64 + b.eq .Ldone4x + + cmp x3,#32 + b.lo .Lone + b.eq .Ltwo +.Lthree: + ext v17.16b,v0.16b,v2.16b,#8 //Karatsuba post-processing + eor v18.16b,v0.16b,v2.16b + eor v1.16b,v1.16b,v17.16b + ld1 {v4.2d,v5.2d,v6.2d},[x2] + eor v1.16b,v1.16b,v18.16b +#ifndef __ARMEB__ + rev64 v5.16b,v5.16b + rev64 v6.16b,v6.16b + rev64 v4.16b,v4.16b +#endif + + pmull v18.1q,v0.1d,v19.1d //1st phase of reduction + ins v2.d[0],v1.d[1] + ins v1.d[1],v0.d[0] + ext v24.16b,v6.16b,v6.16b,#8 + ext v23.16b,v5.16b,v5.16b,#8 + eor v0.16b,v1.16b,v18.16b + + pmull v29.1q,v20.1d,v24.1d //H·Ii+2 + eor v6.16b,v6.16b,v24.16b + + ext v18.16b,v0.16b,v0.16b,#8 //2nd phase of reduction + pmull v0.1q,v0.1d,v19.1d + eor v18.16b,v18.16b,v2.16b + pmull2 v31.1q,v20.2d,v24.2d + pmull v30.1q,v21.1d,v6.1d + eor v0.16b,v0.16b,v18.16b + pmull v7.1q,v22.1d,v23.1d //H^2·Ii+1 + eor v5.16b,v5.16b,v23.16b + ext v0.16b,v0.16b,v0.16b,#8 + + pmull2 v23.1q,v22.2d,v23.2d + eor v16.16b,v4.16b,v0.16b + pmull2 v5.1q,v21.2d,v5.2d + ext v3.16b,v16.16b,v16.16b,#8 + + eor v29.16b,v29.16b,v7.16b + eor v31.16b,v31.16b,v23.16b + eor v30.16b,v30.16b,v5.16b + + pmull v0.1q,v26.1d,v3.1d //H^3·(Xi+Ii) + eor v16.16b,v16.16b,v3.16b + pmull2 v2.1q,v26.2d,v3.2d + pmull v1.1q,v27.1d,v16.1d + + eor v0.16b,v0.16b,v29.16b + eor v2.16b,v2.16b,v31.16b + eor v1.16b,v1.16b,v30.16b + b .Ldone4x + +.align 4 +.Ltwo: + ext v17.16b,v0.16b,v2.16b,#8 //Karatsuba post-processing + eor v18.16b,v0.16b,v2.16b + eor v1.16b,v1.16b,v17.16b + ld1 {v4.2d,v5.2d},[x2] + eor v1.16b,v1.16b,v18.16b +#ifndef __ARMEB__ + rev64 v5.16b,v5.16b + rev64 v4.16b,v4.16b +#endif + + pmull v18.1q,v0.1d,v19.1d //1st phase of reduction + ins v2.d[0],v1.d[1] + ins v1.d[1],v0.d[0] + ext v23.16b,v5.16b,v5.16b,#8 + eor v0.16b,v1.16b,v18.16b + + ext v18.16b,v0.16b,v0.16b,#8 //2nd phase of reduction + pmull v0.1q,v0.1d,v19.1d + eor v18.16b,v18.16b,v2.16b + eor v0.16b,v0.16b,v18.16b + ext v0.16b,v0.16b,v0.16b,#8 + + pmull v29.1q,v20.1d,v23.1d //H·Ii+1 + eor v5.16b,v5.16b,v23.16b + + eor v16.16b,v4.16b,v0.16b + ext v3.16b,v16.16b,v16.16b,#8 + + pmull2 v31.1q,v20.2d,v23.2d + pmull v30.1q,v21.1d,v5.1d + + pmull v0.1q,v22.1d,v3.1d //H^2·(Xi+Ii) + eor v16.16b,v16.16b,v3.16b + pmull2 v2.1q,v22.2d,v3.2d + pmull2 v1.1q,v21.2d,v16.2d + + eor v0.16b,v0.16b,v29.16b + eor v2.16b,v2.16b,v31.16b + eor v1.16b,v1.16b,v30.16b + b .Ldone4x + +.align 4 +.Lone: + ext v17.16b,v0.16b,v2.16b,#8 //Karatsuba post-processing + eor v18.16b,v0.16b,v2.16b + eor v1.16b,v1.16b,v17.16b + ld1 {v4.2d},[x2] + eor v1.16b,v1.16b,v18.16b +#ifndef __ARMEB__ + rev64 v4.16b,v4.16b +#endif + + pmull v18.1q,v0.1d,v19.1d //1st phase of reduction + ins v2.d[0],v1.d[1] + ins v1.d[1],v0.d[0] + eor v0.16b,v1.16b,v18.16b + + ext v18.16b,v0.16b,v0.16b,#8 //2nd phase of reduction + pmull v0.1q,v0.1d,v19.1d + eor v18.16b,v18.16b,v2.16b + eor v0.16b,v0.16b,v18.16b + ext v0.16b,v0.16b,v0.16b,#8 + + eor v16.16b,v4.16b,v0.16b + ext v3.16b,v16.16b,v16.16b,#8 + + pmull v0.1q,v20.1d,v3.1d + eor v16.16b,v16.16b,v3.16b + pmull2 v2.1q,v20.2d,v3.2d + pmull v1.1q,v21.1d,v16.1d + +.Ldone4x: + ext v17.16b,v0.16b,v2.16b,#8 //Karatsuba post-processing + eor v18.16b,v0.16b,v2.16b + eor v1.16b,v1.16b,v17.16b + eor v1.16b,v1.16b,v18.16b + + pmull v18.1q,v0.1d,v19.1d //1st phase of reduction + ins v2.d[0],v1.d[1] + ins v1.d[1],v0.d[0] + eor v0.16b,v1.16b,v18.16b + + ext v18.16b,v0.16b,v0.16b,#8 //2nd phase of reduction + pmull v0.1q,v0.1d,v19.1d + eor v18.16b,v18.16b,v2.16b + eor v0.16b,v0.16b,v18.16b + ext v0.16b,v0.16b,v0.16b,#8 + +#ifndef __ARMEB__ + rev64 v0.16b,v0.16b +#endif + st1 {v0.2d},[x0] //write out Xi + + ret +.size gcm_ghash_v8_4x,.-gcm_ghash_v8_4x .byte 71,72,65,83,72,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,60,97,112,112,114,111,64,111,112,101,110,115,115,108,46,111,114,103,62,0 .align 2 .align 2 #endif +#endif #endif // !OPENSSL_NO_ASM .section .note.GNU-stack,"",%progbits |