aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-01-19pl/math: Add vector/Neon tanJoe Ramsay
New routine uses a similar technique to the single-precision Neon routine, but with an extra reduction to pi/8 using the double-angle formula. It is accurate to 3.5 ULP.
2023-01-10string: Compile memcpy-sve.S for aarch64 if compiler supports itJake Weinstein
This is a partial revert of b7e368fb. If SVE assembly is guarded by __ARM_FEATURE_SVE, it cannot build when SVE is not enabled by the build system. This is ok on AOR, but because Android (bionic) uses ifuncs to select the appropriate assembly at runtime, these need to compile regardless of if the target actually supports the instructions. Check for AArch64 and GCC >= 8 or Clang >= 5 so that SVE is not used on compilers that do not support it. This condition will always be true on future builds of Android for AArch64.
2023-01-10string: Optimize strcpyWilco Dijkstra
Optimize strcpy main loop - large strings are ~22% faster.
2023-01-10string: Improve strrchr-mteWilco Dijkstra
Use shrn for narrowing the mask which simplifies code. Unroll the strchr search loop which improves performance on large strings.
2023-01-09pl/math: Fix benchmark entries for SVE bivariate functionsPierre Blanchard
Variant was wrongly set in structures used to benchmark SVE functions. Before this change only half of the lanes were set as expected. Also reformat for ease of reading.
2023-01-06pl/math: Update copyright yearsJoe Ramsay
All files in pl/math updated to 2023.
2023-01-05Rewrite two abs masks as literalsJoe Ramsay
These were technically undefined behaviour - they have been rewritten without the shift so that their type is unsigned int by default.
2023-01-05pl/math: Add vector/Neon acoshJoe Ramsay
New routine is based on a vector implementation from log1p, which has been reused (with some modification for improved accuracy close to 0) from Neon atanh. Accurate to 3.5 ULP.
2023-01-05pl/math: Add vector/Neon acoshfJoe Ramsay
New routine uses inlined log1pf helper, and is accurate to 3.1 ULP (2.8 ULP if fp exceptions are enabled).
2023-01-05pl/math: Add scalar & vector/Neon tanhJoe Ramsay
New routines use the same algorithm, reliant on a modified version of expm1, and are accurate to 3 ULP.
2022-12-30pl/math: Add vector/SVE log2Pierre Blanchard
The new SVE implementation is a direct port of Neon log2, and is accurate to 2.58 ULPs. Update error threshold and comments for Neon log2 too, new approximate argmax but same threshold.
2022-12-30pl/math: Add vector/SVE log2fPierre Blanchard
New SVE routine is an SVE port of the Neon algorithm and is accurate to 2.48 ULPs.
2022-12-22pl/math: Add scalar & vector/Neon atanhJoe Ramsay
New routines are both based on existing log1p routines. Scalar is accurate to 3 ULP, Neon to 3.5 ULP. Both set fp exceptions correctly regardless of build config.
2022-12-22pl/math: Add scalar atan and set fenv in Neon atanJoe Ramsay
The simplest way to set fenv in Neon atan is by using a scalar fallback for under/overflow cases, however this routine did not have a scalar counterpart so we add a new one, based on the same algorithm and polynomial as the vector variants, and accurate to 2.5 ULP. This is now used as the fallback for all lanes, when any lane of the Neon input is special.
2022-12-22pl/math: Fix fp exceptions in Neon sinhf and sinhJoe Ramsay
Both routines previously relied on the vector expm1(f) routine exposed by the library, which depended on WANT_SIMD_EXCEPT for its fenv behaviour, however both routines were expected to always trigger fp exceptions correctly. To remedy this, both routines now use an inlined helper for expm1 (reused from vector tanhf in the case of sinhf), and special-case small input as well as large when WANT_SIMD_EXCEPT is enabled.
2022-12-20Correct exit code from runulp.shJoe Ramsay
The pipe prevented FAILs and PASSs being counted properly - the while read loop has been rewritten without a pipe, as it was prior to the changes here. fenv checking is temporarily disabled in Neon sinh and sinhf, as they do not get it right. This will be re-enabled once they have been fixed.
2022-12-20pl/math: Update ULP threshold for SVE erfPierre Blanchard
Updated comment and test threshold.
2022-12-20pl/math: Add scalar atanf and set fenv in Neon atanfJoe Ramsay
The simplest way to set fenv in Neon atanf is by using a scalar fallback to under/overflow cases, however this routine did not have a scalar counterpart so we add a new one, based on the same algorithm and polynomial as the vector variants, and accurate to 2.9 ULP. This is now used as the fallback for all lanes, when any lane of the Neon input is special.
2022-12-20pl/math: Add scalar & vector/Neon cbrtJoe Ramsay
New routines use the same algorithm, with simplified argument reduction and recombination in the vector variant. Both are accurate to 2 ULP.
2022-12-19pl/math: Update ULP threshold for Neon asinhJoe Ramsay
New max observed - updated filenames, comments and runulp threshold.
2022-12-19pl/math: Replace WANT_ERRNO with WANT_SIMD_EXCEPT for Neon fenvJoe Ramsay
We were previously misusing the WANT_ERRNO build flag. This is now replaced everywhere appropriate with WANT_SIMD_EXCEPT. A small number of vector routines get fp exceptions right with no modification - the tests have been updated to track this.
2022-12-19pl/math: Improve vector/Neon log2fPierre Blanchard
A new implementation based on the same approach as Neon logf, that is accurate to 2.48 ULPs. Flags set correctly regardless of WANT_ERRNO.
2022-12-15pl/math: Move test intervals to routine source filesJoe Ramsay
To conclude the work on simplifying the runulp.sh script, a new macro has been introduced to specify the intervals in which a routine should be tested in the routine source. This is eventually consumed by runulp.sh.
2022-12-15pl/math: Move fenv expectations out of runulp.shJoe Ramsay
Introduces a new macro, similar to how ULP thresholds are now handled, that emits a list of routines which are expected to correctly trigger fenv exceptions, to be consumed by runulp.sh. All scalar routines are expected to do so. A small number of Neon routines are also expected to, dependent on WANT_ERRNO.
2022-12-15pl/math: Move ULP limits to routine source filesJoe Ramsay
Introduces a new set of macros and Make rules for mechanically generating a list of ULP limits for each routine, to be consumed by runulp.sh. This removes the need to maintain long lists of thresholds in runulp.sh.
2022-12-15pl/math: Auto-generate mathbench and ulp headersJoe Ramsay
Instead of maintaining three separate lists of routines, which are cumbersome and prone to merge conflicts, we provide a new macro, PL_SIG, which by some preprocessor machinery outputs the lists in the required format (macro formats have been changed very slightly to make the generation simpler). Only routines with simple signatures are handled - binary functions still need mathbench wrappers defined manually. As well, routines with non-standard references (i.e. powi/powk) still need entries and wrappers manually defined.
2022-12-13pl/math: Set fenv flags in Neon log1pJoe Ramsay
New behaviour is hidden behind WANT_ERRNO config option.
2022-12-13pl/math: Set fenv flags in Neon tanfJoe Ramsay
New behaviour is hidden behind WANT_ERRNO config option.
2022-12-13pl/math: Set fenv flags in Neon log2fJoe Ramsay
Flags set correctly regardless of WANT_ERRNO.
2022-12-13pl/math: Update ULP threshold for SVE atan2Pierre Blanchard
Test threshold fixed.
2022-12-13pl/math: Set fenv flags in Neon log1pfJoe Ramsay
New behaviour is hidden behind WANT_ERRNO config option.
2022-12-09pl/math: Add polynomial helpersJoe Ramsay
Add macros for simplifying polynomial evaluation using either Horner, pairwise Horner or Estrin. Several routines have been modified to use the new helpers. Readability is improved slightly, and we expect that this will make prototyping new routines simpler.
2022-12-09pl/math/test: Simplify runulp.shJoe Ramsay
Small simplification - pl routines do not support different rounding modes, so there is no need to support them in runulp.sh. As a result we can also remove Ldir.
2022-12-08pl/math: Fix fenv in asinhJoe Ramsay
Special lanes were not being properly masked when a lane was tiny. This is now fixed.
2022-12-08pl/math: Fix vector/SVE erfPierre Blanchard
Fixing a bug that resulted in potentially random results in boring domain by saturating index at an appropriate value.
2022-12-07math: Set fenv exceptions for several Neon routinesJoe Ramsay
In most cases, we mask lanes which should not trigger exceptions with a neutral value, then let the existing special-case handler fix them up later. For exp and exp2 we replace the more complex special-case handler with a simple scalar fallback. All new behaviour is tested in runulp.sh, with a new option to pass -f to the run line. We also extend the fenv testing to Neon log and logf, which already triggered exceptions correctly. New behaviour is mostly hidden behind a new config setting, WANT_SIMD_EXCEPT.
2022-12-07string: arm: Fix cfi restore info for hot loop exitVictor Do Nascimento
The branch out of the core memchr loop to label 60 jumps over the popping of registers r4-r7. The restoration of the cfi state at 60 is adjusted to reflect this fact, avoiding restoring a state where r4-r7 have already been popped off the stack. Built w/ arm-none-linux-gnueabihf, ran make check-string w/ qemu-arm-static.
2022-12-07string: arm: Ensure correct cfi state at strcmp entryVictor Do Nascimento
Move code fragment corresponding to L(fastpath_exit) to after function entry so that a .cfi_remember_state/.cfi_restore_state pair are not needed prior to strcmp start. The resulting reshuffle of code cleans up the entry part, fixing the .size directive calculation, which at present calculates the function size based on the address of __strcmp_arm and not L(strcmp_start_addr).
2022-12-06pl/math: Set fenv flags in Neon asinhfJoe Ramsay
Routine no longer relies on vector log1pf, as this has to become more complex to deal with fenv itself. Instead we re-use a log1pf helper from Neon atanhf which does no special-case handling, instead leaving it all up to the main routine. We now just fall back to the scalar routine for special-case handling. This uncovered a mistake in asinhf's handling of NaNs, which has been fixed.
2022-12-05pl/math: Avoid UB in scalar tanhfJoe Ramsay
The ldexp shortcut was left-shifting a signed value. We now bias the exponent first, will allows the shift to be done on an unsigned value.
2022-11-30pl/math: Add scalar and vector/Neon tanhfJoe Ramsay
Both routines use simplified inline versions of expm1f, and are accurate to 2.6 ULP.
2022-11-29pl/math: Add vector/Neon asinhJoe Ramsay
New routine uses two separate algorithms for input greater and less than 1 (similar to the scalar routine). It is accurate to 2.5 ULP.
2022-11-24pl/math: Update ULP threshold for vector atansJoe Ramsay
New max observed for both Neon and SVE.
2022-11-22pl/math: Add scalar & vector/Neon cbrtfJoe Ramsay
Both routines use the same algorithm - one Newton iteration with the initial guess obtained by a low-order polynomial. Scalar is used as a fallback for subnormal and special cases for the vector routine, which allows vastly simplified argument reduction and reassembly. Both routines accurate to 1.5 ULP.
2022-11-22pl/math: Add scalar and vector/Neon atanhfJoe Ramsay
Both routines are based on a simplified version of log1pf, and are accurate to 3.1 ULP. Also enabled -c flag from runulp.sh - we need this for atanhf so that we can set the control lane to something other than 1, since atanh(1) is infinite.
2022-11-17string: arm: Refactor ENTRY/END macrosSzabolcs Nagy
The .fnstart/.fnend directives can be inlined now that asmdefs.h is arm specific.
2022-11-17string: arm: Use /**/ comments in asmdefs.hSzabolcs Nagy
This is preprocessed asm code, so /**/ style comments are most appropriate.
2022-11-17string: arm: Include asmdefs.h even into empty asm filesSzabolcs Nagy
Currently this is not expected to change behaviour, but if global directives are added in asmdefs.h (like .thumb) those should be in all asm files in case the link ABI is affected.
2022-11-17string: Add separate asmdefs.h per targetSzabolcs Nagy
The definitions in this header are necessarily target specific, so better to have a separate version in each target directory.
2022-11-17string: arm: Fix build failureSzabolcs Nagy
asmdefs.h ifdef logic was wrong: arm only macro definitions were outside of defined(__arm__). Added some ifdef indentation to make the code more readable.