Fix CR.doubleValue() overflow checking

We could accidentally generate a NaN/infinity exponent for an overflow
result without noticing it. This meant that we occasionally produced
a NaN instead of infinity.

Fix bit-rot in README.txt testing instructions.

Test: Ran as part of art/test/1972-math-prec. Ran crcalc tests.

Change-Id: I3b218c22d8ec80557115a74528edfa8747ce6acc

2 files changed, 14 insertions, 8 deletions

diff --git a/src/com/hp/creals/CR.java b/src/com/hp/creals/CR.java
index c5a1c41..20f88e6 100644
--- a/src/com/hp/creals/CR.java
+++ b/src/com/hp/creals/CR.java
@@ -116,6 +116,9 @@
 // hboehm@google.com 11/20/2018.
 // Fix an exception-safety issue in gl_pi_CR.approximate.
 // hboehm@google.com 3/3/2019.
+// Near-overflow floating point exponents were not handled correctly in
+// doubleValue(). Fixed.
+// hboehm@google.com 7/23/2019.
 
 package com.hp.creals;
 
@@ -413,7 +416,7 @@ public volatile static boolean please_stop = false;
       {
         int prec = 0;
 
-        for (;prec > n + 30; prec = (prec * 3)/2 - 16) {
+        for (; prec > n + 30; prec = (prec * 3)/2 - 16) {
             int msd = msd(prec);
             if (msd != Integer.MIN_VALUE) return msd;
             check_prec(prec);
@@ -747,10 +750,11 @@ public volatile static boolean please_stop = false;
         long scaled_int_rep = Double.doubleToLongBits(scaled_int);
         long exp_adj = may_underflow? needed_prec + 96 : needed_prec;
         long orig_exp = (scaled_int_rep >> 52) & 0x7ff;
-        if (((orig_exp + exp_adj) & ~0x7ff) != 0) {
-            // Original unbiased exponent is > 50. Exp_adj > -1050.
-            // Thus this can overflow the 11 bit exponent only if the result
-            // itself overflows.
+        // Original unbiased exponent is > 50. Exp_adj > -1050.
+        // Thus the sum must be > the smallest representable exponent
+        // of -1023.
+        if (orig_exp + exp_adj >= 0x7ff) {
+            // Exponent overflowed.
             if (scaled_int < 0.0) {
                 return Double.NEGATIVE_INFINITY;
             } else {
@@ -760,6 +764,8 @@ public volatile static boolean please_stop = false;
         scaled_int_rep += exp_adj << 52;
         double result = Double.longBitsToDouble(scaled_int_rep);
         if (may_underflow) {
+            // Exponent is too large by 96. Compensate, relying on fp arithmetic
+            // to handle gradual underflow correctly.
             double two48 = (double)(1L << 48);
             return result/two48/two48;
         } else {
diff --git a/tests/README.txt b/tests/README.txt
index 6cedac3..e8aca3a 100644
--- a/tests/README.txt
+++ b/tests/README.txt
@@ -1,14 +1,14 @@
 Run on Android with
 
 1) Build the tests.
-2) adb install <tree root>/out/target/product/<name>/data/app/CRTests/CRTests.apk
+2) adb install <tree root>/out/target/product/<name>/testcases/CRTests/arn64/CRTests.apk
 3) adb shell am instrument -w com.hp.creals.tests/android.test.InstrumentationTestRunner
 
-The last step takes around 10 minutes on a Nexus 5.
+Depending on the device, the last step may take a while.
 (CRTest is quick, SlowCRTest is not, especially not the final trig function
 test.)
 
-Note that Random seeds are not set.  Hence repreated runs should improve
+Note that Random seeds are not set.  Hence repeated runs should improve
 coverage at the cost of reproducibility.  Failing arguments should however
 be printed.