Updating SecureRandomTest.java

This CL adds test for new SecureRandom(seed);
The result is that the behavior depends a lot on which providers are installed.
E.g., if no provider with SecureRandom instances is installed then java uses
SHA1PRNG as default and the result is deterministic.
If for example the SUN provider is installed and jdk11 is being used then
NativePRNG is being used and the result is non-deterministic.

NOKEYCHECK=True
PiperOrigin-RevId: 495312129

1 files changed, 131 insertions, 52 deletions

diff --git a/java/com/google/security/wycheproof/testcases/SecureRandomTest.java b/java/com/google/security/wycheproof/testcases/SecureRandomTest.java
index 86e7bd8..92dc292 100644
--- a/java/com/google/security/wycheproof/testcases/SecureRandomTest.java
+++ b/java/com/google/security/wycheproof/testcases/SecureRandomTest.java
@@ -30,48 +30,75 @@ import org.junit.runners.JUnit4;
 /**
  * Checks whether instances of SecureRandom follow the SecureRandom API.
  *
- * The tests here are quite limited. They only test that instances are non-deterministic, when
+ * <p>The tests here are quite limited. They only test that instances are non-deterministic, when
  * required by the API. The tests do not attempt the determine whether the output is pseudorandom
  * and whether the seeds used are unpredictable.
  *
- * Any instance of SecureRandom must self-seed itself if no seed is provided: hence the following
- * code must never result in deterministic or predictable behaviour:
+ * <p>Any instance of SecureRandom must self-seed itself if no seed is provided: hence the following
+ * code must never result in deterministic or predictable behavior:
+ *
  * <pre>{@code
- *   // Safe use of SecureRandom
- *   SecureRandom secureRandom = SecureRandom.getInstance(ALGORITHM);
- *   byte[] randomOutput = new byte[SIZE];
- *   secureRandom.nextBytes(randomOutput);
- *   ...
+ * // Safe use of SecureRandom
+ * SecureRandom secureRandom = SecureRandom.getInstance(ALGORITHM);
+ * byte[] randomOutput = new byte[SIZE];
+ * secureRandom.nextBytes(randomOutput);
+ * ...
  * }</pre>
- * An important point is that the constructur itself does not necessarily seed the SecureRandom
- * instance and that the self-seeding is only required if caller does not provide any seeds.
- * Thus the following code snippet can lead to deterministic and predictable behaviour:
+ *
+ * The code above is indeed a typical usage pattern and for example recommended here:
+ * https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/security/SecureRandom.html
+ *
+ * <p>A legacy issue is that SecureRandom can sometimes behave deterministically. The constructur of
+ * a SecureRandom instance does not necessarily seed the instance and self-seeding is only required
+ * if the caller does not provide any seeds. For example it is possible to reproduce the output of
+ * `SHA1PRNG` with
+ *
  * <pre>{@code
- *   // Potentially deterministic and predictable outcome.
- *   SecureRandom secureRandom = SecureRandom.getInstance(ALGORITHM);
- *   secureRandom.setSeed(MY_SEED);
- *   byte[] randomOutput = new byte[SIZE];
- *   secureRandom.nextBytes(randomOutput);
- *   ...
+ * // Legacy use case: the following code has deterministic behavior.
+ * String ALGORITHM = "SHA1PRNG";
+ * SecureRandom secureRandom = SecureRandom.getInstance(ALGORITHM);
+ * secureRandom.setSeed(MY_SEED);
+ * byte[] randomOutput = new byte[SIZE];
+ * secureRandom.nextBytes(randomOutput);
+ * ...
  * }</pre>
- * For example "SHA1PRNG" has the property that calling setSeed after the construction gives
- * a SecureRandom instance with output that only depends on the caller provided seeds.
  *
- * Once a SecureRandom instance has been seeded, all further calls of setSeed must add additional
- * randomness. It is not acceptable setSeed overrides the current seed of an instance. Hence the
- * following code must always be non-deterministic.
+ * This behavior is rather unusual. Algorithms other than `SHA1PRNG` often do not use the seed.
+ * `SHA1PRNG` is not well defined and not a stable algorithm. Its output may change between jdk
+ * versions. Different provider may use distinct implementations. Because of it is a bad idea to use
+ * SecureRandom as a pseudorandom function. Applications that require reproducible pseudorandom
+ * output should rather use well defined algorithms such as HMAC, HKDF or SHAKE depending on use
+ * case and availiability.
+ *
+ * <p>The following code should also not be used. The behavior of the code depends on the class used
+ * by `new SecureRandom()`. This class depends on the providers that are installed. E.g., if
+ * `SHA1PRNG` is being used as the underlying PRNG then the behavior is deterministic, if
+ * `NativePRNG` is being used then the behavior is non-deterministic. Thus the behavior can be
+ * easily changed by installing new providers or switching their order.
+ *
  * <pre>{@code
- *   SecureRandom secureRandom = SecureRandom.getInstance(ALGORITHM);
- *   // The next line forces secureRandom to self-seed.
- *   secureRandom.nextBytes(new byte[1]);
- *   // Adding and additional seed. The instance remains properly seeded and unpredictable even if
- *   // MY_SEED is known or constant.
- *   secureRandom.setSeed(MY_SEED);
- *   byte[] randomOutput = new byte[SIZE];
- *   secureRandom.nextBytes(randomOutput);
- *   ...
+ * // Unsafe use of SecureRandom
+ * SecureRandom rand = new SecureRandom(seed);
+ * byte[] bytes = new byte[outputsize];
+ * secureRandom.nextBytes(bytes);
  * }</pre>
  *
+ * <p>Once a SecureRandom instance has been seeded, all further calls of setSeed must add additional
+ * randomness. It is not acceptable if setSeed overrides the current seed of an instance. Hence the
+ * following code must always be non-deterministic.
+ *
+ * <pre>{@code
+ * // Safe way to add additional entropy
+ * SecureRandom secureRandom = SecureRandom.getInstance(ALGORITHM);
+ * // The next line forces secureRandom to self-seed.
+ * secureRandom.nextBytes(new byte[1]);
+ * // Adding an additional seed. The instance remains properly seeded and unpredictable even if
+ * // MY_SEED is known or constant.
+ * secureRandom.setSeed(MY_SEED);
+ * byte[] randomOutput = new byte[SIZE];
+ * secureRandom.nextBytes(randomOutput);
+ * ...
+ * }</pre>
  */
 @RunWith(JUnit4.class)
 public class SecureRandomTest {
@@ -81,7 +108,7 @@ public class SecureRandomTest {
     // TODO(bleichen): Check if all instances of SecureRandom are actually
     //   listed as services. In particular the default SecureRandom() and
     //   SecureRandom.getInstanceStrong() may not not be registered.
-    ArrayList<Provider.Service> result = new ArrayList<Provider.Service>();
+    ArrayList<Provider.Service> result = new ArrayList<>();
     for (Provider p : Security.getProviders()) {
       for (Provider.Service service : p.getServices()) {
         if (service.getType().equals("SecureRandom")) {
@@ -92,10 +119,7 @@ public class SecureRandomTest {
     return result;
   }
 
-  /**
-   * Uninitialized instances or SecureRandom must self-seed before
-   * their first use.
-   */
+  /** Uninitialized instances of SecureRandom must self-seed before their first use. */
   @Test
   public void testSeedUninitializedInstance() throws Exception {
     final int samples = 10;  // the number of samples per SecureRandom.
@@ -104,7 +128,7 @@ public class SecureRandomTest {
     //   2^{-65}*(samples * (#secure random instances))^2.
     // Hence a random failure of this function is unlikely.
     final int outputsize = 8;
-    Set<String> seen = new TreeSet<String>();
+    Set<String> seen = new TreeSet<>();
     for (Provider.Service service : secureRandomServices()) {
       for (int i = 0; i < samples; i++) {
         SecureRandom random = SecureRandom.getInstance(service.getAlgorithm(),
@@ -119,23 +143,29 @@ public class SecureRandomTest {
   }
 
   /**
-   * Calling setSeed directly after the initialization may result in deterministic
-   * results.
+   * Tests calling setSeed directly after constructing a SecureRandom instance.
+   *
+   * <p>If setSeed is called directly after constructing a new SecureRandom instance then its
+   * behavior depends on the class of the instance. A SecureRandom instance may used this seed and
+   * thus become deterministic and reproducable, the instance may mix the seed into its entropy
+   * pool, or may ignore it and use independent seeding.
+   *
+   * <p>For example the provider "SUN" in jdk20 implements SecureRandom classes with the following
+   * behavior:
    *
-   * The test expects that a SecureRandom instance is either completely deterministic if
-   * seeded or non-deterministic and unpredictable (though the test is much too simple
-   * to give any meaningful result).
-   * 
-   * For example the provider "SUN" has the following behaviour:
    * <pre>
-   *   Seeding SHA1PRNG from SUN results in deterministic output.
-   *   Seeding NativePRNG from SUN results in non-deterministic output.
-   *   Seeding NativePRNGBlocking from SUN results in non-deterministic output.
-   *   Seeding NativePRNGNonBlocking from SUN results in non-deterministic output.
+   *   Seeding SHA1PRNG results in deterministic output.
+   *   Seeding NativePRNG results in non-deterministic output.
+   *   Seeding NativePRNGBlocking results in non-deterministic output.
+   *   Seeding NativePRNGNonBlocking results in non-deterministic output.
+   *   Seeding DRBG from SUN results in non-deterministic output.
    * </pre>
    *
-   * jdk9 adds a class java.security.DrbgParameter, which allows to better specify the expected
-   * behaviour of SecureRandom instances. Tests with these parameters are not included here.
+   * Implementations should not depend on the behavior of a given class, since it is easily possible
+   * that the observed behavior changes between versions or platforms.
+   *
+   * <p>jdk9 adds a class java.security.DrbgParameter, which allows to better specify the expected
+   * behavior of SecureRandom instances. Tests with these parameters are not included here.
    */
   @Test
   public void testSetSeedAfterConstruction() throws Exception {
@@ -148,7 +178,7 @@ public class SecureRandomTest {
     final byte[] seed = new byte[32];
     for (Provider.Service service : secureRandomServices()) {
       Provider provider = service.getProvider();
-      Set<String> seen = new TreeSet<String>();
+      Set<String> seen = new TreeSet<>();
       for (int i = 0; i < samples; i++) {
         SecureRandom random = SecureRandom.getInstance(service.getAlgorithm(), provider);
         random.setSeed(seed);
@@ -170,6 +200,55 @@ public class SecureRandomTest {
   }
 
   /**
+   * Checks the default for SecureRandom.
+   *
+   * <p>The test checks the following pattern:
+   *
+   * <pre>{@code
+   * SecureRandom rand = new SecureRandom(seed);
+   * byte[] bytes = new byte[outputsize];
+   * secureRandom.nextBytes(bytes);
+   * }</pre>
+   *
+   * For example under jdk20 with the SUN provider installed at position 1, the code above uses
+   * NativePRNG. This pseudorandom number generator is non-deterministic. If no provider with
+   * pseudorandom number generators is installed then SHA1PRNG is being used. This pseudorandom
+   * number generator can be seeded deterministially, and the code above has indeed deterministic
+   * behavior.
+   *
+   * <p>Hence it is probably a good idea to avoid the pattern above and to consider it as having
+   * undefined or implementation defined behavior. Using no seed (e.g. using new SecureRandom()) is
+   * preferable.
+   */
+  @Test
+  public void testDefaultSecureRandom() throws Exception {
+    final int samples = 10; // the number of samples per SecureRandom.
+    // The size of the generated pseudorandom bytes. An output size of 8 bytes
+    // means that the probability of false positives is about
+    //   2^{-65}*(samples * (#secure random instances))^2.
+    // Hence a random failure of this function is unlikely.
+    final int outputsize = 8;
+    final byte[] seed = new byte[32];
+    Set<String> seen = new TreeSet<>();
+    for (int i = 0; i < samples; i++) {
+      SecureRandom rand = new SecureRandom(seed);
+      byte[] bytes = new byte[outputsize];
+      rand.nextBytes(bytes);
+      String hex = TestUtil.bytesToHex(bytes);
+      seen.add(hex);
+    }
+    String algorithm = new SecureRandom(seed).getAlgorithm();
+    if (seen.size() == 1) {
+      System.out.println("Default SecureRandom " + algorithm + " results in deterministic output.");
+    } else {
+      System.out.println(
+          "Default SecureRandom " + algorithm + " results in non-deterministic output.");
+      // ... and if the implementation is non-determinstic, there should be no repetitions.
+      assertEquals(samples, seen.size());
+    }
+  }
+
+  /**
    * Calling setSeed after use adds the seed to the current state. It must never replace it.
    */
   @Test
@@ -180,7 +259,7 @@ public class SecureRandomTest {
     //   2^{-65}*(samples * (#secure random instances))^2.
     // Hence a random failure of this function is unlikely.
     final int outputsize = 8;
-    Set<String> seen = new TreeSet<String>();
+    Set<String> seen = new TreeSet<>();
     final byte[] seed = new byte[32];
     for (Provider.Service service : secureRandomServices()) {
       for (int i = 0; i < samples; i++) {