summaryrefslogtreecommitdiff
path: root/nn/runtime/test/TestValidation.cpp
AgeCommit message (Collapse)Author
2020-04-30Add more NN priority validation testsMichael Butler
Bug: 147925145 Test: mma Test: NeuralNetworksTest_static Change-Id: Ie636c7cdc84ff7a262925f8ff8ebbd4d4f45201b
2020-04-02Validate if the returned output shapes overflow before updating.Xusong Wang
Additionally, check dimensions overflow in CpuExecutor. Fixes: 152382062 Test: NNT_static Change-Id: I7439ae31799e15c11e3802ae29c8f702a2318e4a
2020-03-26Catch integer overflow in getSizeOfData()Slava Shklyaev
Bug: 146044137 Test: NNT_static Change-Id: Idd6a070b379c6f5b10105647ec93579320c558f3
2020-03-26Disallow operand types where size overflows uint32Slava Shklyaev
Also fixes a bug ValidationTestModel.SetOperandValueFromModel that this change has uncovered. Fix: 146044137 Test: NeuralNetworksTest_utils Test: NeuralNetworksTest_static Change-Id: Ibb8ca42115451eafc9288c679f792a0e3fe790c7
2020-03-26More graph validation, including whole-graph valdation.David Gross
Validate: - operations are in execution order - this is a requirement at the HAL level - at the API level, this is a requirement that there IS an execution order (i.e., the graph is acyclic and leaves no operands unwritten) - operands are not read before written - SUBGRAPH_INPUT/SUBGRAPH_OUTPUT operands are in inputIndexes/outputIndexes Add tests: - detect graph cycle (Cycle) - detect operand read before written (AcyclicReadBeforeWrite) - detect operand never written (MissingWrite, UnwrittenOperand) - detect multiple writes to the same operand (MultipleWrite) Also: - improve some error messages - clean up some comments Bug: 66478689 Test: mma Test: CtsNNAPITestCases Test: NeuralNetworksTest_static Test: VtsHalNeuralnetworksV1_*TargetTest Change-Id: I018b0c195e59b8b89ac8b62e0d80039d673ce81e Merged-In: I018b0c195e59b8b89ac8b62e0d80039d673ce81e (cherry picked from commit 2da722e81bbeacb9c09770d2b4416295efca5b40)
2020-03-23Merge "Add validation tests for ANN_setLoopTimeout" into rvc-devTreeHugger Robot
2020-03-21Merge "Add more CTS validation tests for memory domain." into rvc-devTreeHugger Robot
2020-03-20Add validation tests for ANN_setLoopTimeoutSlava Shklyaev
Also adds mStarted validation. Fix: 149819396 Test: NNT_static Change-Id: I2f947a4ead527b7cff6bf93439c3295e59626f7a
2020-03-18Impose more discipline on uses of ModelArgumentInfoDavid Gross
Change ModelArgumentInfo from a struct to a class. Fields are no longer public, and initialization uses a factory method (e.g., createFromPointer) rather than a setter (e.g., setFromPointer). Accessor methods are guarded by a check against the (already existing) state (POINTER, MEMORY, or both). Calling ANeuralNetworksExecution_setInput*() or ANeuralNetworksExecution_setOutput*() multiple times on the same operand of the same execution is now an error. Bug: 147850002 Test: NeuralNetworksTest_static Change-Id: Idebe6b4df79bff5c62e56860e42463d610d91085
2020-03-16Add more CTS validation tests for memory domain.Xusong Wang
- Validate addInputRoles with different extra parameters. - Validate the case when the device memory is deinitialized between setInputFromMemory and compute. - Validate ANNMemory_copy with conflict dimensions and rank. - Validate ANNMemory_copy with deinitialized source device memory. Bug: 147777318 Test: NNT_static Change-Id: Ic4b518b013f090520c92315b20ec73c8bd857fb3
2020-02-28Cleanup NNAPI QoS DeadlineMichael Butler
This CL does the following changes: * In the case where a timeout duration would cause an overflow of the OptionalTimePoint, this CL instead clamps it to the max time point. Accordingly, the Overflow test result is changed from BAD_DATA to NO_ERROR. * Previously hal::OptionalTimePoint was used throughout the runtime code. However, OptionalTimePoint cannot be directly compared against the current time. Instead, this CL changes the runtime deadline type to std::optional<std::chrono::steady_clock::time_point>, called std::optional<Deadline>. * Implements a simple deadline abort within the sample driver the and runtime's CPU (i.e., non-driver) path. Bug: 147925145 Test: mma Test: CtsNNAPITestCases Test: NeuralNetworksTest_static Change-Id: I54ca961416313334eb3fca3f439db0974044bd6d
2020-02-18Relax setTimeout duration parameterMichael Butler
Prior to this CL, the NNAPI QoS deadline is a strict deadline: a task with a deadline must either complete or abort by the deadline. To avoid as much overhead as possible, it is requested for Android R to have a relaxed deadline, where the task *may* be aborted after the deadline has passed or when the task is estimated to take too long. Because the deadline is now relaxed, the querying functions "supportsCompilationTimeout" and "supportsExecutionTimeout" can be removed. Bug: 149766387 Test: mma Test: CtsNNAPITestCases Test: NeuralNetworksTest_static Change-Id: Ib76bdd8697686b8d6f3414d3742998c67b2f1f64
2020-02-14Merge "Fix NNAPI deadline overflow test"Michael Butler
2020-02-12Fix NNAPI deadline overflow testMichael Butler
Bug: 149441015 Test: mma Test: CtsNNAPITestCases Change-Id: I1fdb9a5d0ab243b35216c2beeca6893b718cd939
2020-02-10Fix ANNExecution_getDuration with new duration codesMiao Wang
- Also add validation tests Bug: 148803008 Test: mm Test: NeuralNetworksTest_static Change-Id: I00df8202f8893a12ae77103ee6730bdf070efc3c
2020-01-29Update sync fence related APIsMiao Wang
- Allow ANeuralNetworksExecution_startComputeWithDependencies to measure execution duration after all dependencies are done. - Allow ANeuralNetworksExecution_startComputeWithDependencies to specify post-gate-release timeout duration. Bug: 142778241 Bug: 136739795 Test: mm Test: NNAPI CTS & VTS tests Change-Id: Ie32e30b7d1cb98882f3084c1741e975d3e39d970
2020-01-27Implement control flow APISlava Shklyaev
Bug: 136735929 Test: N/A Change-Id: Iefbb997cc7275619af21605e16f045ee764aadef
2020-01-22Add tests for sync fence related API.Miao Wang
The following tests are added: - basic validation tests - generated tests for ANeuralNetworksExecution_startComputeWithDependencies. Bug: 142778241 Test: mm Change-Id: I5591db4583e8b752868d06e16874780800d4410e
2020-01-21Create CTS tests for QoS in NNAPIMichael Butler
Bug: 136739795 Bug: 142902514 Bug: 145300530 Test: mma Test: CtsNNAPITestCases Test: NeuralNetworksTest_static Change-Id: I7a9f3644c0a8978031b4a7a3922a10cce5151497
2020-01-17Memory Domain Runtime: Explicit memory copying.Xusong Wang
- Implement ANNMemory_copy. - Add validation tests. Bug: 141353602 Bug: 141363565 Test: NNT_static Change-Id: Id0ec2431cf9cfff650cd8885de4d15abee306a9c
2020-01-17Memory Domain Runtime: Device memory as execution I/O.Xusong Wang
- Add validation checks in NNAPI runtime when using driver-allocated memory as execution I/O. - Add validation tests. Additionally, change the result code of using non-BLOB mode AHWB with ANNModel_setOperandValueFromMemory to BAD_DATA. Bug: 141353602 Bug: 141363565 Test: NNT_static Change-Id: I6f253a0a90a0c1b2baa186034482215435f3c831
2020-01-17Memory Domain Runtime: NDK ANNMemory allocator interface.Xusong Wang
- Implement ANNMemory_createFromDesc. - Add validation tests. Memory allocation with unspecified dimensions or rank will be implemented and tested in later CLs. Bug: 141353602 Bug: 141363565 Test: NNT_static Change-Id: I88cf4b37d9dc69d3466775d10580899080d425a7
2020-01-16Memory Domain Runtime: NDK ANNMemoryDesc.Xusong Wang
- Implement the following NDK APIs: * ANNMemoryDesc_create * ANNMemoryDesc_free * ANNMemoryDesc_setUsageForInput * ANNMemoryDesc_setUsageForOutput * ANNMemoryDesc_setDimensions * ANNMemoryDesc_finish - Add validation tests. Bug: 141353602 Bug: 141363565 Test: NNT_static Change-Id: I2490c0fc44514c7521bb8e5112dbaa43dedd289e
2019-04-08Merge "Finish work on collecting execution duration. Add tests and execution ↵David Gross
state enforcement."
2019-04-01Propagate ANNMemory_free to IBurstContext::freeMemoryMichael Butler
This CL extends nn::Memory to include a reference to all burst objects which use its memory. When the nn:Memory object is destroyed (via ANNMemory_free), it signals all these burst objects so that they can properly clean their memory caches (via IBurstContext::freeMemory). This CL also provides a more intelligent memory slot allocator within ExecutionBurstController to reuse slots after they are freed. This CL includes some additional miscellaneous code cleanup of the neighboring test cases, e.g., closing file descriptors when they are no longer needed. Bug: 128319484 Test: mma Test: atest NeuralNetworksTest_static Change-Id: Ibc19059af5194cd3dd58c9a9d8baa54fa6b26de5
2019-03-28Finish work on collecting execution duration. Add tests and execution state ↵David Gross
enforcement. Fix bug: Execution durations reported by HAL were not reported properly through NDK. Ensure execution duration is reported as UINT64_MAX on error paths. Switch SampleDriver from system_clock to steady_clock. Clarify, enforce, and validate that several ANeuralNetworksExecution_* methods must not be called once execution has been scheduled. Enforce and validate that ANeuralNetworksExecution_setMeasureTiming can only be called in conjunction with ANeuralNetworksCompilation_createForDevices with numDevices = 1. Enforce and validate that ANeuralNetworksExecution_getDuration must not be called until execution has finished. Test: NeuralNetworksTest_static Test: --gtest_filter=IntrospectionControlTest.SimpleAddModel with setprop debug.nn.vlog driver,execution; by inspection of logcat, verify that driver and runtime report the same duration Bug: 123376542 Bug: 79265318 Change-Id: I8a091f3f8207669114c10e89f3ed6a08ea241fbb
2019-03-26NNAPI Burst object cleanupMichael Butler
This CL addresses some follow up comments from ag/6154003 and ag/6575732. Bug: 119570067 Test: mma Test: atest NeuralNetworksTest_static Change-Id: I1a2bd4c9d97296f50d6ef9bb86515ea8e9a54515
2019-03-06Merge "Add a centralized runtime extension information store"TreeHugger Robot
2019-03-06Add a centralized runtime extension information storeSlava Shklyaev
Fix: 124107169 Fix: 123523457 Fix: 124285861 Fix: 124104123 Fix: 123178734 Test: NeuralNetworksTest_static Test: NeuralNetworksTest_utils Test: NeuralNetworksTest_FibonacciExtension (from change Ibe0fc5356baa909bce8424138bd5cfac9f74648f) Change-Id: Id3f105476f42bd747a098f081a07b161036e4922
2019-03-05Add validation test to make sure device names are unique.Miao Wang
Bug: 123024279 Test: mm Test: NeuralNetworksTest_static Change-Id: Ifc5453411c1bfda06448a96da1968664d12e716f
2019-02-26Validate the consistency between dimensionCount and dimensions.Xusong Wang
Also add validation tests exercising the change. Bug: 109666567 Test: NeuralNetworksTest_static Change-Id: I8a3192392a472e93bb70d0643cbf55b40b54fdac
2019-01-30Merge changes Ifdb305d8,I03953c2d,I4b6281cd,Ibeda48f3Xusong Wang
* changes: Regenerate tests. Enable CTS generated DynamicOutputShapeTests. Document, support, and test unspecified dimension. Handle output shapes in ExecutionBuilder.
2019-01-29Fix broken ValidationTests.Xusong Wang
Fix validation tests - ValidationTestExecution.GetOutputOperandRankAndDimensions - ValidationTestBurst.BurstComputeConcurrent - ValidationTestCompilationForDevices.ExecutionTiming Bug: 123572441 Bug: 123573747 Test: NeuralNetworksTest_static Change-Id: I225f55f6c5da0ce21445e3df03a6a9018bcabaf3
2019-01-29Document, support, and test unspecified dimension.David Gross
A tensor operand type with some number of unspecified dimensions is represented by setting each unspecified dimension to 0. A tensor operand type of unspecified rank is represented by setting dimensionCount to 0 and dimensions to NULL. Tensor with unspecified dimensions can not have value and can not be as model input at execution time, but is allowed for output. Enable checks and tests for it. Bug: 77234888 Bug: 73506513 Test: NeuralNetworksTest_static Change-Id: I4b6281cdd7758054a1a43ac34f39fc9c7bfead9b
2019-01-29Merge changes from topic "get-supported-extensions"Slava Shklyaev
* changes: Add Extensions API Add getSupportedExtensions to sample drivers
2019-01-28Merge "NNAPI Burst -- runtime and CTS"Michael Butler
2019-01-28Add Extensions APISlava Shklyaev
Please see the commit message of change Ia9b99015eec7a48bbf969cbe503862271f09adca for motivation. Bug: 118604960 Bug: 118606929 Test: NeuralNetworksTest_static Change-Id: I2703b963f040a846889554888ddd984eac6b6c08
2019-01-27NNAPI Burst -- runtime and CTSMichael Butler
The NNAPI is introducing the notion of an "Execution Burst" object (or more simply a "Burst" object), which is similar to an ANeuralNetworksExecution, but is intended to be reused across multiple executions and has lower IPC overheads. It achieves this low IPC overhead by replacing HIDL HwBinder calls with FMQ messages. This CL implements the NDK burst functions, implements the path through the partitioner/scheduler, and creates CTS tests using the burst object. Bug: 119570067 Test: mma Test: NeuralNetworksTest_static Change-Id: I1d2414f454910ad3ba4b2af728ab95ef8b609c9c
2019-01-25Fix ValidationTestCompilationForDevices.ExecutionTiming.Xusong Wang
Bug: 123377751 Test: NeuralNetworksTest_static Change-Id: I3ebd65117cc702ad5e1e46e350a801e7b493a8c6
2019-01-24Update API to allow collecting execution duration.David Gross
Test: NeuralNetworksTest_static Test: --gtest_filter=IntrospectionControlTest.SimpleAddModel with setprop debug.nn.vlog all; by inspection of logcat, verify that driver and runtime report the same duration TODO: - Ensure duration is reported correctly on error paths and on cpu fallback - Check for more misuses of API: - Ensure that we only request measurement before execution starts - Ensure that we only request measurement for the special introspection case - Ensure that we only request duration after execution completes - Add more extensive timing tests using the introspection/control APIs Bug: 115390094 Change-Id: I1dd61e4128963a22955ebab18bcdf7293419ca20
2019-01-24Merge "Add AHardwareBuffer support for NNAPI"Miao Wang
2019-01-23Add AHardwareBuffer support for NNAPIMiao Wang
- The main purpose for AHardwareBuffer support is for model inputs & outputs. - ANeuralNetworks_setOperandValueFromMemory only accepts BLOB mode AHardwareBuffer. - ANeuralNetworks_set[Input|Output]FromMemory can accepts non-BLOB mode AHardwareBuffer. And for non-BLOB mode buffer, the entire buffer will be used. - There is no guarantee that arbitrary AHardwareBuffer can be used by arbitrary devices. The execution must fail if the driver cannot consume the buffer. - No CPU fallback for AHardwareBuffer except BLOB mode. Test: mm Test: NeuralNetworksTest_static Change-Id: I9d9ac95aea1e56b583c06e4f9c9afdaab1403152
2019-01-23Implement NDK interface for NNAPI compilation caching.Xusong Wang
Add a new NDK API ANeuralNetworksCompilation_setCaching to set caching information for the compilation object. Create validation tests for the new API. Bug: 119616526 Test: NeuralNetworksTest_static Change-Id: I96638073ba39b26a982588e9868e35a49809014b
2019-01-22Implement NDK interface for dynamic output shape.Xusong Wang
Implement the following NDK interfaces - ANeuralNetworksExecution_getOutputOperandDimensions - ANeuralNetworksExecution_getOutputOperandRank Bug: 73506513 Test: NeuralNetworksTest_static Change-Id: I3e0238ec701a0bffbdb5682ee1787df198fe9816
2019-01-18Add NDK API to allow developers to query the device typeMiao Wang
Bug: 111425781 Bug: 112661050 Test: mm Test: NeuralNetworksTest_static Change-Id: Iff756fa8d245c2c18d8ea8682ca648640ab45a3c
2019-01-08Replace ANNOperandType::ExtraParams with a function call.Przemyslaw Szczepaniak
Exending ANeuralNetworkOperandType struct with new fields may be problematic and difficult to maintain in longer term. This change turns recently introduced ANNOperandType.extraParams field into a function call: ANeuralNetworksModel_setOperandSymmPerChannelQuant(...) that associates ANeuralNetworksSymmPerChannelQuant struct with an operand. + added to NeuralNetworks.h an ANDROID_Q ifdef for operand types added in Q. Bug: 119249581 Test: NeuralNetworksTest_static Change-Id: Idebe86ce1ca8f3a31dbfe7b21de4dd45df380a38
2018-12-14Wrap the CpuExecutor as a device during compilation step.Xusong Wang
Make Device a common interface for abstraction of CPU fallback as well as the actual driver devices. Modify the compilation steps, mainly the partitioner, to accommodate the change. Make all possible compilation outcomes plan-based, even for the CPU fallback, which is a SIMPLE body running on CPU only. Add validation for invalid empty model and create validation test. Add validation for empty device list passed to introspection and control API and create tests. Bug: 72506261 Test: NeuralNetworksTest_static Change-Id: Ic63036716afb8a0156c77e0cf43456e490783deb
2018-11-16Implement ANeuralNetworksExecution_compute() and add related properties.David Gross
Change wrapper::Execution::compute() to use ANeuralNetworksExecution_compute() rather than simulating it using ANeuralNetworksExecution_startCompute(). Change test_wrapper::Execution::compute() the same way by default, but to permit simulation by setting global state using test_wrapper::Execution::setComputeUsesSynchronousAPI(). Setting the new property debug.nn.syncexec-runtime to 1 makes ANeuralNetworksExecution_startCompute() behave synchronously (i.e., it will not return until the work is complete). In this case, the runtime itself will not spawn a thread to do the work. This simulates using ANeuralNetworksExecution_compute(). Setting the new property debug.nn.syncexec-cpu to 1 instructs the runtime not to spawn a thread to do CPU execution. Remove the execution path that was under the control of preprocessor symbol DISABLE_PARTITIONED_EXECUTION. Modify NeuralNetworksTest so that instead of iterating over {useCpuOnly=no, useCpuOnly=yes} it instead iterates over {useCpuOnly=no, useCpuOnly=yes} x {computeUsesSynchronousAPI=no, computeUsesSynchronousAPI=yes} Bug: 115920643 Bug: 119272497 Test: NeuralNetworksTest_static (and verify from logcat that we're getting correct threading behavior according to API choices and property settings) Test: cts -m CtsNNAPITestCases Change-Id: I9723aa502197a8d48a95a3ae5d7f053d381001da
2018-10-26Remove invalid test cases for GetSupportedOperationsForDevices andMiao Wang
CreateCompilationForDevices. Test: mm Test: NeuralNetworksTest_static with ag/5370873 Change-Id: I4d67aeca2336b2811d5d6fc2c6381637e464c66b
2018-10-24Implement the following introspection & control APIs:Miao Wang
- ANeuralNetworksModel_getSupportedOperationsForDevices - ANeuralNetworksCompilation_createForDevices The APIs listed above are added to NDK with corresponding NNAPI runtime changes. Validation tests and integration tests with mocked CPU driver are also added. The tests also test APIs to get a device handle and to query basic information of selected devices. The changes in DeviceManager are mainly to facilitate testing. Bug: 112661538 Bug: 112661284 Bug: 111425781 Test: mm Test: NeuralNetworksTest_static Change-Id: If7706447ddee993973397765e2eba15845d8bc51
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-24 06:34:19 +0000