aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2022-07-12Snap for 8820681 from ecb3673040687444c8e6a573b54a3affc4e3a963 to ↵aml_go_adb_330913000android13-mainline-go-adbd-releaseAndroid Build Coastguard Worker
mainline-go-adbd-release Change-Id: If88ec7bbc1523e3a8ff7f4efd4aad226336b792b
2022-06-22Merge remote-tracking branch 'goog/androidx-platform-dev' into tm-mainline-prodAlexander Dorokhine
Changes included: ====================================================================== Refactor DocumentStore::Initialize to improve readability of document store recovery. ====================================================================== Remove non-NDK API usages of ICU4C in libicing. ====================================================================== Move IcuDataFileHelper to the testing directory since it is a test-only util. ====================================================================== Support dump function for DocumentStore ====================================================================== Switch to use PRead rather than MMap in the proto log. ====================================================================== Support dump function for main/lite index and lexicon ====================================================================== Fix LiteIndex::AppendHits ====================================================================== Enable and fix DocumentStoreTest.LoadScoreCacheAndInitializeSuccessfully ====================================================================== Fix MainIndex::GetStorageInfo. ====================================================================== Fix icing-search-engine_fuzz_test by making IcuLanguageSegmenterIterator::Advance non-recursive. ====================================================================== Allow to return additional information for deleted documents in DeleteByQuery ====================================================================== Using enum class in Token::Type for better type safety. ====================================================================== Add some additional logging that will help diagnose b/218413237 ====================================================================== Mark VerbatimTokenizer::ResetToTokenStartingAfter as 'override'. ====================================================================== Support dump function for SchemaStore ====================================================================== Export Icing logging control to JNI ====================================================================== Prepare Icing logging class for JNI export ====================================================================== Export getDebugInfo to JNI ====================================================================== Expose the return_deleted_document_info parameter for deleteByQuery JNI ====================================================================== Enable runtime log control for Icing Library ====================================================================== Fix 1 ClangTidyBuild finding: ====================================================================== Update comments to run benchmarks. ====================================================================== Making icing's own logging class ====================================================================== Convert the string lexicon debug information to a protocol buffer ====================================================================== Fix issue with printing fingerprinted key in our error logs. ====================================================================== Support dump function for IcingSearchEngine ====================================================================== Fix bug in schema store where a failure during RegenerateDerivedFiles would lead to a dangling pointer. ====================================================================== Add RAII class that will create and destroy file directories. ====================================================================== Convert MainIndexDebugInfoProto and LiteIndexDebugInfoProto to string ====================================================================== Make SchemaStore move assignable. ====================================================================== Rollback of "convert the string lexicon debug information to a protocol buffer" ====================================================================== Fix NPE caused by a remap failure. ====================================================================== Unify the name "priority" and "severity" in Icing logging ====================================================================== Avoiding string formatting in Icing logging when we should not log ====================================================================== Switch to use an enum with BASIC/DETAILED to control the verbosity of getDebugInfo ====================================================================== Remove the behavior in the Language Segmenter to filter out non-ascii+non-alphanumeric characters. ====================================================================== Fix the SetSchema bug when we override a schema with nested incompatible types ====================================================================== Wrap __android_log_write with __android_is_loggable ====================================================================== Enable removing expired page tokens to free cache space ====================================================================== Export Icing logging tag to JNI ====================================================================== Update export_to_aosp.sh to change icing log tag to "AppSearchIcing" ====================================================================== Improve the logic of NamespaceChecker. ====================================================================== Step 4.1: Use ScoredDocumentHitsRanker in ResultStateV2 ====================================================================== Step 4.0: Create ScoredDocumentHitsRanker interface and PriorityQueueScoredDocumentHitsRanker ====================================================================== Refactor KeyMapper ====================================================================== Change Icing DEFAULT_LOGGING_LEVEL to INFO ====================================================================== Step 3.4: Create ResultRetrieverV2GroupResultLimiterTest (copied from ResultStateTest) ====================================================================== Add IcingDynamicTrie::IsBranchingTerm to check if a term is branching. ====================================================================== Fix IcingDynamicTrie::Delete bug ====================================================================== Step 3.3: Create ResultRetrieverV2ProjectionTest (copied from ResultRetrieverTest) ====================================================================== Step 3.2: Create ResultRetrieverV2SnippetTest (copied from ResultRetrieverTest) ====================================================================== Step 3.1: Create ResultRetrieverV2Test (copied from ResultRetrieverTest) ====================================================================== Enable legacy multidex ====================================================================== Fix NPE caused by improper handling of return value of GetFileSize. ====================================================================== Step 3.0: Create ResultRetrieverV2 (copied from ResultRetriever) ====================================================================== Step 2: Create PageResult (copied from PageResultState) ====================================================================== Change AppSearch hawkeye testing app multidex to legacy. ====================================================================== Fix icing-search-engine_benchmark bug ====================================================================== (Small fix for step 1) Fix unit test stack memory error ====================================================================== Step 1: Create ResultStateV2 and ResultStateV2Test (copied from ResultState, ResultStateTest) ====================================================================== Bug: 146903474 Bug: 152934343 Bug: 158089703 Bug: 185845269 Bug: 193453081 Bug: 193919210 Bug: 209071710 Bug: 211785521 Bug: 218413237 Bug: 222349894 Bug: 223549255 Bug: 225914361 Bug: 229770338 Bug: 229778472 Bug: 230879098 Bug: 231237897 Bug: 231368517 Bug: 231416401 Bug: 232273174 Bug: 233470404 Bug: 233657885 Test: Presubmit Change-Id: I61a1336455544d12c35ad29f1e79a9a1eb9af14a
2022-06-16Merge remote-tracking branch 'aosp/upstream-master' into androidx-platform-devJiayu Hu
* aosp/upstream-master: Sync from upstream. Sync from upstream. Descriptions: ====================================================================== Export Icing logging tag to JNI ====================================================================== Update export_to_aosp.sh to change icing log tag to "AppSearchIcing" ====================================================================== Improve the logic of NamespaceChecker. ====================================================================== Step 4.1: Use ScoredDocumentHitsRanker in ResultStateV2 ====================================================================== Step 4.0: Create ScoredDocumentHitsRanker interface and PriorityQueueScoredDocumentHitsRanker ====================================================================== Refactor KeyMapper ====================================================================== Change Icing DEFAULT_LOGGING_LEVEL to INFO ====================================================================== Step 3.4: Create ResultRetrieverV2GroupResultLimiterTest (copied from ResultStateTest) ====================================================================== Add IcingDynamicTrie::IsBranchingTerm to check if a term is branching. ====================================================================== Fix IcingDynamicTrie::Delete bug ====================================================================== Step 3.3: Create ResultRetrieverV2ProjectionTest (copied from ResultRetrieverTest) ====================================================================== Step 3.2: Create ResultRetrieverV2SnippetTest (copied from ResultRetrieverTest) ====================================================================== Step 3.1: Create ResultRetrieverV2Test (copied from ResultRetrieverTest) ====================================================================== Enable legacy multidex ====================================================================== Fix NPE caused by improper handling of return value of GetFileSize. ====================================================================== Step 3.0: Create ResultRetrieverV2 (copied from ResultRetriever) ====================================================================== Step 2: Create PageResult (copied from PageResultState) ====================================================================== Change AppSearch hawkeye testing app multidex to legacy. ====================================================================== Fix icing-search-engine_benchmark bug ====================================================================== (Small fix for step 1) Fix unit test stack memory error ====================================================================== Step 1: Create ResultStateV2 and ResultStateV2Test (copied from ResultState, ResultStateTest) ====================================================================== Bug: 146903474 Bug: 152934343 Bug: 193919210 Bug: 231368517 Bug: 232273174 Bug: 233470404 Bug: 233657885 Test: ./gradlew appsearch:appsearch:connectedCheck appsearch:appsearch-local-storage:connectedCheck appsearch:appsearch-platform-storage:connectedCheck Change-Id: Icd76eab5cbe8fd945b7fca84de881c298904ce0c
2022-06-16Snap for 8736029 from 22706bbabc5ddda2b67894b6f3789b587abf2f91 to ↵Android Build Coastguard Worker
mainline-go-adbd-release Change-Id: Ibb1a7ce3c7ea12b90c38811dc4042b0656cc9700
2022-06-15Sync from upstream.Jiayu Hu
Descriptions: ====================================================================== Export Icing logging tag to JNI ====================================================================== Update export_to_aosp.sh to change icing log tag to "AppSearchIcing" ====================================================================== Improve the logic of NamespaceChecker. ====================================================================== Step 4.1: Use ScoredDocumentHitsRanker in ResultStateV2 ====================================================================== Step 4.0: Create ScoredDocumentHitsRanker interface and PriorityQueueScoredDocumentHitsRanker ====================================================================== Refactor KeyMapper ====================================================================== Change Icing DEFAULT_LOGGING_LEVEL to INFO ====================================================================== Step 3.4: Create ResultRetrieverV2GroupResultLimiterTest (copied from ResultStateTest) ====================================================================== Add IcingDynamicTrie::IsBranchingTerm to check if a term is branching. ====================================================================== Fix IcingDynamicTrie::Delete bug ====================================================================== Step 3.3: Create ResultRetrieverV2ProjectionTest (copied from ResultRetrieverTest) ====================================================================== Step 3.2: Create ResultRetrieverV2SnippetTest (copied from ResultRetrieverTest) ====================================================================== Step 3.1: Create ResultRetrieverV2Test (copied from ResultRetrieverTest) ====================================================================== Enable legacy multidex ====================================================================== Fix NPE caused by improper handling of return value of GetFileSize. ====================================================================== Step 3.0: Create ResultRetrieverV2 (copied from ResultRetriever) ====================================================================== Step 2: Create PageResult (copied from PageResultState) ====================================================================== Change AppSearch hawkeye testing app multidex to legacy. ====================================================================== Fix icing-search-engine_benchmark bug ====================================================================== (Small fix for step 1) Fix unit test stack memory error ====================================================================== Step 1: Create ResultStateV2 and ResultStateV2Test (copied from ResultState, ResultStateTest) ====================================================================== Bug: 146903474 Bug: 152934343 Bug: 193919210 Bug: 231368517 Bug: 232273174 Bug: 233470404 Bug: 233657885 Change-Id: Iae5461f8650c2bd58f683128fdd243d90403bc06
2022-06-14Sync from upstream.Jiayu Hu
Re-run the sync from go/oag/2105187 with the new script in cl/454956835 to update icing log tag to "AppSearchIcing". Change-Id: I60ab71d4fb78579407131d22543236f643efdb52
2022-06-06Snap for 8686562 from 5a0dcd8114afa3b0f93bf77a6c989f539555186c to ↵Android Build Coastguard Worker
mainline-go-adbd-release Change-Id: If1f0d7e885345106da637fc619d140e93bf6e058
2022-05-27Fix NPE caused by handling GetFileSize return value. am: 5a0dcd8114Tim Barron
Original change: https://googleplex-android-review.googlesource.com/c/platform/external/icing/+/18618724 Change-Id: I97ff8101b8108697244fc8bd5e692fa8c9710bd5 Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-05-26Fix NPE caused by handling GetFileSize return value.android13-devTim Barron
FileBackedVector wasn't properly checking the return value of GetFileSize when deciding whether or not it needed to grow the underlying file and remap. If GetFileSize failed due to an IO error, then FileBackedVector would conclude (incorrectly) that growing wasn't necessary and would attempt to access the mmapping at the requested index - which could result in either an NPE (if the vector was empty) or an invalid memory access (if the requested index exceeded the actual size of the mmapping). See b/232273174#comment44 for more details. This is a cherrypick of cl/451040413. Test: Builds Change-Id: I7ae0e3119126483c23a85dae3bd7a84712357209
2022-05-24Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devTim
* goog/upstream-master: Sync from upstream. Sync from upstream. Descriptions: ====================================================================== Fix bug in schema store where a failure during RegenerateDerivedFiles would lead to a dangling pointer. ====================================================================== Add RAII class that will create and destroy file directories. ====================================================================== Convert MainIndexDebugInfoProto and LiteIndexDebugInfoProto to string ====================================================================== Make SchemaStore move assignable. ====================================================================== Rollback of "convert the string lexicon debug information to a protocol buffer" ====================================================================== Fix NPE caused by a remap failure. ====================================================================== Unify the name "priority" and "severity" in Icing logging ====================================================================== Avoiding string formatting in Icing logging when we should not log ====================================================================== Switch to use an enum with BASIC/DETAILED to control the verbosity of getDebugInfo ====================================================================== Remove the behavior in the Language Segmenter to filter out non-ascii+non-alphanumeric characters. ====================================================================== Fix the SetSchema bug when we override a schema with nested incompatible types ====================================================================== Wrap __android_log_write with __android_is_loggable ====================================================================== Enable removing expired page tokens to free cache space ====================================================================== Bug: 146903474 Bug: 193453081 Bug: 222349894 Bug: 229770338 Bug: 229778472 Bug: 230879098 Bug: 231416401 Bug: 231237897 Bug: 232273174 Test: ./gradlew appsearch:appsearch:connectedCheck appsearch:appsearch-platform-storage:connectedCheck appsearch:appsearch-local-storage:connectedCheck Change-Id: Icd707c8704e11316d40d0f2cec929e08776d75dc
2022-05-24Sync from upstream.Tim Barron
This sync simply re-runs the sync from go/oag/2101540 , but with a fix in the export script that ensures the newly added RAII classes in the jni folder are properly exported. Change-Id: I67d769bee6a602f5d8ac15c16d8feb49819a246a
2022-05-24Fix dangling pointer from failure in SchemaStore::RegenerateDerivedFiles. ↵Tim Barron
am: c20fd6b697 Original change: https://googleplex-android-review.googlesource.com/c/platform/external/icing/+/18562449 Change-Id: Icc1035bf8244a381238cd5dccb31cca7d7524ebf Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-05-23Fix dangling pointer from failure in SchemaStore::RegenerateDerivedFiles.Tim Barron
Essentially, SchemaStore::SetSchema would work in three steps: 1. Write the new schema 2. Throw out derived data structures 3. Regenerate the derived date structures A failure in the middle of step 3 could cause SectionManager to maintain a a reference to a now-deleted instance of KeyMapper which would then crash on the next access. This resolves that issue via shadowing to ensure that changes are only applied if everything is successfully generated. You can find the full description of the bug at b/231416401#comment10. This is a cherrypick of cl/449537037 and cl/450024848. The changes in cl/449849170 (and replaced with direct calls to delete the temp directories) were excluded to keep the size of cherrypick smaller. Test: Builds Change-Id: Ic018ade45b5aab4b797a0a654eaea6ac424469ab
2022-05-21Snap for 8626064 from 574b723a7ef8a009905919e5286aafccb739f069 to ↵Android Build Coastguard Worker
mainline-go-adbd-release Change-Id: I4426b30c339799617e95b3318d6636aa6536047a
2022-05-20Sync from upstream.Tim Barron
Descriptions: ====================================================================== Fix bug in schema store where a failure during RegenerateDerivedFiles would lead to a dangling pointer. ====================================================================== Add RAII class that will create and destroy file directories. ====================================================================== Convert MainIndexDebugInfoProto and LiteIndexDebugInfoProto to string ====================================================================== Make SchemaStore move assignable. ====================================================================== Rollback of "convert the string lexicon debug information to a protocol buffer" ====================================================================== Fix NPE caused by a remap failure. ====================================================================== Unify the name "priority" and "severity" in Icing logging ====================================================================== Avoiding string formatting in Icing logging when we should not log ====================================================================== Switch to use an enum with BASIC/DETAILED to control the verbosity of getDebugInfo ====================================================================== Remove the behavior in the Language Segmenter to filter out non-ascii+non-alphanumeric characters. ====================================================================== Fix the SetSchema bug when we override a schema with nested incompatible types ====================================================================== Wrap __android_log_write with __android_is_loggable ====================================================================== Enable removing expired page tokens to free cache space ====================================================================== Bug: 146903474 Bug: 193453081 Bug: 222349894 Bug: 229770338 Bug: 229778472 Bug: 230879098 Bug: 231416401 Bug: 231237897 Bug: 232273174 Change-Id: I22f050de16f56dce39e12a7033947519d598c840
2022-05-17Merge "Revert "Merge remote-tracking branch 'goog/upstream-master' into ↵Terry Wang
androidx-platform-dev"" into androidx-platform-dev
2022-05-16Revert "Merge remote-tracking branch 'goog/upstream-master' into ↵Terry Wang
androidx-platform-dev" This reverts commit b992497e56b3dca67d2e8bd618ec40b457b7a21e. Reason for revert: This CL increases AppSearch aar file from 3.76 MiB to 3.92 MiB (+164.48 KiB, +4.27%) in build 8507445. Revert this CL so that we can avoid the size increase and re-work the debug info proto. Change-Id: If75c5ab71306462fba5c495a0231d6b7dd6d67fa
2022-05-14Fix NPE bug caused by a failed Remap. am: 574b723a7eTim Barron
Original change: https://googleplex-android-review.googlesource.com/c/platform/external/icing/+/18385601 Change-Id: I088c0f6702ea49fe05386ec24980b57e6ef40628 Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-05-14Fix NPE bug caused by a failed Remap.Tim Barron
Cherrypick a change that fixes an error in MemoryMappedFile::Remap. Essentially, that error meant that a previously valid mapping could become invalid if Remap were to fail. This is a cherrypick of cl/448387045. Bug: 232273174 Test: builds Change-Id: If794d820583dad347edc319ca72cc8ecf773a100
2022-05-12Merge "Stop dropping nonascii/nonalnum segments." into tm-dev am: 34f7ab9226Tim Barron
Original change: https://googleplex-android-review.googlesource.com/c/platform/external/icing/+/18328740 Change-Id: I9f0f63e7ecb8ce83d1965b72b8547f19b5cdad0c Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-05-12Merge "Stop dropping nonascii/nonalnum segments." into tm-devTim Barron
2022-05-12Fix bug overriding nested incompatible types. am: 5fa917ce71Tim Barron
Original change: https://googleplex-android-review.googlesource.com/c/platform/external/icing/+/18328263 Change-Id: Id7f5471b03eb15647229361b90bb1d4f71c7fd15 Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-05-11Stop dropping nonascii/nonalnum segments.Tim Barron
Cherrypick a change that would remove the existing behavior of the LanguageSegmenter to filer out non-ascii+non-alphanumeric segments. This behavior is more appropriate for a higher level class like the tokenizer. This will allow a consistent hanlding: punctuation and whitespace is not indexed. All other segments are. This is a cherrypick of cl/446831054. Bug: 229770338 Test: builds Change-Id: Id6056354aaea5fca1cdb0e9300065bd3f91e401b
2022-05-11Fix bug overriding nested incompatible types.Tim Barron
Cherrypick a bug fix that would cause SetSchema to fail even with override=true, if a client made an incompatible change to a type that is nested in a newly added type. This is a cherrypick of cl/447787757. Bug: 230879098 Test: builds Change-Id: If8ee54f903f450d938a856d87110fb333aeecb09
2022-05-04Merge branch 'goog/upstream-master' into androidx-platform-devAlexander Dorokhine
These changes have already been merged before automerger paths have been set up, so this is a no-op merge. Test: presubmit Change-Id: Iba37100a39d87d3d59a8c9354ad7bb639192b6c0
2022-05-03Sync from upstream.Jiayu Hu
Descriptions: ====================================================================== Export Icing logging control to JNI ====================================================================== Prepare Icing logging class for JNI export ====================================================================== Export getDebugInfo to JNI ====================================================================== Expose the return_deleted_document_info parameter for deleteByQuery JNI ====================================================================== Enable runtime log control for Icing Library ====================================================================== Fix 1 ClangTidyBuild finding: ====================================================================== Update comments to run benchmarks. ====================================================================== Making icing's own logging class ====================================================================== Convert the string lexicon debug information to a protocol buffer ====================================================================== Fix issue with printing fingerprinted key in our error logs. ====================================================================== Support dump function for IcingSearchEngine ====================================================================== Bug: 146903474 Bug: 229778472 Bug: 209071710 Bug: 222349894 Bug: 225914361 Change-Id: I9750149d1ed0b59f345b8828ff312a62773667fe
2022-04-27Snap for 8505378 from fbf0cddf25ffa157a388d0f717c40a6372531d00 to ↵Android Build Coastguard Worker
mainline-go-adbd-release Change-Id: I88dbbe5ae3dfbfdcfafbe7533467078708ecd3df
2022-04-26Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devJiayu Hu
* goog/upstream-master: Sync from upstream. Descriptions: ====================================================================== Export Icing logging control to JNI ====================================================================== Prepare Icing logging class for JNI export ====================================================================== Export getDebugInfo to JNI ====================================================================== Expose the return_deleted_document_info parameter for deleteByQuery JNI ====================================================================== Enable runtime log control for Icing Library ====================================================================== Fix 1 ClangTidyBuild finding: ====================================================================== Update comments to run benchmarks. ====================================================================== Making icing's own logging class ====================================================================== Convert the string lexicon debug information to a protocol buffer ====================================================================== Fix issue with printing fingerprinted key in our error logs. ====================================================================== Support dump function for IcingSearchEngine ====================================================================== Bug: 146903474 Bug: 229778472 Bug: 209071710 Bug: 222349894 Bug: 225914361 Test: ./gradlew appsearch:appsearch:connectedCheck appsearch:appsearch-local-storage:connectedCheck appsearch:appsearch-platform-storage:connectedCheck Change-Id: I10729bd38efbb342d184762b3d91537d47fb1c2f
2022-04-26Sync from upstream.Jiayu Hu
Descriptions: ====================================================================== Export Icing logging control to JNI ====================================================================== Prepare Icing logging class for JNI export ====================================================================== Export getDebugInfo to JNI ====================================================================== Expose the return_deleted_document_info parameter for deleteByQuery JNI ====================================================================== Enable runtime log control for Icing Library ====================================================================== Fix 1 ClangTidyBuild finding: ====================================================================== Update comments to run benchmarks. ====================================================================== Making icing's own logging class ====================================================================== Convert the string lexicon debug information to a protocol buffer ====================================================================== Fix issue with printing fingerprinted key in our error logs. ====================================================================== Support dump function for IcingSearchEngine ====================================================================== Bug: 146903474 Bug: 229778472 Bug: 209071710 Bug: 222349894 Bug: 225914361 Change-Id: I70056cb261d54cfa869c6bf9b8251752bcfc0142
2022-04-26Fix for memory leaks. am: fbf0cddf25Christopher Ferris
Original change: https://googleplex-android-review.googlesource.com/c/platform/external/icing/+/17902806 Change-Id: Ib0b79ef99623d37031c46c8fcf6741ec4b1556c4 Ignore-AOSP-First: this is an automerge Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-04-21Fix for memory leaks.Christopher Ferris
Bug: 230049434 Test: No more leaks. Change-Id: I0b694c3c1accba109ee18f260da98a0850b8f949
2022-04-12Sync from upstream.Tim Barron
Descriptions: ====================================================================== Add some additional logging that will help diagnose b/218413237 ====================================================================== Mark VerbatimTokenizer::ResetToTokenStartingAfter as 'override'. ====================================================================== Support dump function for SchemaStore ====================================================================== Bug: 218413237 Change-Id: I9efd1dd388cd510df15989c84a8577d4ba56ab3c
2022-04-12Sync from upstream.Tim Barron
====================================================================== Refactor DocumentStore::Initialize to improve readability of document store recovery. ====================================================================== Remove non-NDK API usages of ICU4C in libicing. ====================================================================== Move IcuDataFileHelper to the testing directory since it is a test-only util. ====================================================================== Support dump function for DocumentStore ====================================================================== Switch to use PRead rather than MMap in the proto log. ====================================================================== Support dump function for main/lite index and lexicon ====================================================================== Fix LiteIndex::AppendHits ====================================================================== Enable and fix DocumentStoreTest.LoadScoreCacheAndInitializeSuccessfully ====================================================================== Fix MainIndex::GetStorageInfo. ====================================================================== Fix icing-search-engine_fuzz_test by making IcuLanguageSegmenterIterator::Advance non-recursive. ====================================================================== Allow to return additional information for deleted documents in DeleteByQuery ====================================================================== Using enum class in Token::Type for better type safety. ====================================================================== Bug: 158089703 Bug: 185845269 Bug: 209071710 Bug: 211785521 Bug: 218413237 Bug: 223549255 Change-Id: Id2786047ab279734bdd2aee883e82607b6a0e403
2022-03-29Merge "Move C++ version flag into CMakeLists.txt" into androidx-platform-devAlex Saveliev
2022-03-29[automerge] Switch libicing to depend on libicu. 2p: c885445a01Presubmit Automerger Backend
Original change: https://googleplex-android-review.googlesource.com/c/platform/external/icing/+/17454195 Bug: 223549255 Change-Id: I0f9686eb5d5a5f4ec05d05b67bff849b704eb28e
2022-03-28Switch libicing to depend on libicu.Tim Barron
Bug: 223549255 Test: atest --rebuild-module-info CtsAppSearchTestCases Change-Id: I8747c37b51dd46c2f756a99abb854e3a3d3dab02
2022-03-23Move C++ version flag into CMakeLists.txtAlex Saveliev
This is the better place to set this, as setting it in build.gradle will overwrite other flags we need. Test: Try building libicing.so Change-Id: If265ea1439e888c15c87d48f9176b2fc772ab7c4
2022-03-23[automerge] Merge remote-tracking branch 'goog/androidx-platform-dev' into ↵Presubmit Automerger Backend
tm-dev 2p: 9ab600c39d Original change: https://googleplex-android-review.googlesource.com/c/platform/external/icing/+/17344381 Bug: 152934343 Bug: 158089703 Bug: 185845269 Bug: 203700301 Bug: 204333391 Bug: 205209589 Bug: 206147728 Bug: 209071710 Bug: 209993976 Bug: 218413237 Bug: 223549255 Change-Id: I618e97d4d7ad507eb00778024362f94a40dd65e8
2022-03-23Merge remote-tracking branch 'goog/androidx-platform-dev' into tm-devAlexander Dorokhine
* goog/androidx-platform-dev: Sync from upstream. Sync from upstream. Sync from upstream. Descriptions: ====================================================================== Add some additional logging that will help diagnose b/218413237 ====================================================================== Mark VerbatimTokenizer::ResetToTokenStartingAfter as 'override'. ====================================================================== Support dump function for SchemaStore ====================================================================== Refactor DocumentStore::Initialize to improve readability of document store recovery. ====================================================================== Remove non-NDK API usages of ICU4C in libicing. ====================================================================== Move IcuDataFileHelper to the testing directory since it is a test-only util. ====================================================================== Support dump function for DocumentStore ====================================================================== Switch to use PRead rather than MMap in the proto log. ====================================================================== Support dump function for main/lite index and lexicon ====================================================================== Fix LiteIndex::AppendHits ====================================================================== Enable and fix DocumentStoreTest.LoadScoreCacheAndInitializeSuccessfully ====================================================================== Fix MainIndex::GetStorageInfo. ====================================================================== Fix icing-search-engine_fuzz_test by making IcuLanguageSegmenterIterator::Advance non-recursive. ====================================================================== Allow to return additional information for deleted documents in DeleteByQuery ====================================================================== Using enum class in Token::Type for better type safety. ====================================================================== Normalize Tokens by Token type when retrieving snippets ================ Rename max_window_bytes to max_window_utf32_length, Delete the max_tokens_per_doc field in IcingSearchEngineOptions. ================ Handle suggestion namespace ownership. ================ Fix OkStatus() is not a valid argument to StatusOr in Main_index.RetrieveMoreHits. ================ Allow advancing when current indices are negative in CharacterIterator ================ Adds support for verbatim tokenization and indexing in IcingLib ================ Renames TokenizerIterator Reset functions ================ Add term_match_type to SuggestionSpec proto ================ Unify the C++ proto enum style ================ Allow zero property weights in IcingLib Bug: 152934343 Bug: 158089703 Bug: 185845269 Bug: 203700301 Bug: 204333391 Bug: 205209589 Bug: 206147728 Bug: 209071710 Bug: 209993976 Bug: 218413237 Bug: 218413237 Bug: 223549255 Test: Presubmit Change-Id: I96665ba718f89e69ca99cd833ad80fa555edf436
2022-03-21Merge remote-tracking branch 'goog/upstream-master' into libicuTim Barron
* goog/upstream-master: Sync from upstream. Sync from upstream. Sync from upstream. Descriptions: ====================================================================== Add some additional logging that will help diagnose b/218413237 ====================================================================== Mark VerbatimTokenizer::ResetToTokenStartingAfter as 'override'. ====================================================================== Support dump function for SchemaStore ====================================================================== Refactor DocumentStore::Initialize to improve readability of document store recovery. ====================================================================== Remove non-NDK API usages of ICU4C in libicing. ====================================================================== Move IcuDataFileHelper to the testing directory since it is a test-only util. ====================================================================== Support dump function for DocumentStore ====================================================================== Switch to use PRead rather than MMap in the proto log. ====================================================================== Support dump function for main/lite index and lexicon ====================================================================== Fix LiteIndex::AppendHits ====================================================================== Enable and fix DocumentStoreTest.LoadScoreCacheAndInitializeSuccessfully ====================================================================== Fix MainIndex::GetStorageInfo. ====================================================================== Fix icing-search-engine_fuzz_test by making IcuLanguageSegmenterIterator::Advance non-recursive. ====================================================================== Allow to return additional information for deleted documents in DeleteByQuery ====================================================================== Using enum class in Token::Type for better type safety. ====================================================================== Normalize Tokens by Token type when retrieving snippets ================ Rename max_window_bytes to max_window_utf32_length, Delete the max_tokens_per_doc field in IcingSearchEngineOptions. ================ Handle suggestion namespace ownership. ================ Fix OkStatus() is not a valid argument to StatusOr in Main_index.RetrieveMoreHits. ================ Allow advancing when current indices are negative in CharacterIterator ================ Adds support for verbatim tokenization and indexing in IcingLib ================ Renames TokenizerIterator Reset functions ================ Add term_match_type to SuggestionSpec proto ================ Unify the C++ proto enum style ================ Allow zero property weights in IcingLib ================ Replace refs to c lib headers w/ c++ stdlib equivalents. ================ Update IDF component of BM25F Calculator in IcingLib ================ Expose QuerySuggestions API. ================ Change the tokenizer used in QuerySuggest. ================ Add SectionWeights API to Icing. ================ Apply SectionWeights to BM25F Scoring. ================ Replaces uses of u_strTo/FromUTF32 w/ u_strTo/FromUTF8. ================ Implement ranking in FindTermByPrefix. ================ Fork proto's GzipStream into Icing Lib. ================ Remove token limit behavior from index-processor. ================ Remove no-longer-used write paths for file-backed-proto-log. ================ Modify segmentation rules to consider any segment that begins with a non-Ascii alphanumeric character as valid ================= Implement CalculateNormalizedMatchLength for IcuNormalizer. ================ Add additional benchmark cases that were useful in developing submatching and CalculateNormalizedMatchLength for IcuNormalizer ================= Switch NormalizationMap from static const std::unordered_map<char16_t, char16_t>& to static const std::unordered_map<char16_t, char16_t> *const. ================== Add Initialization Count marker file to break out of crash loops. ================== Delete the Simple Normalizer. ================== Add submatch information to identify a submatch within a document. ================== Bug: 147509515 Bug: 149610413 Bug: 152934343 Bug: 158089703 Bug: 185845269 Bug: 195720764 Bug: 196257995 Bug: 196771754 Bug: 202308641 Bug: 203700301 Bug: 204333391 Bug: 205209589 Bug: 206147728 Bug: 209071710 Bug: 209993976 Bug: 218413237 Bug: 223549255 Test: ./gradlew appsearch:appsearch:connectedCheck appsearch:appsearch-local-storage:connectedCheck appsearch:appsearch-platform-storage:connectedCheck Change-Id: I66c8d78b99ced978feaea3459ccff0cf5b22ff27
2022-03-21Sync from upstream.Tim Barron
Descriptions: ====================================================================== Add some additional logging that will help diagnose b/218413237 ====================================================================== Mark VerbatimTokenizer::ResetToTokenStartingAfter as 'override'. ====================================================================== Support dump function for SchemaStore ====================================================================== Bug: 218413237 Change-Id: I8554dc14b9e9b5aec56e4e9a3b3aa07c0e7d0f1b
2022-03-19Add icing to hwasan-postsubmit am: 37a87538c1 am: c6b7b34736 am: 90f9657024 ↵Florian Mayer
am: 19600c2c36 Original change: https://android-review.googlesource.com/c/platform/external/icing/+/2032711 Change-Id: I04785b123e66c3c274b69a8ef92c3779332d2753
2022-03-19Add icing to hwasan-postsubmit am: 37a87538c1 am: c6b7b34736 am: 90f9657024Florian Mayer
Original change: https://android-review.googlesource.com/c/platform/external/icing/+/2032711 Change-Id: Icca643f192e607083f4c6a0342287d6807d3829c
2022-03-19Add icing to hwasan-postsubmit am: 37a87538c1 am: c6b7b34736Florian Mayer
Original change: https://android-review.googlesource.com/c/platform/external/icing/+/2032711 Change-Id: Id481a4b83694d5dce2397779072ee91ff5a1a2dc
2022-03-19Add icing to hwasan-postsubmit am: 37a87538c1Florian Mayer
Original change: https://android-review.googlesource.com/c/platform/external/icing/+/2032711 Change-Id: I56ed9e691ba86d3e8a0675a0cef1150e1eab4b30
2022-03-18Add icing to hwasan-postsubmitFlorian Mayer
Test: Run tests under HWASan. Bug: 193568145 Change-Id: I9962cc83c3e247d0030029122777b60492b93d25
2022-03-17Sync from upstream.Tim Barron
This sync simply re-runs the sync from ag/17226641, but with a fix in the export script that will prevent proto/icing/query/list_filter/syntax.proto from being exported. Change-Id: Iffbb68b6be93410350859ebeae4737518804ae7f
2022-03-16Sync from upstream.Tim Barron
This sync will bring goog/upstream-master into agreement with aosp/upstream-master. Unfortunately merging any permutation of aosp/upstream-master or androidx-main into goog/upstream-master or androidx-platform-dev isn't doable because I don't own some of the commits in the chain. Descriptions: ====================================================================== Refactor DocumentStore::Initialize to improve readability of document store recovery. ====================================================================== Remove non-NDK API usages of ICU4C in libicing. ====================================================================== Move IcuDataFileHelper to the testing directory since it is a test-only util. ====================================================================== Support dump function for DocumentStore ====================================================================== Switch to use PRead rather than MMap in the proto log. ====================================================================== Support dump function for main/lite index and lexicon ====================================================================== Fix LiteIndex::AppendHits ====================================================================== Enable and fix DocumentStoreTest.LoadScoreCacheAndInitializeSuccessfully ====================================================================== Fix MainIndex::GetStorageInfo. ====================================================================== Fix icing-search-engine_fuzz_test by making IcuLanguageSegmenterIterator::Advance non-recursive. ====================================================================== Allow to return additional information for deleted documents in DeleteByQuery ====================================================================== Using enum class in Token::Type for better type safety. ====================================================================== Normalize Tokens by Token type when retrieving snippets ================ Rename max_window_bytes to max_window_utf32_length, Delete the max_tokens_per_doc field in IcingSearchEngineOptions. ================ Handle suggestion namespace ownership. ================ Fix OkStatus() is not a valid argument to StatusOr in Main_index.RetrieveMoreHits. ================ Allow advancing when current indices are negative in CharacterIterator ================ Adds support for verbatim tokenization and indexing in IcingLib ================ Renames TokenizerIterator Reset functions ================ Add term_match_type to SuggestionSpec proto ================ Unify the C++ proto enum style ================ Allow zero property weights in IcingLib ================ Replace refs to c lib headers w/ c++ stdlib equivalents. ================ Update IDF component of BM25F Calculator in IcingLib ================ Expose QuerySuggestions API. ================ Change the tokenizer used in QuerySuggest. ================ Add SectionWeights API to Icing. ================ Apply SectionWeights to BM25F Scoring. ================ Replaces uses of u_strTo/FromUTF32 w/ u_strTo/FromUTF8. ================ Implement ranking in FindTermByPrefix. ================ Fork proto's GzipStream into Icing Lib. ================ Remove token limit behavior from index-processor. ================ Remove no-longer-used write paths for file-backed-proto-log. ================ Modify segmentation rules to consider any segment that begins with a non-Ascii alphanumeric character as valid ================= Implement CalculateNormalizedMatchLength for IcuNormalizer. ================ Add additional benchmark cases that were useful in developing submatching and CalculateNormalizedMatchLength for IcuNormalizer ================= Switch NormalizationMap from static const std::unordered_map<char16_t, char16_t>& to static const std::unordered_map<char16_t, char16_t> *const. ================== Add Initialization Count marker file to break out of crash loops. ================== Delete the Simple Normalizer. ================== Add submatch information to identify a submatch within a document. ================== Bug: 147509515 Bug: 149610413 Bug: 152934343 Bug: 158089703 Bug: 185845269 Bug: 195720764 Bug: 196257995 Bug: 196771754 Bug: 202308641 Bug: 203700301 Bug: 204333391 Bug: 205209589 Bug: 206147728 Bug: 209071710 Bug: 209993976 Bug: 218413237 Bug: 223549255 Change-Id: I87c5da3a80463d62b8d24e8ff8a9a91952d3fc64
2022-03-09Merge "Add min_sdk_version rules to allow building as an updatable apex." ↵Alexander Dorokhine
into tm-dev am: 9cb1d5d4a5 Original change: https://googleplex-android-review.googlesource.com/c/platform/external/icing/+/16998967 Change-Id: I829e2ccc01f2374c3f0af76ffd07b12637189cc2
2022-03-09Merge "Add min_sdk_version rules to allow building as an updatable apex." ↵Alexander Dorokhine
into tm-dev
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-23 15:31:08 +0000