aboutsummaryrefslogtreecommitdiff
path: root/icing
AgeCommit message (Collapse)Author
2023-12-04Merge remote-tracking branch 'aosp/upstream-master' into androidx-mainTerry Wang
* aosp/upstream-master: Update Icing from upstream. Descriptions: ======================================================================== Rollback "Add a checker to verify the property data type matches the schema." ======================================================================== Bug: 299321977 Change-Id: I569f75db16611e0574b2c8df40508ec2ebd2d498
2023-12-04Update Icing from upstream.Terry Wang
Descriptions: ======================================================================== Rollback "Add a checker to verify the property data type matches the schema." ======================================================================== Bug: 299321977 Change-Id: I9f448463ecaae2b44ce31833a90d9672c33907b1
2023-12-01Merge remote-tracking branch 'aosp/upstream-master' into androidx-mainJiayu Hu
* aosp/upstream-master: Update Icing from upstream. Update Icing from upstream. Update Icing from upstream. Descriptions: ======================================================================== Fix the build error in aosp/2856025 ======================================================================== [Icing][version 3] Bump kVersion to 3 ======================================================================== Make lite index magic dependent on `IcingSearchEngineOptions::build_property_existence_metadata_hits` ======================================================================== Add a flag in IcingSearchEngineOptions to control whether to build property existence metadata hits ======================================================================== Support `hasProperty(property_path)` in the advanced query language ======================================================================== Add PropertyExistenceIndexingHandler to index property existence metadata hit ======================================================================== [JoinIndex Improvement][11/x] Add IcingSearchEngine initialization unit test for switching join index ======================================================================== [JoinIndex Improvement][10/x] Change/Add IcingSearchEngine unit tests ======================================================================== [JoinIndex Improvement][9/x] Integrate QualifiedIdJoinIndexImplV2 with IcingSearchEngine ======================================================================== [JoinIndex Improvement][8/x] Integrate QualifiedIdJoinIndexImplV2 with JoinProcessor ======================================================================== [JoinIndex Improvement][8/x] Integrate QualifiedIdJoinIndexImplV2 with QualifiedIdJoinIndexingHandler ======================================================================== [JoinIndex Improvement][7/x] Create QualifiedIdJoinIndex interface ======================================================================== [JoinIndex Improvement][6.1/x] Unit test (Optimize) ======================================================================== [JoinIndex Improvement][6.0/x] Unit test (General, Put, GetIterator) ======================================================================== [JoinIndex Improvement][5.3/x] Implement Optimize ======================================================================== Remove accents from Greek letters in normalizer ======================================================================== Make arm emulator tests build-only. ======================================================================== [JoinIndex Improvement][5.2/x] Implement GetIterator ======================================================================== [JoinIndex Improvement][5.1/x] Implement Put ======================================================================== [JoinIndex Improvement][5.0/x] Branch QualifiedIdJoinIndex to QualifiedIdJoinIndexImplV2 ======================================================================== [JoinIndex Improvement][4/x] Implement PostingListJoinDataAccessor ======================================================================== [JoinIndex Improvement][3/x] Implement PostingListJoinDataSerializer and DocumentIdToJoinInfo data type ======================================================================== [JoinIndex Improvement][2/x] Create NamespaceFingerprintIdentifier ======================================================================== [JoinIndex Improvement][1/x] Implement namespace_id_old_to_new in Compaction ======================================================================== Update test to also handle ICU 74 segmentation rules. ======================================================================== [Icing][Expand QueryStats][3/x] Add new fields into QueryStats (1) ======================================================================== [Icing][Expand QueryStats][2/x] Refactor QueryStatsProto ======================================================================== [Icing][Expand QueryStats][1/x] Publish DocHitInfoIterator CallStats ======================================================================== Add additional property filter tests ======================================================================== Deprecate hit_intersect_section_ids_mask in DocHitInfoIterator ======================================================================== Change default requires_full_emulation to False for portable_cc_test (third_party/icing/testing) ======================================================================== Cleanup Set requires_full_emulation to True for selective tests ======================================================================== Fix monkey test failures ======================================================================== Complete monkey test logic to change schema during monkey test runtime ======================================================================== Refactor monkey test to prepare for schema update ======================================================================== Fix the schema bug found by monkey test with seed 2551429844 ======================================================================== Move set query stats to the very top of InternalSearch() ======================================================================== Apply section restriction only on leaf nodes ======================================================================== [6/n] Fix callsites in Icing that forgot to check libtextclassifier3::Status (Advanced query parser) ======================================================================== [5/n] Fix callsites in Icing that forgot to check libtextclassifier3::Status (PersistentHashMap) ======================================================================== [4/n] Fix callsites in Icing that forgot to check libtextclassifier3::Status (PostingListIntegerIndexSerializer) ======================================================================== [3/n] Fix callsites in Icing that forgot to check libtextclassifier3::Status (PostingListHitSerializer) ======================================================================== [2/n] Fix callsites in Icing that forgot to check libtextclassifier3::Status (Posting list storage) ======================================================================== [1/n] Fix callsites in Icing that forgot to check libtextclassifier3::Status (Non-functional changes) ======================================================================== Decouple section restriction data from iterators ======================================================================== Fix the crash when a schema type gets more indexable properties than allowed ======================================================================== Add a checker to verify the property data type matches the schema. ======================================================================== Change global std::string in i18n-utils to constexpr std::string_view. ======================================================================== Adjust LiteIndex sort at indexing check conditions. ======================================================================== Add @UsedByNative annotations to native* methods in IcingSearchEngineImpl. ======================================================================== Bug: 305098009 Bug: 307508735 Bug: 291130542 Bug: 275121148 Bug: 303239901 Bug: 301116242 Bug: 299321977 Bug: 300135897 Bug: 297549761 Bug: 309826655 Bug: 296349369 Bug: 302192690 Bug: 302609704 Bug: 301566713 Bug: 296938196 NO_IFTTT="False Alarm: The path is only valid in G3. kVersion is changed to 3, and schema is compatible with version 1." Change-Id: I1d50cf70261c8977e52047c8051e8d143f62ba2c
2023-12-01Update Icing from upstream.Jiayu Hu
Descriptions: ======================================================================== Fix the build error in aosp/2856025 ======================================================================== Change-Id: I8a1030d95d3221cc8484b98e0401b61b4e62a289
2023-11-30Update Icing from upstream.Jiayu Hu
Descriptions: ======================================================================== [Icing][version 3] Bump kVersion to 3 ======================================================================== Make lite index magic dependent on `IcingSearchEngineOptions::build_property_existence_metadata_hits` ======================================================================== Add a flag in IcingSearchEngineOptions to control whether to build property existence metadata hits ======================================================================== Support `hasProperty(property_path)` in the advanced query language ======================================================================== Add PropertyExistenceIndexingHandler to index property existence metadata hit ======================================================================== [JoinIndex Improvement][11/x] Add IcingSearchEngine initialization unit test for switching join index ======================================================================== [JoinIndex Improvement][10/x] Change/Add IcingSearchEngine unit tests ======================================================================== [JoinIndex Improvement][9/x] Integrate QualifiedIdJoinIndexImplV2 with IcingSearchEngine ======================================================================== [JoinIndex Improvement][8/x] Integrate QualifiedIdJoinIndexImplV2 with JoinProcessor ======================================================================== [JoinIndex Improvement][8/x] Integrate QualifiedIdJoinIndexImplV2 with QualifiedIdJoinIndexingHandler ======================================================================== [JoinIndex Improvement][7/x] Create QualifiedIdJoinIndex interface ======================================================================== [JoinIndex Improvement][6.1/x] Unit test (Optimize) ======================================================================== [JoinIndex Improvement][6.0/x] Unit test (General, Put, GetIterator) ======================================================================== [JoinIndex Improvement][5.3/x] Implement Optimize ======================================================================== Remove accents from Greek letters in normalizer ======================================================================== Make arm emulator tests build-only. ======================================================================== [JoinIndex Improvement][5.2/x] Implement GetIterator ======================================================================== [JoinIndex Improvement][5.1/x] Implement Put ======================================================================== [JoinIndex Improvement][5.0/x] Branch QualifiedIdJoinIndex to QualifiedIdJoinIndexImplV2 ======================================================================== [JoinIndex Improvement][4/x] Implement PostingListJoinDataAccessor ======================================================================== [JoinIndex Improvement][3/x] Implement PostingListJoinDataSerializer and DocumentIdToJoinInfo data type ======================================================================== [JoinIndex Improvement][2/x] Create NamespaceFingerprintIdentifier ======================================================================== [JoinIndex Improvement][1/x] Implement namespace_id_old_to_new in Compaction ======================================================================== Update test to also handle ICU 74 segmentation rules. ======================================================================== [Icing][Expand QueryStats][3/x] Add new fields into QueryStats (1) ======================================================================== [Icing][Expand QueryStats][2/x] Refactor QueryStatsProto ======================================================================== [Icing][Expand QueryStats][1/x] Publish DocHitInfoIterator CallStats ======================================================================== Add additional property filter tests ======================================================================== Deprecate hit_intersect_section_ids_mask in DocHitInfoIterator ======================================================================== Change default requires_full_emulation to False for portable_cc_test (third_party/icing/testing) ======================================================================== Cleanup Set requires_full_emulation to True for selective tests ======================================================================== Fix monkey test failures ======================================================================== Complete monkey test logic to change schema during monkey test runtime ======================================================================== Refactor monkey test to prepare for schema update ======================================================================== Fix the schema bug found by monkey test with seed 2551429844 ======================================================================== Move set query stats to the very top of InternalSearch() ======================================================================== Apply section restriction only on leaf nodes ======================================================================== [6/n] Fix callsites in Icing that forgot to check libtextclassifier3::Status (Advanced query parser) ======================================================================== [5/n] Fix callsites in Icing that forgot to check libtextclassifier3::Status (PersistentHashMap) ======================================================================== [4/n] Fix callsites in Icing that forgot to check libtextclassifier3::Status (PostingListIntegerIndexSerializer) ======================================================================== [3/n] Fix callsites in Icing that forgot to check libtextclassifier3::Status (PostingListHitSerializer) ======================================================================== [2/n] Fix callsites in Icing that forgot to check libtextclassifier3::Status (Posting list storage) ======================================================================== [1/n] Fix callsites in Icing that forgot to check libtextclassifier3::Status (Non-functional changes) ======================================================================== Decouple section restriction data from iterators ======================================================================== Fix the crash when a schema type gets more indexable properties than allowed ======================================================================== Add a checker to verify the property data type matches the schema. ======================================================================== Change global std::string in i18n-utils to constexpr std::string_view. ======================================================================== Adjust LiteIndex sort at indexing check conditions. ======================================================================== Bug: 305098009 Bug: 307508735 Bug: 291130542 Bug: 275121148 Bug: 303239901 Bug: 301116242 Bug: 299321977 Bug: 300135897 Bug: 297549761 Bug: 309826655 Bug: 296349369 Bug: 302192690 Bug: 302609704 Bug: 301566713 NO_IFTTT="False Alarm: The path is only valid in G3. kVersion is changed to 3, and schema is compatible with version 1." Change-Id: I8c4c3cd9b93e5240bd774f0a3d6d812f7a9ec198
2023-08-31Merge remote-tracking branch 'aosp/upstream-master' into androidx-mainandroidx-wear-wear-tooling-preview-releaseandroidx-core-performance-releaseGrace Zhao
* aosp/upstream-master: Update Icing from upstream. Descriptions: ======================================================================== [fixit] Remove incorrect reserve usage for STL data structure ======================================================================== Move LiteIndex::SortHits() from the query path to the indexing path ======================================================================== Bug: 286418010 Bug: 297549761 Bug: 248295995 Change-Id: I6fcfd150036dc47a42181dfa14b410220da93d31
2023-08-30Update Icing from upstream.Grace Zhao
Descriptions: ======================================================================== [fixit] Remove incorrect reserve usage for STL data structure ======================================================================== Move LiteIndex::SortHits() from the query path to the indexing path ======================================================================== Bug: 286418010 Bug: 297549761 Bug: 248295995 Change-Id: I46168178e0227451b0f7dbf394bb6cc5db94e1e8
2023-08-30Merge remote-tracking branch 'aosp/upstream-master' into androidx-mainTim Barron
* aosp/upstream-master: Update Icing from upstream. Descriptions: ======================================================================== Fix term frequency bug ======================================================================== Delete dead JNI functions. ======================================================================== Switch Icing JNI implementation to use RegisterNatives ======================================================================== Avoid unnecessary GetObjectClass and GetFieldID calls. ======================================================================== Bug: 296938196 Change-Id: Idaaf78068bcfa8d9a34efd9b892c20049646874f
2023-08-23Update Icing from upstream.Tim Barron
Descriptions: ======================================================================== Fix term frequency bug ======================================================================== Delete dead JNI functions. ======================================================================== Switch Icing JNI implementation to use RegisterNatives ======================================================================== Avoid unnecessary GetObjectClass and GetFieldID calls. ======================================================================== Bug: 296938196 Change-Id: I773bcc05e768a7ac80f1950985b618047d009641
2023-08-07Merge remote-tracking branch 'aosp/upstream-master' into androidx-mainYogesh Singh
* aosp/upstream-master: Update Icing from upstream. Descriptions: ======================================================================== Add join test for SetSchemaNewIndexedDocumentPropertyTriggersIndexRestorationAndReturnsOk ======================================================================== Support different sets of properties to search for different schema types. ======================================================================== Do not cache the section mask for a schema type if no property filters apply while processing the type property filters. ======================================================================== Bug: 288310393 Bug: 291019114 Change-Id: If510ff1ef81f95c2dc10027c1faef0bee6b49aa6
2023-08-07Update Icing from upstream.Yogesh Singh
Descriptions: ======================================================================== Add join test for SetSchemaNewIndexedDocumentPropertyTriggersIndexRestorationAndReturnsOk ======================================================================== Support different sets of properties to search for different schema types. ======================================================================== Do not cache the section mask for a schema type if no property filters apply while processing the type property filters. ======================================================================== Bug: 288310393 Bug: 291019114 Change-Id: Icb867b40a603c0d63e2d96a0e2ec641e424aad7e
2023-07-17Merge remote-tracking branch 'aosp/upstream-master' into androidx-mainGrace Zhao
* aosp/upstream-master: Update Icing from upstream. Descriptions: ======================================================================== [Icing][adhoc] Add backup schema producer unit tests for properties using indexable nested properties list ======================================================================== Fix SetSchema bug where adding an indexable nested document property does not trigger index-rebuild ======================================================================== Add join test for SetSchemaNewIndexedDocumentPropertyTriggersIndexRestorationAndReturnsOk ======================================================================== Bug: 291019114 NO_IFTTT="Path is only valid in G3." Change-Id: Ib0f38921783a9ea2a964523439f3547250853cb0
2023-07-14Update Icing from upstream.Grace Zhao
Descriptions: ======================================================================== [Icing][adhoc] Add backup schema producer unit tests for properties using indexable nested properties list ======================================================================== Fix SetSchema bug where adding an indexable nested document property does not trigger index-rebuild ======================================================================== Add join test for SetSchemaNewIndexedDocumentPropertyTriggersIndexRestorationAndReturnsOk ======================================================================== Bug: 291019114 NO_IFTTT="Path is only valid in G3." Change-Id: I443f0afe79b63c483ca01b96975229f0f926d80d
2023-07-07Merge remote-tracking branch 'aosp/upstream-master' into androidx-mainGrace Zhao
* aosp/upstream-master: Update Icing from upstream. Descriptions: ======================================================================== [Icing] Extend document store uri mapper ======================================================================== Implement dirty flag in PersistentStorage ======================================================================== [Icing][version 2][1/x] Implement ShouldRebuildDerivedFiles ======================================================================== [Icing][version 2][2/x] Bump kVersion to 2 ======================================================================== [NumericSearch][optimization] Change numeric index bucket splitting threshold ======================================================================== [NumericSearch][optimization] Flag guard integer index bucket splitting threshold ======================================================================== Fix MainIndex::TransferAndAddHits by adding a safety check ======================================================================== [Indexable_nested_properties_list 4.5/5] Modify sectionId assignment so that all property paths in the indexable_nested_properties_list consume sectionIds. ======================================================================== Bug: 288934779 Bug: 289150947 Bug: 193919210 Bug: 280094535 Bug: 288969109 Bug: 259743562 NO_IFTTT="Path is only valid in G3." Change-Id: I093500e50399dc90fc2e85cee8e5a48a3c50b1d5
2023-07-07Update Icing from upstream.Grace Zhao
Descriptions: ======================================================================== Fix MainIndex::TransferAndAddHits by adding a safety check ======================================================================== [Indexable_nested_properties_list 4.5/5] Modify sectionId assignment so that all property paths in the indexable_nested_properties_list consume sectionIds. ======================================================================== Bug: 288934779 Bug: 289150947 NO_IFTTT="Path is only valid in G3." Change-Id: Ibad4a09d2026ad038f6b64391497e4248fb891ed
2023-07-06Update Icing from upstream.Yen-Chieh Sung
Descriptions: ======================================================================== [Icing] Extend document store uri mapper ======================================================================== Implement dirty flag in PersistentStorage ======================================================================== [Icing][version 2][1/x] Implement ShouldRebuildDerivedFiles ======================================================================== [Icing][version 2][2/x] Bump kVersion to 2 ======================================================================== [NumericSearch][optimization] Change numeric index bucket splitting threshold ======================================================================== [NumericSearch][optimization] Flag guard integer index bucket splitting threshold ======================================================================== Bug: 193919210 Bug: 280094535 Bug: 288969109 Bug: 259743562 NO_IFTTT="Path is only valid in G3." Change-Id: Ie1e6f5969549f6c945ea32d03c724850ff5600a9
2023-06-28Merge remote-tracking branch 'aosp/upstream-master' into androidx-mainGrace Zhao
* aosp/upstream-master: Update Icing from upstream. Descriptions: ======================================================================== [Join][cleanup][ez] Rename QualifiedIdTypeJoinableIndex as QualifiedIdJoinIndex ======================================================================== Better evaluation for persistent hash map size in join index ======================================================================== Add num_to_score into ResultSpecProto ======================================================================== [Indexable_nested_properties_list 1/5] Add indexable_nested_properties to DocumentIndexingConfig and check it during schema validation. ======================================================================== [Indexable_nested_properties_list 2/5] Add check for indexable_neseted_properties when computing SchemaDelta ======================================================================== [Indexable_nested_properties_list 3/5] Add indexable_nested_properties to schema-property-iterator ======================================================================== [Indexable_nested_properties_list 4/5] Integrate circular schema definitions with schema-property-iterator ======================================================================== Bug: 272145329 Bug: 284232550 Bug: 285636503 Bug: 265304217 Bug: 289150947 Change-Id: Icfcd4901015b75f4af2d231f8a8613c6e33034a0
2023-06-28Update Icing from upstream.Grace Zhao
Descriptions: ======================================================================== [Join][cleanup][ez] Rename QualifiedIdTypeJoinableIndex as QualifiedIdJoinIndex ======================================================================== Better evaluation for persistent hash map size in join index ======================================================================== Add num_to_score into ResultSpecProto ======================================================================== [Indexable_nested_properties_list 1/5] Add indexable_nested_properties to DocumentIndexingConfig and check it during schema validation. ======================================================================== [Indexable_nested_properties_list 2/5] Add check for indexable_neseted_properties when computing SchemaDelta ======================================================================== [Indexable_nested_properties_list 3/5] Add indexable_nested_properties to schema-property-iterator ======================================================================== [Indexable_nested_properties_list 4/5] Integrate circular schema definitions with schema-property-iterator ======================================================================== Bug: 272145329 Bug: 284232550 Bug: 285636503 Bug: 265304217 Bug: 289150947 Change-Id: I0bc3282400e6c69f8551dbddf47585864bfdec06
2023-06-02Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devTim Barron
* goog/upstream-master: Update Icing from upstream. Update Icing from upstream. Descriptions: ======================================================================== Eliminate unnecessary GetCurrentTime syscalls in various hotspots. ======================================================================== Fix MSAN failure in SchemaStore::Header::Write. ======================================================================== Downgrade log priorities on filesystem errors. ======================================================================== Bug: 282308801 Bug: 282997030 Change-Id: I0c95300007aeab6460f6bb495dc38e548a52abbd
2023-06-02Update Icing from upstream.Tim Barron
Descriptions: ======================================================================== Eliminate unnecessary GetCurrentTime syscalls in various hotspots. ======================================================================== Fix MSAN failure in SchemaStore::Header::Write. ======================================================================== Bug: 282308801 Change-Id: Ifb11f890485329e6b0bc1a9cbf2bf046e39b7854
2023-05-23Update Icing from upstream.Tim Barron
Descriptions: ======================================================================== Downgrade log priorities on filesystem errors. ======================================================================== Bug: 282997030 Change-Id: Ia6dd9c9c1f3bdb68458cd1e9dba83e01a4b25123
2023-05-20Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devTim
* goog/upstream-master: Update Icing from upstream. Descriptions: ======================================================================== Rename `this.childrenScores` to `this.childrenRankingSignals`. ======================================================================== Change-Id: Ic4d6c549f7504b68caea87795ac291d88106ccd7
2023-05-19Update Icing from upstream.Tim Barron
Descriptions: ======================================================================== Rename `this.childrenScores` to `this.childrenRankingSignals`. ======================================================================== Change-Id: I6ba4087a6de76ea13d149f4af1b7a778e6623a5d
2023-05-19Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devTim Barron
* goog/upstream-master: Update Icing from upstream. Descriptions: ======================================================================== Add flag to guard PersistentHashMapKeyMapper and DynamicTrieKeyMapper ======================================================================== Add flag to guard FileBackedVector premapping ======================================================================== Check tokenizer type when assigning section id ======================================================================== Bug: 193919210 Change-Id: Ibb064fd62a9e5b867e72701e89448d67215c3c9f
2023-05-19Update Icing from upstream.Tim Barron
Descriptions: ======================================================================== Add flag to guard PersistentHashMapKeyMapper and DynamicTrieKeyMapper ======================================================================== Add flag to guard FileBackedVector premapping ======================================================================== Check tokenizer type when assigning section id ======================================================================== Bug:193919210 Change-Id: I682e22d4ba647220b7222fcc63c41bd0e01bbeb9
2023-05-15Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devTerry Wang
* goog/upstream-master: Update Icing from upstream. Descriptions: ======================================================================== Minor fix for export to framework. ======================================================================== [Icing][version][3/x][ez] Call DataSync for version file ======================================================================== Add `use_read_only_search` flag for flag-guarding IcingSearchEngine::Search locking change in cl/506671441. ======================================================================== Bug: 146008613 Change-Id: Ia22ef83345bdbdd73bf9896191cc91db2a3a0f4c
2023-05-15Update Icing from upstream.Terry Wang
Descriptions: ======================================================================== Minor fix for export to framework. ======================================================================== [Icing][version][3/x][ez] Call DataSync for version file ======================================================================== Add `use_read_only_search` flag for flag-guarding IcingSearchEngine::Search locking change in cl/506671441. ======================================================================== Bug: 146008613 Test: presubmit Change-Id: Id863275cf5cc1ad7bcc69649527cc17981cc6cd2
2023-05-11Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devTim
Update Icing from upstream. Descriptions: ======================================================================== Handle version changes in the schema store. ======================================================================== Modify the definition of propertyDefined: ======================================================================== Remove default args in SchemaStore::SetSchema and fix calls ======================================================================== Add allow_circular_schema_definitions flag ======================================================================== Onboard version detection to Icing ======================================================================== Add version util to help read/write version info ======================================================================== Add support for the overlay schema. ======================================================================== Allow cycles in schema-property-iterator ======================================================================== Add joinable properties into schema definition cycle restrictions. ======================================================================== Loosen circular references restriction for Schema Definitions. ======================================================================== Implement BackupSchemaProducer to generate a backup schema ======================================================================== Minor fix: remove a redundant log ======================================================================== Allow schema types to inherit from more than one parent ======================================================================== allow nested document properties to accept documents of subtype ======================================================================== Support polymorphism for Icing projection in Search and Get API ======================================================================== Add max_joined_child_per_parent into ResultSpec and change behavior ======================================================================== Support Icing schema type polymorphism for the search filter API ======================================================================== Verify that every child type's property set has included all compatible properties from parent types ======================================================================== Add individual type index latency ======================================================================== Build the iterator node for the propertyDefined() custom function ======================================================================== Advance all hits with same doc id from and merge sections once for the same bucket iter ======================================================================== Introduce DocHitInfoIteratorPropertyInSchema for property existence check ======================================================================== Add SchemaUtil::BuildTransitiveInheritanceGraph to build an inheritance map from schema ======================================================================== Introduce a lookup method for a property defined in a schema ======================================================================== Rollback of: Allow LanguageSegmenter::Iterators to declare AccessType. ======================================================================== Adds join info to QueryStatsProto ======================================================================== Bug:280698419 Bug:280698125 Bug:280698121 Bug:280697513 Bug:276349029 Bug:272145329 Bug:270102295 Bug:269295094 Bug:268680462 Bug:265304217 Bug:259744228 Bug:259743562 Bug:256022027 * goog/upstream-master: Update Icing from upstream. Update Icing from upstream. Change-Id: Ia9c5c88bf8e43122204acc8f5231fb8bf65019e3
2023-05-11Update Icing from upstream.Tim Barron
Descriptions: ======================================================================== Handle version changes in the schema store. ======================================================================== Bug: 280697513 Change-Id: I6da14f236cd8e4cf2af75c1fb2c115d1df512114
2023-05-11Update Icing from upstream.Tim Barron
Descriptions: ======================================================================== Modify the definition of propertyDefined: ======================================================================== Remove default args in SchemaStore::SetSchema and fix calls ======================================================================== Add allow_circular_schema_definitions flag ======================================================================== Onboard version detection to Icing ======================================================================== Add version util to help read/write version info ======================================================================== Add support for the overlay schema. ======================================================================== Allow cycles in schema-property-iterator ======================================================================== Add joinable properties into schema definition cycle restrictions. ======================================================================== Loosen circular references restriction for Schema Definitions. ======================================================================== Implement BackupSchemaProducer to generate a backup schema ======================================================================== Minor fix: remove a redundant log ======================================================================== Allow schema types to inherit from more than one parent ======================================================================== allow nested document properties to accept documents of subtype ======================================================================== Support polymorphism for Icing projection in Search and Get API ======================================================================== Add max_joined_child_per_parent into ResultSpec and change behavior ======================================================================== Support Icing schema type polymorphism for the search filter API ======================================================================== Verify that every child type's property set has included all compatible properties from parent types ======================================================================== Add individual type index latency ======================================================================== Build the iterator node for the propertyDefined() custom function ======================================================================== Advance all hits with same doc id from and merge sections once for the same bucket iter ======================================================================== Introduce DocHitInfoIteratorPropertyInSchema for property existence check ======================================================================== Add SchemaUtil::BuildTransitiveInheritanceGraph to build an inheritance map from schema ======================================================================== Introduce a lookup method for a property defined in a schema ======================================================================== Rollback of: Allow LanguageSegmenter::Iterators to declare AccessType. ======================================================================== Adds join info to QueryStatsProto ======================================================================== Bug:280698419 Bug:280698125 Bug:280698121 Bug:280697513 Bug:276349029 Bug:272145329 Bug:270102295 Bug:269295094 Bug:268680462 Bug:265304217 Bug:259744228 Bug:259743562 Bug:256022027 Change-Id: I54cd1d22121c314f8c238d2d49f0809165dc0ca3
2023-04-20Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devJiayu Hu
* goog/upstream-master: Update Icing from upstream. Descriptions: ======================================================================== Adds join info to QueryStatsProto ======================================================================== [Join][cleanup][1/x] Add Icing benchmark for qualified id join ======================================================================== [NumericSearch][benchmark][2/x] Modify run-benchmarks.py to onboard numeric search benchmark ======================================================================== [NumericSearch][benchmark][1/x] Add numeric search icing search engine benchmark ======================================================================== Switch Iterator access type used in SnippetRetriever back to kForwardIterator. ======================================================================== Fix `DocumentStore::OptimizeInto` by passing `options_.document_store_namespace_id_fingerprint()` ======================================================================== Add file size check when initializing mmap for existing metadata file. ======================================================================== Add compression level option to Icing and propagate down to PortableFileBackedProtoLog ======================================================================== Add portable_equals_proto namespace for EqualsProto in tokenized-document_test. ======================================================================== Refactor schema to check consistency of polymorphism by dependency map ======================================================================== Support the `this.propertyWeights()` function in advanced scoring ======================================================================== Randomly set optimize_rebuild_index_threshold in monkey test ======================================================================== Change benchmark batch query latency to average ======================================================================== Performance improvements to SnippetRetriever. ======================================================================== Bug: 270102295 Bug: 260625837 Bug: 269295094 Bug: 193244409 Bug: 269513147 Bug: 276345448 Bug: 259744228 Bug: 272145329 Bug: 276349029 Change-Id: Ibef24574826d6a2c7ea24caa13a073c10a490ceb
2023-04-19Update Icing from upstream.Jiayu Hu
Descriptions: ======================================================================== Adds join info to QueryStatsProto ======================================================================== [Join][cleanup][1/x] Add Icing benchmark for qualified id join ======================================================================== [NumericSearch][benchmark][2/x] Modify run-benchmarks.py to onboard numeric search benchmark ======================================================================== [NumericSearch][benchmark][1/x] Add numeric search icing search engine benchmark ======================================================================== Switch Iterator access type used in SnippetRetriever back to kForwardIterator. ======================================================================== Fix `DocumentStore::OptimizeInto` by passing `options_.document_store_namespace_id_fingerprint()` ======================================================================== Add file size check when initializing mmap for existing metadata file. ======================================================================== Add compression level option to Icing and propagate down to PortableFileBackedProtoLog ======================================================================== Add portable_equals_proto namespace for EqualsProto in tokenized-document_test. ======================================================================== Refactor schema to check consistency of polymorphism by dependency map ======================================================================== Support the `this.propertyWeights()` function in advanced scoring ======================================================================== Randomly set optimize_rebuild_index_threshold in monkey test ======================================================================== Change benchmark batch query latency to average ======================================================================== Performance improvements to SnippetRetriever. ======================================================================== Bug: 270102295 Bug: 260625837 Bug: 269295094 Bug: 193244409 Bug: 269513147 Bug: 276345448 Bug: 259744228 Bug: 272145329 Bug: 276349029 Change-Id: Ief1cb2c21f67aac420d6b79d9b946bbecdc39a5c
2023-03-31Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devTim Barron
* goog/upstream-master: Update Icing from upstream. Descriptions: ======================================================================== Make int64_t the CreateUsageReport timestamp type ======================================================================== Create ResultAdjustmentInfo and refactor ResultStateV2 ======================================================================== Wrap parent/child adjustment info to std::unique_ptr ======================================================================== Apply join child snippet ======================================================================== Apply join child projection ======================================================================== Add IntegerSectionIndexingHandlerTest ======================================================================== Delete Result Retriever this class is dead code. ======================================================================== Introduce a placeholder for the custom function hasPropertyDefined(member) ======================================================================== Fix libtextclassifier3::StatusOr ======================================================================== Performance improvements to SnippetRetriever. ======================================================================== Bug: 193244409 Bug: 256022027 Bug: 259744228 Bug: 268680462 Bug: 270102295 Bug: 271015984 Bug: 274627497 Change-Id: Ib0e26cb79fac7b5b6d1c26bd8b2910c517e9e2ab
2023-03-31Update Icing from upstream.Tim Barron
Descriptions: ======================================================================== Make int64_t the CreateUsageReport timestamp type ======================================================================== Create ResultAdjustmentInfo and refactor ResultStateV2 ======================================================================== Wrap parent/child adjustment info to std::unique_ptr ======================================================================== Apply join child snippet ======================================================================== Apply join child projection ======================================================================== Add IntegerSectionIndexingHandlerTest ======================================================================== Delete Result Retriever this class is dead code. ======================================================================== Introduce a placeholder for the custom function hasPropertyDefined(member) ======================================================================== Fix libtextclassifier3::StatusOr ======================================================================== Performance improvements to SnippetRetriever. ======================================================================== Bug: 193244409 Bug: 256022027 Bug: 259744228 Bug: 268680462 Bug: 270102295 Bug: 271015984 Bug: 274627497 Change-Id: I6bad316b28bb289fa8e3f5b0982d6aaa9e0d135f
2023-03-16Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devTim Barron
* goog/upstream-master: Update Icing from upstream. Descriptions: ======================================================================== Set overall index latency ======================================================================== Change GetUsageScores return type to optional. ======================================================================== Upstream fix to thread-safety annotation for cached_break_iterator_ ======================================================================== Bug: 259744228 Change-Id: I11212f22a3620289f28f5b12dbc6aab2d0b27fbc
2023-03-16Update Icing from upstream.Tim Barron
Descriptions: ======================================================================== Set overall index latency ======================================================================== Change GetUsageScores return type to optional. ======================================================================== Upstream fix to thread-safety annotation for cached_break_iterator_ ======================================================================== Bug: 259744228 Change-Id: Ia7a5032fd7655db773e311173f5735e4451b30c1
2023-03-14Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devTim Barron
* goog/upstream-master: Update Icing from upstream. Update Icing from upstream. Descriptions: ======================================================================== Cache an instance of UBreakIterator to reduce unnecessary creations. ======================================================================== Cap number of individual IntegerIndexStorages that IntegerIndex creates. ======================================================================== Change error in trimRightMostNode from Unimplemented to InvalidArgument. ======================================================================== Add detection for new language features of List Filters Query Language. ======================================================================== Add option to control threshold to rebuild index during optimize by flag ======================================================================== Add option to control use of namespace id to build urimapper by flag. ======================================================================== Enforce schema validation for joinable config. ======================================================================== Adopt bucket splitting for IntegerIndexStorage. ======================================================================== Implement bucket splitting function. ======================================================================== Add Icing initialization unit tests for QualifiedIdTypeJoinableIndex. ======================================================================== Add Icing schema change unit tests for QualifiedIdTypeJoinableIndex. ======================================================================== Add Icing optimization unit tests for QualifiedIdTypeJoinableIndex. ======================================================================== Integrate QualifiedIdTypeJoinableIndex into IcingSearchEngine. ======================================================================== Implement QualifiedIdJoinablePropertyIndexingHandler. ======================================================================== Change QualifiedIdTypeJoinableIndex to store raw qualified id string. ======================================================================== Pass info about unnormalized query terms through lexer/parser/visitor. ======================================================================== Integrate Advanced Query w/ Suggest, make ADVANCED_QUERY default parser. ====================================================================== Bug: 208654892 Bug: 263890397 Bug: 259743562 Bug: 272145329 Bug: 227356108 Change-Id: I44de5853bb6c55b42800ae34d8071016be6c87cd
2023-03-14Update Icing from upstream.Tim Barron
Descriptions: ======================================================================== Cache an instance of UBreakIterator to reduce unnecessary creations. ======================================================================== Cap number of individual IntegerIndexStorages that IntegerIndex creates. ======================================================================== Change error in trimRightMostNode from Unimplemented to InvalidArgument. ======================================================================== Add detection for new language features of List Filters Query Language. ======================================================================== Add option to control threshold to rebuild index during optimize by flag ======================================================================== Add option to control use of namespace id to build urimapper by flag. ======================================================================== Enforce schema validation for joinable config. ======================================================================== Adopt bucket splitting for IntegerIndexStorage. ======================================================================== Implement bucket splitting function. ======================================================================== Add Icing initialization unit tests for QualifiedIdTypeJoinableIndex. ======================================================================== Add Icing schema change unit tests for QualifiedIdTypeJoinableIndex. ======================================================================== Add Icing optimization unit tests for QualifiedIdTypeJoinableIndex. ======================================================================== Integrate QualifiedIdTypeJoinableIndex into IcingSearchEngine. ======================================================================== Implement QualifiedIdJoinablePropertyIndexingHandler. ======================================================================== Change QualifiedIdTypeJoinableIndex to store raw qualified id string. ======================================================================== Pass info about unnormalized query terms through lexer/parser/visitor. ======================================================================== Bug: 208654892 Bug: 263890397 Bug: 259743562 Bug: 272145329 Bug: 227356108 Change-Id: I438a390ddda5673cf2b5781af502f2b7cfeaee74
2023-03-09Update Icing from upstream.Tim Barron
Descriptions: ====================================================================== Integrate Advanced Query w/ Suggest, make ADVANCED_QUERY default parser. ====================================================================== BUG: 208654892 Change-Id: I53889482e844648ec65565f9a7f7c8faa89baa7c
2023-03-07Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devTim Barron
* goog/upstream-master: Update Icing from upstream. Descriptions: ====================================================================== Refactor IndexProcessor ====================================================================== Rename Joinable Cache as Joinable Index ====================================================================== Implement Optimize and Clear for QualifiedIdTypeJoinableCache ====================================================================== Add JoinablePropertyMetadata reverse lookup ====================================================================== Allow code creating LanguageSegmenter::Iterators to declare AccessType ====================================================================== Further codifies the escape behavior in the parser test ====================================================================== Bug: 263890397 Bug: 268680462 Bug: 270102295 Change-Id: I8e45ef63c202aa83c0a4cda9c59b31d46daa4056
2023-03-06Update Icing from upstream.Tim Barron
Descriptions: ====================================================================== Refactor IndexProcessor ====================================================================== Rename Joinable Cache as Joinable Index ====================================================================== Implement Optimize and Clear for QualifiedIdTypeJoinableCache ====================================================================== Add JoinablePropertyMetadata reverse lookup ====================================================================== Allow code creating LanguageSegmenter::Iterators to declare AccessType ====================================================================== Further codifies the escape behavior in the parser test ====================================================================== Bug: 263890397 Bug: 268680462 Bug: 270102295 Change-Id: I3233733b40e985e11c4a6d75c1528cd6a72c1173
2023-03-03Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devTerry Wang
* goog/upstream-master: Update Icing from upstream. Descriptions: ====================================================================== Modify the lexer to consider internal minus characters to be a part of TEXT segments (as hyphens). ====================================================================== Modify the query language to support an individual prefix operator '*'. ====================================================================== Fix the fingerprint issue for document key mapper in document store ====================================================================== Resolve issues around the handling of negative numbers. ====================================================================== Fix doc join info compiler warning/error ====================================================================== Codifies how colons are tokenized by the lexer Bug: 208654892 Bug: 268680462 Change-Id: I08d95f9058d5642e66ef4805a5599de41d2bbcfc
2023-03-03Update Icing from upstream.Terry Wang
Descriptions: ====================================================================== Modify the lexer to consider internal minus characters to be a part of TEXT segments (as hyphens). ====================================================================== Modify the query language to support an individual prefix operator '*'. ====================================================================== Fix the fingerprint issue for document key mapper in document store ====================================================================== Resolve issues around the handling of negative numbers. ====================================================================== Fix doc join info compiler warning/error ====================================================================== Codifies how colons are tokenized by the lexer Bug: 208654892 Bug: 268680462 Change-Id: I67169b2e656eeafb6500b5b03a6c46c3cc9c3d66
2023-03-01Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devTerry Wang
* goog/upstream-master: Update Icing from upstream. Descriptions: ====================================================================== Add PropertyUtil for all property name/path related operations ====================================================================== [JoinableCache][2.0/x] Create SchemaPropertyIterator ====================================================================== [JoinableCache][2.1/x] Handle nested indexable flag ====================================================================== [JoinableCache][2.2/x] Add schema cycle dependency detection for SchemaPropertyIterator ====================================================================== [JoinableCache][3.0/x] Refactor SectionManager ====================================================================== [JoinableCache][3.1/x] Add unit tests for SectionManager::Builder and SchemaTypeManager ====================================================================== [NumericSearch][Storage][12/x] Implement Edit and GetIterator for IntegerIndex ====================================================================== [NumericSearch][Storage][13.0/x] Rename numeric-index_test as integer-index_test ====================================================================== [NumericSearch][Storage][13.1/x] Add IntegerIndexTest ====================================================================== Support the "len", "sum" and "avg" functions in advanced scoring. ====================================================================== Support the "this.childrenScores()" function to allow expressing children scores of joins in advanced scoring. ====================================================================== Create an integration test for Join with advanced scoring ====================================================================== Rename the word "children" to "args" for function related ScoreExpression ====================================================================== Improve IndexBlock by PRead/PWrite instead of repeating mmap/msync/unmap ====================================================================== Refactor QueryVisitor to prepare for support for function calls. ====================================================================== Add support for function calls. ====================================================================== Fix breakage in score-and-rank_benchmark. ====================================================================== [NumericSearch][Storage][adhoc][ez] Fix comment for IntegerIndex ====================================================================== [NumericSearch][Storage][14/x] Create first IntegerIndexStorage benchmark ====================================================================== Rename Icing schema related terminology to prepare for polymorphism support ====================================================================== [JoinableCache][4.0/x] Move common methods from SectionManager to PropertyUtil ====================================================================== [JoinableCache][4.1/x] Retire GetSectionContent ====================================================================== [JoinableCache][4.2/x] Polish SectionManagerTest ====================================================================== Modify QueryVisitor to do: ====================================================================== [NumericSearch][Storage][15/x] Implement TransferIndex for IntegerIndexStorage ====================================================================== [NumericSearch][Storage][16/x] Implement Optimize and last added document id for IntegerIndex ====================================================================== [NumericSearch][rollout][1/x] Include indexable int64 into SchemaDelta and backward compatibility ====================================================================== Add backwards compatibility test for Icing schema storage migration. ====================================================================== Implement trim the right-most node from the doc-hit-info-iterator. ====================================================================== Add TrimmedNode structure into doc-hit-info-iterator. ====================================================================== [JoinableCache][5/x] Implement JoinableProperty and JoinablePropertyManager ====================================================================== [JoinableCache][6/x] Add JoinablePropertyManager into SchemaTypeManager ====================================================================== [JoinableCache][7/x] Implement ExtractJoinableProperties ====================================================================== [JoinableCache][8/x] Create class QualifiedIdTypeJoinableCache ====================================================================== [JoinableCache][9/x] Implement factory method for QualifiedIdTypeJoinableCache ====================================================================== [JoinableCache][10/x] Implement Get and Put for QualifiedIdTypeJoinableCache ====================================================================== [JoinableCache][11/x] Add unit tests for QualifiedIdTypeJoinableCache ====================================================================== Modify DocHitInfoIteratorSectionRestrict to allow multi-property restricts ====================================================================== Fix the definition of LiteIndex::WantsMerge. ====================================================================== [NumericSearch][rollout][2.0/x] Rollout persistent IntegerIndex ====================================================================== [NumericSearch][rollout][2.1/x] Add more tests for integer index restoration and optimization ====================================================================== [JoinableCache][adhoc][ez] Remove qualified id type joinable cache size info from document storage info ====================================================================== Integrate trim right node into suggestion processor. Bug: 208654892 Bug: 228240987 Bug: 249829533 Bug: 256081830 Bug: 259744228 Bug: 261474063 Bug: 263890397 Bug: 266103594 Bug: 268738297 Bug: 269295094 Change-Id: Id4b23bf397ca189c5050bfa6c5fd64e89138e321
2023-03-01Update Icing from upstream.Terry Wang
Descriptions: ====================================================================== Add PropertyUtil for all property name/path related operations ====================================================================== [JoinableCache][2.0/x] Create SchemaPropertyIterator ====================================================================== [JoinableCache][2.1/x] Handle nested indexable flag ====================================================================== [JoinableCache][2.2/x] Add schema cycle dependency detection for SchemaPropertyIterator ====================================================================== [JoinableCache][3.0/x] Refactor SectionManager ====================================================================== [JoinableCache][3.1/x] Add unit tests for SectionManager::Builder and SchemaTypeManager ====================================================================== [NumericSearch][Storage][12/x] Implement Edit and GetIterator for IntegerIndex ====================================================================== [NumericSearch][Storage][13.0/x] Rename numeric-index_test as integer-index_test ====================================================================== [NumericSearch][Storage][13.1/x] Add IntegerIndexTest ====================================================================== Support the "len", "sum" and "avg" functions in advanced scoring. ====================================================================== Support the "this.childrenScores()" function to allow expressing children scores of joins in advanced scoring. ====================================================================== Create an integration test for Join with advanced scoring ====================================================================== Rename the word "children" to "args" for function related ScoreExpression ====================================================================== Improve IndexBlock by PRead/PWrite instead of repeating mmap/msync/unmap ====================================================================== Refactor QueryVisitor to prepare for support for function calls. ====================================================================== Add support for function calls. ====================================================================== Fix breakage in score-and-rank_benchmark. ====================================================================== [NumericSearch][Storage][adhoc][ez] Fix comment for IntegerIndex ====================================================================== [NumericSearch][Storage][14/x] Create first IntegerIndexStorage benchmark ====================================================================== Rename Icing schema related terminology to prepare for polymorphism support ====================================================================== [JoinableCache][4.0/x] Move common methods from SectionManager to PropertyUtil ====================================================================== [JoinableCache][4.1/x] Retire GetSectionContent ====================================================================== [JoinableCache][4.2/x] Polish SectionManagerTest ====================================================================== Modify QueryVisitor to do: ====================================================================== [NumericSearch][Storage][15/x] Implement TransferIndex for IntegerIndexStorage ====================================================================== [NumericSearch][Storage][16/x] Implement Optimize and last added document id for IntegerIndex ====================================================================== [NumericSearch][rollout][1/x] Include indexable int64 into SchemaDelta and backward compatibility ====================================================================== Add backwards compatibility test for Icing schema storage migration. ====================================================================== Implement trim the right-most node from the doc-hit-info-iterator. ====================================================================== Add TrimmedNode structure into doc-hit-info-iterator. ====================================================================== [JoinableCache][5/x] Implement JoinableProperty and JoinablePropertyManager ====================================================================== [JoinableCache][6/x] Add JoinablePropertyManager into SchemaTypeManager ====================================================================== [JoinableCache][7/x] Implement ExtractJoinableProperties ====================================================================== [JoinableCache][8/x] Create class QualifiedIdTypeJoinableCache ====================================================================== [JoinableCache][9/x] Implement factory method for QualifiedIdTypeJoinableCache ====================================================================== [JoinableCache][10/x] Implement Get and Put for QualifiedIdTypeJoinableCache ====================================================================== [JoinableCache][11/x] Add unit tests for QualifiedIdTypeJoinableCache ====================================================================== Modify DocHitInfoIteratorSectionRestrict to allow multi-property restricts ====================================================================== Fix the definition of LiteIndex::WantsMerge. ====================================================================== [NumericSearch][rollout][2.0/x] Rollout persistent IntegerIndex ====================================================================== [NumericSearch][rollout][2.1/x] Add more tests for integer index restoration and optimization ====================================================================== [JoinableCache][adhoc][ez] Remove qualified id type joinable cache size info from document storage info ====================================================================== Integrate trim right node into suggestion processor. Bug: 208654892 Bug: 228240987 Bug: 249829533 Bug: 256081830 Bug: 259744228 Bug: 261474063 Bug: 263890397 Bug: 266103594 Bug: 268738297 Bug: 269295094 Change-Id: I5f1b3f3ed0b5d6933dc8c2ab3279904f7706b23e
2023-02-08Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devAlex Saveliev
* goog/upstream-master: Update icing from upstream ====================================================================== Adds a proto change for the delete propagation option ====================================================================== [ez] Change version to magic for PersistentHashMap ====================================================================== [iOS][testing][nitro] Disabling ICU language segmenter_test. ====================================================================== 1. Add support for segmentation in the QueryVisitor. ====================================================================== Support the new double list type in ScoreExpression ====================================================================== Pass JoinChildrenFetcher from IcingSearchEngine all the way down to ScoringVisitor ====================================================================== Refactor the logic of Icing Joins so that nested search and scoring will be performed before the parent ====================================================================== Add lite-index thread-safety tests. ====================================================================== Split IcingSearchEngineTest into separate tests to cover specific apis: ====================================================================== Fix index tests TearDown method. ====================================================================== Improve query concurrency by providing a finer-grained lock around the LiteIndex. ====================================================================== Fix Icing normalization bug ====================================================================== [ez] Fix integer overflow error for IntegerIndexStorage ====================================================================== [NumericSearch][Storage][11/x] Implement Reset and destructor for IntegerIndex ====================================================================== [NumericSearch][Storage][10/x] Add class IntegerIndex ====================================================================== Refactor NumericIndex based on PersistentStorage ====================================================================== Refactor IntegerIndexStorage based on PersistentStorage ====================================================================== Add "working_path" into PersistentStorage ====================================================================== Refactor PersistentHashMap based on PersistentStorage ====================================================================== Create virtual class PersistentStorage for refactoring ====================================================================== Avoids returning reference to local temporary object. ====================================================================== LSC: Clean up references to the legacy protobuf compat library ====================================================================== Fix time complexity regression for snippet retriever Bug: 256022027 Bug: 193919210 Bug: 266132035 Bug: 208654892 Bug: 261474063 Bug: 266103594 Bug: 146008613 Bug: 253182853 Bug: 266204868 Bug: 249829533 Bug: 266665956 Bug: 265258364 Change-Id: Icff89dee8ed04c09597c08c22e1d180b71c7d396
2023-02-07Update icing from upstreamAlex Saveliev
====================================================================== Adds a proto change for the delete propagation option ====================================================================== [ez] Change version to magic for PersistentHashMap ====================================================================== [iOS][testing][nitro] Disabling ICU language segmenter_test. ====================================================================== 1. Add support for segmentation in the QueryVisitor. ====================================================================== Support the new double list type in ScoreExpression ====================================================================== Pass JoinChildrenFetcher from IcingSearchEngine all the way down to ScoringVisitor ====================================================================== Refactor the logic of Icing Joins so that nested search and scoring will be performed before the parent ====================================================================== Add lite-index thread-safety tests. ====================================================================== Split IcingSearchEngineTest into separate tests to cover specific apis: ====================================================================== Fix index tests TearDown method. ====================================================================== Improve query concurrency by providing a finer-grained lock around the LiteIndex. ====================================================================== Fix Icing normalization bug ====================================================================== [ez] Fix integer overflow error for IntegerIndexStorage ====================================================================== [NumericSearch][Storage][11/x] Implement Reset and destructor for IntegerIndex ====================================================================== [NumericSearch][Storage][10/x] Add class IntegerIndex ====================================================================== Refactor NumericIndex based on PersistentStorage ====================================================================== Refactor IntegerIndexStorage based on PersistentStorage ====================================================================== Add "working_path" into PersistentStorage ====================================================================== Refactor PersistentHashMap based on PersistentStorage ====================================================================== Create virtual class PersistentStorage for refactoring ====================================================================== Avoids returning reference to local temporary object. ====================================================================== LSC: Clean up references to the legacy protobuf compat library ====================================================================== Fix time complexity regression for snippet retriever Bug: 256022027 Bug: 193919210 Bug: 266132035 Bug: 208654892 Bug: 261474063 Bug: 266103594 Bug: 146008613 Bug: 253182853 Bug: 266204868 Bug: 249829533 Bug: 266665956 Bug: 265258364 Change-Id: Ib2398c5097b6a2a57900e2ad4e3737502aa13820
2023-01-23Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devTerry Wang
* goog/upstream-master: Update Icing from upstream. Descriptions: ====================================================================== Fix time complexity regression for snippet retriever ====================================================================== Add support for QueryTerms SectionRestrict Map in Advanced Query. ====================================================================== [NumericSearch][Storage][9/x] Create unit tests for AddKeys and GetIterator ====================================================================== [NumericSearch][Storage][8/x] Add options ====================================================================== Fix the stack overflow issue for icing advanced scoring/query parsers/visitors by limiting the number of tokens allowed ====================================================================== Improves the has joinspec check ====================================================================== Fix the bug of DocHitInfoIteratorTermMain found by monkey test ====================================================================== Rename tests defined in DestructibleDirectoryTest to include "directory" instead of "file". ====================================================================== Support projection in icing monkey test ====================================================================== [NumericSearch][Storage][7/x] Implement iterator for query ====================================================================== [NumericSearch][Storage][6/x] Implement AddKeys ====================================================================== Add support for relevance scoring by enabling term_frequency retrieval in DocHitInfoIterators and by populating the QueryTermIteratorsMap. ====================================================================== [JoinableCache][1/x] Compute SchemaDelta for joinable properties ====================================================================== [NumericSearch][Storage][5/x] PersistToDisk ====================================================================== [NumericSearch][Storage][4.1/x] Create unit tests for initialize ====================================================================== [NumericSearch][Storage][4.0/x] Implement Create and Initialize functions ====================================================================== Rename PostingListUsedSerializer related classes Bug: 193244409 Bug: 208654892 Bug: 246984163 Bug: 249829533 Bug: 256022027 Bug: 265258364 Bug: 265834832 Change-Id: I0b74286176356fb88602f775a431b5a489ff4045
2023-01-23Update Icing from upstream.Terry Wang
Descriptions: ====================================================================== Fix time complexity regression for snippet retriever ====================================================================== Add support for QueryTerms SectionRestrict Map in Advanced Query. ====================================================================== [NumericSearch][Storage][9/x] Create unit tests for AddKeys and GetIterator ====================================================================== [NumericSearch][Storage][8/x] Add options ====================================================================== Fix the stack overflow issue for icing advanced scoring/query parsers/visitors by limiting the number of tokens allowed ====================================================================== Improves the has joinspec check ====================================================================== Fix the bug of DocHitInfoIteratorTermMain found by monkey test ====================================================================== Rename tests defined in DestructibleDirectoryTest to include "directory" instead of "file". ====================================================================== Support projection in icing monkey test ====================================================================== [NumericSearch][Storage][7/x] Implement iterator for query ====================================================================== [NumericSearch][Storage][6/x] Implement AddKeys ====================================================================== Add support for relevance scoring by enabling term_frequency retrieval in DocHitInfoIterators and by populating the QueryTermIteratorsMap. ====================================================================== [JoinableCache][1/x] Compute SchemaDelta for joinable properties ====================================================================== [NumericSearch][Storage][5/x] PersistToDisk ====================================================================== [NumericSearch][Storage][4.1/x] Create unit tests for initialize ====================================================================== [NumericSearch][Storage][4.0/x] Implement Create and Initialize functions ====================================================================== Rename PostingListUsedSerializer related classes Bug: 193244409 Bug: 208654892 Bug: 246984163 Bug: 249829533 Bug: 256022027 Bug: 265258364 Bug: 265834832 Change-Id: Ibc7011a793ef5ed09eace0fb05d168adf5b4faca
2023-01-09Merge remote-tracking branch 'aosp/upstream-master' into androidx-mainAlex Saveliev
Descriptions: ====================================================================== Apply aggregation score to join result and create JoinProcessorTest ====================================================================== Create AggregationScorerTest ====================================================================== Add an EqualsTermMatchInfo matcher to simplify tests that interact with this struct. ====================================================================== Refactor the QueryVisitor to return QueryResults. ====================================================================== Simplify ScoreExpression to ConstantScoreExpression if its evaluation does not depend on a document ====================================================================== Use rvalue and std::move for DocumentStore methods ====================================================================== Downgrade log messages for NOT_FOUND errors in Delete calls. ====================================================================== [NumericSearch][Storage][3.1/x] Add metadata file declaration (Crcs, Info) for IntegerIndexStorage ====================================================================== [NumericSearch][Storage][3.0/x] Declare class IntegerIndexStorage ====================================================================== Cleanup Import from //tools/build_defs/objc ====================================================================== Cleanup LSC: Remove `portable_proto_library()` targets ====================================================================== Bug: 256022027 Bug: 208654892 Bug: 261474063 Bug: 246628980 Bug: 249829533 Bug: 263373214 Change-Id: Ic3238f17efa72dd1c6dbc559d32c11f475cebf88
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-05 08:35:49 +0000