aboutsummaryrefslogtreecommitdiff
path: root/icing/file/persistent-hash-map_test.cc
AgeCommit message (Collapse)Author
2023-05-19Update Icing from upstream.Tim Barron
Descriptions: ======================================================================== Add flag to guard PersistentHashMapKeyMapper and DynamicTrieKeyMapper ======================================================================== Add flag to guard FileBackedVector premapping ======================================================================== Check tokenizer type when assigning section id ======================================================================== Bug:193919210 Change-Id: I682e22d4ba647220b7222fcc63c41bd0e01bbeb9
2023-02-07Update icing from upstreamAlex Saveliev
====================================================================== Adds a proto change for the delete propagation option ====================================================================== [ez] Change version to magic for PersistentHashMap ====================================================================== [iOS][testing][nitro] Disabling ICU language segmenter_test. ====================================================================== 1. Add support for segmentation in the QueryVisitor. ====================================================================== Support the new double list type in ScoreExpression ====================================================================== Pass JoinChildrenFetcher from IcingSearchEngine all the way down to ScoringVisitor ====================================================================== Refactor the logic of Icing Joins so that nested search and scoring will be performed before the parent ====================================================================== Add lite-index thread-safety tests. ====================================================================== Split IcingSearchEngineTest into separate tests to cover specific apis: ====================================================================== Fix index tests TearDown method. ====================================================================== Improve query concurrency by providing a finer-grained lock around the LiteIndex. ====================================================================== Fix Icing normalization bug ====================================================================== [ez] Fix integer overflow error for IntegerIndexStorage ====================================================================== [NumericSearch][Storage][11/x] Implement Reset and destructor for IntegerIndex ====================================================================== [NumericSearch][Storage][10/x] Add class IntegerIndex ====================================================================== Refactor NumericIndex based on PersistentStorage ====================================================================== Refactor IntegerIndexStorage based on PersistentStorage ====================================================================== Add "working_path" into PersistentStorage ====================================================================== Refactor PersistentHashMap based on PersistentStorage ====================================================================== Create virtual class PersistentStorage for refactoring ====================================================================== Avoids returning reference to local temporary object. ====================================================================== LSC: Clean up references to the legacy protobuf compat library ====================================================================== Fix time complexity regression for snippet retriever Bug: 256022027 Bug: 193919210 Bug: 266132035 Bug: 208654892 Bug: 261474063 Bug: 266103594 Bug: 146008613 Bug: 253182853 Bug: 266204868 Bug: 249829533 Bug: 266665956 Bug: 265258364 Change-Id: Ib2398c5097b6a2a57900e2ad4e3737502aa13820
2022-12-12Sync from upstream.Tim Barron
Descriptions: ====================================================================== Add ScoringSpec into JoinSpec. Rename joined_document to child_document. ====================================================================== Create JoinedScoredDocumentHit class and refactor ScoredDocumentHitsRanker. ====================================================================== Implement initial Join workflow ====================================================================== Implement the Lexer for Icing Advanced Query Language ====================================================================== Create struct Options for PersistentHashMap ====================================================================== Premapping FileBackedVector ====================================================================== Create class PersistentHashMapKeyMapper ====================================================================== Add integer sections into TokenizedDocument and rename string sections ====================================================================== Create NumericIndex interface and DocHitInfoIteratorNumeric ====================================================================== Implement DummyNumericIndex and unit test ====================================================================== Change PostingListAccessor::Finalize to rvalue member function ====================================================================== Define the Abstract Syntax Tree for Icing's list_filter parser. ====================================================================== Refactor query processing and score ====================================================================== Refactor IcingSearchEngine for AppSearch Dynamite Module 0p APIs ====================================================================== Implement the Lexer for Icing Advanced Scoring Language ====================================================================== Add a common interface for IcingSearchEngine and dynamite client ====================================================================== Implement a subset of the query grammar. ====================================================================== Refactor index processor ====================================================================== Add integer index into IcingSearchEngine and IndexProcessor ====================================================================== Implement the parser for Icing Advanced Scoring Language ====================================================================== Implement IntegerIndexData and PostingListUsedIntegerIndexDataSerializer ====================================================================== Add PostingListAccessor abstract class for common components and methods ====================================================================== Implement PostingListIntegerIndexDataAccessor ====================================================================== Create PostingListIntegerIndexDataAccessorTest ====================================================================== Fix Icing Segmentation tests for word connectors that changed in ICU 72. ====================================================================== Modify the Advanced Query grammar to allow functions to accept expressions. ====================================================================== Implement QueryVisitor. ====================================================================== Enable the Advanced Query Parser to handle member functions ====================================================================== Refactor the Scorer class to support the Advanced Scoring Language ====================================================================== Integrate advanced query parser with the query processor. ====================================================================== Implement support for JoinSpec in Icing. ====================================================================== Implement the Advanced Scoring Language for basic functions and operators ====================================================================== Bug: 208654892 Bug: 249829533 Bug: 256022027 Bug: 261474063 Bug: 240333360 Bug: 193919210 Change-Id: I5f5bdc6249282ecc4b014b4fbdf8e2d1f8b20c19
2022-09-30Update Icing from upstream.Terry Wang
Descriptions: ====================================================================== Integrate ANTLR-based advanced query prototype with query processor. ====================================================================== [PersistentHashMap][6.1/x] Wrap the return value of KeyMapper::ComputeChecksum by StatusOr ====================================================================== [PersistentHashMap][6.0/x] Replace GetValuesToKeys with iterator for KeyMapper ====================================================================== [PersistentHashMap][5.1/x] Allow client to specify initial num buckets ====================================================================== [PersistentHashMap][5/x] Implement rehashing ====================================================================== Add SchemaType filter and Document Id filter in Search Suggestion API. ====================================================================== Follow up to cl/463377778 ====================================================================== Add Document Id filters in Search Suggestion API. ====================================================================== Cleanup LSC: Replace inclusion of *_proto_portable.pb.h files coming from portable_proto_library() with *(!_proto_portable).pb.h coming from cc_proto_library(). ====================================================================== Cleanup LSC: Replace inclusion of *_proto_portable.pb.h files coming from portable_proto_library() with *(!_proto_portable).pb.h coming from cc_proto_library(). ====================================================================== Cloned from CL 464902284 by 'g4 patch'. ====================================================================== Cleanup LSC: Replace inclusion of *_proto_portable.pb.h files coming from portable_proto_library() with *(!_proto_portable).pb.h coming from cc_proto_library(). ====================================================================== Cleanup Remove unused visibility specs (last referenced in codebase over 132 days ago). ====================================================================== Cleanup LSC: Replace inclusion of *_proto_portable.pb.h files coming from portable_proto_library() with *(!_proto_portable).pb.h coming from cc_proto_library(). ====================================================================== Cleanup LSC: Replace inclusion of *_proto_portable.pb.h files coming from portable_proto_library() with *(!_proto_portable).pb.h coming from cc_proto_library(). ====================================================================== Remove dsaadati@ from third_party/icing OWNERS ====================================================================== Cleanup Move package level default_copts attribute to copts. ====================================================================== Refactor QueryProcessor and QueryProcessTest in preparation for adding ANTLR prototype to parse queries with search_type EXPERIMENTAL_ICING_ADVANCED_QUERY. ====================================================================== Bug: 208654892 Bug: 230553264 Bug: 237324702 Bug: 193919210 Change-Id: I2f0a612747ccb754502489a9b168406532cffaee
2022-09-08Sync from upstream.Grace Zhao
Descriptions: ====================================================================== [FileBackedVector Consolidation][4/x] Fix potential PWrite bug in GrowIfNecessary ====================================================================== [FileBackedVector Consolidation][5/x] Create benchmark for FileBackedVector ====================================================================== [FileBackedVector Consolidation][6/x] Avoid calling GetFileSize in GrowIfNecessary ====================================================================== [PersistentHashMap][3.3/x] Implement Delete ====================================================================== Fix the PopulateMatchedTermsStats bug ====================================================================== Add JNI latency for query latency stats breakdown. ====================================================================== [ResultStateManager] Thread safety test1 ====================================================================== [ResultStateManager][2/x] Thread safety test2 ====================================================================== Add native lock contention latency for measuring query latency ====================================================================== Fix implementation of HasMember operator in ANTLR-based list-filter prototype. ====================================================================== Fix improper uses of std::string_view ====================================================================== Extend the scale of Icing ====================================================================== Decouple the term frequency array from DocHitInfo ====================================================================== Disable hit_term_frequency for non-relevance queries ====================================================================== [ResultStateManager][3/x] Thread safety test3 ====================================================================== [PersistentHashMap][4/x] Implement iterator ======================================================================= Fix the lite index compaction bug ======================================================================= Change-Id: I0edad67affed97af107e2d7cd73770e0268c0903
2022-08-11Sync from upstream.Tim Barron
Descriptions: ====================================================================== Implement new version of ResultState and ResultStateManager to 1) enforce a page byte size limit and 2) improve handling of pagination when we encounter deleted documents. ====================================================================== Fix bugs in IcingDynamicTrie::Delete. ====================================================================== Implement IcingDynamicTrie::IsBranchingTerm. ====================================================================== Change Icing default logging level to INFO ====================================================================== Refactor KeyMapper class to be an interface. ====================================================================== Improve NamespaceChecker logic to improve Suggest latency. ====================================================================== Change icing native log tag to "AppSearchIcing" ====================================================================== Implement Index Compaction rather than rebuilding index during Compaction. ====================================================================== Implement reverse iterator for IcingDynamicTrie ====================================================================== Avoid adding unnecessary branch points during index compaction ====================================================================== Invalidate expired result states when adding to/retrieving from ResultStateManager. ====================================================================== Add new methods (MutableView, MutableArrayView, Append, Allocate) to FileBackedVector ====================================================================== Create and implement PersistentHashMap class. ====================================================================== Implement RFC822 Tokenizer ====================================================================== Remove uses of StringPrintf in ICING_LOG statements ====================================================================== Properly set query latency when an error is encountered or results are empty. ====================================================================== Bug: 146903474 Bug: 152934343 Bug: 193919210 Bug: 193453081 Bug: 231368517 Bug: 235395538 Bug: 236412165 Change-Id: I8aa278cebb12b25b39deb0ef584c0f198952659d