aboutsummaryrefslogtreecommitdiff
path: root/icing
AgeCommit message (Collapse)Author
2021-08-13Merge remote-tracking branch 'aosp/upstream-master' into androidx-mainTim Barron
* aosp/upstream-master: Sync from upstream. Descriptions: ================== Add Initialization Count marker file to break out of crash loops. ================== Delete the Simple Normalizer. ================== Add submatch information to identify a submatch within a document. ================== Bug: 195720764 Change-Id: I12500fdc0e852b81a623e54e9991ac8414f0a770
2021-08-13Sync from upstream.Tim Barron
Descriptions: ================== Add Initialization Count marker file to break out of crash loops. ================== Delete the Simple Normalizer. ================== Add submatch information to identify a submatch within a document. ================== Bug: 195720764 Change-Id: I27bcc66c400da625d595609049d281b809dd6028
2021-08-13Merge androidx-platform-dev/external/icing upstream-master into upstream-masterAlexander Dorokhine
Change-Id: Id39bea14b3a3d378d722dc552e4f3bd4249f3f94
2021-07-20Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devCassie Wang
* goog/upstream-master: Sync from upstream. Sync from upstream. Includes ag/15291302 and ag/15329694 Change-Id: I90c45c6f61975b6a8d2df2183de8f304b54e3251
2021-07-19Sync from upstream.Cassie Wang
Descriptions: ================== Switch to PRId64 string formatting. ================== Conditionally recompute the checksum for PortableFileBackedProtoLog. ================== Fix spammy error log encountered in hit iterator. =================== Fix -Wunused-but-set-variable warnings Bug: 193141896 Change-Id: I46ca2daa90efe442e9ebeac389e91df21ddc81d6
2021-07-14Sync from upstream.Cassie Wang
Descriptions: ============ Lower the memory usage cap from 16MB to 4MB. ============ Modify file-backed-vector to 1) include a check that the size of the file is greater than or equal to the number of elements * sizeof(element) and 2) use PWrite to force the system to allocate disk blocks so that we avoid the risk of trying to mmap a region of the file that the system hasn't allocated a block for (and is unable to do so). ============ Fix unused-error and reorder-ctor issues that surfaced in framework ============ Bug: 191444782 Bug: 193715176 Change-Id: Ic76a46700e770b6740a69c4777f88639c8c4e946
2021-06-30Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devCassie Wang
* goog/upstream-master: Sync from upstream Change-Id: If757496e9cfbd1dc9724bfa46902f4d009745e31
2021-06-30Sync from upstreamCassie Wang
Descriptions: ============== Enforce kMaxDocumentId when inserting new documents. ============== Resets members before reinitializing in IcingSearchEngine Reset() ============== Exclude a data files for tests under the icing/testdata directory. Bug: 192464981 Change-Id: I1f8855550791a93022068ea78b6cba2a0758bfb1
2021-06-24Delete testdata files from external/icingTim Barron
This files are only used to run a migration test case in DocumentStoreTest. Change-Id: I5eeecd4f9942cf29dda981e0bcab537ff43807fc
2021-06-23Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devTim Barron
* goog/upstream-master: Sync from upstream. Change-Id: I0778164b75e1552451290f1b8fe4a0e07b3dc844
2021-06-23Sync from upstream.Tim Barron
Descriptions: ========== Switch DocumentStore to using PortableFileBackedProtoLog instead of FileBackedProtoLog, and migrate any existing FileBackedProtoLog's to the Portable* version. ========== Add more fields to SchemaStats to capture the types that were newly added, modified and modified with an index rebuild. ========== Create a new DeleteByQueryStatsProto to capture query-specific stats related to the performance of DeleteByQuery. ========== Fix bugs in SetSchema: 1. Property id reassignments when a property is deleted and a new one added was not being detected 2. Property id reassignments due to changes to the index_nested_properties field were not detected. 3. Backward incompatible changes should invalidate dependent types. ========== Fix IcingSearchEngineTest.CJKSnippets test by choosing a simpler multi-byte character (when encoded in UTF-8) string that ICU will always segment in the same way regardless of api level. Bug: 185806837 Bug: 187205195 Bug: 187726282 Change-Id: I07cc8bf3fc5b3a7c5e8f0d24d1724297f3bfba5d
2021-06-17Merge remote-tracking branch 'goog/upstream-master' into androidx-platform-devTim Barron
* goog/upstream-master: Sync from upstream. Change-Id: I85b373c56933f2525e3ff928484da874b299d831
2021-06-16Sync from upstream.Tim Barron
Descriptions: ========== Add a dirty bit flag to document store to avoid checksumming during initialization. ========== Change the interpretation of snippet window size to be measured in unicode characters rather than bytes. ========== Implement snippet window shifting to more closely match the requested size. ========== Make GetDocumentFilterData check if a document exists first, and if not, return NOT_FOUND. ========== Move reverse-jni-break-iterator to the tokenization/reverse_jni directory. Bug: 185806837 Bug: 188826120 Bug: 188914218 Change-Id: I908e4b5eedc85332a318ce0e5b891a7a950720db
2021-06-08Compile libicing.so with a version script.Tim Barron
This will make it easier for the linker to eliminate dead code. Change-Id: I9c26f27fe585453e10585546ef1ce1d8602eea93
2021-05-25Pull upstream changes.Cassie Wang
Descriptions: ========== Create a portable file backed proto log ========== Fix bug that was causing us to not update the last_added_document_id of the index ========== Remove dependency on icu dat file. ========== Create a simple dump utility that dumps all schemas, namespaces and documents. ========== Fix usage score handling for nonexistent documents. ========== Allow querying in dumper.cc. ========== Fix bit-util to actually set fields. ========== Bug: 185804696 Bug: 189136429 Bug: 187879464 Change-Id: I7a8f7b223884a647dc0fe4d70de99d586a6b6e33
2021-05-11Sync from upstream.Cassie Wang
Descriptions: ========== Fix our call to Delete() when Put()-ing. ========== Change-Id: I044cc75b99771c07e3f2222a621c2681b5b44957
2021-05-07Sync from upstream.Tim Barron
Descriptions: ========== [b/185445694] Return snippetting information as both byte indices and unicode character indices. ========== [b/185804696] Copy over endian logic to handle ntoh* and hton* conversions. ========== [b/185822483] Rewrite DoesDocumentExist to avoid constructing Status objects and error messages. ========== Change-Id: I64767fb975acc49ee15822d676c87bcafb4a8b4d
2021-04-28Sync from upstream.Tim Barron
Descriptions: ========== Delete soft delete code. This code is unused and can never be used per PWG policy. ========== Replace the overall checksum with a marker file. ========== Switch FileBackedProtoLog ComputeChecksum to mmap the entire section (on 64-bit devices) or in 4MB chunks (on 32-bit devices) ========== Adjust snippet max_window_bytes to include a window if the max_window_bytes is the exact length of the query match as well. ========== Scale last time usage scorers to return score in milliseconds ========== Remove includes of *.proto.h files. ========== Add query length to IcingQueryStats ========== Change-Id: I4bb375cb5ceec475dec62680fb9da026aed6f032
2021-04-14Sync from upstream.Tim Barron
Descriptions: ============ Fix bug in snippet-retriever that was mixing up num_to_snippet with num_snippets_per_property. Change-Id: I0382b1bab2884f1dad89fc78aa17dfd051f64685
2021-04-06Sync from upstreamCassie Wang
Changes: ============ Add a method to return a copy of the data in FileBackedVector. Change-Id: I586e6eaae316de94658e853f1a4cf6a5913c419d
2021-03-31Sync from upstream.Cassie Wang
Descriptions: === Switch from recursively advancing DocHitInfoIterators to iteratively advancing DocHitInfoIterators. === Remove redundant expiration check in DocHitInfoIteratorFilter. === Downgrade URI NOT FOUND message from ERROR to VERBOSE level 1. === Update MDI's blueprints === Add comment that I accidentally removed === Fix num_results_snippeted, which was incorrectly created as a bool, by replacing it with an int field set_num_results_with_snippets. === Migrate SchemaStoreTest, SchemaUtilTest to use SchemaBuilder === Add TAP projects for arm7 and arm64 on ios. === Migrate QueryProcessorTest, DocumentStoreTest and DocumentValidatorTest to use SchemaBuilder Change-Id: Ibd4cf572ea123b78036bdb1e972fdd565a6caf75
2021-03-15Sync from upstream.Tim Barron
Descriptions: ============ Add result grouping to the ResultSpecProto to allow callers to limit the number of results returned per group of namespace. ============ Implement PersistToDisk(LITE) to expose a LITE option for PersistToDisk that only persists the ground truth. ============ Add per-namespace storage info. ============ Add score to SearchResult. ============ Migrate IndexProcessorTest, DocHitInfoIteratorFilterTest, DocHitInfoIteratorSectionRestrictTest to use SchemaBuilder ============ Migrate ScorerTest and ScoringProcessorTest to use SchemaBuilder ============ Add licensing header for storage.proto. Change-Id: I5cbdd8045036dc38cf00191a2eac56cc6f2d3443
2021-03-03Merge aosp upstream-master into internal upstream-masterAlexander Dorokhine
* ../../../icing-upstream-aosp/external/icing: (22 commits) Sync from upstream. Sync from upstream. Update Icing from upstream. Update Icing from upstream. Update Icing from upstream. Update Icing from upstream. Update icing from upstream. Pull upstream changes. Pull upstream changes. Pull upstream changes. Pull upstream changes. Pull upstream changes. Pull upstream changes. Update from upstream, including proto location changes. Copy over changes made to Google3 codebase in Icing. Pull upstream changes. Copy over changes made to Google3 codebase in Icing. Partial CMakeLists to compile the real icing C++ lib. Create JNI layer for Icing Add build.gradle for compiling protos for Jetpack. Change-Id: I88fa4018245b94fcec06e5dedfd45ffbbd067fe8
2021-03-03Sync from upstream.Cassie Wang
Descriptions: ========== New java package name for internal/optimize.proto ========== Migrate IcingSearchEngineWithIcuFileTest and IcingSearchEngineFuzzTest to use SchemaBuilder Change-Id: I4113bf2d7cff536538e45c3afea2c4ced2dd29ee
2021-03-02Sync from upstream.Cassie Wang
Descriptions: ========== Add last optimized time to GetOptimizeInfo. ========== Update the implementation of snippeting to return property paths with value indices and remove the values_index field. ========== Create builders for SchemaProto, SchemaTypeConfigProto and PropertyConfigProto. ========== Rename some protos with the rules: - Remove "Native" prefix - Add "Proto" suffix for consistency with other protos ========== Upgrade your minimum iOS version to 11.4. ========== Fix PersistToDisk definitions to ensure that they properly call datasync. This change is meant to address the first part of ptd doc - that certain functions that claim to persist data don't actually explicitly flush. ========== Change function call from has_field() to field() ========== Add IcingStorageInfo. ========== Add IndexStorageStats. ========== Add SchemaStoreStorageStats. ========== Add DocumentStoreStorageStats. ========== Implement OptimizeStats. ========== Remove the max number of results per query limit (1000) and replace it with a more flexible way to limit memory use by the result-state-manager. ========== Add a test case to ensure we don't add UsageStore's checksum in DocumentStore's ComputeChecksum. ========== Account for UsageStore in GetDiskUsage. ========== Ensure that SchemaStore properly handles function calls when the schema isn't set. ========== Remove jlpl_strict_deps feature from package declarations. ========== Qualifies std::string in 3p directories ========== Section restricts should influence the relevance score. ========== Apply fixes upstream that were necessary to sync changes downstream. Also added a METADATA check to prevent any accidental adds of foo.proto.h includes. ========== Remove the 'com.google.protobuf' to 'com.google.android.icing.protobuf' translation in the export_to_aosp script. ========== Include usage store size in GetOptimizeInfo. This helps clients get a better idea of what savings they could get back if they called Optimize. Change-Id: Ia2339c7987267a73c49dadf1ced4a0a8ef001d4c
2021-01-14Update Icing from upstream.Tim Barron
Change-Id: I43038a59e7170fb8ecbaf6098a37221b3682ce09
2021-01-05Update Icing from upstream.Tim Barron
Change-Id: Iff50aebffb83529b0454e7c3a6dc6864e7a85f4a
2020-12-09Update Icing from upstream.Tim Barron
Change-Id: I662cd2f97794722eca9cc43b73549babed884be4
2020-11-18Update Icing from upstream.Alexander Dorokhine
Change-Id: Ic022a44e876a6060a47e0db991e63b2b73807769
2020-11-05Update icing from upstream.Alexander Dorokhine
Change-Id: Ia63a77142ec717c0d9a81ec0a5c1267381858200
2020-10-28Pull upstream changes.Terry Wang
Change-Id: I73ea5f80ccf16a02519f6f7ccfc993e9b0f39f86
2020-10-01Pull upstream changes.Terry Wang
Change-Id: I794757716961569b5c02171cfc82785efb2cf106
2020-09-24Pull upstream changes.Terry Wang
Change-Id: I44831fdadcdb67f2e19570a35cb4c76faf8397f9
2020-07-28Pull upstream changes.Terry Wang
Change-Id: Iaff79337219850eb1bbede8fc1738f3051e9005e
2020-07-24Pull upstream changes.Cassie Wang
Change-Id: Ieed20fd00a7c00778045434ae1b7c9e019a6c369
2020-06-25Pull upstream changes.Cassie Wang
Change-Id: I8a1e76e3e42188364ac40c0c51efb1e49292c015
2020-06-09Update from upstream, including proto location changes.Alexander Dorokhine
Bug: 158587690 Change-Id: Ie20d5515288cf3a1bdbc19ff5ec62f936456e85d
2020-06-05Copy over changes made to Google3 codebase in Icing.Tim Barron
Change-Id: Ia36edb0a1b085e249dabfc220a5b72418063604f
2020-05-04Pull upstream changes.Cassie Wang
Change-Id: I28e569082d59404e23e554a9aa7d8751328009e8
2020-03-23Copy over changes made to Google3 codebase in Icing.Tim Barron
Change-Id: Ic4e7a964ced111738cca80e078b3e03759c19ccf
2020-02-28Create JNI layer for IcingCassie Wang
Bug: 146383629 Test: manual 'atest java/tests/instrumentation/src/com/google/android/icing/IcingSearchEngineTest.java' Change-Id: I3868ca19a477bd6996f268d2a73a1dc906afaee6
2020-01-16Pull upstream changes.Cassie Wang
Upstream synced @290094995 Bug: 146383629 Test: manual, ran 'm -j libicing' with a local Android.bp Change-Id: I63eb12e93f3de5d6607ba7006fd3ea532dc079e4
2020-01-09Pull upstream changes and copy libtextclassifier classes.Cassie Wang
Upstream synced @288789500. Copied text_classifier dependencies (hash, status, logging) into icing/. Bug: 146383629 Test: manual - ran 'm -j libicing' with a working Android.bp locally Change-Id: I187a98af95b362745a09d605ed8334a6ff6971bb
2019-12-20Port over Icing c++ code from upstreamCassie Wang
Change-Id: Ia3981fed7e0e70589efc027d4123f306cdfbe990
2019-11-26Update Icing protos from piper upstream.Alexander Dorokhine
Change-Id: If0117363362baedc564e4c128b6d0a1009ace9b4
2019-11-22Import platform/external/icingAlexander Dorokhine
source_repository_url: "user/adorokhine/icing-search-ag" source_branch: "staging" source_tip_commit: "67ceaa1d6b6d7da5fa5ed3512ada340c2060250e" target_repository_name: "platform/external/icing" Bug: 145010258 Change-Id: I948c0d7979563d688ba08c13614ab6a54ad3a7ff
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-17 23:05:49 +0000