index
:
external/licenseclassifier.git
aml_tz5_341510010
android14-d1-release
android14-d1-s1-release
android14-d1-s2-release
android14-d1-s3-release
android14-d1-s4-release
android14-d1-s5-release
android14-d1-s6-release
android14-d1-s7-release
android14-d2-release
android14-d2-s1-release
android14-d2-s2-release
android14-d2-s3-release
android14-d2-s4-release
android14-d2-s5-release
android14-dev
android14-gsi
android14-mainline-adbd-release
android14-mainline-adservices-release
android14-mainline-appsearch-release
android14-mainline-art-release
android14-mainline-cellbroadcast-release
android14-mainline-conscrypt-release
android14-mainline-extservices-release
android14-mainline-healthfitness-release
android14-mainline-media-release
android14-mainline-media-swcodec-release
android14-mainline-mediaprovider-release
android14-mainline-networking-release
android14-mainline-os-statsd-release
android14-mainline-permission-release
android14-mainline-resolv-release
android14-mainline-sdkext-release
android14-mainline-tethering-release
android14-mainline-uwb-release
android14-mainline-wifi-release
android14-platform-release
android14-qpr1-release
android14-qpr1-s2-release
android14-qpr2-release
android14-qpr2-s1-release
android14-qpr2-s2-release
android14-qpr2-s3-release
android14-qpr2-s4-release
android14-qpr2-s5-release
android14-qpr3-release
android14-release
android14-s1-release
android14-s2-release
android14-security-release
android14-tests-release
build-tools-release
busytown-mac-infra-release
main
main-16k
master
sdk-release
simpleperf-release
upstream-main
[no description]
about
summary
refs
log
tree
commit
diff
log msg
author
committer
range
path:
root
/
v2
/
tokenizer.go
Age
Commit message (
Expand
)
Author
2022-09-16
Make the public facing API be implemented in terms of io.Reader rather than
Bill Neubauer
2022-09-16
Rewrite the tokenization process to work on streams rather than requiring the
Bill Neubauer
2022-09-16
Removing the Index field from the token structures.
Bill Neubauer
2022-09-16
Adds Copyright detection to the report generated by the classifier.
Bill Neubauer
2022-03-16
Fixes handling of newline characters so that Normalize preserves the newline
Bill Neubauer
2022-03-16
Automated g4 rollback of changelist 415285962.
Google Open Source
2022-03-16
Fixes handling of newline characters so that Normalize preserves the newline
Bill Neubauer
2022-03-16
API implementation for the Normalize method.
Bill Neubauer
2022-03-16
Adding old style MIT license C-Ares.
Google Open Source
2021-03-24
remove reduntant check
Bharat Biradar
2021-03-22
Fix typo
Bharat Biradar
2020-11-13
Change the public API to use []byte instead of string.
Bill Neubauer
2020-11-13
Scope use of phrase induction to fix some bugs.
Bill Neubauer
2020-11-13
Fix handling of extremely long lines by inserting EOL tokens at sentence
Bill Neubauer
2020-11-13
This CL fixes a bug in text handling due to iterating over a slice of runes
Bill Neubauer
2020-11-13
Add new testing scenarios and testing functionality.
Bill Neubauer
2020-11-13
Needed to make a change to number tokenization to resolve an issue
Bill Neubauer
2020-11-13
The tokenizer for the new version of the licenseclassifier.
Bill Neubauer