| Age | Commit message (Collapse) | Author |
|---|
|
Turn license into a stand alone script that uses a helper library to do
it's work. Split up it's 'main' method into distinct stages.
Also, fix nit's in hook script from a previous CL.
BUG=None
TEST=cros lint + manual testing.
CQ-DEPEND=CL:212124
Change-Id: Ic81d50260a6b7603904aabc38b3877ad59abb2b7
Reviewed-on: https://chromium-review.googlesource.com/212137
Tested-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Don Garrett <dgarrett@chromium.org>
|
|
Since we call the license script with two totally seperate code paths,
split it into two scripts with a shared library to make them easier to
follow.
This creates the new ebuild_licensing_hook script to be called during
ebuild processing. We can't remove the equivalent code from the
current script until the ebuild hook script (gen-package-licenses.sh)
is also updated.
BUG=None
TEST=cros lint + manual testing.
Change-Id: I0dcbe61d6379ba22a141d5e9edd048792934bf71
Reviewed-on: https://chromium-review.googlesource.com/212131
Tested-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Commit-Queue: Don Garrett <dgarrett@chromium.org>
|
|
The licenses script could not generate HTML output if not invoked from the
same directory as the script, since it couldn't find it's template files.
This CL uses the location of the script file to locate the templates,
reguardless.
BUG=None
TEST=Ran from src/scripts.
Change-Id: I737f61f3de114c25074300e7bc2b9ff46de404d4
Reviewed-on: https://chromium-review.googlesource.com/211740
Reviewed-by: Yu-Ju Hong <yjhong@chromium.org>
Commit-Queue: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
|
|
Some packages contain proprietary binary blobs for which we have no
requirement to list license information.
Add a new license value for they can be used to be skipped (without
error) by the licensing process.
Adjusted ebuild verification code to ignore SKIPPED_LICENSES.
BUG=chromium:386758
TEST=pytlint + tryjob with peach_pit exynos-pre-boot using new license.
repo uploaded an ebuild which uses a SKIPPED_LICENSE.
Change-Id: I769411cc40943116be52e429d6bd569b04116b09
Reviewed-on: https://chromium-review.googlesource.com/208242
Tested-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Don Garrett <dgarrett@chromium.org>
|
|
Determine if a given ebuild actually installed anything before
deciding if we need to look for and require license information.
BUG=chromium:392566
TEST=cbuildbot --remote -g 204685 -g 207416 panther-release
(expecting to see linux-firmware pass, and coreboot fail)
Change-Id: I4842294c04b072d279094723644d55607e8870b9
Reviewed-on: https://chromium-review.googlesource.com/207416
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
|
|
The cbuildbot script lives in a directory named buildbot. Which is a
bit confusing.
BUG=chromium:373277
TEST=Unitests + cros lint.
CQ-DEPEND=CL:200157
CQ-DEPEND=CL:200165
CQ-DEPEND=CL:200149
CQ-DEPEND=CL:*163598
CQ-DEPEND=CL:*163760
CQ-DEPEND=CL:*163786
Change-Id: Ia5953a4506e8b47d27e1a6908ecb938a439da8c2
Reviewed-on: https://chromium-review.googlesource.com/199664
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
|
|
The script doesn't save license dumps for virtual packages, so don't
try to load them.
BUG=None
TEST=Generated licensing for m36
Change-Id: I4d3d7f018f1ba673e0a8be06b0d6fa33b2480f76
Reviewed-on: https://chromium-review.googlesource.com/200825
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Yu-Ju Hong <yjhong@chromium.org>
Tested-by: Yu-Ju Hong <yjhong@chromium.org>
|
|
BUG=chromium:372975
TEST=None
Change-Id: I39be6b08b841161566737fd99d6a07f60ce3501d
Reviewed-on: https://chromium-review.googlesource.com/199621
Reviewed-by: Zelidrag Hornung <zelidrag@chromium.org>
Tested-by: Yu-Ju Hong <yjhong@chromium.org>
|
|
Removed gather-licenses.sh process-pkg.py (unfinished prototypes) now
that they are replaced by this script.
- updated documentation in comments
- cleaned up yaml import as recommended by David.
- ebuild_dir and description were unused in PackageInfo. Removed them.
- build_source_tree is used to create a PackageInfo from an unpackage
package (as opposed to the other way that queries an ebuild).
- _BuildInfo was added to retrieve package information from the
build-info fork present at package build time when called as a hook.
- Licensing object can now be initialized without visibility on the
template files (when run as a hook), which is fine since they are not
used anyway.
- Per package license bits are now saved/loaded in
/var/db/pkg/cat/name/license.yaml to match the location we get from
portage when we install a prebuilt.
- HookPackageProcess is a new simpler way to enter/generate a license
object and output a per package license bit in its build-info
directory.
- If a licensing run realizes that per package licensing bits are
missing, it creates them on the fly, but only if it is run under sudo
(of course that also slows down image generation from <20sec to
minutes).
BUG=chromium:197970 chromium:271832 chromium:207004 chromium:356539
TEST=(cr) (license-generation) merlin@polgara ~/trunk/chromite/licensing $ FEATURES='noclean' emerge-x86-alex libc-bench
Calculating dependencies... done!
>>> Emerging (1 of 1) dev-util/libc-bench-0.0.1-r8 from chromiumos for /build/x86-alex/
(...)
File not built with -Wl,-z,now: /build/x86-alex/tmp/portage/dev-util/libc-bench-0.0.1-r8/image/usr/local/libc-bench/libc-bench
Generating license for dev-util/libc-bench-0.0.1-r8 in /build/x86-alex/tmp/portage/dev-util/libc-bench-0.0.1-r8
13:34:39: INFO: Read licenses for dev-util/libc-bench-0.0.1-r8: MIT
13:34:39: INFO: dev-util/libc-bench-0.0.1-r8: can't use MIT, will scan source code for copyright
13:34:39: INFO: License(s) for dev-util/libc-bench-0.0.1-r8: COPYRIGHT
13:34:39: INFO: Adding License /build/x86-alex/tmp/portage/dev-util/libc-bench-0.0.1-r8/work/libc-bench/COPYRIGHT (UTF-8)
strip: i686-pc-linux-gnu-strip --strip-unneeded -R .comment -R .GCC.command.line
(...)
>>> Auto-cleaning packages...
>>> Using system located in ROOT tree /build/x86-alex/
>>> No outdated packages were found on your system.
(cr) (license-generation) merlin@polgara ~/trunk/chromite/licensing $ qtbz2 -x -O /build/x86-alex/packages/dev-util/libc-bench-0.0.1-r8.tbz2 | qxpak -x -O - license.yaml
- [category, !!python/unicode 'dev-util']
- [licensing_failed, false]
- [name, !!python/unicode 'libc-bench']
- - ebuild_license_names
- [!!python/unicode 'MIT']
- - homepages
- [!!python/unicode 'http://www.etalabs.net/libc-bench.html', !!python/unicode 'http://git.musl-libc.org/cgit/libc-bench/']
- [ebuild_path, null]
- [need_copyright_attribution, true]
- - license_names
- !!set {}
- [version, !!python/unicode '0.0.1']
- [board, /build/x86-alex]
- [skip, false]
- [scan_source_for_licenses, true]
- - license_text_scanned
- ["Scanned Source License COPYRIGHT:\n\nlibc-bench, a performance and memory usage\
\ benchmark for comparing\nlibc implementations\n\nCopyright \xA9 2011 Rich\
\ Felker\n\nPermission is hereby granted, free of charge, to any person obtaining\n\
a copy of this software and associated documentation files (the\n\"Software\"\
), to deal in the Software without restriction, including\nwithout limitation\
\ the rights to use, copy, modify, merge, publish,\ndistribute, sublicense,\
\ and/or sell copies of the Software, and to\npermit persons to whom the Software\
\ is furnished to do so, subject to\nthe following conditions:\n\nThe above\
\ copyright notice and this permission notice shall be\nincluded in all copies\
\ or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS\
\ IS\", WITHOUT WARRANTY OF ANY KIND,\nEXPRESS OR IMPLIED, INCLUDING BUT NOT\
\ LIMITED TO THE WARRANTIES OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE\
\ AND NONINFRINGEMENT.\nIN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE\
\ LIABLE FOR ANY\nCLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF\
\ CONTRACT,\nTORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE\n\
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n"]
- [build_source_tree, /build/x86-alex/tmp/portage/dev-util/libc-bench-0.0.1-r8]
- [revision, '8']
(cr) (license-generation) merlin@polgara ~/trunk/chromite/licensing $ ls -l /build/x86-alex/var/db/pkg/dev-util/libc-bench-0.0.1-r8/license.yaml
-rw-r--r-- 1 root root 1983 Apr 10 13:34 /build/x86-alex/var/db/pkg/dev-util/libc-bench-0.0.1-r8/license.yaml
Change-Id: I787edaaf63c848e0e2dd76dcd687c8358c92152a
Reviewed-on: https://chromium-review.googlesource.com/194178
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Marc MERLIN <merlin@chromium.org>
Tested-by: Marc MERLIN <merlin@chromium.org>
|
|
This is mechanical -- virtual/target-os depends on chromeos,
so there shouldn't be any real difference in the build.
BUG=chromium:329920
TEST=`cbuildbot beaglebone` passes
TEST=`./buildbot/run_tests` passes
CQ-DEPEND=CL:192991
Change-Id: I876268086b715f7cff868e630cb05fd91067b11e
Reviewed-on: https://chromium-review.googlesource.com/192487
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
|
|
src/repohooks/pre-upload.py imports us directly from outside the chroot
where yaml may not be there. Therefore delay loading yaml to inside
main if we get called the normal way.
BUG=N/A
TESTS=ran inside the chroot, yaml gets loaded as it should.
Change-Id: I000211fc76f3d1259e3798ce8913e202e417b962
Reviewed-on: https://chromium-review.googlesource.com/193976
Reviewed-by: David Riley <davidriley@chromium.org>
Commit-Queue: Marc MERLIN <merlin@chromium.org>
Tested-by: Marc MERLIN <merlin@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
|
|
about_credits.tmpl fixes a bug with license folding/unfolding
licenses.py:
- Added yaml as the safest way to save objects for licensing bits
- PER_PKG_LICENSE_DIR is now where per package bits will get written to
and read back from.
- Removed everything from SKIPPED_PACKAGES, we'll list all X11 packages
from now on as per discussion with vapier
- PackageSkipped is not an exception anymore, it's a per package state
that can be written to disk and read back later (self.skip).
- Similarly licensing_failed can be saved for a package.
- GetPackageInfo now only retrieves basic package information from
portage
- GetLicenses does the slow work of retrieving 'HOMEPAGE', 'LICENSE',
'DESCRIPTION'
- In turn this now allows main to get a package list, compute what gets
filtered and show the real filtered package list before work is done.
- SaveLicenseDump/LoadLicenseDump take care of saving and loading per
package license bits
- gen_licenses is a new option to generate all the license bits when we
have a tree where they have not been generated. Without it, licensing
now retrieves the bits and makes a license file in under 20 seconds.
- When called with gen-licenses ProcessPackageLicenses will first
generate all the per package licensing bits, and then reload them as
if it had been run without --generate-licenses. This shares the common
codepath we'll use during normal runs were the licensing script is run
in build_image and all the bits are already generated.
BUG=chromium:197970 chromium:271812 chromium:207004 chromium:319909
TEST=Generated and inspected output file.
Change-Id: I29384dd2253d68c5ace5e849153e2b6eda13c948
Reviewed-on: https://chromium-review.googlesource.com/191320
Reviewed-by: Marc MERLIN <merlin@chromium.org>
Tested-by: Marc MERLIN <merlin@chromium.org>
|
|
Found a bug that happened if you didn't run eclean, so added an
exception for the bug (package is marked as upgrade), and improved
comments to explain eclean should be run.
Remind the submitter to use Publish+Mail in the svn interface.
Added regex support for licenses, and manually inspected the diff to
verify we do get more licenses than we used to.
Fixed typo on the override of chromeos-chrome.
BUG=chromium:197970 chromium:271812
TEST=Ran licensing, inspected the new output, confirmed we have new
licenses only.
Change-Id: Iffe08d67b25e37cecdd2549dda8c6f0ac61dc073
Reviewed-on: https://chromium-review.googlesource.com/189392
Reviewed-by: Marc MERLIN <merlin@chromium.org>
Tested-by: Marc MERLIN <merlin@chromium.org>
|
|
TEST=generated license file, tested buttons
BUG=chromium:319909
Change-Id: Ic943651367be988edd42a66b2c836372fb05aa93
Reviewed-on: https://chromium-review.googlesource.com/189391
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Marc MERLIN <merlin@chromium.org>
|
|
BUG=chromium:319907
TEST=Checked that AFL license is properly wrapped with this change.
Change-Id: I132ee211c401bde95ee46b08f3262a3815c2e159
Reviewed-on: https://chromium-review.googlesource.com/188601
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Marc MERLIN <merlin@chromium.org>
Tested-by: Marc MERLIN <merlin@chromium.org>
|
|
- Added --all-packages which allows running against all packages in the
tree.
- chromeos-base/* packages are now part of the packages that are
checked for licenses. Until their ebuilds are all fixed, their license
is auto fixed from BSD to BSD-Google.
- Removed exceptions that listed Proprietary as a license since this is
not a real license.
- Packages that use Proprietary as a license are now rewritten as
Google-TOS and a logging.error is output to fix the source
- Put a temporary static mapping for dev-python/pycairo until we can
parse its license type, as well as for chromeos-chrome.
http://crbug.com/348779
- Having '||' in the middle of a license raises an error but lets the
script continue doing its job 'till the end.
- ListInstalledPackages() has improved/cleaned up documentation and now
supports --all-packages
- SKIPPED_PACKAGES is now a static list, not passed to main anymore.
BmUG=chromium:197970,chromium:271812
TEST=Ran licensing against current tree, no errors and new packages
added.
Change-Id: I40c0887a98160f52990cb8309a79cd1fa1d36062
Reviewed-on: https://chromium-review.googlesource.com/188206
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Marc MERLIN <merlin@chromium.org>
Tested-by: Marc MERLIN <merlin@chromium.org>
|
|
Many existing scripts use '/build/board' directly as the sysroot. We
should unify this and have a function that returns the sysroot of a
board.
BUG=None
TEST=`buildbot/run_tests` passes
Change-Id: I13bc130c6f156f70a09cbed9d38ff82204cca0d7
Reviewed-on: https://chromium-review.googlesource.com/182923
Reviewed-by: Yu-Ju Hong <yjhong@chromium.org>
Commit-Queue: Yu-Ju Hong <yjhong@chromium.org>
Tested-by: Yu-Ju Hong <yjhong@chromium.org>
|
|
Also fix a bug I introduced in CL:178129.
BUG=None
TEST=M33 licensing run
Change-Id: Ib8b898d17138f1d30692080b8e0eeb0bdd99ab07
Reviewed-on: https://chromium-review.googlesource.com/180875
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Yu-Ju Hong <yjhong@chromium.org>
Tested-by: Yu-Ju Hong <yjhong@chromium.org>
|
|
Make sure:
- correct ordering
- correct name usage
- new lines before each section
Maybe one or two other random fixes too like whitespace alignment.
BUG=chromium:327969
TEST=`cros lint` doesn't complain as much
TEST=`./buildbot/run_tests` passes
Change-Id: Ia57f06d5c0709103b2373568989d03d8fbaceeb5
Reviewed-on: https://chromium-review.googlesource.com/179802
Reviewed-by: Matt Tennant <mtennant@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
|
|
This CL allows passing a list of packages to licenses.py to check for
licenses. We need this feature to test packages at the BuildPackages
stage. Previously, this was not possible without generating an HTML
output file.
BUG=chromium:323801
TEST=./licenses -p $PACKAGE -b x86-alex
Change-Id: Ic00eb9ab1374b5d2498b00605ad352bdddbf2501
Reviewed-on: https://chromium-review.googlesource.com/178129
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Yu-Ju Hong <yjhong@chromium.org>
Tested-by: Yu-Ju Hong <yjhong@chromium.org>
|
|
To implement a presubmission check for licensing, we need a fast way
to get all license types from an ebuild file without having to access
chroot (to use portageq). This CL implements one such function.
BUG=chromium:318364
TEST=Run against all ebuild files.
TEST=Unittest for cros_bulid_lib changes.
Change-Id: Ie248230fb93d6aa40d8cba1c85633351fbef175d
Reviewed-on: https://chromium-review.googlesource.com/177693
Reviewed-by: Yu-Ju Hong <yjhong@chromium.org>
Tested-by: Yu-Ju Hong <yjhong@chromium.org>
Commit-Queue: Yu-Ju Hong <yjhong@chromium.org>
|
|
Allow calling of FindLicenseType() from inside and outside chroot. An
example usage is to add presubmission check of license-existence,
which is run from outside chroot.
BUG=chromium:318364
TEST=Invoke the class methods from inside and outside chroot.
Change-Id: Ia88e66571e29749d85f69b604ab83d5054b51a8d
Reviewed-on: https://chromium-review.googlesource.com/177398
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Yu-Ju Hong <yjhong@chromium.org>
Tested-by: Yu-Ju Hong <yjhong@chromium.org>
|
|
This CL fixes two HTML escaping issues complained by xmllint. In
licenses.py, HTML escaping should be done right before evaluting the
template to make sure it is performed once (and only once) for every
license file. This change ensures that all URLs and author emails in
the license files are properly displayed. In the template itself, the
block of code surrounded by tag <script></script> should be marked as
a CDATA section so that it can be ignored by the XML parser.
BUG=chromium:317103
BUG=chromium:318909
TEST=Run the script locally and verify the output, pass xmllint
Change-Id: I1b1a7ac35532f2918bcf70031db7b7cb69b7f4fa
Reviewed-on: https://chromium-review.googlesource.com/177287
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Yu-Ju Hong <yjhong@chromium.org>
Commit-Queue: Yu-Ju Hong <yjhong@chromium.org>
|
|
Explain error messages better to guide users to fix the errors. Also
switch to to use ReadUnknownEncodedFile throughout licenses.py.
BUG=chromium:271812
BUG=chromium:320779
TEST=Run the script locally and verify the output
Change-Id: I98c858989c74c1648132e140f6fe69ad5c1e90ef
Reviewed-on: https://chromium-review.googlesource.com/177276
Reviewed-by: Yu-Ju Hong <yjhong@chromium.org>
Commit-Queue: Yu-Ju Hong <yjhong@chromium.org>
Tested-by: Yu-Ju Hong <yjhong@chromium.org>
|
|
Rename directory license-generation to licensing. Create a symlink
licenses to use scripts/wrapper.py
BUG=chromium:271812
BUG=chromium:320779
TEST=Run the script locally and verify the output
Change-Id: I5d3780ac0eec5b55d93c614aee3bda3918be0aab
Reviewed-on: https://chromium-review.googlesource.com/177150
Reviewed-by: Yu-Ju Hong <yjhong@chromium.org>
Tested-by: Yu-Ju Hong <yjhong@chromium.org>
Commit-Queue: Yu-Ju Hong <yjhong@chromium.org>
|
