Move all chromite bin/* content into scripts/

The purpose of this change is to have all chromite scripts run through
a common wrapper.  Via this design, all scripts are accessible in python
namespace by default, and it gives us a single point to inject in
the common boilerplate for scripts (signal handling in particular).

Additionally, since PATH w/in the chroot no longer points into a python
namespace, we can expose only what we wish to be accessible- unittests
for example no longer are exposed.  We no longer have parallel_emerge and
parallel_emerge.py; just parallel_emerge.

The filtering of what is/isn't exposed was done via checking each/every
script name in the tree to see what consumed it.  Certain cases, we
can't yet clean it fully- for chrome-set_ver.py the chrome ebuild
hardcodes the .py (fixed via I9f2d7a14).  For cros_sdk.py, we
have to expose that (along with __init__.py) to allow depot_tools
chromite_wrapper to continue working.  Once that is fixed/deployed,
__init__.py and cros_sdk.py will be removed, and bin/ will no longer
be a valid python namespace.

Since these scripts are now invoked via a wrapper, all sys.path
modification has been removed- it's no longer necessary.

Finally, permissions were cleaned up.  Things that don't need +x no
longer have it.

For reviewing, by its nature this has to be a semi large change.  It's
in the reviewers interest to first look at scripts/wrapper.py, then
start looking through the rest.

For reference, the steps taken to generate this change are roughly thus:
1) Move all bin/ content into scripts as proper modules.
2) Update all pathways to access chromite.scripts.
3) Add a generic wrapper script that loads the script and executes it
 (scripts.wrapper).
4) For each bin/ entry that is desired/required to be accessible,
 add a symlink to bin/<the-name> pointing at ../scripts/wrapper.py.
 Exempting where required (code made use of it), we no longer expose
 the .py part of the name.  For example, you invoke check_gdata_token,
 not check_gdata_token.py; the consumer doesn't care about the implementation,
 the .py was exposed purely because we had unittests in the same directory.

BUG=chromium-os:26916
TEST=run each unittest in chromite.scripts
TEST=for x in chromite/scripts/*; do [ -L $x ] && \
  $x --help &> /dev/null || echo $x; done # basic check of import machinery.
  # Note that a few will fail because the script will bail out w/ need to be
  # ran as root, for example.  Need to run that from the chroot also.
TEST=cros_sdk --replace
TEST=cbuildbot x86-generic-full

Change-Id: I527ef6dd61e95b3fa3c89b61c6cbaf9022332d29
Reviewed-on: https://gerrit.chromium.org/gerrit/17104
Commit-Ready: Brian Harring <ferringb@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>

1 files changed, 109 insertions, 0 deletions

diff --git a/scripts/cros_extract_deps.py b/scripts/cros_extract_deps.py
new file mode 100644
index 000000000..5e6bd8008
--- /dev/null
+++ b/scripts/cros_extract_deps.py
@@ -0,0 +1,109 @@
+#!/usr/bin/python2.6
+# Copyright (c) 2010 The Chromium OS Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+import json
+import portage
+import sys
+from parallel_emerge import DepGraphGenerator
+
+def FlattenDepTree(deptree, pkgtable={}, parentcpv=None):
+  """
+  Turn something like this (the parallel_emerge DepsTree format):
+{
+  "app-admin/eselect-1.2.9": {
+    "action": "merge",
+    "deps": {
+      "sys-apps/coreutils-7.5-r1": {
+        "action": "merge",
+        "deps": {},
+        "deptype": "runtime"
+      },
+      ...
+    }
+  }
+}
+  ...into something like this (the cros_extract_deps format):
+{
+  "app-admin/eselect-1.2.9": {
+    "deps": ["coreutils-7.5-r1"],
+    "rev_deps": [],
+    "name": "eselect",
+    "category": "app-admin",
+    "version": "1.2.9",
+    "full_name": "app-admin/eselect-1.2.9",
+    "action": "merge"
+  },
+  "sys-apps/coreutils-7.5-r1": {
+    "deps": [],
+    "rev_deps": ["app-admin/eselect-1.2.9"],
+    "name": "coreutils",
+    "category": "sys-apps",
+    "version": "7.5-r1",
+    "full_name": "sys-apps/coreutils-7.5-r1",
+    "action": "merge"
+  }
+}
+  """
+  for cpv, record in deptree.items():
+    if cpv not in pkgtable:
+      cat, nam, ver, rev = portage.catpkgsplit(cpv)
+      pkgtable[cpv] = {"deps": [],
+                       "rev_deps": [],
+                       "name": nam,
+                       "category": cat,
+                       "version": "%s-%s" % (ver, rev),
+                       "full_name": cpv,
+                       "action": record["action"]}
+    # If we have a parent, that is a rev_dep for the current package.
+    if parentcpv:
+      pkgtable[cpv]["rev_deps"].append(parentcpv)
+    # If current package has any deps, record those.
+    for childcpv in record["deps"]:
+      pkgtable[cpv]["deps"].append(childcpv)
+    # Visit the subtree recursively as well.
+    FlattenDepTree(record["deps"], pkgtable=pkgtable, parentcpv=cpv)
+  return pkgtable
+
+
+def Usage():
+  """Print usage."""
+  print """Usage:
+cros_extract_deps [--board=BOARD] [emerge args] package
+
+This extracts the dependency tree for the specified package, and outputs it
+to stdout, in a serialized JSON format. Emerge flags such as --root-deps
+can be used to influence what sort of dependency tree is extracted.
+"""
+
+
+def main(argv):
+  if len(argv) == 1:
+    Usage()
+    sys.exit(1)
+
+  # We want the toolchain to be quiet because we are going to output
+  # a well-formed json payload.
+  argv = ['--quiet', '--pretend']
+
+  # cros_extract_deps defaults to rdeps. However, only use this if
+  # there was no explicit --root-deps command line switch.
+  default_rootdeps_arg = ['--root-deps=rdeps']
+  for arg in argv:
+    if arg.startswith('--root-deps'):
+      default_rootdeps_arg = []
+
+  # Now, assemble the overall argv as the concatenation of the
+  # default list + possible rootdeps-default + actual command line.
+  argv.extend(default_rootdeps_arg)
+  argv.extend(sys.argv[1:])
+
+  deps = DepGraphGenerator()
+  deps.Initialize(argv)
+  deps_tree, deps_info = deps.GenDependencyTree()
+  print json.dumps(FlattenDepTree(deps_tree), sort_keys=True, indent=2)
+
+
+if __name__ == "__main__":
+  main(sys.argv[1:])