diff options
Diffstat (limited to 'README.rst')
| -rw-r--r-- | README.rst | 433 |
1 files changed, 371 insertions, 62 deletions
@@ -6,12 +6,12 @@ YAPF :target: https://badge.fury.io/py/yapf :alt: PyPI version -.. image:: https://travis-ci.org/google/yapf.svg?branch=master - :target: https://travis-ci.org/google/yapf +.. image:: https://github.com/google/yapf/actions/workflows/ci.yml/badge.svg + :target: https://github.com/google/yapf/actions :alt: Build status -.. image:: https://coveralls.io/repos/google/yapf/badge.svg?branch=master - :target: https://coveralls.io/r/google/yapf?branch=master +.. image:: https://coveralls.io/repos/google/yapf/badge.svg?branch=main + :target: https://coveralls.io/r/google/yapf?branch=main :alt: Coverage status @@ -23,21 +23,20 @@ are made to remove lint errors from code. This has some obvious limitations. For instance, code that conforms to the PEP 8 guidelines may not be reformatted. But it doesn't mean that the code looks good. -YAPF takes a different approach. It's based off of 'clang-format', developed by -Daniel Jasper. In essence, the algorithm takes the code and reformats it to the -best formatting that conforms to the style guide, even if the original code -didn't violate the style guide. The idea is also similar to the 'gofmt' tool for -the Go programming language: end all holy wars about formatting - if the whole -codebase of a project is simply piped through YAPF whenever modifications are -made, the style remains consistent throughout the project and there's no point -arguing about style in every code review. +YAPF takes a different approach. It's based off of `'clang-format' <https://cl +ang.llvm.org/docs/ClangFormat.html>`_, developed by Daniel Jasper. In essence, +the algorithm takes the code and reformats it to the best formatting that +conforms to the style guide, even if the original code didn't violate the +style guide. The idea is also similar to the `'gofmt' <https://golang.org/cmd/ +gofmt/>`_ tool for the Go programming language: end all holy wars about +formatting - if the whole codebase of a project is simply piped through YAPF +whenever modifications are made, the style remains consistent throughout the +project and there's no point arguing about style in every code review. The ultimate goal is that the code YAPF produces is as good as the code that a programmer would write if they were following the style guide. It takes away some of the drudgery of maintaining your code. -Try out YAPF with this `online demo <https://yapf.now.sh>`_. - .. footer:: YAPF is not an official Google product (experimental or otherwise), it is @@ -114,10 +113,10 @@ Options:: --style STYLE specify formatting style: either a style name (for example "pep8" or "google"), or the name of a file with style settings. The default is pep8 unless a - .style.yapf or setup.cfg file located in the same - directory as the source or one of its parent - directories (for stdin, the current directory is - used). + .style.yapf or setup.cfg or pyproject.toml file + located in the same directory as the source or one of + its parent directories (for stdin, the current + directory is used). --style-help show style settings and exit; this output can be saved to .style.yapf to make your settings permanent --no-local-style don't search for local style definition @@ -136,6 +135,33 @@ If ``--diff`` is supplied, YAPF returns zero when no changes were necessary, non otherwise (including program error). You can use this in a CI workflow to test that code has been YAPF-formatted. +--------------------------------------------- +Excluding files from formatting (.yapfignore or pyproject.toml) +--------------------------------------------- + +In addition to exclude patterns provided on commandline, YAPF looks for additional +patterns specified in a file named ``.yapfignore`` or ``pyproject.toml`` located in the +working directory from which YAPF is invoked. + +``.yapfignore``'s syntax is similar to UNIX's filename pattern matching:: + + * matches everything + ? matches any single character + [seq] matches any character in seq + [!seq] matches any character not in seq + +Note that no entry should begin with `./`. + +If you use ``pyproject.toml``, exclude patterns are specified by ``ignore_pattens`` key +in ``[tool.yapfignore]`` section. For example: + +.. code-block:: ini + + [tool.yapfignore] + ignore_patterns = [ + "temp/**/*.py", + "temp2/*.py" + ] Formatting style ================ @@ -159,26 +185,41 @@ with a ``[style]`` heading. For example: split_before_logical_operator = true The ``based_on_style`` setting determines which of the predefined styles this -custom style is based on (think of it like subclassing). +custom style is based on (think of it like subclassing). Four +styles are predefined: + +- ``pep8`` (default) +- ``google`` (based off of the `Google Python Style Guide`_) +- ``yapf`` (for use with Google open source projects) +- ``facebook`` + +.. _`Google Python Style Guide`: https://github.com/google/styleguide/blob/gh-pages/pyguide.md + +See ``_STYLE_NAME_TO_FACTORY`` in style.py_ for details. + +.. _style.py: https://github.com/google/yapf/blob/main/yapf/yapflib/style.py It's also possible to do the same on the command line with a dictionary. For example: .. code-block:: shell - --style='{based_on_style: chromium, indent_width: 4}' + --style='{based_on_style: pep8, indent_width: 2}' -This will take the ``chromium`` base style and modify it to have four space +This will take the ``pep8`` base style and modify it to have two space indentations. YAPF will search for the formatting style in the following manner: 1. Specified on the command line -2. In the `[style]` section of a `.style.yapf` file in either the current +2. In the ``[style]`` section of a ``.style.yapf`` file in either the current + directory or one of its parent directories. +3. In the ``[yapf]`` section of a ``setup.cfg`` file in either the current directory or one of its parent directories. -3. In the `[yapf]` section of a `setup.cfg` file in either the current +4. In the ``[tool.yapf]`` section of a ``pyproject.toml`` file in either the current directory or one of its parent directories. -4. In the `~/.config/yapf/style` file in your home directory. +5. In the ``[style]`` section of a ``~/.config/yapf/style`` file in your home + directory. If none of those files are found, the default style is used (PEP8). @@ -239,8 +280,11 @@ share several arguments which are described below: >>> from yapf.yapflib.yapf_api import FormatCode # reformat a string of code - >>> FormatCode("f ( a = 1, b = 2 )") + >>> formatted_code, changed = FormatCode("f ( a = 1, b = 2 )") + >>> formatted_code 'f(a=1, b=2)\n' + >>> changed + True A ``style_config`` argument: Either a style name or a path to a file that contains formatting style settings. If None is specified, use the default style @@ -248,7 +292,7 @@ as set in ``style.DEFAULT_STYLE_FACTORY``. .. code-block:: python - >>> FormatCode("def g():\n return True", style_config='pep8') + >>> FormatCode("def g():\n return True", style_config='pep8')[0] 'def g():\n return True\n' A ``lines`` argument: A list of tuples of lines (ints), [start, end], @@ -258,15 +302,15 @@ than a whole file. .. code-block:: python - >>> FormatCode("def g( ):\n a=1\n b = 2\n return a==b", lines=[(1, 1), (2, 3)]) + >>> FormatCode("def g( ):\n a=1\n b = 2\n return a==b", lines=[(1, 1), (2, 3)])[0] 'def g():\n a = 1\n b = 2\n return a==b\n' A ``print_diff`` (bool): Instead of returning the reformatted source, return a -diff that turns the formatted source into reformatter source. +diff that turns the formatted source into reformatted source. .. code-block:: python - >>> print(FormatCode("a==b", filename="foo.py", print_diff=True)) + >>> print(FormatCode("a==b", filename="foo.py", print_diff=True)[0]) --- foo.py (original) +++ foo.py (reformatted) @@ -1 +1 @@ @@ -285,19 +329,61 @@ the diff, the default is ``<unknown>``. >>> print(open("foo.py").read()) # contents of file a==b - >>> FormatFile("foo.py") - ('a == b\n', 'utf-8') + >>> reformatted_code, encoding, changed = FormatFile("foo.py") + >>> formatted_code + 'a == b\n' + >>> encoding + 'utf-8' + >>> changed + True -The ``in-place`` argument saves the reformatted code back to the file: +The ``in_place`` argument saves the reformatted code back to the file: .. code-block:: python - >>> FormatFile("foo.py", in_place=True) + >>> FormatFile("foo.py", in_place=True)[:2] (None, 'utf-8') >>> print(open("foo.py").read()) # contents of file (now fixed) a == b +Formatting diffs +================ + +Options:: + + usage: yapf-diff [-h] [-i] [-p NUM] [--regex PATTERN] [--iregex PATTERN][-v] + [--style STYLE] [--binary BINARY] + + This script reads input from a unified diff and reformats all the changed + lines. This is useful to reformat all the lines touched by a specific patch. + Example usage for git/svn users: + + git diff -U0 --no-color --relative HEAD^ | yapf-diff -i + svn diff --diff-cmd=diff -x-U0 | yapf-diff -p0 -i + + It should be noted that the filename contained in the diff is used + unmodified to determine the source file to update. Users calling this script + directly should be careful to ensure that the path in the diff is correct + relative to the current working directory. + + optional arguments: + -h, --help show this help message and exit + -i, --in-place apply edits to files instead of displaying a diff + -p NUM, --prefix NUM strip the smallest prefix containing P slashes + --regex PATTERN custom pattern selecting file paths to reformat + (case sensitive, overrides -iregex) + --iregex PATTERN custom pattern selecting file paths to reformat + (case insensitive, overridden by -regex) + -v, --verbose be more verbose, ineffective without -i + --style STYLE specify formatting style: either a style name (for + example "pep8" or "google"), or the name of a file + with style settings. The default is pep8 unless a + .style.yapf or setup.cfg or pyproject.toml file + located in the same directory as the source or one of + its parent directories (for stdin, the current + directory is used). + --binary BINARY location of binary to use for yapf Knobs ===== @@ -319,9 +405,35 @@ Knobs value, } +``ALLOW_SPLIT_BEFORE_DEFAULT_OR_NAMED_ASSIGNS`` + Allow splitting before a default / named assignment in an argument list. + ``ALLOW_SPLIT_BEFORE_DICT_VALUE`` Allow splits before the dictionary value. +``ARITHMETIC_PRECEDENCE_INDICATION`` + Let spacing indicate operator precedence. For example: + + .. code-block:: python + + a = 1 * 2 + 3 / 4 + b = 1 / 2 - 3 * 4 + c = (1 + 2) * (3 - 4) + d = (1 - 2) / (3 + 4) + e = 1 * 2 - 3 + f = 1 + 2 + 3 + 4 + + will be formatted as follows to indicate precedence: + + .. code-block:: python + + a = 1*2 + 3/4 + b = 1/2 - 3*4 + c = (1+2) * (3-4) + d = (1-2) / (3+4) + e = 1*2 - 3 + f = 1 + 2 + 3 + 4 + ``BLANK_LINE_BEFORE_NESTED_CLASS_OR_DEF`` Insert a blank line before a ``def`` or ``class`` immediately nested within another ``def`` or ``class``. For example: @@ -352,9 +464,14 @@ Knobs class Bar: pass +``BLANK_LINES_BETWEEN_TOP_LEVEL_IMPORTS_AND_VARIABLES`` + Sets the number of desired blank lines between top-level imports and + variable definitions. Useful for compatibility with tools like isort. + ``COALESCE_BRACKETS`` Do not split consecutive brackets. Only relevant when - ``DEDENT_CLOSING_BRACKETS`` is set. For example: + ``DEDENT_CLOSING_BRACKETS`` or ``INDENT_CLOSING_BRACKETS`` + is set. For example: .. code-block:: python @@ -381,16 +498,14 @@ Knobs ``CONTINUATION_ALIGN_STYLE`` The style for continuation alignment. Possible values are: - - SPACE: Use spaces for continuation alignment. This is default behavior. - - FIXED: Use fixed number (CONTINUATION_INDENT_WIDTH) of columns - (ie: CONTINUATION_INDENT_WIDTH/INDENT_WIDTH tabs) for continuation - alignment. - - VALIGN-RIGHT: Vertically align continuation lines with indent characters. - Slightly right (one more indent character) if cannot vertically align - continuation lines with indent characters. - - For options ``FIXED``, and ``VALIGN-RIGHT`` are only available when - ``USE_TABS`` is enabled. + - ``SPACE``: Use spaces for continuation alignment. This is default + behavior. + - ``FIXED``: Use fixed number (CONTINUATION_INDENT_WIDTH) of columns + (ie: CONTINUATION_INDENT_WIDTH/INDENT_WIDTH tabs or CONTINUATION_INDENT_WIDTH + spaces) for continuation alignment. + - ``VALIGN-RIGHT``: Vertically align continuation lines to multiple of + INDENT_WIDTH columns. Slightly right (one tab or a few spaces) if cannot + vertically align continuation lines with indent characters. ``CONTINUATION_INDENT_WIDTH`` Indent width used for line continuations. @@ -422,6 +537,10 @@ Knobs ``EACH_DICT_ENTRY_ON_SEPARATE_LINE`` Place each dictionary entry onto its own line. +``FORCE_MULTILINE_DICT`` + Respect EACH_DICT_ENTRY_ON_SEPARATE_LINE even if the line is shorter than + COLUMN_LIMIT. + ``I18N_COMMENT`` The regex for an internationalization comment. The presence of this comment stops reformatting of that line, because the comments are required to be @@ -448,12 +567,32 @@ Knobs ``INDENT_WIDTH`` The number of columns to use for indentation. +``INDENT_BLANK_LINES`` + Set to ``True`` to prefer indented blank lines rather than empty + +``INDENT_CLOSING_BRACKETS`` + Put closing brackets on a separate line, indented, if the bracketed + expression can't fit in a single line. Applies to all kinds of brackets, + including function definitions and calls. For example: + + .. code-block:: python + + config = { + 'key1': 'value1', + 'key2': 'value2', + } # <--- this bracket is indented and on a separate line + + time_series = self.remote_client.query_entity_counters( + entity='dev3246.region1', + key='dns.query_latency_tcp', + transform=Transformation.AVERAGE(window=timedelta(seconds=60)), + start_ts=now()-timedelta(days=3), + end_ts=now(), + ) # <--- this bracket is indented and on a separate line + ``JOIN_MULTIPLE_LINES`` Join short lines into one line. E.g., single line ``if`` statements. -``SPACES_AROUND_POWER_OPERATOR`` - Set to ``True`` to prefer using spaces around ``**``. - ``NO_SPACES_AROUND_SELECTED_BINARY_OPERATORS`` Do not include spaces around selected binary operators. For example: @@ -461,36 +600,163 @@ Knobs 1 + 2 * 3 - 4 / 5 - will be formatted as follows when configured with ``*,/``: + will be formatted as follows when configured with ``*``, ``/``: .. code-block:: python 1 + 2*3 - 4/5 +``SPACES_AROUND_POWER_OPERATOR`` + Set to ``True`` to prefer using spaces around ``**``. + ``SPACES_AROUND_DEFAULT_OR_NAMED_ASSIGN`` Set to ``True`` to prefer spaces around the assignment operator for default or keyword arguments. +``SPACES_AROUND_DICT_DELIMITERS`` + Adds a space after the opening '{' and before the ending '}' dict delimiters. + + .. code-block:: python + + {1: 2} + + will be formatted as: + + .. code-block:: python + + { 1: 2 } + +``SPACES_AROUND_LIST_DELIMITERS`` + Adds a space after the opening '[' and before the ending ']' list delimiters. + + .. code-block:: python + + [1, 2] + + will be formatted as: + + .. code-block:: python + + [ 1, 2 ] + +``SPACES_AROUND_SUBSCRIPT_COLON`` + Use spaces around the subscript / slice operator. For example: + + .. code-block:: python + + my_list[1 : 10 : 2] + +``SPACES_AROUND_TUPLE_DELIMITERS`` + Adds a space after the opening '(' and before the ending ')' tuple delimiters. + + .. code-block:: python + + (1, 2, 3) + + will be formatted as: + + .. code-block:: python + + ( 1, 2, 3 ) + ``SPACES_BEFORE_COMMENT`` The number of spaces required before a trailing comment. + This can be a single value (representing the number of spaces + before each trailing comment) or list of of values (representing + alignment column values; trailing comments within a block will + be aligned to the first column value that is greater than the maximum + line length within the block). For example: + + With ``spaces_before_comment=5``: + + .. code-block:: python + + 1 + 1 # Adding values + + will be formatted as: + + .. code-block:: python + + 1 + 1 # Adding values <-- 5 spaces between the end of the statement and comment + + With ``spaces_before_comment=15, 20``: + + .. code-block:: python + + 1 + 1 # Adding values + two + two # More adding + + longer_statement # This is a longer statement + short # This is a shorter statement + + a_very_long_statement_that_extends_beyond_the_final_column # Comment + short # This is a shorter statement + + will be formatted as: + + .. code-block:: python + + 1 + 1 # Adding values <-- end of line comments in block aligned to col 15 + two + two # More adding + + longer_statement # This is a longer statement <-- end of line comments in block aligned to col 20 + short # This is a shorter statement + + a_very_long_statement_that_extends_beyond_the_final_column # Comment <-- the end of line comments are aligned based on the line length + short # This is a shorter statement ``SPACE_BETWEEN_ENDING_COMMA_AND_CLOSING_BRACKET`` Insert a space between the ending comma and closing bracket of a list, etc. +``SPACE_INSIDE_BRACKETS`` + Use spaces inside brackets, braces, and parentheses. For example: + + .. code-block:: python + + method_call( 1 ) + my_dict[ 3 ][ 1 ][ get_index( *args, **kwargs ) ] + my_set = { 1, 2, 3 } + ``SPLIT_ARGUMENTS_WHEN_COMMA_TERMINATED`` Split before arguments if the argument list is terminated by a comma. ``SPLIT_ALL_COMMA_SEPARATED_VALUES`` - If a comma separated list (dict, list, tuple, or function def) is on a - line that is too long, split such that all elements are on a single line. + If a comma separated list (``dict``, ``list``, ``tuple``, or function + ``def``) is on a line that is too long, split such that each element + is on a separate line. + +``SPLIT_ALL_TOP_LEVEL_COMMA_SEPARATED_VALUES`` + Variation on ``SPLIT_ALL_COMMA_SEPARATED_VALUES`` in which, if a + subexpression with a comma fits in its starting line, then the + subexpression is not split. This avoids splits like the one for + ``b`` in this code: + + .. code-block:: python + + abcdef( + aReallyLongThing: int, + b: [Int, + Int]) + + With the new knob this is split as: + + .. code-block:: python + + abcdef( + aReallyLongThing: int, + b: [Int, Int]) ``SPLIT_BEFORE_BITWISE_OPERATOR`` Set to ``True`` to prefer splitting before ``&``, ``|`` or ``^`` rather than after. +``SPLIT_BEFORE_ARITHMETIC_OPERATOR`` + Set to ``True`` to prefer splitting before ``+``, ``-``, ``*``, ``/``, ``//``, + or ``@`` rather than after. + ``SPLIT_BEFORE_CLOSING_BRACKET`` - Split before the closing bracket if a list or dict literal doesn't fit on - a single line. + Split before the closing bracket if a ``list`` or ``dict`` literal doesn't + fit on a single line. ``SPLIT_BEFORE_DICT_SET_GENERATOR`` Split before a dictionary or set generator (comp_for). For example, note @@ -503,6 +769,20 @@ Knobs for variable in bar if variable != 42 } +``SPLIT_BEFORE_DOT`` + Split before the ``.`` if we need to split a longer expression: + + .. code-block:: python + + foo = ('This is a really long string: {}, {}, {}, {}'.format(a, b, c, d)) + + would reformat to something like: + + .. code-block:: python + + foo = ('This is a really long string: {}, {}, {}, {}' + .format(a, b, c, d)) + ``SPLIT_BEFORE_EXPRESSION_AFTER_OPENING_PAREN`` Split after the opening paren which surrounds an expression if it doesn't fit on a single line. @@ -520,8 +800,8 @@ Knobs ``SPLIT_COMPLEX_COMPREHENSION`` For list comprehensions and generator expressions with multiple clauses - (e.g multiple "for" calls, "if" filter expressions) and which need to be - reflowed, split each clause onto its own line. For example: + (e.g multiple ``for`` calls, ``if`` filter expressions) and which need to + be reflowed, split each clause onto its own line. For example: .. code-block:: python @@ -545,6 +825,10 @@ Knobs ``SPLIT_PENALTY_AFTER_UNARY_OPERATOR`` The penalty for splitting the line after a unary operator. +``SPLIT_PENALTY_ARITHMETIC_OPERATOR`` + The penalty of splitting the line around the ``+``, ``-``, ``*``, ``/``, + ``//``, ``%``, and ``@`` operators. + ``SPLIT_PENALTY_BEFORE_IF_EXPR`` The penalty for splitting right before an ``if`` expression. @@ -559,7 +843,7 @@ Knobs The penalty for characters over the column limit. ``SPLIT_PENALTY_FOR_ADDED_LINE_SPLIT`` - The penalty incurred by adding a line split to the unwrapped line. The more + The penalty incurred by adding a line split to the logical line. The more line splits added the higher the penalty. ``SPLIT_PENALTY_IMPORT_NAMES`` @@ -645,6 +929,31 @@ Can I Use YAPF In My Program? Please do! YAPF was designed to be used as a library as well as a command line tool. This means that a tool or IDE plugin is free to use YAPF. +----------------------------------------- +I still get non Pep8 compliant code! Why? +----------------------------------------- + +YAPF tries very hard to be fully PEP 8 compliant. However, it is paramount +to not risk altering the semantics of your code. Thus, YAPF tries to be as +safe as possible and does not change the token stream +(e.g., by adding parentheses). +All these cases however, can be easily fixed manually. For instance, + +.. code-block:: python + + from my_package import my_function_1, my_function_2, my_function_3, my_function_4, my_function_5 + + FOO = my_variable_1 + my_variable_2 + my_variable_3 + my_variable_4 + my_variable_5 + my_variable_6 + my_variable_7 + my_variable_8 + +won't be split, but you can easily get it right by just adding parentheses: + +.. code-block:: python + + from my_package import (my_function_1, my_function_2, my_function_3, + my_function_4, my_function_5) + + FOO = (my_variable_1 + my_variable_2 + my_variable_3 + my_variable_4 + + my_variable_5 + my_variable_6 + my_variable_7 + my_variable_8) Gory Details ============ @@ -653,15 +962,15 @@ Gory Details Algorithm Design ---------------- -The main data structure in YAPF is the ``UnwrappedLine`` object. It holds a list -of ``FormatToken``\s, that we would want to place on a single line if there were -no column limit. An exception being a comment in the middle of an expression -statement will force the line to be formatted on more than one line. The -formatter works on one ``UnwrappedLine`` object at a time. +The main data structure in YAPF is the ``LogicalLine`` object. It holds a list +of ``FormatToken``\s, that we would want to place on a single line if there +were no column limit. An exception being a comment in the middle of an +expression statement will force the line to be formatted on more than one line. +The formatter works on one ``LogicalLine`` object at a time. -An ``UnwrappedLine`` typically won't affect the formatting of lines before or +An ``LogicalLine`` typically won't affect the formatting of lines before or after it. There is a part of the algorithm that may join two or more -``UnwrappedLine``\s into one line. For instance, an if-then statement with a +``LogicalLine``\s into one line. For instance, an if-then statement with a short body can be placed on a single line: .. code-block:: python |