aboutsummaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorPirama Arumuga Nainar <pirama@google.com>2015-07-01 12:25:36 -0700
committerPirama Arumuga Nainar <pirama@google.com>2015-07-01 12:25:36 -0700
commitb6d6993e6e6d3daf4d9876794254d20a134e37c2 (patch)
tree56c1af986f9588ac13a9f88597ce0738a574c6b9 /docs
parente428a2cde8a5a5aba99db38a7d7f6a0b7ee823d2 (diff)
downloadclang-b6d6993e6e6d3daf4d9876794254d20a134e37c2.tar.gz
Update aosp/master clang for rebase to r239765
Change-Id: I0393bcc952590a7226af8c4b58534a8ee5fd2d99
Diffstat (limited to 'docs')
-rw-r--r--docs/ClangFormatStyleOptions.rst15
-rw-r--r--docs/CrossCompilation.rst2
-rw-r--r--docs/DriverInternals.rst4
-rw-r--r--docs/InternalsManual.rst422
-rw-r--r--docs/LanguageExtensions.rst3
-rw-r--r--docs/LibASTMatchersTutorial.rst2
-rw-r--r--docs/MSVCCompatibility.rst25
-rw-r--r--docs/ReleaseNotes.rst18
-rw-r--r--docs/SafeStack.rst163
-rw-r--r--docs/SanitizerCoverage.rst355
-rw-r--r--docs/ThreadSafetyAnalysis.rst3
-rw-r--r--docs/UsersManual.rst97
-rw-r--r--docs/index.rst2
-rw-r--r--docs/tools/clang.pod14
14 files changed, 933 insertions, 192 deletions
diff --git a/docs/ClangFormatStyleOptions.rst b/docs/ClangFormatStyleOptions.rst
index ce6fae19c0..edec058b40 100644
--- a/docs/ClangFormatStyleOptions.rst
+++ b/docs/ClangFormatStyleOptions.rst
@@ -160,6 +160,17 @@ the configuration (without a prefix: ``Auto``).
argument2);
\endcode
+**AlignConsecutiveAssignments** (``bool``)
+ If ``true``, aligns consecutive assignments.
+
+ This will align the assignment operators of consecutive lines. This
+ will result in formattings like
+ \code
+ int aaaa = 12;
+ int b = 23;
+ int ccc = 23;
+ \endcode
+
**AlignEscapedNewlinesLeft** (``bool``)
If ``true``, aligns escaped newlines as far left as possible.
Otherwise puts them into the right-most column.
@@ -191,10 +202,10 @@ the configuration (without a prefix: ``Auto``).
* ``SFS_None`` (in configuration: ``None``)
Never merge functions into a single line.
- * ``SFS_Inline`` (in configuration: ``Inline``)
- Only merge functions defined inside a class.
* ``SFS_Empty`` (in configuration: ``Empty``)
Only merge empty functions.
+ * ``SFS_Inline`` (in configuration: ``Inline``)
+ Only merge functions defined inside a class. Implies "empty".
* ``SFS_All`` (in configuration: ``All``)
Merge all functions fitting on a single line.
diff --git a/docs/CrossCompilation.rst b/docs/CrossCompilation.rst
index fd42856ec3..d3a775b4fd 100644
--- a/docs/CrossCompilation.rst
+++ b/docs/CrossCompilation.rst
@@ -110,7 +110,7 @@ will be chosen, so you'll almost always have to change it via flags.
Typical flags include:
* ``-mcpu=<cpu-name>``, like x86-64, swift, cortex-a15
- * ``-fpu=<fpu-name>``, like SSE3, NEON, controlling the FP unit available
+ * ``-mfpu=<fpu-name>``, like SSE3, NEON, controlling the FP unit available
* ``-mfloat-abi=<fabi>``, like soft, hard, controlling which registers
to use for floating-point
diff --git a/docs/DriverInternals.rst b/docs/DriverInternals.rst
index c779555dae..4cd2e5dfe6 100644
--- a/docs/DriverInternals.rst
+++ b/docs/DriverInternals.rst
@@ -138,12 +138,12 @@ The driver functionality is conceptually divided into five stages:
this vector instead of storing its values directly.
The clang driver can dump the results of this stage using the
- ``-ccc-print-options`` flag (which must precede any actual command
+ ``-###`` flag (which must precede any actual command
line arguments). For example:
.. code-block:: console
- $ clang -ccc-print-options -Xarch_i386 -fomit-frame-pointer -Wa,-fast -Ifoo -I foo t.c
+ $ clang -### -Xarch_i386 -fomit-frame-pointer -Wa,-fast -Ifoo -I foo t.c
Option 0 - Name: "-Xarch_", Values: {"i386", "-fomit-frame-pointer"}
Option 1 - Name: "-Wa,", Values: {"-fast"}
Option 2 - Name: "-I", Values: {"foo"}
diff --git a/docs/InternalsManual.rst b/docs/InternalsManual.rst
index 7bb34b788a..7f2a8fafcc 100644
--- a/docs/InternalsManual.rst
+++ b/docs/InternalsManual.rst
@@ -1621,192 +1621,304 @@ How to change Clang
How to add an attribute
-----------------------
+Attributes are a form of metadata that can be attached to a program construct,
+allowing the programmer to pass semantic information along to the compiler for
+various uses. For example, attributes may be used to alter the code generation
+for a program construct, or to provide extra semantic information for static
+analysis. This document explains how to add a custom attribute to Clang.
+Documentation on existing attributes can be found `here
+<//clang.llvm.org/docs/AttributeReference.html>`_.
Attribute Basics
^^^^^^^^^^^^^^^^
-
-Attributes in clang come in two forms: parsed form, and semantic form. Both
-forms are represented via a tablegen definition of the attribute, specified in
-Attr.td.
+Attributes in Clang are handled in three stages: parsing into a parsed attribute
+representation, conversion from a parsed attribute into a semantic attribute,
+and then the semantic handling of the attribute.
+
+Parsing of the attribute is determined by the various syntactic forms attributes
+can take, such as GNU, C++11, and Microsoft style attributes, as well as other
+information provided by the table definition of the attribute. Ultimately, the
+parsed representation of an attribute object is an ``AttributeList`` object.
+These parsed attributes chain together as a list of parsed attributes attached
+to a declarator or declaration specifier. The parsing of attributes is handled
+automatically by Clang, except for attributes spelled as keywords. When
+implementing a keyword attribute, the parsing of the keyword and creation of the
+``AttributeList`` object must be done manually.
+
+Eventually, ``Sema::ProcessDeclAttributeList()`` is called with a ``Decl`` and
+an ``AttributeList``, at which point the parsed attribute can be transformed
+into a semantic attribute. The process by which a parsed attribute is converted
+into a semantic attribute depends on the attribute definition and semantic
+requirements of the attribute. The end result, however, is that the semantic
+attribute object is attached to the ``Decl`` object, and can be obtained by a
+call to ``Decl::getAttr<T>()``.
+
+The structure of the semantic attribute is also governed by the attribute
+definition given in Attr.td. This definition is used to automatically generate
+functionality used for the implementation of the attribute, such as a class
+derived from ``clang::Attr``, information for the parser to use, automated
+semantic checking for some attributes, etc.
``include/clang/Basic/Attr.td``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+The first step to adding a new attribute to Clang is to add its definition to
+`include/clang/Basic/Attr.td
+<http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/Attr.td?view=markup>`_.
+This tablegen definition must derive from the ``Attr`` (tablegen, not
+semantic) type, or one of its derivatives. Most attributes will derive from the
+``InheritableAttr`` type, which specifies that the attribute can be inherited by
+later redeclarations of the ``Decl`` it is associated with.
+``InheritableParamAttr`` is similar to ``InheritableAttr``, except that the
+attribute is written on a parameter instead of a declaration. If the attribute
+is intended to apply to a type instead of a declaration, such an attribute
+should derive from ``TypeAttr``, and will generally not be given an AST
+representation. (Note that this document does not cover the creation of type
+attributes.) An attribute that inherits from ``IgnoredAttr`` is parsed, but will
+generate an ignored attribute diagnostic when used, which may be useful when an
+attribute is supported by another vendor but not supported by clang.
+
+The definition will specify several key pieces of information, such as the
+semantic name of the attribute, the spellings the attribute supports, the
+arguments the attribute expects, and more. Most members of the ``Attr`` tablegen
+type do not require definitions in the derived definition as the default
+suffice. However, every attribute must specify at least a spelling list, a
+subject list, and a documentation list.
+
+Spellings
+~~~~~~~~~
+All attributes are required to specify a spelling list that denotes the ways in
+which the attribute can be spelled. For instance, a single semantic attribute
+may have a keyword spelling, as well as a C++11 spelling and a GNU spelling. An
+empty spelling list is also permissible and may be useful for attributes which
+are created implicitly. The following spellings are accepted:
+
+ ============ ================================================================
+ Spelling Description
+ ============ ================================================================
+ ``GNU`` Spelled with a GNU-style ``__attribute__((attr))`` syntax and
+ placement.
+ ``CXX11`` Spelled with a C++-style ``[[attr]]`` syntax. If the attribute
+ is meant to be used by Clang, it should set the namespace to
+ ``"clang"``.
+ ``Declspec`` Spelled with a Microsoft-style ``__declspec(attr)`` syntax.
+ ``Keyword`` The attribute is spelled as a keyword, and required custom
+ parsing.
+ ``GCC`` Specifies two spellings: the first is a GNU-style spelling, and
+ the second is a C++-style spelling with the ``gnu`` namespace.
+ Attributes should only specify this spelling for attributes
+ supported by GCC.
+ ``Pragma`` The attribute is spelled as a ``#pragma``, and requires custom
+ processing within the preprocessor. If the attribute is meant to
+ be used by Clang, it should set the namespace to ``"clang"``.
+ Note that this spelling is not used for declaration attributes.
+ ============ ================================================================
+
+Subjects
+~~~~~~~~
+Attributes appertain to one or more ``Decl`` subjects. If the attribute attempts
+to attach to a subject that is not in the subject list, a diagnostic is issued
+automatically. Whether the diagnostic is a warning or an error depends on how
+the attribute's ``SubjectList`` is defined, but the default behavior is to warn.
+The diagnostics displayed to the user are automatically determined based on the
+subjects in the list, but a custom diagnostic parameter can also be specified in
+the ``SubjectList``. The diagnostics generated for subject list violations are
+either ``diag::warn_attribute_wrong_decl_type`` or
+``diag::err_attribute_wrong_decl_type``, and the parameter enumeration is found
+in `include/clang/Sema/AttributeList.h
+<http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Sema/AttributeList.h?view=markup>`_
+If a previously unused Decl node is added to the ``SubjectList``, the logic used
+to automatically determine the diagnostic parameter in `utils/TableGen/ClangAttrEmitter.cpp
+<http://llvm.org/viewvc/llvm-project/cfe/trunk/utils/TableGen/ClangAttrEmitter.cpp?view=markup>`_
+may need to be updated.
+
+By default, all subjects in the SubjectList must either be a Decl node defined
+in ``DeclNodes.td``, or a statement node defined in ``StmtNodes.td``. However,
+more complex subjects can be created by creating a ``SubsetSubject`` object.
+Each such object has a base subject which it appertains to (which must be a
+Decl or Stmt node, and not a SubsetSubject node), and some custom code which is
+called when determining whether an attribute appertains to the subject. For
+instance, a ``NonBitField`` SubsetSubject appertains to a ``FieldDecl``, and
+tests whether the given FieldDecl is a bit field. When a SubsetSubject is
+specified in a SubjectList, a custom diagnostic parameter must also be provided.
-First, add your attribute to the `include/clang/Basic/Attr.td
-<http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/Attr.td?view=markup>`_
-file.
-
-Each attribute gets a ``def`` inheriting from ``Attr`` or one of its
-subclasses. ``InheritableAttr`` means that the attribute also applies to
-subsequent declarations of the same name. ``InheritableParamAttr`` is similar
-to ``InheritableAttr``, except that the attribute is written on a parameter
-instead of a declaration, type or statement. Attributes inheriting from
-``TypeAttr`` are pure type attributes which generally are not given a
-representation in the AST. Attributes inheriting from ``TargetSpecificAttr``
-are attributes specific to one or more target architectures. An attribute that
-inherits from ``IgnoredAttr`` is parsed, but will generate an ignored attribute
-diagnostic when used. The attribute type may be useful when an attribute is
-supported by another vendor, but not supported by clang.
-
-``Spellings`` lists the strings that can appear in ``__attribute__((here))`` or
-``[[here]]``. All such strings will be synonymous. Possible ``Spellings``
-are: ``GNU`` (for use with GNU-style __attribute__ spellings), ``Declspec``
-(for use with Microsoft Visual Studio-style __declspec spellings), ``CXX11`
-(for use with C++11-style [[foo]] and [[foo::bar]] spellings), and ``Keyword``
-(for use with attributes that are implemented as keywords, like C++11's
-``override`` or ``final``). If you want to allow the ``[[]]`` C++11 syntax, you
-have to define a list of ``Namespaces``, which will let users write
-``[[namespace::spelling]]``. Using the empty string for a namespace will allow
-users to write just the spelling with no "``::``". Attributes which g++-4.8
-or later accepts should also have a ``CXX11<"gnu", "spelling">`` spelling.
-
-``Subjects`` restricts what kinds of AST node to which this attribute can
-appertain (roughly, attach). The subjects are specified via a ``SubjectList``,
-which specify the list of subjects. Additionally, subject-related diagnostics
-can be specified to be warnings or errors, with the default being a warning.
-The diagnostics displayed to the user are automatically determined based on
-the subjects in the list, but a custom diagnostic parameter can also be
-specified in the ``SubjectList``. The diagnostics generated for subject list
-violations are either ``diag::warn_attribute_wrong_decl_type`` or
-``diag::err_attribute_wrong_decl_type``, and the parameter enumeration is
-found in `include/clang/Sema/AttributeList.h
-<http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Sema/AttributeList.h?view=markup>`_
-If you add new Decl nodes to the ``SubjectList``, you may need to update the
-logic used to automatically determine the diagnostic parameter in `utils/TableGen/ClangAttrEmitter.cpp
-<http://llvm.org/viewvc/llvm-project/cfe/trunk/utils/TableGen/ClangAttrEmitter.cpp?view=markup>`_.
-
-Diagnostic checking for attribute subject lists is automated except when
+Diagnostic checking for attribute subject lists is automated except when
``HasCustomParsing`` is set to ``1``.
-By default, all subjects in the SubjectList must either be a Decl node defined
-in ``DeclNodes.td``, or a statement node defined in ``StmtNodes.td``. However,
-more complex subjects can be created by creating a ``SubsetSubject`` object.
-Each such object has a base subject which it appertains to (which must be a
-Decl or Stmt node, and not a SubsetSubject node), and some custom code which is
-called when determining whether an attribute appertains to the subject. For
-instance, a ``NonBitField`` SubsetSubject appertains to a ``FieldDecl``, and
-tests whether the given FieldDecl is a bit field. When a SubsetSubject is
-specified in a SubjectList, a custom diagnostic parameter must also be provided.
-
-``Args`` names the arguments the attribute takes, in order. If ``Args`` is
+Documentation
+~~~~~~~~~~~~~
+All attributes must have some form of documentation associated with them.
+Documentation is table generated on the public web server by a server-side
+process that runs daily. Generally, the documentation for an attribute is a
+stand-alone definition in `include/clang/Basic/AttrDocs.td
+<http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/AttdDocs.td?view=markup>`_
+that is named after the attribute being documented.
+
+If the attribute is not for public consumption, or is an implicitly-created
+attribute that has no visible spelling, the documentation list can specify the
+``Undocumented`` object. Otherwise, the attribute should have its documentation
+added to AttrDocs.td.
+
+Documentation derives from the ``Documentation`` tablegen type. All derived
+types must specify a documentation category and the actual documentation itself.
+Additionally, it can specify a custom heading for the attribute, though a
+default heading will be chosen when possible.
+
+There are four predefined documentation categories: ``DocCatFunction`` for
+attributes that appertain to function-like subjects, ``DocCatVariable`` for
+attributes that appertain to variable-like subjects, ``DocCatType`` for type
+attributes, and ``DocCatStmt`` for statement attributes. A custom documentation
+category should be used for groups of attributes with similar functionality.
+Custom categories are good for providing overview information for the attributes
+grouped under it. For instance, the consumed annotation attributes define a
+custom category, ``DocCatConsumed``, that explains what consumed annotations are
+at a high level.
+
+Documentation content (whether it is for an attribute or a category) is written
+using reStructuredText (RST) syntax.
+
+After writing the documentation for the attribute, it should be locally tested
+to ensure that there are no issues generating the documentation on the server.
+Local testing requires a fresh build of clang-tblgen. To generate the attribute
+documentation, execute the following command::
+
+ clang-tblgen -gen-attr-docs -I /path/to/clang/include /path/to/clang/include/clang/Basic/Attr.td -o /path/to/clang/docs/AttributeReference.rst
+
+When testing locally, *do not* commit changes to ``AttributeReference.rst``.
+This file is generated by the server automatically, and any changes made to this
+file will be overwritten.
+
+Arguments
+~~~~~~~~~
+Attributes may optionally specify a list of arguments that can be passed to the
+attribute. Attribute arguments specify both the parsed form and the semantic
+form of the attribute. For example, if ``Args`` is
``[StringArgument<"Arg1">, IntArgument<"Arg2">]`` then
-``__attribute__((myattribute("Hello", 3)))`` will be a valid use. Attribute
-arguments specify both the parsed form and the semantic form of the attribute.
-The previous example shows an attribute which requires two attributes while
-parsing, and the Attr subclass' constructor for the attribute will require a
-string and integer argument.
-
-Diagnostic checking for argument counts is automated except when
-``HasCustomParsing`` is set to ``1``, or when the attribute uses an optional or
-variadic argument. Diagnostic checking for argument semantics is not automated.
-
-If the parsed form of the attribute is more complex, or differs from the
-semantic form, the ``HasCustomParsing`` bit can be set to ``1`` for the class,
-and the parsing code in `Parser::ParseGNUAttributeArgs
-<http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/ParseDecl.cpp?view=markup>`_
-can be updated for the special case. Note that this only applies to arguments
-with a GNU spelling -- attributes with a __declspec spelling currently ignore
+``__attribute__((myattribute("Hello", 3)))`` will be a valid use; it requires
+two arguments while parsing, and the Attr subclass' constructor for the
+semantic attribute will require a string and integer argument.
+
+All arguments have a name and a flag that specifies whether the argument is
+optional. The associated C++ type of the argument is determined by the argument
+definition type. If the existing argument types are insufficient, new types can
+be created, but it requires modifying `utils/TableGen/ClangAttrEmitter.cpp
+<http://llvm.org/viewvc/llvm-project/cfe/trunk/utils/TableGen/ClangAttrEmitter.cpp?view=markup>`_
+to properly support the type.
+
+Other Properties
+~~~~~~~~~~~~~~~~
+The ``Attr`` definition has other members which control the behavior of the
+attribute. Many of them are special-purpose and beyond the scope of this
+document, however a few deserve mention.
+
+If the parsed form of the attribute is more complex, or differs from the
+semantic form, the ``HasCustomParsing`` bit can be set to ``1`` for the class,
+and the parsing code in `Parser::ParseGNUAttributeArgs()
+<http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/ParseDecl.cpp?view=markup>`_
+can be updated for the special case. Note that this only applies to arguments
+with a GNU spelling -- attributes with a __declspec spelling currently ignore
this flag and are handled by ``Parser::ParseMicrosoftDeclSpec``.
-Custom accessors can be generated for an attribute based on the spelling list
-for that attribute. For instance, if an attribute has two different spellings:
-'Foo' and 'Bar', accessors can be created:
-``[Accessor<"isFoo", [GNU<"Foo">]>, Accessor<"isBar", [GNU<"Bar">]>]``
-These accessors will be generated on the semantic form of the attribute,
-accepting no arguments and returning a Boolean.
+Note that setting this member to 1 will opt out of common attribute semantic
+handling, requiring extra implementation efforts to ensure the attribute
+appertains to the appropriate subject, etc.
-Attributes which do not require an AST node should set the ``ASTNode`` field to
-``0`` to avoid polluting the AST. Note that anything inheriting from
-``TypeAttr`` or ``IgnoredAttr`` automatically do not generate an AST node. All
-other attributes generate an AST node by default. The AST node is the semantic
+If the attribute should not be propagated from from a template declaration to an
+instantiation of the template, set the ``Clone`` member to 0. By default, all
+attributes will be cloned to template instantiations.
+
+Attributes that do not require an AST node should set the ``ASTNode`` field to
+``0`` to avoid polluting the AST. Note that anything inheriting from
+``TypeAttr`` or ``IgnoredAttr`` automatically do not generate an AST node. All
+other attributes generate an AST node by default. The AST node is the semantic
representation of the attribute.
-Attributes which do not require custom semantic handling should set the
-``SemaHandler`` field to ``0``. Note that anything inheriting from
-``IgnoredAttr`` automatically do not get a semantic handler. All other
-attributes are assumed to use a semantic handler by default. Attributes
-without a semantic handler are not given a parsed attribute Kind enumeration.
-
-The ``LangOpts`` field can be used to specify a list of language options
-required by the attribute. For instance, all of the CUDA-specific attributes
-specify ``[CUDA]`` for the ``LangOpts`` field, and when the CUDA language
-option is not enabled, an "attribute ignored" warning diagnostic is emitted.
-Since language options are not table generated nodes, new language options must
-be created manually and should specify the spelling used by ``LangOptions`` class.
-
-Target-specific attribute sometimes share a spelling with other attributes in
-different targets. For instance, the ARM and MSP430 targets both have an
-attribute spelled ``GNU<"interrupt">``, but with different parsing and semantic
-requirements. To support this feature, an attribute inheriting from
-``TargetSpecificAttribute`` make specify a ``ParseKind`` field. This field
-should be the same value between all arguments sharing a spelling, and
-corresponds to the parsed attribute's Kind enumeration. This allows attributes
-to share a parsed attribute kind, but have distinct semantic attribute classes.
-For instance, ``AttributeList::AT_Interrupt`` is the shared parsed attribute
-kind, but ARMInterruptAttr and MSP430InterruptAttr are the semantic attributes
-generated.
-
-By default, when declarations are merging attributes, an attribute will not be
-duplicated. However, if an attribute can be duplicated during this merging
-stage, set ``DuplicatesAllowedWhileMerging`` to ``1``, and the attribute will
+The ``LangOpts`` field specifies a list of language options required by the
+attribute. For instance, all of the CUDA-specific attributes specify ``[CUDA]``
+for the ``LangOpts`` field, and when the CUDA language option is not enabled, an
+"attribute ignored" warning diagnostic is emitted. Since language options are
+not table generated nodes, new language options must be created manually and
+should specify the spelling used by ``LangOptions`` class.
+
+Custom accessors can be generated for an attribute based on the spelling list
+for that attribute. For instance, if an attribute has two different spellings:
+'Foo' and 'Bar', accessors can be created:
+``[Accessor<"isFoo", [GNU<"Foo">]>, Accessor<"isBar", [GNU<"Bar">]>]``
+These accessors will be generated on the semantic form of the attribute,
+accepting no arguments and returning a ``bool``.
+
+Attributes that do not require custom semantic handling should set the
+``SemaHandler`` field to ``0``. Note that anything inheriting from
+``IgnoredAttr`` automatically do not get a semantic handler. All other
+attributes are assumed to use a semantic handler by default. Attributes
+without a semantic handler are not given a parsed attribute ``Kind`` enumerator.
+
+Target-specific attributes may share a spelling with other attributes in
+different targets. For instance, the ARM and MSP430 targets both have an
+attribute spelled ``GNU<"interrupt">``, but with different parsing and semantic
+requirements. To support this feature, an attribute inheriting from
+``TargetSpecificAttribute`` may specify a ``ParseKind`` field. This field
+should be the same value between all arguments sharing a spelling, and
+corresponds to the parsed attribute's ``Kind`` enumerator. This allows
+attributes to share a parsed attribute kind, but have distinct semantic
+attribute classes. For instance, ``AttributeList::AT_Interrupt`` is the shared
+parsed attribute kind, but ARMInterruptAttr and MSP430InterruptAttr are the
+semantic attributes generated.
+
+By default, when declarations are merging attributes, an attribute will not be
+duplicated. However, if an attribute can be duplicated during this merging
+stage, set ``DuplicatesAllowedWhileMerging`` to ``1``, and the attribute will
be merged.
-By default, attribute arguments are parsed in an evaluated context. If the
-arguments for an attribute should be parsed in an unevaluated context (akin to
-the way the argument to a ``sizeof`` expression is parsed), you can set
+By default, attribute arguments are parsed in an evaluated context. If the
+arguments for an attribute should be parsed in an unevaluated context (akin to
+the way the argument to a ``sizeof`` expression is parsed), set
``ParseArgumentsAsUnevaluated`` to ``1``.
-If additional functionality is desired for the semantic form of the attribute,
-the ``AdditionalMembers`` field specifies code to be copied verbatim into the
-semantic attribute class object.
-
-All attributes must have one or more form of documentation, which is provided
-in the ``Documentation`` list. Generally, the documentation for an attribute
-is a stand-alone definition in `include/clang/Basic/AttrDocs.td
-<http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/AttdDocs.td?view=markup>`_
-that is named after the attribute being documented. Each documentation element
-is given a ``Category`` (variable, function, or type) and ``Content``. A single
-attribute may contain multiple documentation elements for distinct categories.
-For instance, an attribute which can appertain to both function and types (such
-as a calling convention attribute), should contain two documentation elements.
-The ``Content`` for an attribute uses reStructuredText (RST) syntax.
-
-If an attribute is used internally by the compiler, but is not written by users
-(such as attributes with an empty spelling list), it can use the
-``Undocumented`` documentation element.
+If additional functionality is desired for the semantic form of the attribute,
+the ``AdditionalMembers`` field specifies code to be copied verbatim into the
+semantic attribute class object, with ``public`` access.
Boilerplate
^^^^^^^^^^^
-
All semantic processing of declaration attributes happens in `lib/Sema/SemaDeclAttr.cpp
-<http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/SemaDeclAttr.cpp?view=markup>`_,
-and generally starts in the ``ProcessDeclAttribute`` function. If your
-attribute is a "simple" attribute -- meaning that it requires no custom
-semantic processing aside from what is automatically provided for you, you can
-add a call to ``handleSimpleAttribute<YourAttr>(S, D, Attr);`` to the switch
-statement. Otherwise, write a new ``handleYourAttr()`` function, and add that
-to the switch statement.
-
-If your attribute causes extra warnings to fire, define a ``DiagGroup`` in
+<http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/SemaDeclAttr.cpp?view=markup>`_,
+and generally starts in the ``ProcessDeclAttribute()`` function. If the
+attribute is a "simple" attribute -- meaning that it requires no custom semantic
+processing aside from what is automatically provided, add a call to
+``handleSimpleAttribute<YourAttr>(S, D, Attr);`` to the switch statement.
+Otherwise, write a new ``handleYourAttr()`` function, and add that to the switch
+statement. Please do not implement handling logic directly in the ``case`` for
+the attribute.
+
+Unless otherwise specified by the attribute definition, common semantic checking
+of the parsed attribute is handled automatically. This includes diagnosing
+parsed attributes that do not appertain to the given ``Decl``, ensuring the
+correct minimum number of arguments are passed, etc.
+
+If the attribute adds additional warnings, define a ``DiagGroup`` in
`include/clang/Basic/DiagnosticGroups.td
<http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/DiagnosticGroups.td?view=markup>`_
-named after the attribute's ``Spelling`` with "_"s replaced by "-"s. If you're
-only defining one diagnostic, you can skip ``DiagnosticGroups.td`` and use
-``InGroup<DiagGroup<"your-attribute">>`` directly in `DiagnosticSemaKinds.td
+named after the attribute's ``Spelling`` with "_"s replaced by "-"s. If there
+is only a single diagnostic, it is permissible to use ``InGroup<DiagGroup<"your-attribute">>``
+directly in `DiagnosticSemaKinds.td
<http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td?view=markup>`_
All semantic diagnostics generated for your attribute, including automatically-
-generated ones (such as subjects and argument counts), should have a
+generated ones (such as subjects and argument counts), should have a
corresponding test case.
-The meat of your attribute
-^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Find an appropriate place in Clang to do whatever your attribute needs to do.
-Check for the attribute's presence using ``Decl::getAttr<YourAttr>()``.
-
-Update the :doc:`LanguageExtensions` document to describe your new attribute.
+Semantic handling
+^^^^^^^^^^^^^^^^^
+Most attributes are implemented to have some effect on the compiler. For
+instance, to modify the way code is generated, or to add extra semantic checks
+for an analysis pass, etc. Having added the attribute definition and conversion
+to the semantic representation for the attribute, what remains is to implement
+the custom logic requiring use of the attribute.
+
+The ``clang::Decl`` object can be queried for the presence or absence of an
+attribute using ``hasAttr<T>()``. To obtain a pointer to the semantic
+representation of the attribute, ``getAttr<T>`` may be used.
How to add an expression or statement
-------------------------------------
diff --git a/docs/LanguageExtensions.rst b/docs/LanguageExtensions.rst
index bd5992e3d6..0b4775ba6a 100644
--- a/docs/LanguageExtensions.rst
+++ b/docs/LanguageExtensions.rst
@@ -1845,6 +1845,9 @@ with :doc:`ThreadSanitizer`.
Use ``__has_feature(memory_sanitizer)`` to check if the code is being built
with :doc:`MemorySanitizer`.
+Use ``__has_feature(safe_stack)`` to check if the code is being built
+with :doc:`SafeStack`.
+
Extensions for selectively disabling optimization
=================================================
diff --git a/docs/LibASTMatchersTutorial.rst b/docs/LibASTMatchersTutorial.rst
index 1e88ec203c..fe36511a0c 100644
--- a/docs/LibASTMatchersTutorial.rst
+++ b/docs/LibASTMatchersTutorial.rst
@@ -169,7 +169,7 @@ You should now be able to run the syntax checker, which is located in
.. code-block:: console
- cat "int main() { return 0; }" > test.cpp
+ echo "int main() { return 0; }" > test.cpp
bin/loop-convert test.cpp --
Note the two dashes after we specify the source file. The additional
diff --git a/docs/MSVCCompatibility.rst b/docs/MSVCCompatibility.rst
index 1dba9e865b..3794813db8 100644
--- a/docs/MSVCCompatibility.rst
+++ b/docs/MSVCCompatibility.rst
@@ -84,14 +84,23 @@ The status of major ABI-impacting C++ features:
* RTTI: :good:`Complete`. Generation of RTTI data structures has been
finished, along with support for the ``/GR`` flag.
-* Exceptions and SEH: :partial:`Minimal`. Clang can parse both constructs, but
- does not know how to emit compatible handlers. Clang can throw and rethrow
- C++ exceptions.
-
-* Thread-safe initialization of local statics: :none:`Unstarted`. We are ABI
- compatible with MSVC 2013, which does not support thread-safe local statics.
- MSVC "14" changed the ABI to make initialization of local statics thread safe,
- and we have not yet implemented this.
+* Exceptions and SEH: :partial:`Partial`.
+ C++ exceptions (``try`` / ``catch`` / ``throw``) and
+ structured exceptions (``__try`` / ``__except`` / ``__finally``) mostly
+ work on x64. 32-bit exception handling support is being worked on. LLVM does
+ not model asynchronous exceptions, so it is currently impossible to catch an
+ asynchronous exception generated in the same frame as the catching ``__try``.
+ C++ exception specifications are ignored, but this is `consistent with Visual
+ C++`_.
+
+.. _consistent with Visual C++:
+ https://msdn.microsoft.com/en-us/library/wfa0edys.aspx
+
+* Thread-safe initialization of local statics: :good:`Complete`. MSVC 2015
+ added support for thread-safe initialization of such variables by taking an
+ ABI break.
+ We are ABI compatible with both the MSVC 2013 and 2015 ABI for static local
+ variables.
* Lambdas: :good:`Mostly complete`. Clang is compatible with Microsoft's
implementation of lambdas except for providing overloads for conversion to
diff --git a/docs/ReleaseNotes.rst b/docs/ReleaseNotes.rst
index 90655d4d6f..c6dc6aa8ba 100644
--- a/docs/ReleaseNotes.rst
+++ b/docs/ReleaseNotes.rst
@@ -11,8 +11,8 @@ Written by the `LLVM Team <http://llvm.org/>`_
.. warning::
These are in-progress notes for the upcoming Clang 3.7 release. You may
- prefer the `Clang 3.5 Release Notes
- <http://llvm.org/releases/3.5.0/tools/clang/docs/ReleaseNotes.html>`_.
+ prefer the `Clang 3.6 Release Notes
+ <http://llvm.org/releases/3.6.0/tools/clang/docs/ReleaseNotes.html>`_.
Introduction
============
@@ -47,8 +47,10 @@ sections with improvements to Clang's support for those languages.
Major New Features
------------------
-- Feature ...
-
+- Use of the ``__declspec`` language extension for declaration attributes now
+ requires passing the -fms-extensions or -fborland compiler flag. This language
+ extension is also enabled when compiling CUDA code, but its use should be
+ viewed as an implementation detail that is subject to change.
Improvements to Clang's diagnostics
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -118,7 +120,13 @@ These are major API changes that have happened since the 3.6 release of
Clang. If upgrading an external codebase that uses Clang as a library,
this section should help get you past the largest hurdles of upgrading.
-...
+- Some of the `PPCallbacks` interface now deals in `MacroDefinition`
+ objects instead of `MacroDirective` objects. This allows preserving
+ full information on macros imported from modules.
+
+- `clang-c/Index.h` no longer `#include`\s `clang-c/Documentation.h`.
+ You now need to explicitly `#include "clang-c/Documentation.h"` if
+ you use the libclang documentation API.
libclang
--------
diff --git a/docs/SafeStack.rst b/docs/SafeStack.rst
new file mode 100644
index 0000000000..5115d95995
--- /dev/null
+++ b/docs/SafeStack.rst
@@ -0,0 +1,163 @@
+=========
+SafeStack
+=========
+
+.. contents::
+ :local:
+
+Introduction
+============
+
+SafeStack is an instrumentation pass that protects programs against attacks
+based on stack buffer overflows, without introducing any measurable performance
+overhead. It works by separating the program stack into two distinct regions:
+the safe stack and the unsafe stack. The safe stack stores return addresses,
+register spills, and local variables that are always accessed in a safe way,
+while the unsafe stack stores everything else. This separation ensures that
+buffer overflows on the unsafe stack cannot be used to overwrite anything
+on the safe stack, which includes return addresses.
+
+Performance
+-----------
+
+The performance overhead of the SafeStack instrumentation is less than 0.1% on
+average across a variety of benchmarks (see the `Code-Pointer Integrity
+<http://dslab.epfl.ch/pubs/cpi.pdf>`_ paper for details). This is mainly
+because most small functions do not have any variables that require the unsafe
+stack and, hence, do not need unsafe stack frames to be created. The cost of
+creating unsafe stack frames for large functions is amortized by the cost of
+executing the function.
+
+In some cases, SafeStack actually improves the performance. Objects that end up
+being moved to the unsafe stack are usually large arrays or variables that are
+used through multiple stack frames. Moving such objects away from the safe
+stack increases the locality of frequently accessed values on the stack, such
+as register spills, return addresses, and small local variables.
+
+Limitations
+-----------
+
+SafeStack has not been subjected to a comprehensive security review, and there
+exist known weaknesses, including but not limited to the following.
+
+In its current state, the separation of local variables provides protection
+against stack buffer overflows, but the safe stack itself is not protected
+from being corrupted through a pointer dereference. The Code-Pointer
+Integrity paper describes two ways in which we may protect the safe stack:
+hardware segmentation on the 32-bit x86 architecture or information hiding
+on other architectures.
+
+Even with information hiding, the safe stack would merely be hidden
+from attackers by being somewhere in the address space. Depending on the
+application, the address could be predictable even on 64-bit address spaces
+because not all the bits are addressable, multiple threads each have their
+stack, the application could leak the safe stack address to memory via
+``__builtin_frame_address``, bugs in the low-level runtime support etc.
+Safe stack leaks could be mitigated by writing and deploying a static binary
+analysis or a dynamic binary instrumentation based tool to find leaks.
+
+This approach doesn't prevent an attacker from "imbalancing" the safe
+stack by say having just one call, and doing two rets (thereby returning
+to an address that wasn't meant as a return address). This can be at least
+partially mitigated by deploying SafeStack alongside a forward control-flow
+integrity mechanism to ensure that calls are made using the correct calling
+convention. Clang does not currently implement a comprehensive forward
+control-flow integrity protection scheme; there exists one that protects
+:doc:`virtual calls <ControlFlowIntegrity>` but not non-virtual indirect calls.
+
+Compatibility
+-------------
+
+Most programs, static libraries, or individual files can be compiled
+with SafeStack as is. SafeStack requires basic runtime support, which, on most
+platforms, is implemented as a compiler-rt library that is automatically linked
+in when the program is compiled with SafeStack.
+
+Linking a DSO with SafeStack is not currently supported.
+
+Known compatibility limitations
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Certain code that relies on low-level stack manipulations requires adaption to
+work with SafeStack. One example is mark-and-sweep garbage collection
+implementations for C/C++ (e.g., Oilpan in chromium/blink), which must be
+changed to look for the live pointers on both safe and unsafe stacks.
+
+SafeStack supports linking together modules that are compiled with and without
+SafeStack, both statically and dynamically. One corner case that is not
+supported is using ``dlopen()`` to load a dynamic library that uses SafeStack into
+a program that is not compiled with SafeStack but uses threads.
+
+Signal handlers that use ``sigaltstack()`` must not use the unsafe stack (see
+``__attribute__((no_sanitize("safe-stack")))`` below).
+
+Programs that use APIs from ``ucontext.h`` are not supported yet.
+
+Usage
+=====
+
+To enable SafeStack, just pass ``-fsanitize=safe-stack`` flag to both compile and link
+command lines.
+
+Supported Platforms
+-------------------
+
+SafeStack was tested on Linux, FreeBSD and MacOSX.
+
+Low-level API
+-------------
+
+``__has_feature(safe_stack)``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In some rare cases one may need to execute different code depending on
+whether SafeStack is enabled. The macro ``__has_feature(safe_stack)`` can
+be used for this purpose.
+
+.. code-block:: c
+
+ #if __has_feature(safe_stack)
+ // code that builds only under SafeStack
+ #endif
+
+``__attribute__((no_sanitize("safe-stack")))``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Use ``__attribute__((no_sanitize("safe-stack")))`` on a function declaration
+to specify that the safe stack instrumentation should not be applied to that
+function, even if enabled globally (see ``-fsanitize=safe-stack`` flag). This
+attribute may be required for functions that make assumptions about the
+exact layout of their stack frames.
+
+Care should be taken when using this attribute. The return address is not
+protected against stack buffer overflows, and it is easier to leak the
+address of the safe stack to memory by taking the address of a local variable.
+
+
+``__builtin___get_unsafe_stack_ptr()``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This builtin function returns current unsafe stack pointer of the current
+thread.
+
+``__builtin___get_unsafe_stack_start()``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This builtin function returns a pointer to the start of the unsafe stack of the
+current thread.
+
+Design
+======
+
+Please refer to
+`http://dslab.epfl.ch/proj/cpi/ <http://dslab.epfl.ch/proj/cpi/>`_ for more
+information about the design of the SafeStack and its related technologies.
+
+
+Publications
+------------
+
+`Code-Pointer Integrity <http://dslab.epfl.ch/pubs/cpi.pdf>`_.
+Volodymyr Kuznetsov, Laszlo Szekeres, Mathias Payer, George Candea, R. Sekar, Dawn Song.
+USENIX Symposium on Operating Systems Design and Implementation
+(`OSDI <https://www.usenix.org/conference/osdi14>`_), Broomfield, CO, October 2014
diff --git a/docs/SanitizerCoverage.rst b/docs/SanitizerCoverage.rst
new file mode 100644
index 0000000000..65af6ffbf0
--- /dev/null
+++ b/docs/SanitizerCoverage.rst
@@ -0,0 +1,355 @@
+=================
+SanitizerCoverage
+=================
+
+.. contents::
+ :local:
+
+Introduction
+============
+
+Sanitizer tools have a very simple code coverage tool built in. It allows to
+get function-level, basic-block-level, and edge-level coverage at a very low
+cost.
+
+How to build and run
+====================
+
+SanitizerCoverage can be used with :doc:`AddressSanitizer`,
+:doc:`LeakSanitizer`, :doc:`MemorySanitizer`, and UndefinedBehaviorSanitizer.
+In addition to ``-fsanitize=``, pass one of the following compile-time flags:
+
+* ``-fsanitize-coverage=func`` for function-level coverage (very fast).
+* ``-fsanitize-coverage=bb`` for basic-block-level coverage (may add up to 30%
+ **extra** slowdown).
+* ``-fsanitize-coverage=edge`` for edge-level coverage (up to 40% slowdown).
+
+You may also specify ``-fsanitize-coverage=indirect-calls`` for
+additional `caller-callee coverage`_.
+
+At run time, pass ``coverage=1`` in ``ASAN_OPTIONS``, ``LSAN_OPTIONS``,
+``MSAN_OPTIONS`` or ``UBSAN_OPTIONS``, as appropriate.
+
+To get `Coverage counters`_, add ``-fsanitize-coverage=8bit-counters``
+to one of the above compile-time flags. At runtime, use
+``*SAN_OPTIONS=coverage=1:coverage_counters=1``.
+
+Example:
+
+.. code-block:: console
+
+ % cat -n cov.cc
+ 1 #include <stdio.h>
+ 2 __attribute__((noinline))
+ 3 void foo() { printf("foo\n"); }
+ 4
+ 5 int main(int argc, char **argv) {
+ 6 if (argc == 2)
+ 7 foo();
+ 8 printf("main\n");
+ 9 }
+ % clang++ -g cov.cc -fsanitize=address -fsanitize-coverage=func
+ % ASAN_OPTIONS=coverage=1 ./a.out; ls -l *sancov
+ main
+ -rw-r----- 1 kcc eng 4 Nov 27 12:21 a.out.22673.sancov
+ % ASAN_OPTIONS=coverage=1 ./a.out foo ; ls -l *sancov
+ foo
+ main
+ -rw-r----- 1 kcc eng 4 Nov 27 12:21 a.out.22673.sancov
+ -rw-r----- 1 kcc eng 8 Nov 27 12:21 a.out.22679.sancov
+
+Every time you run an executable instrumented with SanitizerCoverage
+one ``*.sancov`` file is created during the process shutdown.
+If the executable is dynamically linked against instrumented DSOs,
+one ``*.sancov`` file will be also created for every DSO.
+
+Postprocessing
+==============
+
+The format of ``*.sancov`` files is very simple: the first 8 bytes is the magic,
+one of ``0xC0BFFFFFFFFFFF64`` and ``0xC0BFFFFFFFFFFF32``. The last byte of the
+magic defines the size of the following offsets. The rest of the data is the
+offsets in the corresponding binary/DSO that were executed during the run.
+
+A simple script
+``$LLVM/projects/compiler-rt/lib/sanitizer_common/scripts/sancov.py`` is
+provided to dump these offsets.
+
+.. code-block:: console
+
+ % sancov.py print a.out.22679.sancov a.out.22673.sancov
+ sancov.py: read 2 PCs from a.out.22679.sancov
+ sancov.py: read 1 PCs from a.out.22673.sancov
+ sancov.py: 2 files merged; 2 PCs total
+ 0x465250
+ 0x4652a0
+
+You can then filter the output of ``sancov.py`` through ``addr2line --exe
+ObjectFile`` or ``llvm-symbolizer --obj ObjectFile`` to get file names and line
+numbers:
+
+.. code-block:: console
+
+ % sancov.py print a.out.22679.sancov a.out.22673.sancov 2> /dev/null | llvm-symbolizer --obj a.out
+ cov.cc:3
+ cov.cc:5
+
+How good is the coverage?
+=========================
+
+It is possible to find out which PCs are not covered, by subtracting the covered
+set from the set of all instrumented PCs. The latter can be obtained by listing
+all callsites of ``__sanitizer_cov()`` in the binary. On Linux, ``sancov.py``
+can do this for you. Just supply the path to binary and a list of covered PCs:
+
+.. code-block:: console
+
+ % sancov.py print a.out.12345.sancov > covered.txt
+ sancov.py: read 2 64-bit PCs from a.out.12345.sancov
+ sancov.py: 1 file merged; 2 PCs total
+ % sancov.py missing a.out < covered.txt
+ sancov.py: found 3 instrumented PCs in a.out
+ sancov.py: read 2 PCs from stdin
+ sancov.py: 1 PCs missing from coverage
+ 0x4cc61c
+
+Edge coverage
+=============
+
+Consider this code:
+
+.. code-block:: c++
+
+ void foo(int *a) {
+ if (a)
+ *a = 0;
+ }
+
+It contains 3 basic blocks, let's name them A, B, C:
+
+.. code-block:: none
+
+ A
+ |\
+ | \
+ | B
+ | /
+ |/
+ C
+
+If blocks A, B, and C are all covered we know for certain that the edges A=>B
+and B=>C were executed, but we still don't know if the edge A=>C was executed.
+Such edges of control flow graph are called
+`critical <http://en.wikipedia.org/wiki/Control_flow_graph#Special_edges>`_. The
+edge-level coverage (``-fsanitize-coverage=edge``) simply splits all critical
+edges by introducing new dummy blocks and then instruments those blocks:
+
+.. code-block:: none
+
+ A
+ |\
+ | \
+ D B
+ | /
+ |/
+ C
+
+Bitset
+======
+
+When ``coverage_bitset=1`` run-time flag is given, the coverage will also be
+dumped as a bitset (text file with 1 for blocks that have been executed and 0
+for blocks that were not).
+
+.. code-block:: console
+
+ % clang++ -fsanitize=address -fsanitize-coverage=edge cov.cc
+ % ASAN_OPTIONS="coverage=1:coverage_bitset=1" ./a.out
+ main
+ % ASAN_OPTIONS="coverage=1:coverage_bitset=1" ./a.out 1
+ foo
+ main
+ % head *bitset*
+ ==> a.out.38214.bitset-sancov <==
+ 01101
+ ==> a.out.6128.bitset-sancov <==
+ 11011%
+
+For a given executable the length of the bitset is always the same (well,
+unless dlopen/dlclose come into play), so the bitset coverage can be
+easily used for bitset-based corpus distillation.
+
+Caller-callee coverage
+======================
+
+(Experimental!)
+Every indirect function call is instrumented with a run-time function call that
+captures caller and callee. At the shutdown time the process dumps a separate
+file called ``caller-callee.PID.sancov`` which contains caller/callee pairs as
+pairs of lines (odd lines are callers, even lines are callees)
+
+.. code-block:: console
+
+ a.out 0x4a2e0c
+ a.out 0x4a6510
+ a.out 0x4a2e0c
+ a.out 0x4a87f0
+
+Current limitations:
+
+* Only the first 14 callees for every caller are recorded, the rest are silently
+ ignored.
+* The output format is not very compact since caller and callee may reside in
+ different modules and we need to spell out the module names.
+* The routine that dumps the output is not optimized for speed
+* Only Linux x86_64 is tested so far.
+* Sandboxes are not supported.
+
+Coverage counters
+=================
+
+This experimental feature is inspired by
+`AFL <http://lcamtuf.coredump.cx/afl/technical_details.txt>`_'s coverage
+instrumentation. With additional compile-time and run-time flags you can get
+more sensitive coverage information. In addition to boolean values assigned to
+every basic block (edge) the instrumentation will collect imprecise counters.
+On exit, every counter will be mapped to a 8-bit bitset representing counter
+ranges: ``1, 2, 3, 4-7, 8-15, 16-31, 32-127, 128+`` and those 8-bit bitsets will
+be dumped to disk.
+
+.. code-block:: console
+
+ % clang++ -g cov.cc -fsanitize=address -fsanitize-coverage=edge,8bit-counters
+ % ASAN_OPTIONS="coverage=1:coverage_counters=1" ./a.out
+ % ls -l *counters-sancov
+ ... a.out.17110.counters-sancov
+ % xxd *counters-sancov
+ 0000000: 0001 0100 01
+
+These counters may also be used for in-process coverage-guided fuzzers. See
+``include/sanitizer/coverage_interface.h``:
+
+.. code-block:: c++
+
+ // The coverage instrumentation may optionally provide imprecise counters.
+ // Rather than exposing the counter values to the user we instead map
+ // the counters to a bitset.
+ // Every counter is associated with 8 bits in the bitset.
+ // We define 8 value ranges: 1, 2, 3, 4-7, 8-15, 16-31, 32-127, 128+
+ // The i-th bit is set to 1 if the counter value is in the i-th range.
+ // This counter-based coverage implementation is *not* thread-safe.
+
+ // Returns the number of registered coverage counters.
+ uintptr_t __sanitizer_get_number_of_counters();
+ // Updates the counter 'bitset', clears the counters and returns the number of
+ // new bits in 'bitset'.
+ // If 'bitset' is nullptr, only clears the counters.
+ // Otherwise 'bitset' should be at least
+ // __sanitizer_get_number_of_counters bytes long and 8-aligned.
+ uintptr_t
+ __sanitizer_update_counter_bitset_and_clear_counters(uint8_t *bitset);
+
+Output directory
+================
+
+By default, .sancov files are created in the current working directory.
+This can be changed with ``ASAN_OPTIONS=coverage_dir=/path``:
+
+.. code-block:: console
+
+ % ASAN_OPTIONS="coverage=1:coverage_dir=/tmp/cov" ./a.out foo
+ % ls -l /tmp/cov/*sancov
+ -rw-r----- 1 kcc eng 4 Nov 27 12:21 a.out.22673.sancov
+ -rw-r----- 1 kcc eng 8 Nov 27 12:21 a.out.22679.sancov
+
+Sudden death
+============
+
+Normally, coverage data is collected in memory and saved to disk when the
+program exits (with an ``atexit()`` handler), when a SIGSEGV is caught, or when
+``__sanitizer_cov_dump()`` is called.
+
+If the program ends with a signal that ASan does not handle (or can not handle
+at all, like SIGKILL), coverage data will be lost. This is a big problem on
+Android, where SIGKILL is a normal way of evicting applications from memory.
+
+With ``ASAN_OPTIONS=coverage=1:coverage_direct=1`` coverage data is written to a
+memory-mapped file as soon as it collected.
+
+.. code-block:: console
+
+ % ASAN_OPTIONS="coverage=1:coverage_direct=1" ./a.out
+ main
+ % ls
+ 7036.sancov.map 7036.sancov.raw a.out
+ % sancov.py rawunpack 7036.sancov.raw
+ sancov.py: reading map 7036.sancov.map
+ sancov.py: unpacking 7036.sancov.raw
+ writing 1 PCs to a.out.7036.sancov
+ % sancov.py print a.out.7036.sancov
+ sancov.py: read 1 PCs from a.out.7036.sancov
+ sancov.py: 1 files merged; 1 PCs total
+ 0x4b2bae
+
+Note that on 64-bit platforms, this method writes 2x more data than the default,
+because it stores full PC values instead of 32-bit offsets.
+
+In-process fuzzing
+==================
+
+Coverage data could be useful for fuzzers and sometimes it is preferable to run
+a fuzzer in the same process as the code being fuzzed (in-process fuzzer).
+
+You can use ``__sanitizer_get_total_unique_coverage()`` from
+``<sanitizer/coverage_interface.h>`` which returns the number of currently
+covered entities in the program. This will tell the fuzzer if the coverage has
+increased after testing every new input.
+
+If a fuzzer finds a bug in the ASan run, you will need to save the reproducer
+before exiting the process. Use ``__asan_set_death_callback`` from
+``<sanitizer/asan_interface.h>`` to do that.
+
+An example of such fuzzer can be found in `the LLVM tree
+<http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/README.txt?view=markup>`_.
+
+Performance
+===========
+
+This coverage implementation is **fast**. With function-level coverage
+(``-fsanitize-coverage=func``) the overhead is not measurable. With
+basic-block-level coverage (``-fsanitize-coverage=bb``) the overhead varies
+between 0 and 25%.
+
+============== ========= ========= ========= ========= ========= =========
+ benchmark cov0 cov1 diff 0-1 cov2 diff 0-2 diff 1-2
+============== ========= ========= ========= ========= ========= =========
+ 400.perlbench 1296.00 1307.00 1.01 1465.00 1.13 1.12
+ 401.bzip2 858.00 854.00 1.00 1010.00 1.18 1.18
+ 403.gcc 613.00 617.00 1.01 683.00 1.11 1.11
+ 429.mcf 605.00 582.00 0.96 610.00 1.01 1.05
+ 445.gobmk 896.00 880.00 0.98 1050.00 1.17 1.19
+ 456.hmmer 892.00 892.00 1.00 918.00 1.03 1.03
+ 458.sjeng 995.00 1009.00 1.01 1217.00 1.22 1.21
+462.libquantum 497.00 492.00 0.99 534.00 1.07 1.09
+ 464.h264ref 1461.00 1467.00 1.00 1543.00 1.06 1.05
+ 471.omnetpp 575.00 590.00 1.03 660.00 1.15 1.12
+ 473.astar 658.00 652.00 0.99 715.00 1.09 1.10
+ 483.xalancbmk 471.00 491.00 1.04 582.00 1.24 1.19
+ 433.milc 616.00 627.00 1.02 627.00 1.02 1.00
+ 444.namd 602.00 601.00 1.00 654.00 1.09 1.09
+ 447.dealII 630.00 634.00 1.01 653.00 1.04 1.03
+ 450.soplex 365.00 368.00 1.01 395.00 1.08 1.07
+ 453.povray 427.00 434.00 1.02 495.00 1.16 1.14
+ 470.lbm 357.00 375.00 1.05 370.00 1.04 0.99
+ 482.sphinx3 927.00 928.00 1.00 1000.00 1.08 1.08
+============== ========= ========= ========= ========= ========= =========
+
+Why another coverage?
+=====================
+
+Why did we implement yet another code coverage?
+ * We needed something that is lightning fast, plays well with
+ AddressSanitizer, and does not significantly increase the binary size.
+ * Traditional coverage implementations based in global counters
+ `suffer from contention on counters
+ <https://groups.google.com/forum/#!topic/llvm-dev/cDqYgnxNEhY>`_.
diff --git a/docs/ThreadSafetyAnalysis.rst b/docs/ThreadSafetyAnalysis.rst
index 0a1b8049e4..19ec2352ff 100644
--- a/docs/ThreadSafetyAnalysis.rst
+++ b/docs/ThreadSafetyAnalysis.rst
@@ -857,6 +857,9 @@ implementation.
// Assert that is mutex is currently held for read operations.
void AssertReaderHeld() ASSERT_SHARED_CAPABILITY(this);
+
+ // For negative capabilities.
+ const Mutex& operator!() const { return *this; }
};
diff --git a/docs/UsersManual.rst b/docs/UsersManual.rst
index bf8ce78036..fdc52f8fb1 100644
--- a/docs/UsersManual.rst
+++ b/docs/UsersManual.rst
@@ -589,6 +589,25 @@ Current limitations
translated from debug annotations. That translation can be lossy,
which results in some remarks having no location information.
+Other Options
+-------------
+Clang options that that don't fit neatly into other categories.
+
+.. option:: -MV
+
+ When emitting a dependency file, use formatting conventions appropriate
+ for NMake or Jom. Ignored unless another option causes Clang to emit a
+ dependency file.
+
+When Clang emits a dependency file (e.g., you supplied the -M option)
+most filenames can be written to the file without any special formatting.
+Different Make tools will treat different sets of characters as "special"
+and use different conventions for telling the Make tool that the character
+is actually part of the filename. Normally Clang uses backslash to "escape"
+a special character, which is the convention used by GNU Make. The -MV
+option tells Clang to put double-quotes around the entire filename, which
+is the convention used by NMake and Jom.
+
Language and Target-Independent Features
========================================
@@ -911,6 +930,8 @@ number of cases where the compilation environment is tightly controlled
and the precompiled header cannot be generated after headers have been
installed.
+.. _controlling-code-generation:
+
Controlling Code Generation
---------------------------
@@ -959,6 +980,8 @@ are listed below.
flow analysis.
- ``-fsanitize=cfi``: :doc:`control flow integrity <ControlFlowIntegrity>`
checks. Implies ``-flto``.
+ - ``-fsanitize=safe-stack``: :doc:`safe stack <SafeStack>`
+ protection against stack-based memory corruption errors.
The following more fine-grained checks are also available:
@@ -1078,6 +1101,11 @@ are listed below.
sanitizers (e.g. :doc:`AddressSanitizer`) may not support recovery,
and always crash the program after the issue is detected.
+**-f[no-]sanitize-coverage=[type,features,...]**
+
+ Enable simple code coverage in addition to certain sanitizers.
+ See :doc:`SanitizerCoverage` for more details.
+
.. option:: -fno-assume-sane-operator-new
Don't assume that the C++'s new operator is sane.
@@ -1190,6 +1218,32 @@ behavior. Code that is not exercised in the profile will be optimized as if it
is unimportant, and the compiler may make poor optimization choices for code
that is disproportionately used while profiling.
+Differences Between Sampling and Instrumentation
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Although both techniques are used for similar purposes, there are important
+differences between the two:
+
+1. Profile data generated with one cannot be used by the other, and there is no
+ conversion tool that can convert one to the other. So, a profile generated
+ via ``-fprofile-instr-generate`` must be used with ``-fprofile-instr-use``.
+ Similarly, sampling profiles generated by external profilers must be
+ converted and used with ``-fprofile-sample-use``.
+
+2. Instrumentation profile data can be used for code coverage analysis and
+ optimization.
+
+3. Sampling profiles can only be used for optimization. They cannot be used for
+ code coverage analysis. Although it would be technically possible to use
+ sampling profiles for code coverage, sample-based profiles are too
+ coarse-grained for code coverage purposes; it would yield poor results.
+
+4. Sampling profiles must be generated by an external tool. The profile
+ generated by that tool must then be converted into a format that can be read
+ by LLVM. The section on sampling profilers describes one of the supported
+ sampling profile formats.
+
+
Using Sampling Profilers
^^^^^^^^^^^^^^^^^^^^^^^^
@@ -1257,17 +1311,38 @@ usual build cycle when using sample profilers for optimization:
$ clang++ -O2 -gline-tables-only -fprofile-sample-use=code.prof code.cc -o code
-Sample Profile Format
-"""""""""""""""""""""
+Sample Profile Formats
+""""""""""""""""""""""
+
+Since external profilers generate profile data in a variety of custom formats,
+the data generated by the profiler must be converted into a format that can be
+read by the backend. LLVM supports three different sample profile formats:
+
+1. ASCII text. This is the easiest one to generate. The file is divided into
+ sections, which correspond to each of the functions with profile
+ information. The format is described below.
+
+2. Binary encoding. This uses a more efficient encoding that yields smaller
+ profile files, which may be useful when generating large profiles. It can be
+ generated from the text format using the ``llvm-profdata`` tool.
+
+3. GCC encoding. This is based on the gcov format, which is accepted by GCC. It
+ is only interesting in environments where GCC and Clang co-exist. Similarly
+ to the binary encoding, it can be generated using the ``llvm-profdata`` tool.
+
+If you are using Linux Perf to generate sampling profiles, you can use the
+conversion tool ``create_llvm_prof`` described in the previous section.
+Otherwise, you will need to write a conversion tool that converts your
+profiler's native format into one of these three.
+
-If you are not using Linux Perf to collect profiles, you will need to
-write a conversion tool from your profiler to LLVM's format. This section
-explains the file format expected by the backend.
+Sample Profile Text Format
+""""""""""""""""""""""""""
-Sample profiles are written as ASCII text. The file is divided into sections,
-which correspond to each of the functions executed at runtime. Each
-section has the following format (taken from
-https://github.com/google/autofdo/blob/master/profile_writer.h):
+This section describes the ASCII text format for sampling profiles. It is,
+arguably, the easiest one to generate. If you are interested in generating any
+of the other two, consult the ``ProfileData`` library in in LLVM's source tree
+(specifically, ``llvm/lib/ProfileData/SampleProfWriter.cpp``).
.. code-block:: console
@@ -1374,8 +1449,8 @@ instrumentation:
$ LLVM_PROFILE_FILE="code-%p.profraw" ./code
3. Combine profiles from multiple runs and convert the "raw" profile format to
- the input expected by clang. Use the ``merge`` command of the llvm-profdata
- tool to do this.
+ the input expected by clang. Use the ``merge`` command of the
+ ``llvm-profdata`` tool to do this.
.. code-block:: console
diff --git a/docs/index.rst b/docs/index.rst
index 67bdf689e0..dec2bc828c 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -26,8 +26,10 @@ Using Clang as a Compiler
MemorySanitizer
DataFlowSanitizer
LeakSanitizer
+ SanitizerCoverage
SanitizerSpecialCaseList
ControlFlowIntegrity
+ SafeStack
Modules
MSVCCompatibility
FAQ
diff --git a/docs/tools/clang.pod b/docs/tools/clang.pod
index f7d2eaf683..153f97b2ca 100644
--- a/docs/tools/clang.pod
+++ b/docs/tools/clang.pod
@@ -39,12 +39,12 @@ the other tools.
This stage handles tokenization of the input source file, macro expansion,
#include expansion and handling of other preprocessor directives. The output of
this stage is typically called a ".i" (for C), ".ii" (for C++), ".mi" (for
-Objective-C) , or ".mii" (for Objective-C++) file.
+Objective-C), or ".mii" (for Objective-C++) file.
=item B<Parsing and Semantic Analysis>
This stage parses the input file, translating preprocessor tokens into a parse
-tree. Once in the form of a parser tree, it applies semantic analysis to compute
+tree. Once in the form of a parse tree, it applies semantic analysis to compute
types for expressions as well and determine whether the code is well formed. This
stage is responsible for generating most of the compiler warnings as well as
parse errors. The output of this stage is an "Abstract Syntax Tree" (AST).
@@ -330,13 +330,13 @@ all by the program.
=item B<-fexceptions>
-Enable generation of unwind information, this allows exceptions to be thrown
+Enable generation of unwind information. This allows exceptions to be thrown
through Clang compiled stack frames. This is on by default in x86-64.
=item B<-ftrapv>
Generate code to catch integer overflow errors. Signed integer overflow is
-undefined in C, with this flag, extra code is generated to detect this and abort
+undefined in C. With this flag, extra code is generated to detect this and abort
when it happens.
@@ -389,7 +389,7 @@ Display available options.
=item B<-Qunused-arguments>
-Don't emit warning for unused driver arguments.
+Do not emit any warnings for unused driver arguments.
=item B<-Wa,>I<args>
@@ -578,7 +578,7 @@ write temporary files used during the compilation process.
If this environment variable is present, it is treated as a delimited
list of paths to be added to the default system include path list. The
-delimiter is the platform dependent delimitor, as used in the I<PATH>
+delimiter is the platform dependent delimiter, as used in the I<PATH>
environment variable.
Empty components in the environment variable are ignored.
@@ -592,7 +592,7 @@ which are only used when processing the appropriate language.
=item B<MACOSX_DEPLOYMENT_TARGET>
If -mmacosx-version-min is unspecified, the default deployment target
-is read from this environment variable. This option only affects darwin
+is read from this environment variable. This option only affects Darwin
targets.
=back