diff options
Diffstat (limited to 'doc/source')
| -rw-r--r-- | doc/source/cdef.rst | 46 | ||||
| -rw-r--r-- | doc/source/conf.py | 4 | ||||
| -rw-r--r-- | doc/source/embedding.rst | 9 | ||||
| -rw-r--r-- | doc/source/goals.rst | 6 | ||||
| -rw-r--r-- | doc/source/index.rst | 3 | ||||
| -rw-r--r-- | doc/source/installation.rst | 54 | ||||
| -rw-r--r-- | doc/source/overview.rst | 72 | ||||
| -rw-r--r-- | doc/source/ref.rst | 37 | ||||
| -rw-r--r-- | doc/source/using.rst | 24 | ||||
| -rw-r--r-- | doc/source/whatsnew.rst | 207 |
10 files changed, 340 insertions, 122 deletions
diff --git a/doc/source/cdef.rst b/doc/source/cdef.rst index f0bc6ba..0662668 100644 --- a/doc/source/cdef.rst +++ b/doc/source/cdef.rst @@ -235,7 +235,12 @@ specifies that all structure fields should have an alignment of one byte. (Note that the packed attribute has no effect on bit fields so far, which mean that they may be packed differently than on GCC. Also, this has no effect on structs declared with ``"...;"``---more -about it later in `Letting the C compiler fill the gaps`_.) +about it later in `Letting the C compiler fill the gaps`_. In +particular, if your C source uses other attributes like +``__attribute__((aligned(16)))``, there is no way to declare this fact +in the ``cdef()``, but you can generally just declare the struct with +``"...;"`` as the last field.) + *New in version 1.12:* In ABI mode, you can also pass ``pack=n``, with an integer ``n`` which must be a power of two. Then the alignment of any field is limited to ``n`` if it would otherwise be @@ -302,6 +307,7 @@ strings as arguments instead of byte strings. .. _loading-libraries: +.. _dlopen: ffi.dlopen(): loading libraries in ABI mode ------------------------------------------- @@ -364,6 +370,18 @@ look for the library's filename. This also means that ``ffi.dlopen(None)`` no longer work on Windows; try instead ``ffi.dlopen(ctypes.util.find_library('c'))``. +*New in version 1.14:* ``ffi.dlopen(handle)``: instead of a file path, +you can give an already-opened library handle, as a cdata of type +``void *``. Such a call converts this handle into a regular FFI object +with the functions and global variables declared by ``ffi.cdef()``. +Useful if you have special needs (e.g. you need the GNU extension +``dlmopen()``, which you can itself declare and call using a different +``ffi`` object). Note that in this variant, ``dlclose()`` is not called +automatically if the FFI object is garbage-collected (but you can still +call ``ffi.dlclose()`` explicitly if needed). + + +.. _set_source: ffibuilder.set_source(): preparing out-of-line modules ------------------------------------------------------ @@ -471,10 +489,12 @@ Moreover, you can use "``...``" (literally, dot-dot-dot) in the ``cdef()`` at various places, in order to ask the C compiler to fill in the details. These places are: -* structure declarations: any ``struct { }`` that ends with "``...;``" as - the last "field" is - partial: it may be missing fields and/or have them declared out of order. - This declaration will be corrected by the compiler. (But note that you +* structure declarations: any ``struct { }`` or ``union { }`` that ends + with "``...;``" as the last "field" is partial: it may be missing + fields, have them declared out of order, use non-standard alignment, + etc. Precisely, the field offsets, total struct size, and total + struct alignment deduced by looking at the ``cdef`` are not relied + upon and will instead be corrected by the compiler. (But note that you can only access fields that you declared, not others.) Any ``struct`` declaration which doesn't use "``...``" is assumed to be exact, but this is checked: you get an error if it is not correct. @@ -512,14 +532,14 @@ in the details. These places are: field; then you would use "``typedef struct { ...; } foo_t;``". * array lengths: when used as structure fields or in global variables, - arrays can have an unspecified length, as in "``int n[...];``". The + arrays can have an unspecified length, as in "``extern int n[...];``". The length is completed by the C compiler. - This is slightly different from "``int n[];``", because the latter + This is slightly different from "``extern int n[];``", because the latter means that the length is not known even to the C compiler, and thus no attempt is made to complete it. This supports - multidimensional arrays: "``int n[...][...];``". + multidimensional arrays: "``extern int n[...][...];``". - *New in version 1.2:* "``int m[][...];``", i.e. ``...`` can be used + *New in version 1.2:* "``extern int m[][...];``", i.e. ``...`` can be used in the innermost dimensions without being also used in the outermost dimension. In the example given, the length of the ``m`` array is assumed not to be known to the C compiler, but the length of every @@ -568,12 +588,12 @@ pointer types; currently, it also does not work for variadic functions. For more complex types, you have no choice but be precise. For example, you cannot misdeclare a ``int *`` argument as ``long *``, or a global -array ``int a[5];`` as ``long a[5];``. CFFI considers `all types listed -above`_ as primitive (so ``long long a[5];`` and ``int64_t a[5]`` are +array ``extern int a[5];`` as ``extern long a[5];``. CFFI considers `all types listed +above`_ as primitive (so ``extern long long a[5];`` and ``extern int64_t a[5]`` are different declarations). The reason for that is detailed in `a comment about an issue.`__ -.. __: https://bitbucket.org/cffi/cffi/issues/265/cffi-doesnt-allow-creating-pointers-to#comment-28406958 +.. __: https://foss.heptapod.net/pypy/cffi/-/issues/265#note_50393 ffibuilder.compile() etc.: compiling out-of-line modules @@ -806,7 +826,7 @@ write the rest of the build script): print lib.mysize Extra arguments to ``ffi.verify()``: - + * ``tmpdir`` controls where the C files are created and compiled. Unless the ``CFFI_TMPDIR`` environment variable is set, the default is diff --git a/doc/source/conf.py b/doc/source/conf.py index 3400cd1..33e8c11 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -45,9 +45,9 @@ copyright = u'2012-2018, Armin Rigo, Maciej Fijalkowski' # built documents. # # The short X.Y version. -version = '1.12' +version = '1.15' # The full version, including alpha/beta/rc tags. -release = '1.12.2' +release = '1.15.0' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. diff --git a/doc/source/embedding.rst b/doc/source/embedding.rst index 181249c..8020f21 100644 --- a/doc/source/embedding.rst +++ b/doc/source/embedding.rst @@ -380,7 +380,7 @@ inaccuracies in this paragraph or better ways to do things.) ``dlopen("libpythonX.Y.so", RTLD_LAZY|RTLD_GLOBAL)``, which will force ``libpythonX.Y.so`` to be loaded first. -.. __: https://bitbucket.org/cffi/cffi/issues/264/ +.. __: https://foss.heptapod.net/pypy/cffi/-/issues/264 Using multiple CFFI-made DLLs @@ -522,3 +522,10 @@ calling the C function ``cffi_start_python()`` with no argument. It returns an integer, 0 or -1, to tell if the initialization succeeded or not. Currently there is no way to prevent a failing initialization from also dumping a traceback and more information to stderr. +Note that the function ``cffi_start_python()`` is static: it must be +called from C source written inside ``ffibuilder.set_source()``. To +call it from somewhere else, you need to make a function (with a +different non-static name) in the ``ffibuilder.set_source()`` that just +calls ``cffi_start_python()``. The reason it is static is to avoid +naming conflicts in case you are ultimately trying to link a large C +program with more than one cffi embedded module in it. diff --git a/doc/source/goals.rst b/doc/source/goals.rst index 0fda659..df4877c 100644 --- a/doc/source/goals.rst +++ b/doc/source/goals.rst @@ -55,8 +55,8 @@ Get started by reading `the overview`__. Comments and bugs ----------------- -The best way to contact us is on the IRC ``#pypy`` channel of -``irc.freenode.net``. Feel free to discuss matters either there or in +The best way to contact us is on the IRC ``#cffi`` or ``#pypy`` channels of +``irc.libera.chat``. Feel free to discuss matters either there or in the `mailing list`_. Please report to the `issue tracker`_ any bugs. As a general rule, when there is a design issue to resolve, we pick the @@ -65,5 +65,5 @@ everything you need to access C code and nothing more. --- the authors, Armin Rigo and Maciej Fijalkowski -.. _`issue tracker`: https://bitbucket.org/cffi/cffi/issues +.. _`issue tracker`: https://foss.heptapod.net/pypy/cffi/issues .. _`mailing list`: https://groups.google.com/forum/#!forum/python-cffi diff --git a/doc/source/index.rst b/doc/source/index.rst index 1126318..54934f2 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -17,6 +17,3 @@ copy-paste from header files or documentation. ref cdef embedding - - - diff --git a/doc/source/installation.rst b/doc/source/installation.rst index 4af17d5..6d55eb5 100644 --- a/doc/source/installation.rst +++ b/doc/source/installation.rst @@ -17,7 +17,7 @@ platform as well as on Windows 32 and 64. (It relies occasionally on libffi, so it depends on libffi being bug-free; this may not be fully the case on some of the more exotic platforms.) -CFFI supports CPython 2.6, 2.7, 3.x (tested with 3.2 to 3.4); and is +CFFI supports CPython 2.7, 3.x (tested with 3.6 to 3.9); and is distributed with PyPy (CFFI 1.0 is distributed with and requires PyPy 2.6). @@ -31,7 +31,7 @@ libraries on PyPy. Requirements: -* CPython 2.6 or 2.7 or 3.x, or PyPy (PyPy 2.0 for the earliest +* CPython 2.7 or 3.x, or PyPy (PyPy 2.0 for the earliest versions of CFFI; or PyPy 2.6 for CFFI 1.0). * in some cases you need to be able to compile C extension modules. @@ -52,26 +52,26 @@ Download and Installation: * https://pypi.python.org/pypi/cffi -* Checksums of the "source" package version 1.12.2: +* Checksums of the "source" package version 1.15.0: - - MD5: 4d7dcb6c7c738c15d2ece9bd4c5f86da + - MD5: f3a3f26cd3335fc597479c9475da0a0b - - SHA: 5f579d4980cbcc8aac592721f714ef6a64370ab1 + - SHA1: 9c51c29e35510adf7f94542e1f8e05611930b07b - - SHA256: e113878a446c6228669144ae8a56e268c91b7f1fafae927adc4879d9849e0ea7 + - SHA256: 920f0d66a896c2d99f0adbb391f990a84091179542c205fa53ce5787aff87954 -* Or grab the most current version from the `Bitbucket page`_: - ``hg clone https://bitbucket.org/cffi/cffi`` +* Or grab the most current version from the `Heptapod page`_: + ``hg clone https://foss.heptapod.net/pypy/cffi`` * ``python setup.py install`` or ``python setup_base.py install`` (should work out of the box on Linux or Windows; see below for - `MacOS X`_ or `Windows 64`_.) + `MacOS X`_.) * running the tests: ``py.test c/ testing/`` (if you didn't install cffi yet, you need first ``python setup_base.py build_ext -f -i``) -.. _`Bitbucket page`: https://bitbucket.org/cffi/cffi +.. _`Heptapod page`: https://foss.heptapod.net/pypy/cffi Demos: @@ -82,9 +82,9 @@ Demos: ultimate reference is given by the tests, notably `testing/cffi1/test_verify1.py`_ and `testing/cffi0/backend_tests.py`_. -.. _`demo`: https://bitbucket.org/cffi/cffi/src/default/demo -.. _`testing/cffi1/test_verify1.py`: https://bitbucket.org/cffi/cffi/src/default/testing/cffi1/test_verify1.py -.. _`testing/cffi0/backend_tests.py`: https://bitbucket.org/cffi/cffi/src/default/testing/cffi0/backend_tests.py +.. _`demo`: https://foss.heptapod.net/pypy/cffi/-/tree/branch/default/demo +.. _`testing/cffi1/test_verify1.py`: https://foss.heptapod.net/pypy/cffi/-/blob/branch/default/testing/cffi1/test_verify1.py +.. _`testing/cffi0/backend_tests.py`: https://foss.heptapod.net/pypy/cffi/-/blob/branch/default/testing/cffi0/backend_tests.py Platform-specific instructions @@ -133,39 +133,23 @@ as described here_. .. _here: http://superuser.com/questions/259278/python-2-6-1-pycrypto-2-3-pypi-package-broken-pipe-during-build -Windows (regular 32-bit) -++++++++++++++++++++++++ +Windows (32/64-bit) ++++++++++++++++++++ -Win32 works and is tested at least each official release. +Win32 and Win64 work and are tested at least each official release. The recommended C compiler compatible with Python 2.7 is this one: http://www.microsoft.com/en-us/download/details.aspx?id=44266 -There is a known problem with distutils on Python 2.7, as -explained in https://bugs.python.org/issue23246, and the same +There is a known problem with distutils on Python 2.7, as +explained in https://bugs.python.org/issue23246, and the same problem applies whenever you want to run compile() to build a dll with -this specific compiler suite download. +this specific compiler suite download. ``import setuptools`` might help, but YMMV For Python 3.4 and beyond: https://www.visualstudio.com/en-us/downloads/visual-studio-2015-ctp-vs -Windows 64 -++++++++++ - -Win64 received very basic testing and we applied a few essential -fixes in cffi 0.7. The comment above applies for Python 2.7 on -Windows 64 as well. Please report any other issue. - -Note as usual that this is only about running the 64-bit version of -Python on the 64-bit OS. If you're running the 32-bit version (the -common case apparently), then you're running Win32 as far as we're -concerned. - -.. _`issue 9`: https://bitbucket.org/cffi/cffi/issue/9 -.. _`Python issue 7546`: http://bugs.python.org/issue7546 - - Linux and OS/X: UCS2 versus UCS4 ++++++++++++++++++++++++++++++++ diff --git a/doc/source/overview.rst b/doc/source/overview.rst index bcc5663..dbc3540 100644 --- a/doc/source/overview.rst +++ b/doc/source/overview.rst @@ -3,7 +3,7 @@ Overview ======================================================= .. contents:: - + The first section presents a simple working example of using CFFI to call a C function in a compiled shared object @@ -61,9 +61,9 @@ Create the file ``piapprox_build.py``: Execute this script. If everything is OK, it should produce ``_pi_cffi.c``, and then invoke the compiler on it. The produced -``_pi_cffi.c`` contains a copy of the string given in ``set_source()``, +``_pi_cffi.c`` contains a copy of the string given in :ref:`set_source() <set_source>`, in this example the ``#include "pi.h"``. Afterwards, it contains glue code -for all the functions, types and globals declared in the ``cdef()`` above. +for all the functions, types and globals declared in the :ref:`cdef() <cdef>` above. At runtime, you use the extension module like this: @@ -126,7 +126,7 @@ May look familiar to those who have used ctypes_. >>> ffi = FFI() >>> ffi.cdef(""" ... int printf(const char *format, ...); // copy-pasted from the man page - ... """) + ... """) >>> C = ffi.dlopen(None) # loads the entire C namespace >>> arg = ffi.new("char[]", b"world") # equivalent to C code: char arg[] = "world"; >>> C.printf(b"hi there, %s.\n", arg) # call printf @@ -138,7 +138,7 @@ Note that ``char *`` arguments expect a ``bytes`` object. If you have a ``str`` (or a ``unicode`` on Python 2) you need to encode it explicitly with ``somestring.encode(myencoding)``. -*Python 3 on Windows:* ``ffi.dlopen(None)`` does not work. This problem +*Python 3 on Windows:* :ref:`ffi.dlopen(None) <dlopen>` does not work. This problem is messy and not really fixable. The problem does not occur if you try to call a function from a specific DLL that exists on your system: then you use ``ffi.dlopen("path.dll")``. @@ -179,7 +179,7 @@ Struct/Array Example (minimal, in-line) f.close() This can be used as a more flexible replacement of the struct_ and -array_ modules, and replaces ctypes_. You could also call ``ffi.new("pixel_t[600][800]")`` +array_ modules, and replaces ctypes_. You could also call :ref:`ffi.new("pixel_t[600][800]") <new>` and get a two-dimensional array. .. _struct: http://docs.python.org/library/struct.html @@ -191,7 +191,7 @@ and get a two-dimensional array. This example also admits an out-of-line equivalent. It is similar to the first example `Main mode of usage`_ above, but passing ``None`` as the second argument to -``ffibuilder.set_source()``. Then in the main program you write +:ref:`ffibuilder.set_source() <set_source>`. Then in the main program you write ``from _simple_example import ffi`` and then the same content as the in-line example above starting from the line ``image = ffi.new("pixel_t[]", 800*600)``. @@ -244,7 +244,7 @@ You need to run the ``example_build.py`` script once to generate "source code" into the file ``_example.c`` and compile this to a regular C extension module. (CFFI selects either Python or C for the module to generate based on whether the second argument to -``set_source()`` is ``None`` or not.) +:ref:`set_source() <set_source>` is ``None`` or not.) *You need a C compiler for this single step. It produces a file called e.g. _example.so or _example.pyd. If needed, it can be distributed in @@ -265,7 +265,7 @@ Note that this works independently of the exact C layout of ``struct passwd`` (it is "API level", as opposed to "ABI level"). It requires a C compiler in order to run ``example_build.py``, but it is much more portable than trying to get the details of the fields of ``struct -passwd`` exactly right. Similarly, in the ``cdef()`` we declared +passwd`` exactly right. Similarly, in the :ref:`cdef() <cdef>` we declared ``getpwuid()`` as taking an ``int`` argument; on some platforms this might be slightly incorrect---but it does not matter. @@ -301,28 +301,28 @@ with the files ``pi.c`` and ``pi.h``: /* filename: pi.c*/ # include <stdlib.h> # include <math.h> - + /* Returns a very crude approximation of Pi given a int: a number of iteration */ float pi_approx(int n){ - + double i,x,y,sum=0; - + for(i=0;i<n;i++){ - + x=rand(); y=rand(); - + if (sqrt(x*x+y*y) < sqrt((double)RAND_MAX*RAND_MAX)) sum++; } - + return 4*(float)sum/(float)n; } .. code-block:: C /* filename: pi.h*/ float pi_approx(int n); - + Create a script named ``pi_extension_build.py``, building the C extension: @@ -330,21 +330,21 @@ the C extension: from cffi import FFI ffibuilder = FFI() - + ffibuilder.cdef("float pi_approx(int n);") - + ffibuilder.set_source("_pi", # name of the output C extension """ - #include "pi.h"', + #include "pi.h" """, sources=['pi.c'], # includes pi.c as additional sources libraries=['m']) # on Unix, link with the math library - + if __name__ == "__main__": ffibuilder.compile(verbose=True) Build the extension: - + .. code-block:: shell python pi_extension_build.py @@ -354,14 +354,14 @@ Observe, in the working directory, the generated output files: Linux for example). It can be called from Python: .. code-block:: python - + from _pi.lib import pi_approx - + approx = pi_approx(10) - assert str(pi_approximation).startswith("3.") - + assert str(approx).startswith("3.") + approx = pi_approx(10000) - assert str(approx).startswith("3.1") + assert str(approx).startswith("3.1") .. _performance: @@ -428,7 +428,7 @@ easily). This mixture mode lets you massively reduces the import times, because it is slow to parse a large C header. It also allows you to do more detailed checkings during build-time without worrying about performance -(e.g. calling ``cdef()`` many times with small pieces of declarations, +(e.g. calling :ref:`cdef() <cdef>` many times with small pieces of declarations, based on the version of libraries detected on the system). .. code-block:: python @@ -461,7 +461,7 @@ any more: lib.printf(b"hi there, number %d\n", ffi.cast("int", 2)) -Note that this ``ffi.dlopen()``, unlike the one from in-line mode, +Note that this :ref:`ffi.dlopen() <dlopen>`, unlike the one from in-line mode, does not invoke any additional magic to locate the library: it must be a path name (with or without a directory), as required by the C ``dlopen()`` or ``LoadLibrary()`` functions. This means that @@ -496,7 +496,7 @@ In-line, API level The "API level + in-line" mode combination exists but is long deprecated. It used to be done with ``lib = ffi.verify("C header")``. -The out-of-line variant with ``set_source("modname", "C header")`` is +The out-of-line variant with :ref:`set_source("modname", "C header") <set_source>` is preferred and avoids a number of problems when the project grows in size. @@ -558,13 +558,13 @@ means that most of the documentation or examples can be copied straight from the man pages. The declarations can contain **types, functions, constants** -and **global variables.** What you pass to the ``cdef()`` must not +and **global variables.** What you pass to the :ref:`cdef() <cdef>` must not contain more than that; in particular, ``#ifdef`` or ``#include`` directives are not supported. The cdef in the above examples are just that - they declared "there is a function in the C level with this given signature", or "there is a struct type with this shape". -In the ABI examples, the ``dlopen()`` calls load libraries manually. +In the ABI examples, the :ref:`dlopen() <dlopen>` calls load libraries manually. At the binary level, a program is split into multiple namespaces---a global one (on some platforms), plus one namespace per library. So ``dlopen()`` returns a ``<FFILibrary>`` object, and this object has @@ -576,13 +576,13 @@ you would call ``cdef()`` only once but ``dlopen()`` several times. By opposition, the API mode works more closely like a C program: the C linker (static or dynamic) is responsible for finding any symbol used. You name the libraries in the ``libraries`` keyword argument to -``set_source()``, but never need to say which symbol comes +:ref:`set_source() <set_source>`, but never need to say which symbol comes from which library. Other common arguments to ``set_source()`` include ``library_dirs`` and ``include_dirs``; all these arguments are passed to the standard distutils/setuptools. -The ``ffi.new()`` lines allocate C objects. They are filled +The :ref:`ffi.new() <new>` lines allocate C objects. They are filled with zeroes initially, unless the optional second argument is used. If specified, this argument gives an "initializer", like you can use with C code to initialize global variables. @@ -609,10 +609,10 @@ The more fundamental reason to prefer the API mode is that *the C libraries are typically meant to be used with a C compiler.* You are not supposed to do things like guess where fields are in the structures. The "real example" above shows how CFFI uses a C compiler under the -hood: this example uses ``set_source(..., "C source...")`` and never -``dlopen()``. When using this approach, +hood: this example uses :ref:`set_source(..., "C source...") <set_source>` and never +:ref:`dlopen() <dlopen>`. When using this approach, we have the advantage that we can use literally "``...``" at various places in -the ``cdef()``, and the missing information will be completed with the +the :ref:`cdef() <cdef>`, and the missing information will be completed with the help of the C compiler. CFFI will turn this into a single C source file, which contains the "C source" part unmodified, followed by some "magic" C code and declarations derived from the ``cdef()``. When diff --git a/doc/source/ref.rst b/doc/source/ref.rst index 3dc8e4b..05c0f7c 100644 --- a/doc/source/ref.rst +++ b/doc/source/ref.rst @@ -30,6 +30,8 @@ ffi.error confuse it with ``ffi.errno``.) +.. _new: + ffi.new() +++++++++ @@ -37,7 +39,7 @@ ffi.new() allocate an instance according to the specified C type and return a pointer to it. The specified C type must be either a pointer or an array: ``new('X *')`` allocates an X and returns a pointer to it, -whereas ``new('X[n]')`` allocates an array of n X'es and returns an +whereas ``new('X[10]')`` allocates an array of 10 X'es and returns an array referencing it (which works mostly like a pointer, like in C). You can also use ``new('X[]', n)`` to allocate an array of a non-constant length n. See the `detailed documentation`__ for other @@ -51,9 +53,13 @@ the value of type ``cdecl`` that it points to. This means that the raw data can be used as long as this object is kept alive, but must not be used for a longer time. Be careful about that when copying the pointer to the memory somewhere else, e.g. into another structure. -Also, this means that a line like ``x = ffi.new(...)[0]`` is *always -wrong:* the newly allocated object goes out of scope instantly, and so -is freed immediately, and ``x`` is garbage. +Also, this means that a line like ``x = ffi.cast("B *", ffi.new("A *"))`` +or ``x = ffi.new("struct s[1]")[0]`` is wrong: the newly allocated object +goes out of scope instantly, and so is freed immediately, and ``x`` is +garbage. The only case where this is fine comes from a special case for +pointers-to-struct and pointers-to-union types: after +``p = ffi.new("struct-or-union *", ..)``, then either ``p`` or ``p[0]`` +keeps the memory alive. The returned memory is initially cleared (filled with zeroes), before the optional initializer is applied. For performance, see @@ -231,7 +237,8 @@ memory); and byte strings were supported in version 1.8 onwards. *New in version 1.12:* added the optional *first* argument ``cdecl``, and the keyword argument ``require_writable``: -* ``cdecl`` defaults to ``"char[]"``, but a different array type can be +* ``cdecl`` defaults to ``"char[]"``, but a different array + or (from version 1.13) pointer type can be specified for the result. A value like ``"int[]"`` will return an array of ints instead of chars, and its length will be set to the number of ints that fit in the buffer (rounded down if the division is not exact). Values @@ -243,6 +250,12 @@ the keyword argument ``require_writable``: keeps the underlying Python object alive and locked. (In addition, ``ffi.from_buffer("int[]", x)`` gives better array bound checking.) + *New in version 1.13:* ``cdecl`` can be a pointer type. If it points + to a struct or union, you can, as usual, write ``p.field`` instead of + ``p[0].field``. You can also access ``p[n]``; note that CFFI does not + perform any bounds checking in this case. Note also that ``p[0]`` cannot + be used to keep the buffer alive (unlike what occurs with ``ffi.new()``). + * if ``require_writable`` is set to True, the function fails if the buffer obtained from ``python_buffer`` is read-only (e.g. if ``python_buffer`` is a byte string). The exact exception is raised by the object itself, and @@ -297,7 +310,7 @@ performance difference). It can still be useful in writing typechecks, e.g.: .. code-block:: python - + def myfunction(ptr): assert ffi.typeof(ptr) is ffi.typeof("foo_t*") ... @@ -448,7 +461,7 @@ destructors will be called in a random order. If you need a particular order, see the discussion in `issue 340`__. .. __: http://bugs.python.org/issue31105 -.. __: https://bitbucket.org/cffi/cffi/issues/340/resources-release-issues +.. __: https://foss.heptapod.net/pypy/cffi/-/issues/340 .. _ffi-new-handle: @@ -608,6 +621,14 @@ called at a known point. The above is equivalent to this code:: with my_new("int[]", n) as my_array: ... +**Warning:** due to a bug, ``p = ffi.new_allocator(..)("struct-or-union *")`` +might not follow the rule that either ``p`` or ``p[0]`` keeps the memory +alive, which holds for the normal ``ffi.new("struct-or-union *")`` allocator. +It may sometimes be the case that if there is only a reference to ``p[0]``, +the memory is freed. The cause is that the rule doesn't hold for +``ffi.gc()``, which is sometimes used in the implementation of +``ffi.new_allocator()()``; this might be fixed in a future release. + .. _ffi-release: @@ -712,7 +733,7 @@ example:: raise IndexError("index too large!") ... -.. __: https://bitbucket.org/cffi/cffi/issues/233/ +.. __: https://foss.heptapod.net/pypy/cffi/-/issues/233 .. _ffi-getctype: diff --git a/doc/source/using.rst b/doc/source/using.rst index ff8e5f1..38c96ba 100644 --- a/doc/source/using.rst +++ b/doc/source/using.rst @@ -459,7 +459,7 @@ that you really meant the 42 to be passed as a C ``int``, and not a if necessary with ``ffi.cast()``: .. code-block:: python - + lib.printf("hello, %d\n", ffi.cast("int", 42)) lib.printf("hello, %ld\n", ffi.cast("long", 42)) lib.printf("hello, %f\n", ffi.cast("double", 42)) @@ -787,7 +787,7 @@ The ``extern "Python"`` functions cannot be variadic for now. This may be implemented in the future. (`This demo`__ shows how to do it anyway, but it is a bit lengthy.) -.. __: https://bitbucket.org/cffi/cffi/src/default/demo/extern_python_varargs.py +.. __: https://foss.heptapod.net/pypy/cffi/-/blob/branch/default/demo/extern_python_varargs.py Each corresponding Python callback function is defined with the ``@ffi.def_extern()`` decorator. Be careful when writing this @@ -876,11 +876,27 @@ ffi.callback() and the result is the same. protections can interfere (for example, on SELinux you need to run with ``deny_execmem`` set to ``off``). - Note also that a cffi fix for the latter issue was attempted---see + - `On Mac OS X,`__ you need to give your application the entitlement + ``com.apple.security.cs.allow-unsigned-executable-memory``. + + Note also that a cffi fix for this issue was attempted---see the ``ffi_closure_alloc`` branch---but was not merged because it creates potential `memory corruption`__ with ``fork()``. + In other words: yes, it is dangerous to allow write+execute memory in your + program; that's why the various "hardening" options above exist. But at + the same time, these options open wide the door to another attack: if the + program forks and then attempts to call any of the ``ffi.callback()``, then + this immediately results in a crash---or, with a minimal amount of work + from an attacker, arbitrary code execution. To me it sounds even more + dangerous than the original problem, and that's why cffi is not playing + along. + + To fix the issue once and for all on the affected platforms, you need + to refactor the involved code so that it no longer uses ``ffi.callback()``. + .. __: https://github.com/pyca/pyopenssl/issues/596 +.. __: https://foss.heptapod.net/pypy/cffi/-/issues/391 .. __: https://bugzilla.redhat.com/show_bug.cgi?id=1249685 Warning: like ffi.new(), ffi.callback() returns a cdata that has @@ -950,7 +966,7 @@ arguments are passed: ffibuilder.compile(verbose=True) .. code-block:: python - + # file "example.py" from _example import ffi, lib diff --git a/doc/source/whatsnew.rst b/doc/source/whatsnew.rst index 18bb218..aa7f2fe 100644 --- a/doc/source/whatsnew.rst +++ b/doc/source/whatsnew.rst @@ -2,15 +2,191 @@ What's New ====================== +v1.15.0 +======= -v1.12.2 +* Fixed MANIFEST.in to include missing file for Windows arm64 support + +* Fixed Linux wheel build to use gcc default ISA for libffi + +* Updated setup.py Python trove specifiers to currently-tested Python versions + +* CPython 3.10 support (including wheels) + +* MacOS arm64 support (including wheels) + +* Initial Windows arm64 support + +* Misc. doc and test updates + +v1.14.6 +======= + +* Test fixes for CPython 3.10.0b3 + +* Support for `sys.unraisablehook()` on Python >= 3.8 + +* Fix two minor memory leaks (thanks Sebastian!) + +* Like many projects that had an IRC channel on freenode, we moved it to + ``irc.libera.chat``. + +v1.14.5 +======= + +* Source fix for old gcc versions + +* This and future releases should include wheels on more platforms, + thanks to our new release managers Matt and Matt! + +v1.14.4 +======= + +Release done for pip reasons. + +v1.14.3 ======= +Release done for pip reasons. + +v1.14.2 +======= + +* CPython 3 on Windows: we again try to compile with ``Py_LIMITED_API`` + by default. This flag is not added if you run the compilation with + CPython 3.4, as it only works with CPython >= 3.5, but by now this + version of Python is quite old (and we no longer distribute cffi + wheels for it). + + This may require that you upgrade ``virtualenv`` (requires version 16 + or newer) or at least copy manually ``python3.dll`` into your existing + virtualenvs. For distributing wheels with your cffi modules, you may + also need to upgrade ``wheel`` to the just-released version 0.35. + + You can manually disable ``Py_LIMITED_API`` by calling + ``ffi.set_source(..., py_limited_api=False)``. + + +v1.14.1 +======= + +* CFFI source code is now `hosted on Heptapod`_. + +* Improved support for ``typedef int my_array_t[...];`` with an explicit + dot-dot-dot in API mode (`issue #453`_) + +* Windows (32 and 64 bits): multiple fixes for ABI-mode call to functions + that return a structure. + +* Experimental support for MacOS 11 on aarch64. + +* and a few other minor changes and bug fixes. + +.. _`hosted on Heptapod`: https://foss.heptapod.net/pypy/cffi/ +.. _`issue #453`: https://foss.heptapod.net/pypy/cffi/issues/453 + + +v1.14 +===== + +* ``ffi.dlopen()`` can now be called with a handle (as a ``void *``) to an + already-opened C library. + +* CPython only: fixed a stack overflow issue for calls like + ``lib.myfunc([large list])``. If the function is declared as taking a + ``float *`` argument, for example, then the array is temporarily converted + into a C array of floats---however, the code used to use ``alloca()`` for + this temporary storage, no matter how large. This is now fixed. + + The fix concerns all modes: in-line/out-of-line API/ABI. Also note that your + API-mode C extension modules need to be regenerated with cffi 1.14 in order + to get the fix; i.e. for API mode, the fix is in the generated C sources. + (The C sources generated from cffi 1.14 should also work when running in + a different environment in which we have an older version of cffi. Also, + this change makes no difference on PyPy.) + + As a workaround that works on all versions of cffi, you can write + ``lib.myfunc(ffi.new("float[]", [large list]))``, which is + equivalent but explicity builds the intermediate array as a regular + Python object on the heap. + +* fixed a memory leak inside ``ffi.getwinerror()`` on CPython 3.x. + + +v1.13.2 +======= + +* re-release because the Linux wheels came with an attached version of libffi + that was very old and buggy (`issue #432`_). + +.. _`issue #432`: https://foss.heptapod.net/pypy/cffi/-/issues/432 + + + +v1.13.1 +======= + +* deprecate the way to declare in ``cdef()`` a global variable with only + ``void *foo;``. You should always use a storage class, like ``extern void + *foo;`` or maybe ``static void *foo;``. These are all equivalent for + the purposes of ``cdef()``, but the reason for deprecating the bare version + is that (as far as I know) it would always be mistake in a real C header. + +* fix the regression ``RuntimeError: found a situation in which we try + to build a type recursively`` (`issue #429`_). + +* fixed `issue #427`_ where a multithreading mistake in the embedding logic + initialization code would cause deadlocks on CPython 3.7. + +.. _`issue #429`: https://foss.heptapod.net/pypy/cffi/-/issues/429 +.. _`issue #427`: https://foss.heptapod.net/pypy/cffi/-/issues/427 + + +v1.13 +===== + +* ``ffi.from_buffer("type *", ..)`` is now supported, in addition to + ``"type[]"``. You can then write ``p.field`` to access the items, instead + of only ``p[0].field``. Be careful that no bounds checking is performed, so + ``p[n]`` might access data out of bounds. + +* fix for structs containing unnamed bitfields like ``int : 1;``. + +* when calling cdata of "function pointer" type, give a RuntimeError instead + of a crash if the pointer happens to be NULL + +* support some more binary operations between constants in enum definitions + (PR #96) + +* silence a warning incorrectly emitted if you use a quote in a preprocessor + line + +* detect a corner case that would throw the C code into an infinite + recursion, with ``ffi.cdef("""struct X { void(*fnptr)(struct X); };""")`` + + +Older Versions +============== + +v1.12.3 +------- + +* Fix for nested struct types that end in a var-sized array (#405). + +* Add support for using ``U`` and ``L`` characters at the end of integer + constants in ``ffi.cdef()`` (thanks Guillaume). + +* More 3.8 fixes. + + +v1.12.2 +------- + * Added temporary workaround to compile on CPython 3.8.0a2. v1.12.1 -======= +------- * CPython 3 on Windows: we again no longer compile with ``Py_LIMITED_API`` by default because such modules *still* cannot be used with virtualenv. @@ -21,12 +197,12 @@ v1.12.1 Like before, `Issue #350`_ mentions a workaround if you still want the ``Py_LIMITED_API`` flag and *either* you are not concerned about virtualenv *or* you are sure your module will not be used on CPython - <= 3.4: pass ``define_macros=[("Py_LIMITED_API", None)]`` to the + <= 3.4: pass ``define_macros=[("Py_LIMITED_API", None)]`` as a keyword to the ``ffibuilder.set_source()`` call. v1.12 -===== +----- * `Direct support for pkg-config`__. @@ -61,11 +237,8 @@ v1.12 to 1 byte instead of 4). .. __: cdef.html#pkgconfig -.. _`issue #362`: https://bitbucket.org/cffi/cffi/issues/362/ - +.. _`issue #362`: https://foss.heptapod.net/pypy/cffi/-/issues/362 -Older Versions -============== v1.11.5 ------- @@ -93,13 +266,13 @@ v1.11.5 * CPython 3 on Windows: we no longer compile with ``Py_LIMITED_API`` by default because such modules cannot be used with virtualenv. `Issue #350`_ mentions a workaround if you still want that and are not - concerned about virtualenv: pass a ``define_macros=[("Py_LIMITED_API", - None)]`` to the ``ffibuilder.set_source()`` call. + concerned about virtualenv: pass ``define_macros=[("Py_LIMITED_API", + None)]`` as a keyword to the ``ffibuilder.set_source()`` call. -.. _`Issue #345`: https://bitbucket.org/cffi/cffi/issues/345/ -.. _`Issue #350`: https://bitbucket.org/cffi/cffi/issues/350/ -.. _`Issue #358`: https://bitbucket.org/cffi/cffi/issues/358/ -.. _`Issue #357`: https://bitbucket.org/cffi/cffi/issues/357/ +.. _`Issue #345`: https://foss.heptapod.net/pypy/cffi/-/issues/345 +.. _`Issue #350`: https://foss.heptapod.net/pypy/cffi/-/issues/350 +.. _`Issue #358`: https://foss.heptapod.net/pypy/cffi/-/issues/358 +.. _`Issue #357`: https://foss.heptapod.net/pypy/cffi/-/issues/357 v1.11.4 @@ -112,7 +285,7 @@ v1.11.4 ``foo.cp36-win32.pyd``, to make it clear that they are regular CPython modules depending on ``python36.dll``. -.. _`Issue #355`: https://bitbucket.org/cffi/cffi/issues/355/ +.. _`Issue #355`: https://foss.heptapod.net/pypy/cffi/-/issues/355 v1.11.3 @@ -201,9 +374,9 @@ v1.11 that are *slower* to call than the API mode does. For some reason it is often thought to be faster. It is not! -.. __: https://bitbucket.org/cffi/cffi/issues/321/cffi-191-segmentation-fault-during-self +.. __: https://foss.heptapod.net/pypy/cffi/-/issues/321 .. __: ref.html#ffi-gc -.. __: https://bitbucket.org/cffi/cffi/issues/320/improve-memory_pressure-management +.. __: https://foss.heptapod.net/pypy/cffi/-/issues/320 .. __: http://bugs.python.org/issue31105 |