diff options
Diffstat (limited to 'doc/source')
| -rw-r--r-- | doc/source/cdef.rst | 1012 | ||||
| -rw-r--r-- | doc/source/conf.py | 194 | ||||
| -rw-r--r-- | doc/source/embedding.rst | 531 | ||||
| -rw-r--r-- | doc/source/goals.rst | 69 | ||||
| -rw-r--r-- | doc/source/index.rst | 19 | ||||
| -rw-r--r-- | doc/source/installation.rst | 190 | ||||
| -rw-r--r-- | doc/source/overview.rst | 651 | ||||
| -rw-r--r-- | doc/source/ref.rst | 1028 | ||||
| -rw-r--r-- | doc/source/using.rst | 1043 | ||||
| -rw-r--r-- | doc/source/whatsnew.rst | 887 |
10 files changed, 0 insertions, 5624 deletions
diff --git a/doc/source/cdef.rst b/doc/source/cdef.rst deleted file mode 100644 index 0662668..0000000 --- a/doc/source/cdef.rst +++ /dev/null @@ -1,1012 +0,0 @@ -====================================== -Preparing and Distributing modules -====================================== - -.. contents:: - -There are three or four different ways to use CFFI in a project. -In order of complexity: - -* The **"in-line", "ABI mode"**: - - .. code-block:: python - - import cffi - - ffi = cffi.FFI() - ffi.cdef("C-like declarations") - lib = ffi.dlopen("libpath") - - # use ffi and lib here - -.. _out-of-line-abi: - -* The **"out-of-line",** but still **"ABI mode",** useful to organize - the code and reduce the import time: - - .. code-block:: python - - # in a separate file "package/foo_build.py" - import cffi - - ffibuilder = cffi.FFI() - ffibuilder.set_source("package._foo", None) - ffibuilder.cdef("C-like declarations") - - if __name__ == "__main__": - ffibuilder.compile() - - Running ``python foo_build.py`` produces a file ``_foo.py``, which - can then be imported in the main program: - - .. code-block:: python - - from package._foo import ffi - lib = ffi.dlopen("libpath") - - # use ffi and lib here - -.. _out-of-line-api: - -* The **"out-of-line", "API mode"** gives you the most flexibility - and speed to access a C library at the level of C, instead of at the - binary level: - - .. code-block:: python - - # in a separate file "package/foo_build.py" - import cffi - - ffibuilder = cffi.FFI() - ffibuilder.set_source("package._foo", r"""real C code""") # <= - ffibuilder.cdef("C-like declarations with '...'") - - if __name__ == "__main__": - ffibuilder.compile(verbose=True) - - Running ``python foo_build.py`` produces a file ``_foo.c`` and - invokes the C compiler to turn it into a file ``_foo.so`` (or - ``_foo.pyd`` or ``_foo.dylib``). It is a C extension module which - can be imported in the main program: - - .. code-block:: python - - from package._foo import ffi, lib - # no ffi.dlopen() - - # use ffi and lib here - -.. _distutils-setuptools: - -* Finally, you can (but don't have to) use CFFI's **Distutils** or - **Setuptools integration** when writing a ``setup.py``. For - Distutils (only in out-of-line API mode): - - .. code-block:: python - - # setup.py (requires CFFI to be installed first) - from distutils.core import setup - - import foo_build # possibly with sys.path tricks to find it - - setup( - ..., - ext_modules=[foo_build.ffibuilder.distutils_extension()], - ) - - For Setuptools (out-of-line, but works in ABI or API mode; - recommended): - - .. code-block:: python - - # setup.py (with automatic dependency tracking) - from setuptools import setup - - setup( - ..., - setup_requires=["cffi>=1.0.0"], - cffi_modules=["package/foo_build.py:ffibuilder"], - install_requires=["cffi>=1.0.0"], - ) - - Note again that the ``foo_build.py`` example contains the following - lines, which mean that the ``ffibuilder`` is not actually compiled - when ``package.foo_build`` is merely imported---it will be compiled - independently by the Setuptools logic, using compilation parameters - provided by Setuptools: - - .. code-block:: python - - if __name__ == "__main__": # not when running with setuptools - ffibuilder.compile(verbose=True) - -* Note that some bundler tools that try to find all modules used by a - project, like PyInstaller, will miss ``_cffi_backend`` in the - out-of-line mode because your program contains no explicit ``import - cffi`` or ``import _cffi_backend``. You need to add - ``_cffi_backend`` explicitly (as a "hidden import" in PyInstaller, - but it can also be done more generally by adding the line ``import - _cffi_backend`` in your main program). - -Note that CFFI actually contains two different ``FFI`` classes. The -page `Using the ffi/lib objects`_ describes the common functionality. -It is what you get in the ``from package._foo import ffi`` lines above. -On the other hand, the extended ``FFI`` class is the one you get from -``import cffi; ffi_or_ffibuilder = cffi.FFI()``. It has the same -functionality (for in-line use), but also the extra methods described -below (to prepare the FFI). NOTE: We use the name ``ffibuilder`` -instead of ``ffi`` in the out-of-line context, when the code is about -producing a ``_foo.so`` file; this is an attempt to distinguish it -from the different ``ffi`` object that you get by later saying -``from _foo import ffi``. - -.. _`Using the ffi/lib objects`: using.html - -The reason for this split of functionality is that a regular program -using CFFI out-of-line does not need to import the ``cffi`` pure -Python package at all. (Internally it still needs ``_cffi_backend``, -a C extension module that comes with CFFI; this is why CFFI is also -listed in ``install_requires=..`` above. In the future this might be -split into a different PyPI package that only installs -``_cffi_backend``.) - -Note that a few small differences do exist: notably, ``from _foo import -ffi`` returns an object of a type written in C, which does not let you -add random attributes to it (nor does it have all the -underscore-prefixed internal attributes of the Python version). -Similarly, the ``lib`` objects returned by the C version are read-only, -apart from writes to global variables. Also, ``lib.__dict__`` does -not work before version 1.2 or if ``lib`` happens to declare a name -called ``__dict__`` (use instead ``dir(lib)``). The same is true -for ``lib.__class__``, ``lib.__all__`` and ``lib.__name__`` added -in successive versions. - - -.. _cdef: - -ffi/ffibuilder.cdef(): declaring types and functions ----------------------------------------------------- - -**ffi/ffibuilder.cdef(source)**: parses the given C source. -It registers all the functions, types, constants and global variables in -the C source. The types can be used immediately in ``ffi.new()`` and -other functions. Before you can access the functions and global -variables, you need to give ``ffi`` another piece of information: where -they actually come from (which you do with either ``ffi.dlopen()`` or -``ffi.set_source()``). - -.. _`all types listed above`: - -The C source is parsed internally (using ``pycparser``). This code -cannot contain ``#include``. It should typically be a self-contained -piece of declarations extracted from a man page. The only things it -can assume to exist are the standard types: - -* char, short, int, long, long long (both signed and unsigned) - -* float, double, long double - -* intN_t, uintN_t (for N=8,16,32,64), intptr_t, uintptr_t, ptrdiff_t, - size_t, ssize_t - -* wchar_t (if supported by the backend). *New in version 1.11:* - char16_t and char32_t. - -* _Bool and bool (equivalent). If not directly supported by the C - compiler, this is declared with the size of ``unsigned char``. - -* FILE. `See here.`__ - -* all `common Windows types`_ are defined if you run - on Windows (``DWORD``, ``LPARAM``, etc.). Exception: - ``TBYTE TCHAR LPCTSTR PCTSTR LPTSTR PTSTR PTBYTE PTCHAR`` are - not automatically defined; see `ffi.set_unicode()`_. - -* the other standard integer types from - stdint.h, like ``intmax_t``, as long as they map to integers of 1, - 2, 4 or 8 bytes. Larger integers are not supported. - -.. __: ref.html#file -.. _`common Windows types`: http://msdn.microsoft.com/en-us/library/windows/desktop/aa383751%28v=vs.85%29.aspx - -The declarations can also contain "``...``" at various places; these are -placeholders that will be completed by the compiler. More information -about it below in `Letting the C compiler fill the gaps`_. - -Note that all standard type names listed above are handled as -*defaults* only (apart from the ones that are keywords in the C -language). If your ``cdef`` contains an explicit typedef that -redefines one of the types above, then the default described above is -ignored. (This is a bit hard to implement cleanly, so in some corner -cases it might fail, notably with the error ``Multiple type specifiers -with a type tag``. Please report it as a bug if it does.) - -Multiple calls to ``ffi.cdef()`` are possible. Beware that it can be -slow to call ``ffi.cdef()`` a lot of times, a consideration that is -important mainly in in-line mode. - -The ``ffi.cdef()`` call optionally takes an extra argument: either -``packed`` or ``pack``. If you pass ``packed=True``, -then all structs declared within -this cdef are "packed". (If you need both packed and non-packed -structs, use several cdefs in sequence.) This -has a meaning similar to ``__attribute__((packed))`` in GCC. It -specifies that all structure fields should have an alignment of one -byte. (Note that the packed attribute has no effect on bit fields so -far, which mean that they may be packed differently than on GCC. -Also, this has no effect on structs declared with ``"...;"``---more -about it later in `Letting the C compiler fill the gaps`_. In -particular, if your C source uses other attributes like -``__attribute__((aligned(16)))``, there is no way to declare this fact -in the ``cdef()``, but you can generally just declare the struct with -``"...;"`` as the last field.) - -*New in version 1.12:* In ABI mode, you can also pass ``pack=n``, -with an integer ``n`` which must be a power of two. Then the -alignment of any field is limited to ``n`` if it would otherwise be -greater than ``n``. Passing ``pack=1`` is equivalent to passing -``packed=True``. This is meant to emulate ``#pragma pack(n)`` from -the MSVC compiler. On Windows, the default is ``pack=8`` (from cffi -1.12 onwards); on other platforms, the default is ``pack=None``. - -Note that you can use the type-qualifiers ``const`` and ``restrict`` -(but not ``__restrict`` or ``__restrict__``) in the ``cdef()``, but -this has no effect on the cdata objects that you get at run-time (they -are never ``const``). The effect is limited to knowing if a global -variable is meant to be a constant or not. Also, *new in version -1.3:* when using ``set_source()`` or ``verify()``, these two -qualifiers are copied from the cdef to the generated C code; this -fixes warnings by the C compiler. - -Note a trick if you copy-paste code from sources in which there are -extra macros (for example, the Windows documentation uses SAL -annotations like ``_In_`` or ``_Out_``). These hints must be removed -in the string given to cdef(), but it can be done programmatically -like this:: - - ffi.cdef(re.sub(r"\b(_In_|_Inout_|_Out_|_Outptr_)(opt_)?\b", " ", - """ - DWORD WINAPI GetModuleFileName( - _In_opt_ HMODULE hModule, - _Out_ LPTSTR lpFilename, - _In_ DWORD nSize - ); - """)) - -Note also that pycparser, the underlying C parser, recognizes -preprocessor-like directives in the following format: ``# NUMBER -"FILE"``. For example, if you put ``# 42 "foo.h"`` in the middle of the -string passed to ``cdef()`` and there is an error two lines later, then -it is reported with an error message that starts with ``foo.h:43:`` (the -line which is given the number 42 is the line immediately after the -directive). *New in version 1.10.1:* CFFI automatically puts the line -``# 1 "<cdef source string>"`` just before the string you give to -``cdef()``. - - -.. _`ffi.set_unicode()`: - -**ffi.set_unicode(enabled_flag)**: Windows: if ``enabled_flag`` is -True, enable the ``UNICODE`` and ``_UNICODE`` defines in C, and -declare the types ``TBYTE TCHAR LPCTSTR PCTSTR LPTSTR PTSTR PTBYTE -PTCHAR`` to be (pointers to) ``wchar_t``. If ``enabled_flag`` is -False, declare these types to be (pointers to) plain 8-bit characters. -(These types are not predeclared at all if you don't call -``set_unicode()``.) - -The reason behind this method is that a lot of standard functions have -two versions, like ``MessageBoxA()`` and ``MessageBoxW()``. The -official interface is ``MessageBox()`` with arguments like -``LPTCSTR``. Depending on whether ``UNICODE`` is defined or not, the -standard header renames the generic function name to one of the two -specialized versions, and declares the correct (unicode or not) types. - -Usually, the right thing to do is to call this method with True. Be -aware (particularly on Python 2) that, afterwards, you need to pass unicode -strings as arguments instead of byte strings. - - -.. _loading-libraries: -.. _dlopen: - -ffi.dlopen(): loading libraries in ABI mode -------------------------------------------- - -``ffi.dlopen(libpath, [flags])``: this function opens a shared library and -returns a module-like library object. Use this when you are fine with -the limitations of ABI-level access to the system (dependency on ABI -details, getting crashes instead of C compiler errors/warnings, and -higher overhead to call the C functions). In case of doubt, read again -`ABI versus API`_ in the overview. - -.. _`ABI versus API`: overview.html#abi-versus-api - -You can use the library object to call the functions previously -declared by ``ffi.cdef()``, to read constants, and to read or write -global variables. Note that you can use a single ``cdef()`` to -declare functions from multiple libraries, as long as you load each of -them with ``dlopen()`` and access the functions from the correct one. - -The ``libpath`` is the file name of the shared library, which can -contain a full path or not (in which case it is searched in standard -locations, as described in ``man dlopen``), with extensions or not. -Alternatively, if ``libpath`` is None, it returns the standard C library -(which can be used to access the functions of glibc, on Linux). Note -that ``libpath`` `cannot be None`__ on Windows with Python 3. - -.. __: http://bugs.python.org/issue23606 - -Let me state it again: this gives ABI-level access to the library, so -you need to have all types declared manually exactly as they were -while the library was made. No checking is done. Mismatches can -cause random crashes. API-level access, on the other hand, is safer. -Speed-wise, API-level access is much faster (it is common to have -the opposite misconception about performance). - -Note that only functions and global variables live in library objects; -the types exist in the ``ffi`` instance independently of library objects. -This is due to the C model: the types you declare in C are not tied to a -particular library, as long as you ``#include`` their headers; but you -cannot call functions from a library without linking it in your program, -as ``dlopen()`` does dynamically in C. - -For the optional ``flags`` argument, see ``man dlopen`` (ignored on -Windows). It defaults to ``ffi.RTLD_NOW``. - -This function returns a "library" object that gets closed when it goes -out of scope. Make sure you keep the library object around as long as -needed. (Alternatively, the out-of-line FFIs have a method -``ffi.dlclose(lib)``.) - -.. _dlopen-note: - -Note: the old version of ``ffi.dlopen()`` from the in-line ABI mode -tries to use ``ctypes.util.find_library()`` if it cannot directly find -the library. The newer out-of-line ``ffi.dlopen()`` no longer does it -automatically; it simply passes the argument it receives to the -underlying ``dlopen()`` or ``LoadLibrary()`` function. If needed, it -is up to you to use ``ctypes.util.find_library()`` or any other way to -look for the library's filename. This also means that -``ffi.dlopen(None)`` no longer work on Windows; try instead -``ffi.dlopen(ctypes.util.find_library('c'))``. - -*New in version 1.14:* ``ffi.dlopen(handle)``: instead of a file path, -you can give an already-opened library handle, as a cdata of type -``void *``. Such a call converts this handle into a regular FFI object -with the functions and global variables declared by ``ffi.cdef()``. -Useful if you have special needs (e.g. you need the GNU extension -``dlmopen()``, which you can itself declare and call using a different -``ffi`` object). Note that in this variant, ``dlclose()`` is not called -automatically if the FFI object is garbage-collected (but you can still -call ``ffi.dlclose()`` explicitly if needed). - - -.. _set_source: - -ffibuilder.set_source(): preparing out-of-line modules ------------------------------------------------------- - -**ffibuilder.set_source(module_name, c_header_source, [\*\*keywords...])**: -prepare the ffi for producing out-of-line an external module called -``module_name``. - -``ffibuilder.set_source()`` by itself does not write any file, but merely -records its arguments for later. It can therefore be called before or -after ``ffibuilder.cdef()``. - -In **ABI mode,** you call ``ffibuilder.set_source(module_name, None)``. The -argument is the name (or dotted name inside a package) of the Python -module to generate. In this mode, no C compiler is called. - -In **API mode,** the ``c_header_source`` argument is a string that -will be pasted into the .c file generated. Typically, it is specified as -``r""" ...multiple lines of C code... """`` (the ``r`` prefix allows these -lines to contain a literal ``\n``, for example). This piece of C code -typically contains some ``#include``, but may also contain more, -like definitions for custom "wrapper" C functions. The goal is that -the .c file can be generated like this:: - - // C file "module_name.c" - #include <Python.h> - - ...c_header_source... - - ...magic code... - -where the "magic code" is automatically generated from the ``cdef()``. -For example, if the ``cdef()`` contains ``int foo(int x);`` then the -magic code will contain logic to call the function ``foo()`` with an -integer argument, itself wrapped inside some CPython or PyPy-specific -code. - -The keywords arguments to ``set_source()`` control how the C compiler -will be called. They are passed directly to distutils_ or setuptools_ -and include at least ``sources``, ``include_dirs``, ``define_macros``, -``undef_macros``, ``libraries``, ``library_dirs``, ``extra_objects``, -``extra_compile_args`` and ``extra_link_args``. You typically need at -least ``libraries=['foo']`` in order to link with ``libfoo.so`` or -``libfoo.so.X.Y``, or ``foo.dll`` on Windows. The ``sources`` is a -list of extra .c files compiled and linked together (the file -``module_name.c`` shown above is always generated and automatically added as the -first argument to ``sources``). See the distutils documentations for -`more information about the other arguments`__. - -.. __: http://docs.python.org/distutils/setupscript.html#library-options -.. _distutils: http://docs.python.org/distutils/setupscript.html#describing-extension-modules -.. _setuptools: https://pythonhosted.org/setuptools/setuptools.html - -An extra keyword argument processed internally is -``source_extension``, defaulting to ``".c"``. The file generated will -be actually called ``module_name + source_extension``. Example for -C++ (but note that there are still a few known issues of C-versus-C++ -compatibility): - -.. code-block:: python - - ffibuilder.set_source("mymodule", r''' - extern "C" { - int somefunc(int somearg) { return real_cpp_func(somearg); } - } - ''', source_extension='.cpp') - -.. _pkgconfig: - -**ffibuilder.set_source_pkgconfig(module_name, pkgconfig_libs, -c_header_source, [\*\*keywords...])**: - -*New in version 1.12.* This is equivalent to ``set_source()`` but it -first calls the system utility ``pkg-config`` with the package names -given in the list ``pkgconfig_libs``. It collects the information -obtained in this way and adds it to the explicitly-provided -``**keywords`` (if any). This should probably not be used on Windows. - -If the ``pkg-config`` program is not installed or does not know about -the requested library, the call fails with ``cffi.PkgConfigError``. If -necessary, you can catch this error and try to call ``set_source()`` -directly. (Ideally, you should also do that if the ``ffibuilder`` -instance has no method ``set_source_pkgconfig()``, to support older -versions of cffi.) - - -Letting the C compiler fill the gaps ------------------------------------- - -If you are using a C compiler ("API mode"), then: - -* functions taking or returning integer or float-point arguments can be - misdeclared: if e.g. a function is declared by ``cdef()`` as taking a - ``int``, but actually takes a ``long``, then the C compiler handles the - difference. - -* other arguments are checked: you get a compilation warning or error - if you pass a ``int *`` argument to a function expecting a ``long *``. - -* similarly, most other things declared in the ``cdef()`` are checked, - to the best we implemented so far; mistakes give compilation - warnings or errors. - -Moreover, you can use "``...``" (literally, dot-dot-dot) in the -``cdef()`` at various places, in order to ask the C compiler to fill -in the details. These places are: - -* structure declarations: any ``struct { }`` or ``union { }`` that ends - with "``...;``" as the last "field" is partial: it may be missing - fields, have them declared out of order, use non-standard alignment, - etc. Precisely, the field offsets, total struct size, and total - struct alignment deduced by looking at the ``cdef`` are not relied - upon and will instead be corrected by the compiler. (But note that you - can only access fields that you declared, not others.) Any ``struct`` - declaration which doesn't use "``...``" is assumed to be exact, but this is - checked: you get an error if it is not correct. - -* integer types: the syntax "``typedef - int... foo_t;``" declares the type ``foo_t`` as an integer type - whose exact size and signedness is not specified. The compiler will - figure it out. (Note that this requires ``set_source()``; it does - not work with ``verify()``.) The ``int...`` can be replaced with - ``long...`` or ``unsigned long long...`` or any other primitive - integer type, with no effect. The type will always map to one of - ``(u)int(8,16,32,64)_t`` in Python, but in the generated C code, - only ``foo_t`` is used. - -* *New in version 1.3:* floating-point types: "``typedef - float... foo_t;``" (or equivalently "``typedef double... foo_t;``") - declares ``foo_t`` as a-float-or-a-double; the compiler will figure - out which it is. Note that if the actual C type is even larger - (``long double`` on some platforms), then compilation will fail. - The problem is that the Python "float" type cannot be used to store - the extra precision. (Use the non-dot-dot-dot syntax ``typedef long - double foo_t;`` as usual, which returns values that are not Python - floats at all but cdata "long double" objects.) - -* unknown types: the syntax "``typedef ... foo_t;``" declares the type - ``foo_t`` as opaque. Useful mainly for when the API takes and returns - ``foo_t *`` without you needing to look inside the ``foo_t``. Also - works with "``typedef ... *foo_p;``" which declares the pointer type - ``foo_p`` without giving a name to the opaque type itself. Note that - such an opaque struct has no known size, which prevents some operations - from working (mostly like in C). *You cannot use this syntax to - declare a specific type, like an integer type! It declares opaque - struct-like types only.* In some cases you need to say that - ``foo_t`` is not opaque, but just a struct where you don't know any - field; then you would use "``typedef struct { ...; } foo_t;``". - -* array lengths: when used as structure fields or in global variables, - arrays can have an unspecified length, as in "``extern int n[...];``". The - length is completed by the C compiler. - This is slightly different from "``extern int n[];``", because the latter - means that the length is not known even to the C compiler, and thus - no attempt is made to complete it. This supports - multidimensional arrays: "``extern int n[...][...];``". - - *New in version 1.2:* "``extern int m[][...];``", i.e. ``...`` can be used - in the innermost dimensions without being also used in the outermost - dimension. In the example given, the length of the ``m`` array is - assumed not to be known to the C compiler, but the length of every - item (like the sub-array ``m[0]``) is always known the C compiler. - In other words, only the outermost dimension can be specified as - ``[]``, both in C and in CFFI, but any dimension can be given as - ``[...]`` in CFFI. - -* enums: if you don't know the exact order (or values) of the declared - constants, then use this syntax: "``enum foo { A, B, C, ... };``" - (with a trailing "``...``"). The C compiler will be used to figure - out the exact values of the constants. An alternative syntax is - "``enum foo { A=..., B, C };``" or even - "``enum foo { A=..., B=..., C=... };``". Like - with structs, an ``enum`` without "``...``" is assumed to - be exact, and this is checked. - -* integer constants and macros: you can write in the ``cdef`` the line - "``#define FOO ...``", with any macro name FOO but with ``...`` as - a value. Provided the macro - is defined to be an integer value, this value will be available via - an attribute of the library object. The - same effect can be achieved by writing a declaration - ``static const int FOO;``. The latter is more general because it - supports other types than integer types (note: the C syntax is then - to write the ``const`` together with the variable name, as in - ``static char *const FOO;``). - -Currently, it is not supported to find automatically which of the -various integer or float types you need at which place---except in the -following case: if such a type is explicitly named. For an integer -type, use ``typedef int... the_type_name;``, or another type like -``typedef unsigned long... the_type_name;``. Both are equivalent and -replaced by the real C type, which must be an integer type. -Similarly, for floating-point types, use ``typedef float... -the_type_name;`` or equivalently ``typedef double... the_type_name;``. -Note that ``long double`` cannot be detected this way. - -In the case of function arguments or return types, when it is a simple -integer/float type, you can simply misdeclare it. If you misdeclare a -function ``void f(long)`` as ``void f(int)``, it still works (but you -have to call it with arguments that fit an int). It works because the C -compiler will do the casting for us. This C-level casting of arguments -and return types only works for regular function, and not for function -pointer types; currently, it also does not work for variadic functions. - -For more complex types, you have no choice but be precise. For example, -you cannot misdeclare a ``int *`` argument as ``long *``, or a global -array ``extern int a[5];`` as ``extern long a[5];``. CFFI considers `all types listed -above`_ as primitive (so ``extern long long a[5];`` and ``extern int64_t a[5]`` are -different declarations). The reason for that is detailed in `a comment -about an issue.`__ - -.. __: https://foss.heptapod.net/pypy/cffi/-/issues/265#note_50393 - - -ffibuilder.compile() etc.: compiling out-of-line modules --------------------------------------------------------- - -You can use one of the following functions to actually generate the -.py or .c file prepared with ``ffibuilder.set_source()`` and -``ffibuilder.cdef()``. - -Note that these function won't overwrite a .py/.c file with exactly -the same content, to preserve the mtime. In some cases where you need -the mtime to be updated anyway, delete the file before calling the -functions. - -*New in version 1.8:* the C code produced by ``emit_c_code()`` or -``compile()`` contains ``#define Py_LIMITED_API``. This means that on -CPython >= 3.2, compiling this source produces a binary .so/.dll that -should work for any version of CPython >= 3.2 (as opposed to only for -the same version of CPython x.y). However, the standard ``distutils`` -package will still produce a file called e.g. -``NAME.cpython-35m-x86_64-linux-gnu.so``. You can manually rename it to -``NAME.abi3.so``, or use setuptools version 26 or later. Also, note -that compiling with a debug version of Python will not actually define -``Py_LIMITED_API``, as doing so makes ``Python.h`` unhappy. - -*New in version 1.12:* ``Py_LIMITED_API`` is now defined on Windows too. -If you use ``virtualenv``, you need a recent version of it: versions -older than 16.0.0 forgot to copy ``python3.dll`` into the virtual -environment. In case upgrading ``virtualenv`` is a real problem, you -can manually edit the C code to remove the first line ``# define -Py_LIMITED_API``. - -**ffibuilder.compile(tmpdir='.', verbose=False, debug=None):** -explicitly generate the .py or .c file, -and (if .c) compile it. The output file is (or are) put in the -directory given by ``tmpdir``. In the examples given here, we use -``if __name__ == "__main__": ffibuilder.compile()`` in the build scripts---if -they are directly executed, this makes them rebuild the .py/.c file in -the current directory. (Note: if a package is specified in the call -to ``set_source()``, then a corresponding subdirectory of the ``tmpdir`` -is used.) - -*New in version 1.4:* ``verbose`` argument. If True, it prints the -usual distutils output, including the command lines that call the -compiler. (This parameter might be changed to True by default in a -future release.) - -*New in version 1.8.1:* ``debug`` argument. If set to a bool, it -controls whether the C code is compiled in debug mode or not. The -default None means to use the host Python's ``sys.flags.debug``. -Starting with version 1.8.1, if you are running a debug-mode Python, the -C code is thus compiled in debug mode by default (note that it is anyway -necessary to do so on Windows). - -**ffibuilder.emit_python_code(filename):** generate the given .py file (same -as ``ffibuilder.compile()`` for ABI mode, with an explicitly-named file to -write). If you choose, you can include this .py file pre-packaged in -your own distributions: it is identical for any Python version (2 or -3). - -**ffibuilder.emit_c_code(filename):** generate the given .c file (for API -mode) without compiling it. Can be used if you have some other method -to compile it, e.g. if you want to integrate with some larger build -system that will compile this file for you. You can also distribute -the .c file: unless the build script you used depends on the OS or -platform, the .c file itself is generic (it would be exactly the same -if produced on a different OS, with a different version of CPython, or -with PyPy; it is done with generating the appropriate ``#ifdef``). - -**ffibuilder.distutils_extension(tmpdir='build', verbose=True):** for -distutils-based ``setup.py`` files. Calling this creates the .c file -if needed in the given ``tmpdir``, and returns a -``distutils.core.Extension`` instance. - -For Setuptools, you use instead the line -``cffi_modules=["path/to/foo_build.py:ffibuilder"]`` in ``setup.py``. This -line asks Setuptools to import and use a helper provided by CFFI, -which in turn executes the file ``path/to/foo_build.py`` (as with -``execfile()``) and looks up its global variable called ``ffibuilder``. You -can also say ``cffi_modules=["path/to/foo_build.py:maker"]``, where -``maker`` names a global function; it is called with no argument and -is supposed to return a ``FFI`` object. - - -ffi/ffibuilder.include(): combining multiple CFFI interfaces ------------------------------------------------------------- - -**ffi/ffibuilder.include(other_ffi)**: includes the typedefs, structs, unions, -enums and constants defined in another FFI instance. This is meant -for large projects where one CFFI-based interface depends on some -types declared in a different CFFI-based interface. - -*Note that you should only use one ffi object per library; the intended -usage of ffi.include() is if you want to interface with several -inter-dependent libraries.* For only one library, make one ``ffi`` -object. (You can write several ``cdef()`` calls over the same ``ffi`` -from several Python files, if one file would be too large.) - -For out-of-line modules, the ``ffibuilder.include(other_ffibuilder)`` -line should -occur in the build script, and the ``other_ffibuilder`` argument should be -another FFI instance that comes from another build script. When the two build -scripts are turned into generated files, say ``_ffi.so`` and -``_other_ffi.so``, then importing ``_ffi.so`` will internally cause -``_other_ffi.so`` to be imported. At that point, the real -declarations from ``_other_ffi.so`` are combined with the real -declarations from ``_ffi.so``. - -The usage of ``ffi.include()`` is the cdef-level equivalent of a -``#include`` in C, where a part of the program might include types and -functions defined in another part for its own usage. You can see on -the ``ffi`` object (and associated ``lib`` objects on the *including* -side) the types and constants declared on the included side. In API -mode, you can also see the functions and global variables directly. -In ABI mode, these must be accessed via the original ``other_lib`` -object returned by the ``dlopen()`` method on ``other_ffi``. - - -ffi.cdef() limitations ----------------------- - -All of the ANSI C *declarations* should be supported in ``cdef()``, -and some of C99. (This excludes any ``#include`` or ``#ifdef``.) -Known missing features that are either in C99, or are GCC or MSVC -extensions: - -* Any ``__attribute__`` or ``#pragma pack(n)`` - -* Additional types: special-size floating and fixed - point types, vector types, and so on. - -* The C99 types ``float _Complex`` and ``double _Complex`` are supported - by cffi since version 1.11, but not libffi: you cannot call C - functions with complex arguments or return value, except if they are - directly API-mode functions. The type ``long double _Complex`` is not - supported at all (declare and use it as if it were an array of two - ``long double``, and write wrapper functions in C with set_source()). - -* ``__restrict__`` or ``__restrict`` are extensions of, respectively, - GCC and MSVC. They are not recognized. But ``restrict`` is a C - keyword and is accepted (and ignored). - -Note that declarations like ``int field[];`` in -structures are interpreted as variable-length structures. Declarations -like ``int field[...];`` on the other hand are arrays whose length is -going to be completed by the compiler. You can use ``int field[];`` -for array fields that are not, in fact, variable-length; it works too, -but in this case, as CFFI -believes it cannot ask the C compiler for the length of the array, you -get reduced safety checks: for example, you risk overwriting the -following fields by passing too many array items in the constructor. - -*New in version 1.2:* -Thread-local variables (``__thread``) can be accessed, as well as -variables defined as dynamic macros (``#define myvar (*fetchme())``). -Before version 1.2, you need to write getter/setter functions. - -Note that if you declare a variable in ``cdef()`` without using -``const``, CFFI assumes it is a read-write variable and generates two -pieces of code, one to read it and one to write it. If the variable -cannot in fact be written to in C code, for one reason or another, it -will not compile. In this case, you can declare it as a constant: for -example, instead of ``foo_t *myglob;`` you would use ``foo_t *const -myglob;``. Note also that ``const foo_t *myglob;`` is a *variable;* it -contains a variable pointer to a constant ``foo_t``. - - -Debugging dlopen'ed C libraries -------------------------------- - -A few C libraries are actually hard to use correctly in a ``dlopen()`` -setting. This is because most C libraries are intended for, and tested -with, a situation where they are *linked* with another program, using -either static linking or dynamic linking --- but from a program written -in C, at start-up, using the linker's capabilities instead of -``dlopen()``. - -This can occasionally create issues. You would have the same issues in -another setting than CFFI, like with ``ctypes`` or even plain C code that -calls ``dlopen()``. This section contains a few generally useful -environment variables (on Linux) that can help when debugging these -issues. - -**export LD_TRACE_LOADED_OBJECTS=all** - - provides a lot of information, sometimes too much depending on the - setting. Output verbose debugging information about the dynamic - linker. If set to ``all`` prints all debugging information it has, if - set to ``help`` prints a help message about which categories can be - specified in this environment variable - -**export LD_VERBOSE=1** - - (glibc since 2.1) If set to a nonempty string, output symbol - versioning information about the program if querying information - about the program (i.e., either ``LD_TRACE_LOADED_OBJECTS`` has been set, - or ``--list`` or ``--verify`` options have been given to the dynamic - linker). - -**export LD_WARN=1** - - (ELF only)(glibc since 2.1.3) If set to a nonempty string, warn - about unresolved symbols. - - -ffi.verify(): in-line API-mode ------------------------------- - -**ffi.verify()** is supported for backward compatibility, but is -deprecated. ``ffi.verify(c_header_source, tmpdir=.., ext_package=.., -modulename=.., flags=.., **kwargs)`` makes and compiles a C file from -the ``ffi.cdef()``, like ``ffi.set_source()`` in API mode, and then -immediately loads and returns the dynamic library object. Some -non-trivial logic is used to decide if the dynamic library must be -recompiled or not; see below for ways to control it. - -The ``c_header_source`` and the extra keyword arguments have the -same meaning as in ``ffi.set_source()``. - -One remaining use case for ``ffi.verify()`` would be the following -hack to find explicitly the size of any type, in bytes, and have it -available in Python immediately (e.g. because it is needed in order to -write the rest of the build script): - -.. code-block:: python - - ffi = cffi.FFI() - ffi.cdef("const int mysize;") - lib = ffi.verify("const int mysize = sizeof(THE_TYPE);") - print lib.mysize - -Extra arguments to ``ffi.verify()``: - -* ``tmpdir`` controls where the C - files are created and compiled. Unless the ``CFFI_TMPDIR`` environment - variable is set, the default is - ``directory_containing_the_py_file/__pycache__`` using the - directory name of the .py file that contains the actual call to - ``ffi.verify()``. (This is a bit of a hack but is generally - consistent with the location of the .pyc files for your library. - The name ``__pycache__`` itself comes from Python 3.) - -* ``ext_package`` controls in which package the - compiled extension module should be looked from. This is - only useful after distributing ffi.verify()-based modules. - -* The ``tag`` argument gives an extra string inserted in the - middle of the extension module's name: ``_cffi_<tag>_<hash>``. - Useful to give a bit more context, e.g. when debugging. - -* The ``modulename`` argument can be used to force a specific module - name, overriding the name ``_cffi_<tag>_<hash>``. Use with care, - e.g. if you are passing variable information to ``verify()`` but - still want the module name to be always the same (e.g. absolute - paths to local files). In this case, no hash is computed and if - the module name already exists it will be reused without further - check. Be sure to have other means of clearing the ``tmpdir`` - whenever you change your sources. - -* ``source_extension`` has the same meaning as in ``ffibuilder.set_source()``. - -* The optional ``flags`` argument (ignored on Windows) defaults to - ``ffi.RTLD_NOW``; see ``man dlopen``. (With - ``ffibuilder.set_source()``, you would use ``sys.setdlopenflags()``.) - -* The optional ``relative_to`` argument is useful if you need to list - local files passed to the C compiler:: - - ext = ffi.verify(..., sources=['foo.c'], relative_to=__file__) - - The line above is roughly the same as:: - - ext = ffi.verify(..., sources=['/path/to/this/file/foo.c']) - - except that the default name of the produced library is built from - the CRC checkum of the argument ``sources``, as well as most other - arguments you give to ``ffi.verify()`` -- but not ``relative_to``. - So if you used the second line, it would stop finding the - already-compiled library after your project is installed, because - the ``'/path/to/this/file'`` suddenly changed. The first line does - not have this problem. - -Note that during development, every time you change the C sources that -you pass to ``cdef()`` or ``verify()``, then the latter will create a -new module file name, based on two CRC32 hashes computed from these -strings. This creates more and more files in the ``__pycache__`` -directory. It is recommended that you clean it up from time to time. -A nice way to do that is to add, in your test suite, a call to -``cffi.verifier.cleanup_tmpdir()``. Alternatively, you can manually -remove the whole ``__pycache__`` directory. - -An alternative cache directory can be given as the ``tmpdir`` argument -to ``verify()``, via the environment variable ``CFFI_TMPDIR``, or by -calling ``cffi.verifier.set_tmpdir(path)`` prior to calling -``verify``. - - -Upgrading from CFFI 0.9 to CFFI 1.0 ------------------------------------ - -CFFI 1.0 is backward-compatible, but it is still a good idea to -consider moving to the out-of-line approach new in 1.0. Here are the -steps. - -**ABI mode** if your CFFI project uses ``ffi.dlopen()``: - -.. code-block:: python - - import cffi - - ffi = cffi.FFI() - ffi.cdef("stuff") - lib = ffi.dlopen("libpath") - -and *if* the "stuff" part is big enough that import time is a concern, -then rewrite it as described in `the out-of-line but still ABI mode`__ -above. Optionally, see also the `setuptools integration`__ paragraph. - -.. __: out-of-line-abi_ -.. __: distutils-setuptools_ - - -**API mode** if your CFFI project uses ``ffi.verify()``: - -.. code-block:: python - - import cffi - - ffi = cffi.FFI() - ffi.cdef("stuff") - lib = ffi.verify("real C code") - -then you should really rewrite it as described in `the out-of-line, -API mode`__ above. It avoids a number of issues that have caused -``ffi.verify()`` to grow a number of extra arguments over time. Then -see the `distutils or setuptools`__ paragraph. Also, remember to -remove the ``ext_package=".."`` from your ``setup.py``, which was -sometimes needed with ``verify()`` but is just creating confusion with -``set_source()``. - -.. __: out-of-line-api_ -.. __: distutils-setuptools_ - -The following example should work both with old (pre-1.0) and new -versions of CFFI---supporting both is important to run on old -versions of PyPy (CFFI 1.0 does not work in PyPy < 2.6): - -.. code-block:: python - - # in a separate file "package/foo_build.py" - import cffi - - ffi = cffi.FFI() - C_HEADER_SRC = r''' - #include "somelib.h" - ''' - C_KEYWORDS = dict(libraries=['somelib']) - - if hasattr(ffi, 'set_source'): - ffi.set_source("package._foo", C_HEADER_SRC, **C_KEYWORDS) - - ffi.cdef(''' - int foo(int); - ''') - - if __name__ == "__main__": - ffi.compile() - -And in the main program: - -.. code-block:: python - - try: - from package._foo import ffi, lib - except ImportError: - from package.foo_build import ffi, C_HEADER_SRC, C_KEYWORDS - lib = ffi.verify(C_HEADER_SRC, **C_KEYWORDS) - -(FWIW, this latest trick can be used more generally to allow the -import to "work" even if the ``_foo`` module was not generated.) - -Writing a ``setup.py`` script that works both with CFFI 0.9 and 1.0 -requires explicitly checking the version of CFFI that we can have---it -is hard-coded as a built-in module in PyPy: - -.. code-block:: python - - if '_cffi_backend' in sys.builtin_module_names: # PyPy - import _cffi_backend - requires_cffi = "cffi==" + _cffi_backend.__version__ - else: - requires_cffi = "cffi>=1.0.0" - -Then we use the ``requires_cffi`` variable to give different arguments to -``setup()`` as needed, e.g.: - -.. code-block:: python - - if requires_cffi.startswith("cffi==0."): - # backward compatibility: we have "cffi==0.*" - from package.foo_build import ffi - extra_args = dict( - ext_modules=[ffi.verifier.get_extension()], - ext_package="...", # if needed - ) - else: - extra_args = dict( - setup_requires=[requires_cffi], - cffi_modules=['package/foo_build.py:ffi'], - ) - setup( - name=..., - ..., - install_requires=[requires_cffi], - **extra_args - ) diff --git a/doc/source/conf.py b/doc/source/conf.py deleted file mode 100644 index 33e8c11..0000000 --- a/doc/source/conf.py +++ /dev/null @@ -1,194 +0,0 @@ -# -*- coding: utf-8 -*- -# -# CFFI documentation build configuration file, created by -# sphinx-quickstart on Thu Jun 14 16:37:47 2012. -# -# This file is execfile()d with the current directory set to its containing dir. -# -# Note that not all possible configuration values are present in this -# autogenerated file. -# -# All configuration values have a default; values that are commented out -# serve to show the default. - -import sys, os - -# If extensions (or modules to document with autodoc) are in another directory, -# add these directories to sys.path here. If the directory is relative to the -# documentation root, use os.path.abspath to make it absolute, like shown here. -#sys.path.append(os.path.abspath('.')) - -# -- General configuration ----------------------------------------------------- - -# Add any Sphinx extension module names here, as strings. They can be extensions -# coming with Sphinx (named 'sphinx.ext.*') or your custom ones. -extensions = ['sphinx.ext.autodoc'] - -# Add any paths that contain templates here, relative to this directory. -templates_path = ['_templates'] - -# The suffix of source filenames. -source_suffix = '.rst' - -# The encoding of source files. -#source_encoding = 'utf-8' - -# The master toctree document. -master_doc = 'index' - -# General information about the project. -project = u'CFFI' -copyright = u'2012-2018, Armin Rigo, Maciej Fijalkowski' - -# The version info for the project you're documenting, acts as replacement for -# |version| and |release|, also used in various other places throughout the -# built documents. -# -# The short X.Y version. -version = '1.15' -# The full version, including alpha/beta/rc tags. -release = '1.15.0' - -# The language for content autogenerated by Sphinx. Refer to documentation -# for a list of supported languages. -#language = None - -# There are two options for replacing |today|: either, you set today to some -# non-false value, then it is used: -#today = '' -# Else, today_fmt is used as the format for a strftime call. -#today_fmt = '%B %d, %Y' - -# List of documents that shouldn't be included in the build. -#unused_docs = [] - -# List of directories, relative to source directory, that shouldn't be searched -# for source files. -exclude_trees = [] - -# The reST default role (used for this markup: `text`) to use for all documents. -#default_role = None - -# If true, '()' will be appended to :func: etc. cross-reference text. -#add_function_parentheses = True - -# If true, the current module name will be prepended to all description -# unit titles (such as .. function::). -#add_module_names = True - -# If true, sectionauthor and moduleauthor directives will be shown in the -# output. They are ignored by default. -#show_authors = False - -# The name of the Pygments (syntax highlighting) style to use. -pygments_style = 'sphinx' - -# A list of ignored prefixes for module index sorting. -#modindex_common_prefix = [] - - -# -- Options for HTML output --------------------------------------------------- - -# The theme to use for HTML and HTML Help pages. Major themes that come with -# Sphinx are currently 'default' and 'sphinxdoc'. -#html_theme = 'default' - -# Theme options are theme-specific and customize the look and feel of a theme -# further. For a list of options available for each theme, see the -# documentation. -#html_theme_options = {} - -# Add any paths that contain custom themes here, relative to this directory. -#html_theme_path = [] - -# The name for this set of Sphinx documents. If None, it defaults to -# "<project> v<release> documentation". -#html_title = None - -# A shorter title for the navigation bar. Default is the same as html_title. -#html_short_title = None - -# The name of an image file (relative to this directory) to place at the top -# of the sidebar. -#html_logo = None - -# The name of an image file (within the static path) to use as favicon of the -# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 -# pixels large. -#html_favicon = None - -# Add any paths that contain custom static files (such as style sheets) here, -# relative to this directory. They are copied after the builtin static files, -# so a file named "default.css" will overwrite the builtin "default.css". -#html_static_path = ['_static'] - -# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, -# using the given strftime format. -#html_last_updated_fmt = '%b %d, %Y' - -# If true, SmartyPants will be used to convert quotes and dashes to -# typographically correct entities. -#html_use_smartypants = True - -# Custom sidebar templates, maps document names to template names. -#html_sidebars = {} - -# Additional templates that should be rendered to pages, maps page names to -# template names. -#html_additional_pages = {} - -# If false, no module index is generated. -#html_use_modindex = True - -# If false, no index is generated. -#html_use_index = True - -# If true, the index is split into individual pages for each letter. -#html_split_index = False - -# If true, links to the reST sources are added to the pages. -#html_show_sourcelink = True - -# If true, an OpenSearch description file will be output, and all pages will -# contain a <link> tag referring to it. The value of this option must be the -# base URL from which the finished HTML is served. -#html_use_opensearch = '' - -# If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml"). -#html_file_suffix = '' - -# Output file base name for HTML help builder. -htmlhelp_basename = 'CFFIdoc' - - -# -- Options for LaTeX output -------------------------------------------------- - -# The paper size ('letter' or 'a4'). -#latex_paper_size = 'letter' - -# The font size ('10pt', '11pt' or '12pt'). -#latex_font_size = '10pt' - -# Grouping the document tree into LaTeX files. List of tuples -# (source start file, target name, title, author, documentclass [howto/manual]). -latex_documents = [ - ('index', 'CFFI.tex', u'CFFI Documentation', - u'Armin Rigo, Maciej Fijalkowski', 'manual'), -] - -# The name of an image file (relative to this directory) to place at the top of -# the title page. -#latex_logo = None - -# For "manual" documents, if this is true, then toplevel headings are parts, -# not chapters. -#latex_use_parts = False - -# Additional stuff for the LaTeX preamble. -#latex_preamble = '' - -# Documents to append as an appendix to all manuals. -#latex_appendices = [] - -# If false, no module index is generated. -#latex_use_modindex = True diff --git a/doc/source/embedding.rst b/doc/source/embedding.rst deleted file mode 100644 index 8020f21..0000000 --- a/doc/source/embedding.rst +++ /dev/null @@ -1,531 +0,0 @@ -================================ -Using CFFI for embedding -================================ - -.. contents:: - -You can use CFFI to generate C code which exports the API of your choice -to any C application that wants to link with this C code. This API, -which you define yourself, ends up as the API of a ``.so/.dll/.dylib`` -library---or you can statically link it within a larger application. - -Possible use cases: - -* Exposing a library written in Python directly to C/C++ programs. - -* Using Python to make a "plug-in" for an existing C/C++ program that is - already written to load them. - -* Using Python to implement part of a larger C/C++ application (with - static linking). - -* Writing a small C/C++ wrapper around Python, hiding the fact that the - application is actually written in Python (to make a custom - command-line interface; for distribution purposes; or simply to make - it a bit harder to reverse-engineer the application). - -The general idea is as follows: - -* You write and execute a Python script, which produces a ``.c`` file - with the API of your choice (and optionally compile it into a - ``.so/.dll/.dylib``). The script also gives some Python code to be - "frozen" inside the ``.so``. - -* At runtime, the C application loads this ``.so/.dll/.dylib`` (or is - statically linked with the ``.c`` source) without having to know that - it was produced from Python and CFFI. - -* The first time a C function is called, Python is initialized and - the frozen Python code is executed. - -* The frozen Python code defines more Python functions that implement the - C functions of your API, which are then used for all subsequent C - function calls. - -One of the goals of this approach is to be entirely independent from -the CPython C API: no ``Py_Initialize()`` nor ``PyRun_SimpleString()`` -nor even ``PyObject``. It works identically on CPython and PyPy. - -This is entirely *new in version 1.5.* (PyPy contains CFFI 1.5 since -release 5.0.) - - -Usage ------ - -.. __: overview.html#embedding - -See the `paragraph in the overview page`__ for a quick introduction. -In this section, we explain every step in more details. We will use -here this slightly expanded example: - -.. code-block:: c - - /* file plugin.h */ - typedef struct { int x, y; } point_t; - extern int do_stuff(point_t *); - -.. code-block:: c - - /* file plugin.h, Windows-friendly version */ - typedef struct { int x, y; } point_t; - - /* When including this file from ffibuilder.set_source(), the - following macro is defined to '__declspec(dllexport)'. When - including this file directly from your C program, we define - it to 'extern __declspec(dllimport)' instead. - - With non-MSVC compilers we simply define it to 'extern'. - (The 'extern' is needed for sharing global variables; - functions would be fine without it. The macros always - include 'extern': you must not repeat it when using the - macros later.) - */ - #ifndef CFFI_DLLEXPORT - # if defined(_MSC_VER) - # define CFFI_DLLEXPORT extern __declspec(dllimport) - # else - # define CFFI_DLLEXPORT extern - # endif - #endif - - CFFI_DLLEXPORT int do_stuff(point_t *); - -.. code-block:: python - - # file plugin_build.py - import cffi - ffibuilder = cffi.FFI() - - with open('plugin.h') as f: - # read plugin.h and pass it to embedding_api(), manually - # removing the '#' directives and the CFFI_DLLEXPORT - data = ''.join([line for line in f if not line.startswith('#')]) - data = data.replace('CFFI_DLLEXPORT', '') - ffibuilder.embedding_api(data) - - ffibuilder.set_source("my_plugin", r''' - #include "plugin.h" - ''') - - ffibuilder.embedding_init_code(""" - from my_plugin import ffi - - @ffi.def_extern() - def do_stuff(p): - print("adding %d and %d" % (p.x, p.y)) - return p.x + p.y - """) - - ffibuilder.compile(target="plugin-1.5.*", verbose=True) - # or: ffibuilder.emit_c_code("my_plugin.c") - -Running the code above produces a *DLL*, i,e, a dynamically-loadable -library. It is a file with the extension ``.dll`` on Windows, -``.dylib`` on Mac OS/X, or ``.so`` on other platforms. As usual, it -is produced by generating some intermediate ``.c`` code and then -calling the regular platform-specific C compiler. See below__ for -some pointers to C-level issues with using the produced library. - -.. __: `Issues about using the .so`_ - -Here are some details about the methods used above: - -* **ffibuilder.embedding_api(source):** parses the given C source, which - declares functions that you want to be exported by the DLL. It can - also declare types, constants and global variables that are part of - the C-level API of your DLL. - - The functions that are found in ``source`` will be automatically - defined in the ``.c`` file: they will contain code that initializes - the Python interpreter the first time any of them is called, - followed by code to call the attached Python function (with - ``@ffi.def_extern()``, see next point). - - The global variables, on the other hand, are not automatically - produced. You have to write their definition explicitly in - ``ffibuilder.set_source()``, as regular C code (see the point after next). - -* **ffibuilder.embedding_init_code(python_code):** this gives - initialization-time Python source code. This code is copied - ("frozen") inside the DLL. At runtime, the code is executed when - the DLL is first initialized, just after Python itself is - initialized. This newly initialized Python interpreter has got an - extra "built-in" module that can be loaded magically without - accessing any files, with a line like "``from my_plugin import ffi, - lib``". The name ``my_plugin`` comes from the first argument to - ``ffibuilder.set_source()``. This module represents "the caller's C world" - from the point of view of Python. - - The initialization-time Python code can import other modules or - packages as usual. You may have typical Python issues like needing - to set up ``sys.path`` somehow manually first. - - For every function declared within ``ffibuilder.embedding_api()``, the - initialization-time Python code or one of the modules it imports - should use the decorator ``@ffi.def_extern()`` to attach a - corresponding Python function to it. - - If the initialization-time Python code fails with an exception, then - you get a traceback printed to stderr, along with more information - to help you identify problems like wrong ``sys.path``. If some - function remains unattached at the time where the C code tries to - call it, an error message is also printed to stderr and the function - returns zero/null. - - Note that the CFFI module never calls ``exit()``, but CPython itself - contains code that calls ``exit()``, for example if importing - ``site`` fails. This may be worked around in the future. - -* **ffibuilder.set_source(c_module_name, c_code):** set the name of the - module from Python's point of view. It also gives more C code which - will be included in the generated C code. In trivial examples it - can be an empty string. It is where you would ``#include`` some - other files, define global variables, and so on. The macro - ``CFFI_DLLEXPORT`` is available to this C code: it expands to the - platform-specific way of saying "the following declaration should be - exported from the DLL". For example, you would put "``extern int - my_glob;``" in ``ffibuilder.embedding_api()`` and "``CFFI_DLLEXPORT int - my_glob = 42;``" in ``ffibuilder.set_source()``. - - Currently, any *type* declared in ``ffibuilder.embedding_api()`` must also - be present in the ``c_code``. This is automatic if this code - contains a line like ``#include "plugin.h"`` in the example above. - -* **ffibuilder.compile([target=...] [, verbose=True]):** make the C code and - compile it. By default, it produces a file called - ``c_module_name.dll``, ``c_module_name.dylib`` or - ``c_module_name.so``, but the default can be changed with the - optional ``target`` keyword argument. You can use - ``target="foo.*"`` with a literal ``*`` to ask for a file called - ``foo.dll`` on Windows, ``foo.dylib`` on OS/X and ``foo.so`` - elsewhere. One reason for specifying an alternate ``target`` is to - include characters not usually allowed in Python module names, like - "``plugin-1.5.*``". - - For more complicated cases, you can call instead - ``ffibuilder.emit_c_code("foo.c")`` and compile the resulting ``foo.c`` - file using other means. CFFI's compilation logic is based on the - standard library ``distutils`` package, which is really developed - and tested for the purpose of making CPython extension modules; it - might not always be appropriate for making general DLLs. Also, just - getting the C code is what you need if you do not want to make a - stand-alone ``.so/.dll/.dylib`` file: this C file can be compiled - and statically linked as part of a larger application. - - -More reading ------------- - -If you're reading this page about embedding and you are not familiar -with CFFI already, here are a few pointers to what you could read -next: - -* For the ``@ffi.def_extern()`` functions, integer C types are passed - simply as Python integers; and simple pointers-to-struct and basic - arrays are all straightforward enough. However, sooner or later you - will need to read about this topic in more details here__. - -* ``@ffi.def_extern()``: see `documentation here,`__ notably on what - happens if the Python function raises an exception. - -* To create Python objects attached to C data, one common solution is - to use ``ffi.new_handle()``. See documentation here__. - -* In embedding mode, the major direction is C code that calls Python - functions. This is the opposite of the regular extending mode of - CFFI, in which the major direction is Python code calling C. That's - why the page `Using the ffi/lib objects`_ talks first about the - latter, and why the direction "C code that calls Python" is - generally referred to as "callbacks" in that page. If you also - need to have your Python code call C code, read more about - `Embedding and Extending`_ below. - -* ``ffibuilder.embedding_api(source)``: follows the same syntax as - ``ffibuilder.cdef()``, `documented here.`__ You can use the "``...``" - syntax as well, although in practice it may be less useful than it - is for ``cdef()``. On the other hand, it is expected that often the - C sources that you need to give to ``ffibuilder.embedding_api()`` would be - exactly the same as the content of some ``.h`` file that you want to - give to users of your DLL. That's why the example above does this:: - - with open('foo.h') as f: - ffibuilder.embedding_api(f.read()) - - Note that a drawback of this approach is that ``ffibuilder.embedding_api()`` - doesn't support ``#ifdef`` directives. You may have to use a more - convoluted expression like:: - - with open('foo.h') as f: - lines = [line for line in f if not line.startswith('#')] - ffibuilder.embedding_api(''.join(lines)) - - As in the example above, you can also use the same ``foo.h`` from - ``ffibuilder.set_source()``:: - - ffibuilder.set_source('module_name', r''' - #include "foo.h" - ''') - - -.. __: using.html#working -.. __: using.html#def-extern -.. __: ref.html#ffi-new-handle -.. __: cdef.html#cdef - -.. _`Using the ffi/lib objects`: using.html - - -Troubleshooting ---------------- - -* The error message - - cffi extension module 'c_module_name' has unknown version 0x2701 - - means that the running Python interpreter located a CFFI version older - than 1.5. CFFI 1.5 or newer must be installed in the running Python. - -* On PyPy, the error message - - debug: pypy_setup_home: directories 'lib-python' and 'lib_pypy' not - found in pypy's shared library location or in any parent directory - - means that the ``libpypy-c.so`` file was found, but the standard library - was not found from this location. This occurs at least on some Linux - distributions, because they put ``libpypy-c.so`` inside ``/usr/lib/``, - instead of the way we recommend, which is: keep that file inside - ``/opt/pypy/bin/`` and put a symlink to there from ``/usr/lib/``. - The quickest fix is to do that change manually. - - -Issues about using the .so --------------------------- - -This paragraph describes issues that are not necessarily specific to -CFFI. It assumes that you have obtained the ``.so/.dylib/.dll`` file as -described above, but that you have troubles using it. (In summary: it -is a mess. This is my own experience, slowly built by using Google and -by listening to reports from various platforms. Please report any -inaccuracies in this paragraph or better ways to do things.) - -* The file produced by CFFI should follow this naming pattern: - ``libmy_plugin.so`` on Linux, ``libmy_plugin.dylib`` on Mac, or - ``my_plugin.dll`` on Windows (no ``lib`` prefix on Windows). - -* First note that this file does not contain the Python interpreter - nor the standard library of Python. You still need it to be - somewhere. There are ways to compact it to a smaller number of files, - but this is outside the scope of CFFI (please report if you used some - of these ways successfully so that I can add some links here). - -* In what we'll call the "main program", the ``.so`` can be either - used dynamically (e.g. by calling ``dlopen()`` or ``LoadLibrary()`` - inside the main program), or at compile-time (e.g. by compiling it - with ``gcc -lmy_plugin``). The former case is always used if you're - building a plugin for a program, and the program itself doesn't need - to be recompiled. The latter case is for making a CFFI library that - is more tightly integrated inside the main program. - -* In the case of compile-time usage: you can add the gcc - option ``-Lsome/path/`` before ``-lmy_plugin`` to describe where the - ``libmy_plugin.so`` is. On some platforms, notably Linux, ``gcc`` - will complain if it can find ``libmy_plugin.so`` but not - ``libpython27.so`` or ``libpypy-c.so``. To fix it, you need to call - ``LD_LIBRARY_PATH=/some/path/to/libpypy gcc``. - -* When actually executing the main program, it needs to find the - ``libmy_plugin.so`` but also ``libpython27.so`` or ``libpypy-c.so``. - For PyPy, unpack a PyPy distribution and you get a full directory - structure with ``libpypy-c.so`` inside a ``bin`` subdirectory, or on - Windows ``pypy-c.dll`` inside the top directory; you must not move - this file around, but just point to it. One way to point to it is by - running the main program with some environment variable: - ``LD_LIBRARY_PATH=/some/path/to/libpypy`` on Linux, - ``DYLD_LIBRARY_PATH=/some/path/to/libpypy`` on OS/X. - -* You can avoid the ``LD_LIBRARY_PATH`` issue if you compile - ``libmy_plugin.so`` with the path hard-coded inside in the first - place. On Linux, this is done by ``gcc -Wl,-rpath=/some/path``. You - would put this option in ``ffibuilder.set_source("my_plugin", ..., - extra_link_args=['-Wl,-rpath=/some/path/to/libpypy'])``. The path can - start with ``$ORIGIN`` to mean "the directory where - ``libmy_plugin.so`` is". You can then specify a path relative to that - place, like ``extra_link_args=['-Wl,-rpath=$ORIGIN/../venv/bin']``. - Use ``ldd libmy_plugin.so`` to look at what path is currently compiled - in after the expansion of ``$ORIGIN``.) - - After this, you don't need ``LD_LIBRARY_PATH`` any more to locate - ``libpython27.so`` or ``libpypy-c.so`` at runtime. In theory it - should also cover the call to ``gcc`` for the main program. I wasn't - able to make ``gcc`` happy without ``LD_LIBRARY_PATH`` on Linux if - the rpath starts with ``$ORIGIN``, though. - -* The same rpath trick might be used to let the main program find - ``libmy_plugin.so`` in the first place without ``LD_LIBRARY_PATH``. - (This doesn't apply if the main program uses ``dlopen()`` to load it - as a dynamic plugin.) You'd make the main program with ``gcc - -Wl,-rpath=/path/to/libmyplugin``, possibly with ``$ORIGIN``. The - ``$`` in ``$ORIGIN`` causes various shell problems on its own: if - using a common shell you need to say ``gcc - -Wl,-rpath=\$ORIGIN``. From a Makefile, you need to say - something like ``gcc -Wl,-rpath=\$$ORIGIN``. - -* On some Linux distributions, notably Debian, the ``.so`` files of - CPython C extension modules may be compiled without saying that they - depend on ``libpythonX.Y.so``. This makes such Python systems - unsuitable for embedding if the embedder uses ``dlopen(..., - RTLD_LOCAL)``. You get an ``undefined symbol`` error. See - `issue #264`__. A workaround is to first call - ``dlopen("libpythonX.Y.so", RTLD_LAZY|RTLD_GLOBAL)``, which will - force ``libpythonX.Y.so`` to be loaded first. - -.. __: https://foss.heptapod.net/pypy/cffi/-/issues/264 - - -Using multiple CFFI-made DLLs ------------------------------ - -Multiple CFFI-made DLLs can be used by the same process. - -Note that all CFFI-made DLLs in a process share a single Python -interpreter. The effect is the same as the one you get by trying to -build a large Python application by assembling a lot of unrelated -packages. Some of these might be libraries that monkey-patch some -functions from the standard library, for example, which might be -unexpected from other parts. - - -Multithreading --------------- - -Multithreading should work transparently, based on Python's standard -Global Interpreter Lock. - -If two threads both try to call a C function when Python is not yet -initialized, then locking occurs. One thread proceeds with -initialization and blocks the other thread. The other thread will be -allowed to continue only when the execution of the initialization-time -Python code is done. - -If the two threads call two *different* CFFI-made DLLs, the Python -initialization itself will still be serialized, but the two pieces of -initialization-time Python code will not. The idea is that there is a -priori no reason for one DLL to wait for initialization of the other -DLL to be complete. - -After initialization, Python's standard Global Interpreter Lock kicks -in. The end result is that when one CPU progresses on executing -Python code, no other CPU can progress on executing more Python code -from another thread of the same process. At regular intervals, the -lock switches to a different thread, so that no single thread should -appear to block indefinitely. - - -Testing -------- - -For testing purposes, a CFFI-made DLL can be imported in a running -Python interpreter instead of being loaded like a C shared library. - -You might have some issues with the file name: for example, on -Windows, Python expects the file to be called ``c_module_name.pyd``, -but the CFFI-made DLL is called ``target.dll`` instead. The base name -``target`` is the one specified in ``ffibuilder.compile()``, and on Windows -the extension is ``.dll`` instead of ``.pyd``. You have to rename or -copy the file, or on POSIX use a symlink. - -The module then works like a regular CFFI extension module. It is -imported with "``from c_module_name import ffi, lib``" and exposes on -the ``lib`` object all C functions. You can test it by calling these -C functions. The initialization-time Python code frozen inside the -DLL is executed the first time such a call is done. - - -Embedding and Extending ------------------------ - -The embedding mode is not incompatible with the non-embedding mode of -CFFI. - -You can use *both* ``ffibuilder.embedding_api()`` and -``ffibuilder.cdef()`` in the -same build script. You put in the former the declarations you want to -be exported by the DLL; you put in the latter only the C functions and -types that you want to share between C and Python, but not export from -the DLL. - -As an example of that, consider the case where you would like to have -a DLL-exported C function written in C directly, maybe to handle some -cases before calling Python functions. To do that, you must *not* put -the function's signature in ``ffibuilder.embedding_api()``. (Note that this -requires more hacks if you use ``ffibuilder.embedding_api(f.read())``.) -You must only write the custom function definition in -``ffibuilder.set_source()``, and prefix it with the macro CFFI_DLLEXPORT: - -.. code-block:: c - - CFFI_DLLEXPORT int myfunc(int a, int b) - { - /* implementation here */ - } - -This function can, if it wants, invoke Python functions using the -general mechanism of "callbacks"---called this way because it is a -call from C to Python, although in this case it is not calling -anything back: - -.. code-block:: python - - ffibuilder.cdef(""" - extern "Python" int mycb(int); - """) - - ffibuilder.set_source("my_plugin", r""" - - static int mycb(int); /* the callback: forward declaration, to make - it accessible from the C code that follows */ - - CFFI_DLLEXPORT int myfunc(int a, int b) - { - int product = a * b; /* some custom C code */ - return mycb(product); - } - """) - -and then the Python initialization code needs to contain the lines: - -.. code-block:: python - - @ffi.def_extern() - def mycb(x): - print "hi, I'm called with x =", x - return x * 10 - -This ``@ffi.def_extern`` is attaching a Python function to the C -callback ``mycb()``, which in this case is not exported from the DLL. -Nevertheless, the automatic initialization of Python occurs when -``mycb()`` is called, if it happens to be the first function called -from C. More precisely, it does not happen when ``myfunc()`` is -called: this is just a C function, with no extra code magically -inserted around it. It only happens when ``myfunc()`` calls -``mycb()``. - -As the above explanation hints, this is how ``ffibuilder.embedding_api()`` -actually implements function calls that directly invoke Python code; -here, we have merely decomposed it explicitly, in order to add some -custom C code in the middle. - -In case you need to force, from C code, Python to be initialized -before the first ``@ffi.def_extern()`` is called, you can do so by -calling the C function ``cffi_start_python()`` with no argument. It -returns an integer, 0 or -1, to tell if the initialization succeeded -or not. Currently there is no way to prevent a failing initialization -from also dumping a traceback and more information to stderr. -Note that the function ``cffi_start_python()`` is static: it must be -called from C source written inside ``ffibuilder.set_source()``. To -call it from somewhere else, you need to make a function (with a -different non-static name) in the ``ffibuilder.set_source()`` that just -calls ``cffi_start_python()``. The reason it is static is to avoid -naming conflicts in case you are ultimately trying to link a large C -program with more than one cffi embedded module in it. diff --git a/doc/source/goals.rst b/doc/source/goals.rst deleted file mode 100644 index df4877c..0000000 --- a/doc/source/goals.rst +++ /dev/null @@ -1,69 +0,0 @@ -Goals ------ - -The interface is based on `LuaJIT's FFI`_, and follows a few principles: - -* The goal is to call C code from Python without learning a 3rd language: - existing alternatives require users to learn domain specific language - (Cython_, SWIG_) or API (ctypes_). The CFFI design requires users to know - only C and Python, minimizing the extra bits of API that need to be learned. - -* Keep all the Python-related logic in Python so that you don't need to - write much C code (unlike `CPython native C extensions`_). - -* The preferred way is to work at the level of the API (Application - Programming Interface): the C compiler is called from the declarations - you write to validate and link to the C language constructs. - Alternatively, it is also possible to work at the ABI level - (Application Binary Interface), the way ctypes_ work. - However, on non-Windows platforms, C libraries typically - have a specified C API but not an ABI (e.g. they may - document a "struct" as having at least these fields, but maybe more). - -* Try to be complete. For now some C99 constructs are not supported, - but all C89 should be, including macros (and including macro "abuses", - which you can `manually wrap`_ in saner-looking C functions). - -* Attempt to support both PyPy and CPython, with a reasonable path - for other Python implementations like IronPython and Jython. - -* Note that this project is **not** about embedding executable C code in - Python, unlike `Weave`_. This is about calling existing C libraries - from Python. - -* There is no C++ support. Sometimes, it is reasonable to write a C - wrapper around the C++ code and then call this C API with CFFI. - Otherwise, look at other projects. I would recommend cppyy_, which - has got some similarities (and also works efficiently on both CPython - and PyPy). - -.. _`LuaJIT's FFI`: http://luajit.org/ext_ffi.html -.. _`Cython`: http://www.cython.org -.. _`SWIG`: http://www.swig.org/ -.. _`CPython native C extensions`: http://docs.python.org/extending/extending.html -.. _`native C extensions`: http://docs.python.org/extending/extending.html -.. _`ctypes`: http://docs.python.org/library/ctypes.html -.. _`Weave`: http://wiki.scipy.org/Weave -.. _`cppyy`: http://cppyy.readthedocs.io/en/latest/ -.. _`manually wrap`: overview.html#abi-versus-api - -Get started by reading `the overview`__. - -.. __: overview.html - - -Comments and bugs ------------------ - -The best way to contact us is on the IRC ``#cffi`` or ``#pypy`` channels of -``irc.libera.chat``. Feel free to discuss matters either there or in -the `mailing list`_. Please report to the `issue tracker`_ any bugs. - -As a general rule, when there is a design issue to resolve, we pick the -solution that is the "most C-like". We hope that this module has got -everything you need to access C code and nothing more. - ---- the authors, Armin Rigo and Maciej Fijalkowski - -.. _`issue tracker`: https://foss.heptapod.net/pypy/cffi/issues -.. _`mailing list`: https://groups.google.com/forum/#!forum/python-cffi diff --git a/doc/source/index.rst b/doc/source/index.rst deleted file mode 100644 index 54934f2..0000000 --- a/doc/source/index.rst +++ /dev/null @@ -1,19 +0,0 @@ -================================ -CFFI documentation -================================ - -C Foreign Function Interface for Python. Interact with almost any C -code from Python, based on C-like declarations that you can often -copy-paste from header files or documentation. - -.. toctree:: - :maxdepth: 2 - - goals - whatsnew - installation - overview - using - ref - cdef - embedding diff --git a/doc/source/installation.rst b/doc/source/installation.rst deleted file mode 100644 index 6d55eb5..0000000 --- a/doc/source/installation.rst +++ /dev/null @@ -1,190 +0,0 @@ -======================================================= -Installation and Status -======================================================= - -Quick installation for CPython (cffi is distributed with PyPy): - -* ``pip install cffi`` - -* or get the source code via the `Python Package Index`__. - -.. __: http://pypi.python.org/pypi/cffi - -In more details: - -This code has been developed on Linux, but should work on any POSIX -platform as well as on Windows 32 and 64. (It relies occasionally on -libffi, so it depends on libffi being bug-free; this may not be fully -the case on some of the more exotic platforms.) - -CFFI supports CPython 2.7, 3.x (tested with 3.6 to 3.9); and is -distributed with PyPy (CFFI 1.0 is distributed with and requires -PyPy 2.6). - -The core speed of CFFI is better than ctypes, with import times being -either lower if you use the post-1.0 features, or much higher if you -don't. The wrapper Python code you typically need to write around the -raw CFFI interface slows things down on CPython, but not unreasonably -so. On PyPy, this wrapper code has a minimal impact thanks to the JIT -compiler. This makes CFFI the recommended way to interface with C -libraries on PyPy. - -Requirements: - -* CPython 2.7 or 3.x, or PyPy (PyPy 2.0 for the earliest - versions of CFFI; or PyPy 2.6 for CFFI 1.0). - -* in some cases you need to be able to compile C extension modules. - On non-Windows platforms, this usually means installing the package - ``python-dev``. Refer to the appropriate docs for your OS. - -* on CPython, on non-Windows platforms, you also need to install - ``libffi-dev`` in order to compile CFFI itself. - -* pycparser >= 2.06: https://github.com/eliben/pycparser (automatically - tracked by ``pip install cffi``). - -* `py.test`_ is needed to run the tests of CFFI itself. - -.. _`py.test`: http://pypi.python.org/pypi/pytest - -Download and Installation: - -* https://pypi.python.org/pypi/cffi - -* Checksums of the "source" package version 1.15.0: - - - MD5: f3a3f26cd3335fc597479c9475da0a0b - - - SHA1: 9c51c29e35510adf7f94542e1f8e05611930b07b - - - SHA256: 920f0d66a896c2d99f0adbb391f990a84091179542c205fa53ce5787aff87954 - -* Or grab the most current version from the `Heptapod page`_: - ``hg clone https://foss.heptapod.net/pypy/cffi`` - -* ``python setup.py install`` or ``python setup_base.py install`` - (should work out of the box on Linux or Windows; see below for - `MacOS X`_.) - -* running the tests: ``py.test c/ testing/`` (if you didn't - install cffi yet, you need first ``python setup_base.py build_ext -f - -i``) - -.. _`Heptapod page`: https://foss.heptapod.net/pypy/cffi - -Demos: - -* The `demo`_ directory contains a number of small and large demos - of using ``cffi``. - -* The documentation below might be sketchy on details; for now the - ultimate reference is given by the tests, notably - `testing/cffi1/test_verify1.py`_ and `testing/cffi0/backend_tests.py`_. - -.. _`demo`: https://foss.heptapod.net/pypy/cffi/-/tree/branch/default/demo -.. _`testing/cffi1/test_verify1.py`: https://foss.heptapod.net/pypy/cffi/-/blob/branch/default/testing/cffi1/test_verify1.py -.. _`testing/cffi0/backend_tests.py`: https://foss.heptapod.net/pypy/cffi/-/blob/branch/default/testing/cffi0/backend_tests.py - - -Platform-specific instructions ------------------------------- - -``libffi`` is notoriously messy to install and use --- to the point that -CPython includes its own copy to avoid relying on external packages. -CFFI does the same for Windows, but not for other platforms (which should -have their own working libffi's). -Modern Linuxes work out of the box thanks to ``pkg-config``. Here are some -(user-supplied) instructions for other platforms. - - -MacOS X -+++++++ - -**Homebrew** (Thanks David Griffin for this) - -1) Install homebrew: http://brew.sh - -2) Run the following commands in a terminal - -:: - - brew install pkg-config libffi - PKG_CONFIG_PATH=/usr/local/opt/libffi/lib/pkgconfig pip install cffi - - -Alternatively, **on OS/X 10.6** (Thanks Juraj Sukop for this) - -For building libffi you can use the default install path, but then, in -``setup.py`` you need to change:: - - include_dirs = [] - -to:: - - include_dirs = ['/usr/local/lib/libffi-3.0.11/include'] - -Then running ``python setup.py build`` complains about "fatal error: error writing to -: Broken pipe", which can be fixed by running:: - - ARCHFLAGS="-arch i386 -arch x86_64" python setup.py build - -as described here_. - -.. _here: http://superuser.com/questions/259278/python-2-6-1-pycrypto-2-3-pypi-package-broken-pipe-during-build - - -Windows (32/64-bit) -+++++++++++++++++++ - -Win32 and Win64 work and are tested at least each official release. - -The recommended C compiler compatible with Python 2.7 is this one: -http://www.microsoft.com/en-us/download/details.aspx?id=44266 -There is a known problem with distutils on Python 2.7, as -explained in https://bugs.python.org/issue23246, and the same -problem applies whenever you want to run compile() to build a dll with -this specific compiler suite download. -``import setuptools`` might help, but YMMV - -For Python 3.4 and beyond: -https://www.visualstudio.com/en-us/downloads/visual-studio-2015-ctp-vs - - -Linux and OS/X: UCS2 versus UCS4 -++++++++++++++++++++++++++++++++ - -This is about getting an ImportError about ``_cffi_backend.so`` with a -message like ``Symbol not found: _PyUnicodeUCS2_AsASCIIString``. This -error occurs in Python 2 as soon as you mix "ucs2" and "ucs4" builds of -Python. It means that you are now running a Python compiled with -"ucs4", but the extension module ``_cffi_backend.so`` was compiled by a -different Python: one that was running "ucs2". (If the opposite problem -occurs, you get an error about ``_PyUnicodeUCS4_AsASCIIString`` -instead.) - -If you are using ``pyenv``, then see -https://github.com/yyuu/pyenv/issues/257. - -More generally, the solution that should always work is to download the -sources of CFFI (instead of a prebuilt binary) and make sure that you -build it with the same version of Python than the one that will use it. -For example, with virtualenv: - -* ``virtualenv ~/venv`` - -* ``cd ~/path/to/sources/of/cffi`` - -* ``~/venv/bin/python setup.py build --force`` # forcing a rebuild to - make sure - -* ``~/venv/bin/python setup.py install`` - -This will compile and install CFFI in this virtualenv, using the -Python from this virtualenv. - - -NetBSD -++++++ - -You need to make sure you have an up-to-date version of libffi, which -fixes some bugs. diff --git a/doc/source/overview.rst b/doc/source/overview.rst deleted file mode 100644 index dbc3540..0000000 --- a/doc/source/overview.rst +++ /dev/null @@ -1,651 +0,0 @@ -======================================================= -Overview -======================================================= - -.. contents:: - - -The first section presents a simple working -example of using CFFI to call a C function in a compiled shared object -(DLL) from Python. CFFI is -flexible and covers several other use cases presented in the second -section. The third section shows how to export Python functions -to a Python interpreter embedded in a C or C++ application. The last -two sections delve deeper in the CFFI library. - -Make sure you have `cffi installed`__. - -.. __: installation.html - -.. _out-of-line-api-level: -.. _real-example: - - -Main mode of usage ------------------- - -The main way to use CFFI is as an interface to some already-compiled -shared object which is provided by other means. Imagine that you have a -system-installed shared object called ``piapprox.dll`` (Windows) or -``libpiapprox.so`` (Linux and others) or ``libpiapprox.dylib`` (OS X), -exporting a function ``float pi_approx(int n);`` that computes some -approximation of pi given a number of iterations. You want to call -this function from Python. Note this method works equally well with a -static library ``piapprox.lib`` (Windows) or ``libpiapprox.a``. - -Create the file ``piapprox_build.py``: - -.. code-block:: python - - from cffi import FFI - ffibuilder = FFI() - - # cdef() expects a single string declaring the C types, functions and - # globals needed to use the shared object. It must be in valid C syntax. - ffibuilder.cdef(""" - float pi_approx(int n); - """) - - # set_source() gives the name of the python extension module to - # produce, and some C source code as a string. This C code needs - # to make the declarated functions, types and globals available, - # so it is often just the "#include". - ffibuilder.set_source("_pi_cffi", - """ - #include "pi.h" // the C header of the library - """, - libraries=['piapprox']) # library name, for the linker - - if __name__ == "__main__": - ffibuilder.compile(verbose=True) - -Execute this script. If everything is OK, it should produce -``_pi_cffi.c``, and then invoke the compiler on it. The produced -``_pi_cffi.c`` contains a copy of the string given in :ref:`set_source() <set_source>`, -in this example the ``#include "pi.h"``. Afterwards, it contains glue code -for all the functions, types and globals declared in the :ref:`cdef() <cdef>` above. - -At runtime, you use the extension module like this: - -.. code-block:: python - - from _pi_cffi import ffi, lib - print(lib.pi_approx(5000)) - -That's all! In the rest of this page, we describe some more advanced -examples and other CFFI modes. In particular, there is a complete -example `if you don't have an already-installed C library to call`_. - -For more information about the ``cdef()`` and ``set_source()`` methods -of the ``FFI`` class, see `Preparing and Distributing modules`__. - -.. __: cdef.html - -When your example works, a common alternative to running the build -script manually is to have it run as part of a ``setup.py``. Here is -an example using the Setuptools distribution: - -.. code-block:: python - - from setuptools import setup - - setup( - ... - setup_requires=["cffi>=1.0.0"], - cffi_modules=["piapprox_build:ffibuilder"], # "filename:global" - install_requires=["cffi>=1.0.0"], - ) - - -Other CFFI modes ----------------- - -CFFI can be used in one of four modes: "ABI" versus "API" level, -each with "in-line" or "out-of-line" preparation (or compilation). - -The **ABI mode** accesses libraries at the binary level, whereas the -faster **API mode** accesses them with a C compiler. We explain the -difference in more details below__. - -.. __: `abi-versus-api`_ - -In the **in-line mode,** everything is set up every time you import -your Python code. In the **out-of-line mode,** you have a separate -step of preparation (and possibly C compilation) that produces a -module which your main program can then import. - - -Simple example (ABI level, in-line) -+++++++++++++++++++++++++++++++++++ - -May look familiar to those who have used ctypes_. - -.. code-block:: python - - >>> from cffi import FFI - >>> ffi = FFI() - >>> ffi.cdef(""" - ... int printf(const char *format, ...); // copy-pasted from the man page - ... """) - >>> C = ffi.dlopen(None) # loads the entire C namespace - >>> arg = ffi.new("char[]", b"world") # equivalent to C code: char arg[] = "world"; - >>> C.printf(b"hi there, %s.\n", arg) # call printf - hi there, world. - 17 # this is the return value - >>> - -Note that ``char *`` arguments expect a ``bytes`` object. If you have a -``str`` (or a ``unicode`` on Python 2) you need to encode it explicitly -with ``somestring.encode(myencoding)``. - -*Python 3 on Windows:* :ref:`ffi.dlopen(None) <dlopen>` does not work. This problem -is messy and not really fixable. The problem does not occur if you try -to call a function from a specific DLL that exists on your system: then -you use ``ffi.dlopen("path.dll")``. - -*This example does not call any C compiler. It works in the so-called -ABI mode, which means that it will crash if you call some function or -access some fields of a structure that was slightly misdeclared in the -cdef().* - -If using a C compiler to install your module is an option, it is highly -recommended to use the API mode instead. (It is also faster.) - - -Struct/Array Example (minimal, in-line) -+++++++++++++++++++++++++++++++++++++++ - -.. code-block:: python - - from cffi import FFI - ffi = FFI() - ffi.cdef(""" - typedef struct { - unsigned char r, g, b; - } pixel_t; - """) - image = ffi.new("pixel_t[]", 800*600) - - f = open('data', 'rb') # binary mode -- important - f.readinto(ffi.buffer(image)) - f.close() - - image[100].r = 255 - image[100].g = 192 - image[100].b = 128 - - f = open('data', 'wb') - f.write(ffi.buffer(image)) - f.close() - -This can be used as a more flexible replacement of the struct_ and -array_ modules, and replaces ctypes_. You could also call :ref:`ffi.new("pixel_t[600][800]") <new>` -and get a two-dimensional array. - -.. _struct: http://docs.python.org/library/struct.html -.. _array: http://docs.python.org/library/array.html -.. _ctypes: http://docs.python.org/library/ctypes.html - -*This example does not call any C compiler.* - -This example also admits an out-of-line equivalent. It is similar to -the first example `Main mode of usage`_ above, -but passing ``None`` as the second argument to -:ref:`ffibuilder.set_source() <set_source>`. Then in the main program you write -``from _simple_example import ffi`` and then the same content as the -in-line example above starting from the line ``image = -ffi.new("pixel_t[]", 800*600)``. - - -API Mode, calling the C standard library -++++++++++++++++++++++++++++++++++++++++ - -.. code-block:: python - - # file "example_build.py" - - # Note: we instantiate the same 'cffi.FFI' class as in the previous - # example, but call the result 'ffibuilder' now instead of 'ffi'; - # this is to avoid confusion with the other 'ffi' object you get below - - from cffi import FFI - ffibuilder = FFI() - - ffibuilder.set_source("_example", - r""" // passed to the real C compiler, - // contains implementation of things declared in cdef() - #include <sys/types.h> - #include <pwd.h> - - // We can also define custom wrappers or other functions - // here (this is an example only): - static struct passwd *get_pw_for_root(void) { - return getpwuid(0); - } - """, - libraries=[]) # or a list of libraries to link with - # (more arguments like setup.py's Extension class: - # include_dirs=[..], extra_objects=[..], and so on) - - ffibuilder.cdef(""" - // declarations that are shared between Python and C - struct passwd { - char *pw_name; - ...; // literally dot-dot-dot - }; - struct passwd *getpwuid(int uid); // defined in <pwd.h> - struct passwd *get_pw_for_root(void); // defined in set_source() - """) - - if __name__ == "__main__": - ffibuilder.compile(verbose=True) - -You need to run the ``example_build.py`` script once to generate -"source code" into the file ``_example.c`` and compile this to a -regular C extension module. (CFFI selects either Python or C for the -module to generate based on whether the second argument to -:ref:`set_source() <set_source>` is ``None`` or not.) - -*You need a C compiler for this single step. It produces a file called -e.g. _example.so or _example.pyd. If needed, it can be distributed in -precompiled form like any other extension module.* - -Then, in your main program, you use: - -.. code-block:: python - - from _example import ffi, lib - - p = lib.getpwuid(0) - assert ffi.string(p.pw_name) == b'root' - p = lib.get_pw_for_root() - assert ffi.string(p.pw_name) == b'root' - -Note that this works independently of the exact C layout of ``struct -passwd`` (it is "API level", as opposed to "ABI level"). It requires -a C compiler in order to run ``example_build.py``, but it is much more -portable than trying to get the details of the fields of ``struct -passwd`` exactly right. Similarly, in the :ref:`cdef() <cdef>` we declared -``getpwuid()`` as taking an ``int`` argument; on some platforms this -might be slightly incorrect---but it does not matter. - -Note also that at runtime, the API mode is faster than the ABI mode. - -To integrate it inside a ``setup.py`` distribution with Setuptools: - -.. code-block:: python - - from setuptools import setup - - setup( - ... - setup_requires=["cffi>=1.0.0"], - cffi_modules=["example_build.py:ffibuilder"], - install_requires=["cffi>=1.0.0"], - ) - - -.. _`if you don't have an already-installed C library to call`: - -API Mode, calling C sources instead of a compiled library -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - -If you want to call some library that is not precompiled, but for which -you have C sources, then the easiest solution is to make a single -extension module that is compiled from both the C sources of this -library, and the additional CFFI wrappers. For example, say you start -with the files ``pi.c`` and ``pi.h``: - - .. code-block:: C - - /* filename: pi.c*/ - # include <stdlib.h> - # include <math.h> - - /* Returns a very crude approximation of Pi - given a int: a number of iteration */ - float pi_approx(int n){ - - double i,x,y,sum=0; - - for(i=0;i<n;i++){ - - x=rand(); - y=rand(); - - if (sqrt(x*x+y*y) < sqrt((double)RAND_MAX*RAND_MAX)) - sum++; } - - return 4*(float)sum/(float)n; } - - .. code-block:: C - - /* filename: pi.h*/ - float pi_approx(int n); - -Create a script named ``pi_extension_build.py``, building -the C extension: - - .. code-block:: python - - from cffi import FFI - ffibuilder = FFI() - - ffibuilder.cdef("float pi_approx(int n);") - - ffibuilder.set_source("_pi", # name of the output C extension - """ - #include "pi.h" - """, - sources=['pi.c'], # includes pi.c as additional sources - libraries=['m']) # on Unix, link with the math library - - if __name__ == "__main__": - ffibuilder.compile(verbose=True) - -Build the extension: - - .. code-block:: shell - - python pi_extension_build.py - -Observe, in the working directory, the generated output files: -``_pi.c``, ``_pi.o`` and the compiled C extension (called ``_pi.so`` on -Linux for example). It can be called from Python: - - .. code-block:: python - - from _pi.lib import pi_approx - - approx = pi_approx(10) - assert str(approx).startswith("3.") - - approx = pi_approx(10000) - assert str(approx).startswith("3.1") - - -.. _performance: - -Purely for performance (API level, out-of-line) -+++++++++++++++++++++++++++++++++++++++++++++++ - -A variant of the `section above`__ where the goal is not to call an -existing C library, but to compile and call some C function written -directly in the build script: - -.. __: real-example_ - -.. code-block:: python - - # file "example_build.py" - - from cffi import FFI - ffibuilder = FFI() - - ffibuilder.cdef("int foo(int *, int *, int);") - - ffibuilder.set_source("_example", - r""" - static int foo(int *buffer_in, int *buffer_out, int x) - { - /* some algorithm that is seriously faster in C than in Python */ - } - """) - - if __name__ == "__main__": - ffibuilder.compile(verbose=True) - -.. code-block:: python - - # file "example.py" - - from _example import ffi, lib - - buffer_in = ffi.new("int[]", 1000) - # initialize buffer_in here... - - # easier to do all buffer allocations in Python and pass them to C, - # even for output-only arguments - buffer_out = ffi.new("int[]", 1000) - - result = lib.foo(buffer_in, buffer_out, 1000) - -*You need a C compiler to run example_build.py, once. It produces a -file called e.g. _example.so or _example.pyd. If needed, it can be -distributed in precompiled form like any other extension module.* - - -.. _out-of-line-abi-level: - -Out-of-line, ABI level -++++++++++++++++++++++ - -The out-of-line ABI mode is a mixture of the regular (API) out-of-line -mode and the in-line ABI mode. It lets you use the ABI mode, with its -advantages (not requiring a C compiler) and problems (crashes more -easily). - -This mixture mode lets you massively reduces the import times, because -it is slow to parse a large C header. It also allows you to do more -detailed checkings during build-time without worrying about performance -(e.g. calling :ref:`cdef() <cdef>` many times with small pieces of declarations, -based on the version of libraries detected on the system). - -.. code-block:: python - - # file "simple_example_build.py" - - from cffi import FFI - - ffibuilder = FFI() - # Note that the actual source is None - ffibuilder.set_source("_simple_example", None) - ffibuilder.cdef(""" - int printf(const char *format, ...); - """) - - if __name__ == "__main__": - ffibuilder.compile(verbose=True) - -Running it once produces ``_simple_example.py``. Your main program -only imports this generated module, not ``simple_example_build.py`` -any more: - -.. code-block:: python - - from _simple_example import ffi - - lib = ffi.dlopen(None) # Unix: open the standard C library - #import ctypes.util # or, try this on Windows: - #lib = ffi.dlopen(ctypes.util.find_library("c")) - - lib.printf(b"hi there, number %d\n", ffi.cast("int", 2)) - -Note that this :ref:`ffi.dlopen() <dlopen>`, unlike the one from in-line mode, -does not invoke any additional magic to locate the library: it must be -a path name (with or without a directory), as required by the C -``dlopen()`` or ``LoadLibrary()`` functions. This means that -``ffi.dlopen("libfoo.so")`` is ok, but ``ffi.dlopen("foo")`` is not. -In the latter case, you could replace it with -``ffi.dlopen(ctypes.util.find_library("foo"))``. Also, None is only -recognized on Unix to open the standard C library. - -For distribution purposes, remember that there is a new -``_simple_example.py`` file generated. You can either include it -statically within your project's source files, or, with Setuptools, -you can say in the ``setup.py``: - -.. code-block:: python - - from setuptools import setup - - setup( - ... - setup_requires=["cffi>=1.0.0"], - cffi_modules=["simple_example_build.py:ffibuilder"], - install_requires=["cffi>=1.0.0"], - ) - -In summary, this mode is useful when you wish to declare many C structures but -do not need fast interaction with a shared object. It is useful for parsing -binary files, for instance. - - -In-line, API level -++++++++++++++++++ - -The "API level + in-line" mode combination exists but is long -deprecated. It used to be done with ``lib = ffi.verify("C header")``. -The out-of-line variant with :ref:`set_source("modname", "C header") <set_source>` is -preferred and avoids a number of problems when the project grows in -size. - - -.. _embedding: - -Embedding ---------- - -*New in version 1.5.* - -CFFI can be used for embedding__: creating a standard -dynamically-linked library (``.dll`` under Windows, ``.so`` elsewhere) -which can be used from a C application. - -.. code-block:: python - - import cffi - ffibuilder = cffi.FFI() - - ffibuilder.embedding_api(""" - int do_stuff(int, int); - """) - - ffibuilder.set_source("my_plugin", "") - - ffibuilder.embedding_init_code(""" - from my_plugin import ffi - - @ffi.def_extern() - def do_stuff(x, y): - print("adding %d and %d" % (x, y)) - return x + y - """) - - ffibuilder.compile(target="plugin-1.5.*", verbose=True) - -This simple example creates ``plugin-1.5.dll`` or ``plugin-1.5.so`` as -a DLL with a single exported function, ``do_stuff()``. You execute -the script above once, with the interpreter you want to have -internally used; it can be CPython 2.x or 3.x or PyPy. This DLL can -then be used "as usual" from an application; the application doesn't -need to know that it is talking with a library made with Python and -CFFI. At runtime, when the application calls ``int do_stuff(int, -int)``, the Python interpreter is automatically initialized and ``def -do_stuff(x, y):`` gets called. `See the details in the documentation -about embedding.`__ - -.. __: embedding.html -.. __: embedding.html - - -What actually happened? ------------------------ - -The CFFI interface operates on the same level as C - you declare types -and functions using the same syntax as you would define them in C. This -means that most of the documentation or examples can be copied straight -from the man pages. - -The declarations can contain **types, functions, constants** -and **global variables.** What you pass to the :ref:`cdef() <cdef>` must not -contain more than that; in particular, ``#ifdef`` or ``#include`` -directives are not supported. The cdef in the above examples are just -that - they declared "there is a function in the C level with this -given signature", or "there is a struct type with this shape". - -In the ABI examples, the :ref:`dlopen() <dlopen>` calls load libraries manually. -At the binary level, a program is split into multiple namespaces---a -global one (on some platforms), plus one namespace per library. So -``dlopen()`` returns a ``<FFILibrary>`` object, and this object has -got as attributes all function, constant and variable symbols that are -coming from this library and that have been declared in the -``cdef()``. If you have several interdependent libraries to load, -you would call ``cdef()`` only once but ``dlopen()`` several times. - -By opposition, the API mode works more closely like a C program: the C -linker (static or dynamic) is responsible for finding any symbol used. -You name the libraries in the ``libraries`` keyword argument to -:ref:`set_source() <set_source>`, but never need to say which symbol comes -from which library. -Other common arguments to ``set_source()`` include ``library_dirs`` and -``include_dirs``; all these arguments are passed to the standard -distutils/setuptools. - -The :ref:`ffi.new() <new>` lines allocate C objects. They are filled -with zeroes initially, unless the optional second argument is used. -If specified, this argument gives an "initializer", like you can use -with C code to initialize global variables. - -The actual ``lib.*()`` function calls should be obvious: it's like C. - - -.. _abi-versus-api: - -ABI versus API --------------- - -Accessing the C library at the binary level ("ABI") is fraught -with problems, particularly on non-Windows platforms. - -The most immediate drawback of the ABI level is that calling functions -needs to go through the very general *libffi* library, which is slow -(and not always perfectly tested on non-standard platforms). The API -mode instead compiles a CPython C wrapper that directly invokes the -target function. It can be massively faster (and works -better than libffi ever will). - -The more fundamental reason to prefer the API mode is that *the C -libraries are typically meant to be used with a C compiler.* You are not -supposed to do things like guess where fields are in the structures. -The "real example" above shows how CFFI uses a C compiler under the -hood: this example uses :ref:`set_source(..., "C source...") <set_source>` and never -:ref:`dlopen() <dlopen>`. When using this approach, -we have the advantage that we can use literally "``...``" at various places in -the :ref:`cdef() <cdef>`, and the missing information will be completed with the -help of the C compiler. CFFI will turn this into a single C source file, -which contains the "C source" part unmodified, followed by some -"magic" C code and declarations derived from the ``cdef()``. When -this C file is compiled, the resulting C extension module will contain -all the information we need---or the C compiler will give warnings or -errors, as usual e.g. if we misdeclare some function's signature. - -Note that the "C source" part from ``set_source()`` can contain -arbitrary C code. You can use this to declare some -more helper functions written in C. To export -these helpers to Python, put their signature in the ``cdef()`` too. -(You can use the ``static`` C keyword in the "C source" part, -as in ``static int myhelper(int x) { return x * 42; }``, -because these helpers are only -referenced from the "magic" C code that is generated afterwards in the -same C file.) - -This can be used for example to wrap "crazy" macros into more standard -C functions. The extra layer of C can be useful for other reasons -too, like calling functions that expect some complicated argument -structures that you prefer to build in C rather than in Python. (On -the other hand, if all you need is to call "function-like" macros, -then you can directly declare them in the ``cdef()`` as if they were -functions.) - -The generated piece of C code should be the same independently on the -platform on which you run it (or the Python version), so in simple cases -you can directly distribute the pre-generated C code and treat it as a -regular C extension module (which depends on the ``_cffi_backend`` -module, on CPython). The special Setuptools lines in the `example -above`__ are meant for the more complicated cases where we need to -regenerate the C sources as well---e.g. because the Python script that -regenerates this file will itself look around the system to know what it -should include or not. - -.. __: real-example_ diff --git a/doc/source/ref.rst b/doc/source/ref.rst deleted file mode 100644 index 05c0f7c..0000000 --- a/doc/source/ref.rst +++ /dev/null @@ -1,1028 +0,0 @@ -================================ -CFFI Reference -================================ - -.. contents:: - - -FFI Interface -------------- - -*This page documents the runtime interface of the two types "FFI" and -"CompiledFFI". These two types are very similar to each other. You get -a CompiledFFI object if you import an out-of-line module. You get a FFI -object from explicitly writing cffi.FFI(). Unlike CompiledFFI, the type -FFI has also got additional methods documented on the* `next page`__. - -.. __: cdef.html - - -ffi.NULL -++++++++ - -**ffi.NULL**: a constant NULL of type ``<cdata 'void *'>``. - - -ffi.error -+++++++++ - -**ffi.error**: the Python exception raised in various cases. (Don't -confuse it with ``ffi.errno``.) - - -.. _new: - -ffi.new() -+++++++++ - -**ffi.new(cdecl, init=None)**: -allocate an instance according to the specified C type and return a -pointer to it. The specified C type must be either a pointer or an -array: ``new('X *')`` allocates an X and returns a pointer to it, -whereas ``new('X[10]')`` allocates an array of 10 X'es and returns an -array referencing it (which works mostly like a pointer, like in C). -You can also use ``new('X[]', n)`` to allocate an array of a -non-constant length n. See the `detailed documentation`__ for other -valid initializers. - -.. __: using.html#working - -When the returned ``<cdata>`` object goes out of scope, the memory is -freed. In other words the returned ``<cdata>`` object has ownership of -the value of type ``cdecl`` that it points to. This means that the raw -data can be used as long as this object is kept alive, but must not be -used for a longer time. Be careful about that when copying the -pointer to the memory somewhere else, e.g. into another structure. -Also, this means that a line like ``x = ffi.cast("B *", ffi.new("A *"))`` -or ``x = ffi.new("struct s[1]")[0]`` is wrong: the newly allocated object -goes out of scope instantly, and so is freed immediately, and ``x`` is -garbage. The only case where this is fine comes from a special case for -pointers-to-struct and pointers-to-union types: after -``p = ffi.new("struct-or-union *", ..)``, then either ``p`` or ``p[0]`` -keeps the memory alive. - -The returned memory is initially cleared (filled with zeroes), before -the optional initializer is applied. For performance, see -`ffi.new_allocator()`_ for a way to allocate non-zero-initialized -memory. - -*New in version 1.12:* see also ``ffi.release()``. - - -ffi.cast() -++++++++++ - -**ffi.cast("C type", value)**: similar to a C cast: returns an -instance of the named C type initialized with the given value. The -value is casted between integers or pointers of any type. - - -.. _ffi-errno: -.. _ffi-getwinerror: - -ffi.errno, ffi.getwinerror() -++++++++++++++++++++++++++++ - -**ffi.errno**: the value of ``errno`` received from the most recent C call -in this thread, and passed to the following C call. (This is a thread-local -read-write property.) - -**ffi.getwinerror(code=-1)**: on Windows, in addition to ``errno`` we -also save and restore the ``GetLastError()`` value across function -calls. This function returns this error code as a tuple ``(code, -message)``, adding a readable message like Python does when raising -WindowsError. If the argument ``code`` is given, format that code into -a message instead of using ``GetLastError()``. -(Note that it is also possible to declare and call the ``GetLastError()`` -function as usual.) - - -.. _ffi-string: -.. _ffi-unpack: - -ffi.string(), ffi.unpack() -++++++++++++++++++++++++++ - -**ffi.string(cdata, [maxlen])**: return a Python string (or unicode -string) from the 'cdata'. - -- If 'cdata' is a pointer or array of characters or bytes, returns the - null-terminated string. The returned string extends until the first - null character. The 'maxlen' argument limits how far we look for a - null character. If 'cdata' is an - array then 'maxlen' defaults to its length. See ``ffi.unpack()`` below - for a way to continue past the first null character. *Python 3:* this - returns a ``bytes``, not a ``str``. - -- If 'cdata' is a pointer or array of wchar_t, returns a unicode string - following the same rules. *New in version 1.11:* can also be - char16_t or char32_t. - -- If 'cdata' is a single character or byte or a wchar_t or charN_t, - returns it as a byte string or unicode string. (Note that in some - situation a single wchar_t or char32_t may require a Python unicode - string of length 2.) - -- If 'cdata' is an enum, returns the value of the enumerator as a string. - If the value is out of range, it is simply returned as the stringified - integer. - -**ffi.unpack(cdata, length)**: unpacks an array of C data of the given -length, returning a Python string/unicode/list. The 'cdata' should be -a pointer; if it is an array it is first converted to the pointer -type. *New in version 1.6.* - -- If 'cdata' is a pointer to 'char', returns a byte string. It does - not stop at the first null. (An equivalent way to do that is - ``ffi.buffer(cdata, length)[:]``.) - -- If 'cdata' is a pointer to 'wchar_t', returns a unicode string. - ('length' is measured in number of wchar_t; it is not the size in - bytes.) *New in version 1.11:* can also be char16_t or char32_t. - -- If 'cdata' is a pointer to anything else, returns a list, of the - given 'length'. (A slower way to do that is ``[cdata[i] for i in - range(length)]``.) - - -.. _ffi-buffer: -.. _ffi-from-buffer: - -ffi.buffer(), ffi.from_buffer() -+++++++++++++++++++++++++++++++ - -**ffi.buffer(cdata, [size])**: return a buffer object that references -the raw C data pointed to by the given 'cdata', of 'size' bytes. What -Python calls "a buffer", or more precisely "an object supporting the -buffer interface", is an object that represents some raw memory and -that can be passed around to various built-in or extension functions; -these built-in functions read from or write to the raw memory directly, -without needing an extra copy. - -The 'cdata' argument -must be a pointer or an array. If unspecified, the size of the -buffer is either the size of what ``cdata`` points to, or the whole size -of the array. - -Here are a few examples of where buffer() would be useful: - -- use ``file.write()`` and ``file.readinto()`` with - such a buffer (for files opened in binary mode) - -- overwrite the content of a struct: if ``p`` is a cdata pointing to - it, use ``ffi.buffer(p)[:] = newcontent``, where ``newcontent`` is - a bytes object (``str`` in Python 2). - -Remember that like in C, you can use ``array + index`` to get the pointer -to the index'th item of an array. (In C you might more naturally write -``&array[index]``, but that is equivalent.) - -The returned object's type is not the builtin ``buffer`` nor ``memoryview`` -types, because these types' API changes too much across Python versions. -Instead it has the following Python API (a subset of Python 2's ``buffer``) -in addition to supporting the buffer interface: - -- ``buf[:]`` or ``bytes(buf)``: copy data out of the buffer, returning a - regular byte string (or ``buf[start:end]`` for a part) - -- ``buf[:] = newstr``: copy data into the buffer (or ``buf[start:end] - = newstr``) - -- ``len(buf)``, ``buf[index]``, ``buf[index] = newchar``: access as a sequence - of characters. - -The buffer object returned by ``ffi.buffer(cdata)`` keeps alive the -``cdata`` object: if it was originally an owning cdata, then its -owned memory will not be freed as long as the buffer is alive. - -Python 2/3 compatibility note: you should avoid using ``str(buf)``, -because it gives inconsistent results between Python 2 and Python 3. -(This is similar to how ``str()`` gives inconsistent results on regular -byte strings). Use ``buf[:]`` instead. - -*New in version 1.10:* ``ffi.buffer`` is now the type of the returned -buffer objects; ``ffi.buffer()`` actually calls the constructor. - -**ffi.from_buffer([cdecl,] python_buffer, require_writable=False)**: -return an array cdata (by default a ``<cdata 'char[]'>``) that -points to the data of the given Python object, which must support the -buffer interface. Note that ``ffi.from_buffer()`` turns a generic -Python buffer object into a cdata object, whereas ``ffi.buffer()`` does -the opposite conversion. Both calls don't actually copy any data. - -``ffi.from_buffer()`` is meant to be used on objects -containing large quantities of raw data, like bytearrays -or ``array.array`` or numpy -arrays. It supports both the old *buffer* API (in Python 2.x) and the -new *memoryview* API. Note that if you pass a read-only buffer object, -you still get a regular ``<cdata 'char[]'>``; it is your responsibility -not to write there if the original buffer doesn't expect you to. -*In particular, never modify byte strings!* - -The original object is kept alive (and, in case -of memoryview, locked) as long as the cdata object returned by -``ffi.from_buffer()`` is alive. - -A common use case is calling a C function with some ``char *`` that -points to the internal buffer of a Python object; for this case you -can directly pass ``ffi.from_buffer(python_buffer)`` as argument to -the call. - -*New in version 1.10:* the ``python_buffer`` can be anything supporting -the buffer/memoryview interface (except unicode strings). Previously, -bytearray objects were supported in version 1.7 onwards (careful, if you -resize the bytearray, the ``<cdata>`` object will point to freed -memory); and byte strings were supported in version 1.8 onwards. - -*New in version 1.12:* added the optional *first* argument ``cdecl``, and -the keyword argument ``require_writable``: - -* ``cdecl`` defaults to ``"char[]"``, but a different array - or (from version 1.13) pointer type can be - specified for the result. A value like ``"int[]"`` will return an array of - ints instead of chars, and its length will be set to the number of ints - that fit in the buffer (rounded down if the division is not exact). Values - like ``"int[42]"`` or ``"int[2][3]"`` will return an array of exactly 42 - (resp. 2-by-3) ints, raising a ValueError if the buffer is too small. The - difference between specifying ``"int[]"`` and using the older code ``p1 = - ffi.from_buffer(x); p2 = ffi.cast("int *", p1)`` is that the older code - needs to keep ``p1`` alive as long as ``p2`` is in use, because only ``p1`` - keeps the underlying Python object alive and locked. (In addition, - ``ffi.from_buffer("int[]", x)`` gives better array bound checking.) - - *New in version 1.13:* ``cdecl`` can be a pointer type. If it points - to a struct or union, you can, as usual, write ``p.field`` instead of - ``p[0].field``. You can also access ``p[n]``; note that CFFI does not - perform any bounds checking in this case. Note also that ``p[0]`` cannot - be used to keep the buffer alive (unlike what occurs with ``ffi.new()``). - -* if ``require_writable`` is set to True, the function fails if the buffer - obtained from ``python_buffer`` is read-only (e.g. if ``python_buffer`` is - a byte string). The exact exception is raised by the object itself, and - for things like bytes it varies with the Python version, so don't rely on - it. (Before version 1.12, the same effect can be achieved with a hack: - call ``ffi.memmove(python_buffer, b"", 0)``. This has no effect if the - object is writable, but fails if it is read-only.) Please keep in mind - that CFFI does not implement the C keyword ``const``: even if you set - ``require_writable`` to False explicitly, you still get a regular - read-write cdata pointer. - -*New in version 1.12:* see also ``ffi.release()``. - - -ffi.memmove() -+++++++++++++ - -**ffi.memmove(dest, src, n)**: copy ``n`` bytes from memory area -``src`` to memory area ``dest``. See examples below. Inspired by the -C functions ``memcpy()`` and ``memmove()``---like the latter, the -areas can overlap. Each of ``dest`` and ``src`` can be either a cdata -pointer or a Python object supporting the buffer/memoryview interface. -In the case of ``dest``, the buffer/memoryview must be writable. -*New in version 1.3.* Examples: - -* ``ffi.memmove(myptr, b"hello", 5)`` copies the 5 bytes of - ``b"hello"`` to the area that ``myptr`` points to. - -* ``ba = bytearray(100); ffi.memmove(ba, myptr, 100)`` copies 100 - bytes from ``myptr`` into the bytearray ``ba``. - -* ``ffi.memmove(myptr + 1, myptr, 100)`` shifts 100 bytes from - the memory at ``myptr`` to the memory at ``myptr + 1``. - -In versions before 1.10, ``ffi.from_buffer()`` had restrictions on the -type of buffer, which made ``ffi.memmove()`` more general. - -.. _ffi-typeof: -.. _ffi-sizeof: -.. _ffi-alignof: - -ffi.typeof(), ffi.sizeof(), ffi.alignof() -+++++++++++++++++++++++++++++++++++++++++ - -**ffi.typeof("C type" or cdata object)**: return an object of type -``<ctype>`` corresponding to the parsed string, or to the C type of the -cdata instance. Usually you don't need to call this function or to -explicitly manipulate ``<ctype>`` objects in your code: any place that -accepts a C type can receive either a string or a pre-parsed ``ctype`` -object (and because of caching of the string, there is no real -performance difference). It can still be useful in writing typechecks, -e.g.: - -.. code-block:: python - - def myfunction(ptr): - assert ffi.typeof(ptr) is ffi.typeof("foo_t*") - ... - -Note also that the mapping from strings like ``"foo_t*"`` to the -``<ctype>`` objects is stored in some internal dictionary. This -guarantees that there is only one ``<ctype 'foo_t *'>`` object, so you -can use the ``is`` operator to compare it. The downside is that the -dictionary entries are immortal for now. In the future, we may add -transparent reclamation of old, unused entries. In the meantime, note -that using strings like ``"int[%d]" % length`` to name a type will -create many immortal cached entries if called with many different -lengths. - -**ffi.sizeof("C type" or cdata object)**: return the size of the -argument in bytes. The argument can be either a C type, or a cdata object, -like in the equivalent ``sizeof`` operator in C. - -For ``array = ffi.new("T[]", n)``, then ``ffi.sizeof(array)`` returns -``n * ffi.sizeof("T")``. *New in version 1.9:* Similar rules apply for -structures with a variable-sized array at the end. More precisely, if -``p`` was returned by ``ffi.new("struct foo *", ...)``, then -``ffi.sizeof(p[0])`` now returns the total allocated size. In previous -versions, it used to just return ``ffi.sizeof(ffi.typeof(p[0]))``, which -is the size of the structure ignoring the variable-sized part. (Note -that due to alignment, it is possible for ``ffi.sizeof(p[0])`` to return -a value smaller than ``ffi.sizeof(ffi.typeof(p[0]))``.) - -**ffi.alignof("C type")**: return the natural alignment size in bytes of -the argument. Corresponds to the ``__alignof__`` operator in GCC. - - -.. _ffi-offsetof: -.. _ffi-addressof: - -ffi.offsetof(), ffi.addressof() -+++++++++++++++++++++++++++++++ - -**ffi.offsetof("C struct or array type", \*fields_or_indexes)**: return the -offset within the struct of the given field. Corresponds to ``offsetof()`` -in C. - -You can give several field names in case of nested structures. You -can also give numeric values which correspond to array items, in case -of a pointer or array type. For example, ``ffi.offsetof("int[5]", 2)`` -is equal to the size of two integers, as is ``ffi.offsetof("int *", 2)``. - - -**ffi.addressof(cdata, \*fields_or_indexes)**: limited equivalent to -the '&' operator in C: - -1. ``ffi.addressof(<cdata 'struct-or-union'>)`` returns a cdata that -is a pointer to this struct or union. The returned pointer is only -valid as long as the original ``cdata`` object is; be sure to keep it -alive if it was obtained directly from ``ffi.new()``. - -2. ``ffi.addressof(<cdata>, field-or-index...)`` returns the address -of a field or array item inside the given structure or array. In case -of nested structures or arrays, you can give more than one field or -index to look recursively. Note that ``ffi.addressof(array, index)`` -can also be expressed as ``array + index``: this is true both in CFFI -and in C, where ``&array[index]`` is just ``array + index``. - -3. ``ffi.addressof(<library>, "name")`` returns the address of the -named function or global variable from the given library object. -For functions, it returns a regular cdata -object containing a pointer to the function. - -Note that the case 1. cannot be used to take the address of a -primitive or pointer, but only a struct or union. It would be -difficult to implement because only structs and unions are internally -stored as an indirect pointer to the data. If you need a C int whose -address can be taken, use ``ffi.new("int[1]")`` in the first place; -similarly, for a pointer, use ``ffi.new("foo_t *[1]")``. - - -.. _ffi-cdata: -.. _ffi-ctype: - -ffi.CData, ffi.CType -++++++++++++++++++++ - -**ffi.CData, ffi.CType**: the Python type of the objects referred to -as ``<cdata>`` and ``<ctype>`` in the rest of this document. Note -that some cdata objects may be actually of a subclass of -``ffi.CData``, and similarly with ctype, so you should check with -``if isinstance(x, ffi.CData)``. Also, ``<ctype>`` objects have -a number of attributes for introspection: ``kind`` and ``cname`` are -always present, and depending on the kind they may also have -``item``, ``length``, ``fields``, ``args``, ``result``, ``ellipsis``, -``abi``, ``elements`` and ``relements``. - -*New in version 1.10:* ``ffi.buffer`` is now `a type`__ as well. - -.. __: #ffi-buffer - - -.. _ffi-gc: - -ffi.gc() -++++++++ - -**ffi.gc(cdata, destructor, size=0)**: -return a new cdata object that points to the -same data. Later, when this new cdata object is garbage-collected, -``destructor(old_cdata_object)`` will be called. Example of usage: -``ptr = ffi.gc(lib.custom_malloc(42), lib.custom_free)``. -Note that like objects -returned by ``ffi.new()``, the returned pointer objects have *ownership*, -which means the destructor is called as soon as *this* exact returned -object is garbage-collected. - -*New in version 1.12:* see also ``ffi.release()``. - -**ffi.gc(ptr, None, size=0)**: -removes the ownership on a object returned by a -regular call to ``ffi.gc``, and no destructor will be called when it -is garbage-collected. The object is modified in-place, and the -function returns ``None``. *New in version 1.7: ffi.gc(ptr, None)* - -Note that ``ffi.gc()`` should be avoided for limited resources, or (with -cffi below 1.11) for large memory allocations. This is particularly -true on PyPy: its GC does not know how much memory or how many resources -the returned ``ptr`` holds. It will only run its GC when enough memory -it knows about has been allocated (and thus run the destructor possibly -later than you would expect). Moreover, the destructor is called in -whatever thread PyPy is at that moment, which might be a problem for -some C libraries. In these cases, consider writing a wrapper class with -custom ``__enter__()`` and ``__exit__()`` methods, allocating and -freeing the C data at known points in time, and using it in a ``with`` -statement. In cffi 1.12, see also ``ffi.release()``. - -*New in version 1.11:* the ``size`` argument. If given, this should be -an estimate of the size (in bytes) that ``ptr`` keeps alive. This -information is passed on to the garbage collector, fixing part of the -problem described above. The ``size`` argument is most important on -PyPy; on CPython, it is ignored so far, but in the future it could be -used to trigger more eagerly the cyclic reference GC, too (see CPython -`issue 31105`__). - -The form ``ffi.gc(ptr, None, size=0)`` can be called with a negative -``size``, to cancel the estimate. It is not mandatory, though: -nothing gets out of sync if the size estimates do not match. It only -makes the next GC start more or less early. - -Note that if you have several ``ffi.gc()`` objects, the corresponding -destructors will be called in a random order. If you need a particular -order, see the discussion in `issue 340`__. - -.. __: http://bugs.python.org/issue31105 -.. __: https://foss.heptapod.net/pypy/cffi/-/issues/340 - - -.. _ffi-new-handle: -.. _ffi-from-handle: - -ffi.new_handle(), ffi.from_handle() -+++++++++++++++++++++++++++++++++++ - -**ffi.new_handle(python_object)**: return a non-NULL cdata of type -``void *`` that contains an opaque reference to ``python_object``. You -can pass it around to C functions or store it into C structures. Later, -you can use **ffi.from_handle(p)** to retrieve the original -``python_object`` from a value with the same ``void *`` pointer. -*Calling ffi.from_handle(p) is invalid and will likely crash if -the cdata object returned by new_handle() is not kept alive!* - -See a `typical usage example`_ below. - -(In case you are wondering, this ``void *`` is not the ``PyObject *`` -pointer. This wouldn't make sense on PyPy anyway.) - -The ``ffi.new_handle()/from_handle()`` functions *conceptually* work -like this: - -* ``new_handle()`` returns cdata objects that contains references to - the Python objects; we call them collectively the "handle" cdata - objects. The ``void *`` value in these handle cdata objects are - random but unique. - -* ``from_handle(p)`` searches all live "handle" cdata objects for the - one that has the same value ``p`` as its ``void *`` value. It then - returns the Python object referenced by that handle cdata object. - If none is found, you get "undefined behavior" (i.e. crashes). - -The "handle" cdata object keeps the Python object alive, similar to -how ``ffi.new()`` returns a cdata object that keeps a piece of memory -alive. If the handle cdata object *itself* is not alive any more, -then the association ``void * -> python_object`` is dead and -``from_handle()`` will crash. - -*New in version 1.4:* two calls to ``new_handle(x)`` are guaranteed to -return cdata objects with different ``void *`` values, even with the -same ``x``. This is a useful feature that avoids issues with unexpected -duplicates in the following trick: if you need to keep alive the -"handle" until explicitly asked to free it, but don't have a natural -Python-side place to attach it to, then the easiest is to ``add()`` it -to a global set. It can later be removed from the set by -``global_set.discard(p)``, with ``p`` any cdata object whose ``void *`` -value compares equal. - -.. _`typical usage example`: - -Usage example: suppose you have a C library where you must call a -``lib.process_document()`` function which invokes some callback. The -``process_document()`` function receives a pointer to a callback and a -``void *`` argument. The callback is then invoked with the ``void -*data`` argument that is equal to the provided value. In this typical -case, you can implement it like this (out-of-line API mode):: - - class MyDocument: - ... - - def process(self): - h = ffi.new_handle(self) - lib.process_document(lib.my_callback, # the callback - h, # 'void *data' - args...) - # 'h' stays alive until here, which means that the - # ffi.from_handle() done in my_callback() during - # the call to process_document() is safe - - def callback(self, arg1, arg2): - ... - - # the actual callback is this one-liner global function: - @ffi.def_extern() - def my_callback(arg1, arg2, data): - return ffi.from_handle(data).callback(arg1, arg2) - - -.. _ffi-dlopen: -.. _ffi-dlclose: - -ffi.dlopen(), ffi.dlclose() -+++++++++++++++++++++++++++ - -**ffi.dlopen(libpath, [flags])**: opens and returns a "handle" to a -dynamic library, as a ``<lib>`` object. See `Preparing and -Distributing modules`_. - -**ffi.dlclose(lib)**: explicitly closes a ``<lib>`` object returned -by ``ffi.dlopen()``. - -**ffi.RLTD_...**: constants: flags for ``ffi.dlopen()``. - - -ffi.new_allocator() -+++++++++++++++++++ - -**ffi.new_allocator(alloc=None, free=None, should_clear_after_alloc=True)**: -returns a new allocator. An "allocator" is a callable that behaves like -``ffi.new()`` but uses the provided low-level ``alloc`` and ``free`` -functions. *New in version 1.2.* - -``alloc()`` is invoked with the size as sole argument. If it returns -NULL, a MemoryError is raised. Later, if ``free`` is not None, it will -be called with the result of ``alloc()`` as argument. Both can be either -Python function or directly C functions. If only ``free`` is None, then no -free function is called. If both ``alloc`` and ``free`` are None, the -default alloc/free combination is used. (In other words, the call -``ffi.new(*args)`` is equivalent to ``ffi.new_allocator()(*args)``.) - -If ``should_clear_after_alloc`` is set to False, then the memory -returned by ``alloc()`` is assumed to be already cleared (or you are -fine with garbage); otherwise CFFI will clear it. Example: for -performance, if you are using ``ffi.new()`` to allocate large chunks of -memory where the initial content can be left uninitialized, you can do:: - - # at module level - new_nonzero = ffi.new_allocator(should_clear_after_alloc=False) - - # then replace `p = ffi.new("char[]", bigsize)` with: - p = new_nonzero("char[]", bigsize) - -**NOTE:** the following is a general warning that applies particularly -(but not only) to PyPy versions 5.6 or older (PyPy > 5.6 attempts to -account for the memory returned by ``ffi.new()`` or a custom allocator; -and CPython uses reference counting). If you do large allocations, then -there is no hard guarantee about when the memory will be freed. You -should avoid both ``new()`` and ``new_allocator()()`` if you want to be -sure that the memory is promptly released, e.g. before you allocate more -of it. - -An alternative is to declare and call the C ``malloc()`` and ``free()`` -functions, or some variant like ``mmap()`` and ``munmap()``. Then you -control exactly when the memory is allocated and freed. For example, -add these two lines to your existing ``ffibuilder.cdef()``:: - - void *malloc(size_t size); - void free(void *ptr); - -and then call these two functions manually:: - - p = lib.malloc(n * ffi.sizeof("int")) - try: - my_array = ffi.cast("int *", p) - ... - finally: - lib.free(p) - -In cffi version 1.12 you can indeed use ``ffi.new_allocator()`` but use the -``with`` statement (see ``ffi.release()``) to force the free function to be -called at a known point. The above is equivalent to this code:: - - my_new = ffi.new_allocator(lib.malloc, lib.free) # at global level - ... - with my_new("int[]", n) as my_array: - ... - -**Warning:** due to a bug, ``p = ffi.new_allocator(..)("struct-or-union *")`` -might not follow the rule that either ``p`` or ``p[0]`` keeps the memory -alive, which holds for the normal ``ffi.new("struct-or-union *")`` allocator. -It may sometimes be the case that if there is only a reference to ``p[0]``, -the memory is freed. The cause is that the rule doesn't hold for -``ffi.gc()``, which is sometimes used in the implementation of -``ffi.new_allocator()()``; this might be fixed in a future release. - - -.. _ffi-release: - -ffi.release() and the context manager -+++++++++++++++++++++++++++++++++++++ - -**ffi.release(cdata)**: release the resources held by a cdata object from -``ffi.new()``, ``ffi.gc()``, ``ffi.from_buffer()`` or -``ffi.new_allocator()()``. The cdata object must not be used afterwards. -The normal Python destructor of the cdata object releases the same resources, -but this allows the releasing to occur at a known time, as opposed as at an -unspecified point in the future. -*New in version 1.12.* - -``ffi.release(cdata)`` is equivalent to ``cdata.__exit__()``, which means that -you can use the ``with`` statement to ensure that the cdata is released at the -end of a block (in version 1.12 and above):: - - with ffi.from_buffer(...) as p: - do something with p - -The effect is more precisely as follows: - -* on an object returned from ``ffi.gc(destructor)``, ``ffi.release()`` will - cause the ``destructor`` to be called immediately. - -* on an object returned from a custom allocator, the custom free function - is called immediately. - -* on CPython, ``ffi.from_buffer(buf)`` locks the buffer, so ``ffi.release()`` - can be used to unlock it at a known time. On PyPy, there is no locking - (so far); the effect of ``ffi.release()`` is limited to removing the link, - allowing the original buffer object to be garbage-collected even if the - cdata object stays alive. - -* on CPython this method has no effect (so far) on objects returned by - ``ffi.new()``, because the memory is allocated inline with the cdata object - and cannot be freed independently. It might be fixed in future releases of - cffi. - -* on PyPy, ``ffi.release()`` frees the ``ffi.new()`` memory immediately. It is - useful because otherwise the memory is kept alive until the next GC occurs. - If you allocate large amounts of memory with ``ffi.new()`` and don't free - them with ``ffi.release()``, PyPy (>= 5.7) runs its GC more often to - compensate, so the total memory allocated should be kept within bounds - anyway; but calling ``ffi.release()`` explicitly should improve performance - by reducing the frequency of GC runs. - -After ``ffi.release(x)``, do not use anything pointed to by ``x`` any longer. -As an exception to this rule, you can call ``ffi.release(x)`` several times -for the exact same cdata object ``x``; the calls after the first one are -ignored. - - -ffi.init_once() -+++++++++++++++ - -**ffi.init_once(function, tag)**: run ``function()`` once. The -``tag`` should be a primitive object, like a string, that identifies -the function: ``function()`` is only called the first time we see the -``tag``. The return value of ``function()`` is remembered and -returned by the current and all future ``init_once()`` with the same -tag. If ``init_once()`` is called from multiple threads in parallel, -all calls block until the execution of ``function()`` is done. If -``function()`` raises an exception, it is propagated and nothing is -cached (i.e. ``function()`` will be called again, in case we catch the -exception and try ``init_once()`` again). *New in version 1.4.* - -Example:: - - from _xyz_cffi import ffi, lib - - def initlib(): - lib.init_my_library() - - def make_new_foo(): - ffi.init_once(initlib, "init") - return lib.make_foo() - -``init_once()`` is optimized to run very quickly if ``function()`` has -already been called. (On PyPy, the cost is zero---the JIT usually -removes everything in the machine code it produces.) - -*Note:* one motivation__ for ``init_once()`` is the CPython notion of -"subinterpreters" in the embedded case. If you are using the -out-of-line API mode, ``function()`` is called only once even in the -presence of multiple subinterpreters, and its return value is shared -among all subinterpreters. The goal is to mimic the way traditional -CPython C extension modules have their init code executed only once in -total even if there are subinterpreters. In the example above, the C -function ``init_my_library()`` is called once in total, not once per -subinterpreter. For this reason, avoid Python-level side-effects in -``function()`` (as they will only be applied in the first -subinterpreter to run); instead, return a value, as in the following -example:: - - def init_get_max(): - return lib.initialize_once_and_get_some_maximum_number() - - def process(i): - if i > ffi.init_once(init_get_max, "max"): - raise IndexError("index too large!") - ... - -.. __: https://foss.heptapod.net/pypy/cffi/-/issues/233 - - -.. _ffi-getctype: -.. _ffi-list-types: - -ffi.getctype(), ffi.list_types() -++++++++++++++++++++++++++++++++ - -**ffi.getctype("C type" or <ctype>, extra="")**: return the string -representation of the given C type. If non-empty, the "extra" string is -appended (or inserted at the right place in more complicated cases); it -can be the name of a variable to declare, or an extra part of the type -like ``"*"`` or ``"[5]"``. For example -``ffi.getctype(ffi.typeof(x), "*")`` returns the string representation -of the C type "pointer to the same type than x"; and -``ffi.getctype("char[80]", "a") == "char a[80]"``. - -**ffi.list_types()**: Returns the user type names known to this FFI -instance. This returns a tuple containing three lists of names: -``(typedef_names, names_of_structs, names_of_unions)``. *New in -version 1.6.* - - -.. _`Preparing and Distributing modules`: cdef.html#loading-libraries - - -Conversions ------------ - -This section documents all the conversions that are allowed when -*writing into* a C data structure (or passing arguments to a function -call), and *reading from* a C data structure (or getting the result of a -function call). The last column gives the type-specific operations -allowed. - -+---------------+------------------------+------------------+----------------+ -| C type | writing into | reading from |other operations| -+===============+========================+==================+================+ -| integers | an integer or anything | a Python int or | int(), bool() | -| and enums | on which int() works | long, depending | `[6]`, | -| `[5]` | (but not a float!). | on the type | ``<`` | -| | Must be within range. | (ver. 1.10: or a | | -| | | bool) | | -+---------------+------------------------+------------------+----------------+ -| ``char`` | a string of length 1 | a string of | int(), bool(), | -| | or another <cdata char>| length 1 | ``<`` | -+---------------+------------------------+------------------+----------------+ -| ``wchar_t``, | a unicode of length 1 | a unicode of | | -| ``char16_t``, | (or maybe 2 if | length 1 | int(), | -| ``char32_t`` | surrogates) or | (or maybe 2 if | bool(), ``<`` | -| `[8]` | another similar <cdata>| surrogates) | | -+---------------+------------------------+------------------+----------------+ -| ``float``, | a float or anything on | a Python float | float(), int(),| -| ``double`` | which float() works | | bool(), ``<`` | -+---------------+------------------------+------------------+----------------+ -|``long double``| another <cdata> with | a <cdata>, to | float(), int(),| -| | a ``long double``, or | avoid loosing | bool() | -| | anything on which | precision `[3]` | | -| | float() works | | | -+---------------+------------------------+------------------+----------------+ -| ``float`` | a complex number | a Python complex | complex(), | -| ``_Complex``, | or anything on which | number | bool() | -| ``double`` | complex() works | | `[7]` | -| ``_Complex`` | | | | -+---------------+------------------------+------------------+----------------+ -| pointers | another <cdata> with | a <cdata> |``[]`` `[4]`, | -| | a compatible type (i.e.| |``+``, ``-``, | -| | same type | |bool() | -| | or ``void*``, or as an | | | -| | array instead) `[1]` | | | -+---------------+------------------------+ | | -| ``void *`` | another <cdata> with | | | -| | any pointer or array | | | -| | type | | | -+---------------+------------------------+ +----------------+ -| pointers to | same as pointers | | ``[]``, ``+``, | -| structure or | | | ``-``, bool(), | -| union | | | and read/write | -| | | | struct fields | -+---------------+------------------------+ +----------------+ -| function | same as pointers | | bool(), | -| pointers | | | call `[2]` | -+---------------+------------------------+------------------+----------------+ -| arrays | a list or tuple of | a <cdata> |len(), iter(), | -| | items | |``[]`` `[4]`, | -| | | |``+``, ``-`` | -+---------------+------------------------+ +----------------+ -| ``char[]``, | same as arrays, or a | | len(), iter(), | -| ``un/signed`` | Python byte string | | ``[]``, ``+``, | -| ``char[]``, | | | ``-`` | -| ``_Bool[]`` | | | | -+---------------+------------------------+ +----------------+ -|``wchar_t[]``, | same as arrays, or a | | len(), iter(), | -|``char16_t[]``,| Python unicode string | | ``[]``, | -|``char32_t[]`` | | | ``+``, ``-`` | -| | | | | -+---------------+------------------------+------------------+----------------+ -| structure | a list or tuple or | a <cdata> | read/write | -| | dict of the field | | fields | -| | values, or a same-type | | | -| | <cdata> | | | -+---------------+------------------------+ +----------------+ -| union | same as struct, but | | read/write | -| | with at most one field | | fields | -+---------------+------------------------+------------------+----------------+ - -`[1]` ``item *`` is ``item[]`` in function arguments: - - In a function declaration, as per the C standard, a ``item *`` - argument is identical to a ``item[]`` argument (and ``ffi.cdef()`` - doesn't record the difference). So when you call such a function, - you can pass an argument that is accepted by either C type, like - for example passing a Python string to a ``char *`` argument - (because it works for ``char[]`` arguments) or a list of integers - to a ``int *`` argument (it works for ``int[]`` arguments). Note - that even if you want to pass a single ``item``, you need to - specify it in a list of length 1; for example, a ``struct point_s - *`` argument might be passed as ``[[x, y]]`` or ``[{'x': 5, 'y': - 10}]``. - - As an optimization, CFFI assumes that a - function with a ``char *`` argument to which you pass a Python - string will not actually modify the array of characters passed in, - and so passes directly a pointer inside the Python string object. - (On PyPy, this optimization is only available since PyPy 5.4 - with CFFI 1.8.) - -`[2]` C function calls are done with the GIL released. - - Note that we assume that the called functions are *not* using the - Python API from Python.h. For example, we don't check afterwards - if they set a Python exception. You may work around it, but mixing - CFFI with ``Python.h`` is not recommended. (If you do that, on - PyPy and on some platforms like Windows, you may need to explicitly - link to ``libpypy-c.dll`` to access the CPython C API compatibility - layer; indeed, CFFI-generated modules on PyPy don't link to - ``libpypy-c.dll`` on their own. But really, don't do that in the - first place.) - -`[3]` ``long double`` support: - - We keep ``long double`` values inside a cdata object to avoid - loosing precision. Normal Python floating-point numbers only - contain enough precision for a ``double``. If you really want to - convert such an object to a regular Python float (i.e. a C - ``double``), call ``float()``. If you need to do arithmetic on - such numbers without any precision loss, you need instead to define - and use a family of C functions like ``long double add(long double - a, long double b);``. - -`[4]` Slicing with ``x[start:stop]``: - - Slicing is allowed, as long as you specify explicitly both ``start`` - and ``stop`` (and don't give any ``step``). It gives a cdata - object that is a "view" of all items from ``start`` to ``stop``. - It is a cdata of type "array" (so e.g. passing it as an argument to a - C function would just convert it to a pointer to the ``start`` item). - As with indexing, negative bounds mean really negative indices, like in - C. As for slice assignment, it accepts any iterable, including a list - of items or another array-like cdata object, but the length must match. - (Note that this behavior differs from initialization: e.g. you can - say ``chararray[10:15] = "hello"``, but the assigned string must be of - exactly the correct length; no implicit null character is added.) - -`[5]` Enums are handled like ints: - - Like C, enum types are mostly int types (unsigned or signed, int or - long; note that GCC's first choice is unsigned). Reading an enum - field of a structure, for example, returns you an integer. To - compare their value symbolically, use code like ``if x.field == - lib.FOO``. If you really want to get their value as a string, use - ``ffi.string(ffi.cast("the_enum_type", x.field))``. - -`[6]` bool() on a primitive cdata: - - *New in version 1.7.* In previous versions, it only worked on - pointers; for primitives it always returned True. - - *New in version 1.10:* The C type ``_Bool`` or ``bool`` converts to - Python booleans now. You get an exception if a C ``_Bool`` happens - to contain a value different from 0 and 1 (this case triggers - undefined behavior in C; if you really have to interface with a - library relying on this, don't use ``_Bool`` in the CFFI side). - Also, when converting from a byte string to a ``_Bool[]``, only the - bytes ``\x00`` and ``\x01`` are accepted. - -`[7]` libffi does not support complex numbers: - - *New in version 1.11:* CFFI now supports complex numbers directly. - Note however that libffi does not. This means that C functions that - take directly as argument types or return type a complex type cannot - be called by CFFI, unless they are directly using the API mode. - -`[8]` ``wchar_t``, ``char16_t`` and ``char32_t`` - - See `Unicode character types`_ below. - - -.. _file: - -Support for FILE -++++++++++++++++ - -You can declare C functions taking a ``FILE *`` argument and -call them with a Python file object. If needed, you can also do ``c_f -= ffi.cast("FILE *", fileobj)`` and then pass around ``c_f``. - -Note, however, that CFFI does this by a best-effort approach. If you -need finer control over buffering, flushing, and timely closing of the -``FILE *``, then you should not use this special support for ``FILE *``. -Instead, you can handle regular ``FILE *`` cdata objects that you -explicitly make using fdopen(), like this: - -.. code-block:: python - - ffi.cdef(''' - FILE *fdopen(int, const char *); // from the C <stdio.h> - int fclose(FILE *); - ''') - - myfile.flush() # make sure the file is flushed - newfd = os.dup(myfile.fileno()) # make a copy of the file descriptor - fp = lib.fdopen(newfd, "w") # make a cdata 'FILE *' around newfd - lib.write_stuff_to_file(fp) # invoke the external function - lib.fclose(fp) # when you're done, close fp (and newfd) - -The special support for ``FILE *`` is anyway implemented in a similar manner -on CPython 3.x and on PyPy, because these Python implementations' files are -not natively based on ``FILE *``. Doing it explicity offers more control. - - -.. _unichar: - -Unicode character types -+++++++++++++++++++++++ - -The ``wchar_t`` type has the same signedness as the underlying -platform's. For example, on Linux, it is a signed 32-bit integer. -However, the types ``char16_t`` and ``char32_t`` (*new in version 1.11*) -are always unsigned. - -Note that CFFI assumes that these types are meant to contain UTF-16 or -UTF-32 characters in the native endianness. More precisely: - -* ``char32_t`` is assumed to contain UTF-32, or UCS4, which is just the - unicode codepoint; - -* ``char16_t`` is assumed to contain UTF-16, i.e. UCS2 plus surrogates; - -* ``wchar_t`` is assumed to contain either UTF-32 or UTF-16 based on its - actual platform-defined size of 4 or 2 bytes. - -Whether this assumption is true or not is unspecified by the C language. -In theory, the C library you are interfacing with could use one of these -types with a different meaning. You would then need to handle it -yourself---for example, by using ``uint32_t`` instead of ``char32_t`` in -the ``cdef()``, and building the expected arrays of ``uint32_t`` -manually. - -Python itself can be compiled with ``sys.maxunicode == 65535`` or -``sys.maxunicode == 1114111`` (Python >= 3.3 is always 1114111). This -changes the handling of surrogates (which are pairs of 16-bit -"characters" which actually stand for a single codepoint whose value is -greater than 65535). If your Python is ``sys.maxunicode == 1114111``, -then it can store arbitrary unicode codepoints; surrogates are -automatically inserted when converting from Python unicodes to UTF-16, -and automatically removed when converting back. On the other hand, if -your Python is ``sys.maxunicode == 65535``, then it is the other way -around: surrogates are removed when converting from Python unicodes -to UTF-32, and added when converting back. In other words, surrogate -conversion is done only when there is a size mismatch. - -Note that Python's internal representations is not specified. For -example, on CPython >= 3.3, it will use 1- or 2- or 4-bytes arrays -depending on what the string actually contains. With CFFI, when you -pass a Python byte string to a C function expecting a ``char*``, then -we pass directly a pointer to the existing data without needing a -temporary buffer; however, the same cannot cleanly be done with -*unicode* string arguments and the ``wchar_t*`` / ``char16_t*`` / -``char32_t*`` types, because of the changing internal -representation. As a result, and for consistency, CFFI always allocates -a temporary buffer for unicode strings. - -**Warning:** for now, if you use ``char16_t`` and ``char32_t`` with -``set_source()``, you have to make sure yourself that the types are -declared by the C source you provide to ``set_source()``. They would be -declared if you ``#include`` a library that explicitly uses them, for -example, or when using C++11. Otherwise, you need ``#include -<uchar.h>`` on Linux, or more generally something like ``typedef -uint16_t char16_t;``. This is not done automatically by CFFI because -``uchar.h`` is not standard across platforms, and writing a ``typedef`` -like above would crash if the type happens to be already defined. diff --git a/doc/source/using.rst b/doc/source/using.rst deleted file mode 100644 index 38c96ba..0000000 --- a/doc/source/using.rst +++ /dev/null @@ -1,1043 +0,0 @@ -================================ -Using the ffi/lib objects -================================ - -.. contents:: - -Keep this page under your pillow. - - -.. _working: - -Working with pointers, structures and arrays --------------------------------------------- - -The C code's integers and floating-point values are mapped to Python's -regular ``int``, ``long`` and ``float``. Moreover, the C type ``char`` -corresponds to single-character strings in Python. (If you want it to -map to small integers, use either ``signed char`` or ``unsigned char``.) - -Similarly, the C type ``wchar_t`` corresponds to single-character -unicode strings. Note that in some situations (a narrow Python build -with an underlying 4-bytes wchar_t type), a single wchar_t character -may correspond to a pair of surrogates, which is represented as a -unicode string of length 2. If you need to convert such a 2-chars -unicode string to an integer, ``ord(x)`` does not work; use instead -``int(ffi.cast('wchar_t', x))``. - -*New in version 1.11:* in addition to ``wchar_t``, the C types -``char16_t`` and ``char32_t`` work the same but with a known fixed size. -In previous versions, this could be achieved using ``uint16_t`` and -``int32_t`` but without automatic conversion to Python unicodes. - -Pointers, structures and arrays are more complex: they don't have an -obvious Python equivalent. Thus, they correspond to objects of type -``cdata``, which are printed for example as -``<cdata 'struct foo_s *' 0xa3290d8>``. - -``ffi.new(ctype, [initializer])``: this function builds and returns a -new cdata object of the given ``ctype``. The ctype is usually some -constant string describing the C type. It must be a pointer or array -type. If it is a pointer, e.g. ``"int *"`` or ``struct foo *``, then -it allocates the memory for one ``int`` or ``struct foo``. If it is -an array, e.g. ``int[10]``, then it allocates the memory for ten -``int``. In both cases the returned cdata is of type ``ctype``. - -The memory is initially filled with zeros. An initializer can be given -too, as described later. - -Example:: - - >>> ffi.new("int *") - <cdata 'int *' owning 4 bytes> - >>> ffi.new("int[10]") - <cdata 'int[10]' owning 40 bytes> - - >>> ffi.new("char *") # allocates only one char---not a C string! - <cdata 'char *' owning 1 bytes> - >>> ffi.new("char[]", "foobar") # this allocates a C string, ending in \0 - <cdata 'char[]' owning 7 bytes> - -Unlike C, the returned pointer object has *ownership* on the allocated -memory: when this exact object is garbage-collected, then the memory is -freed. If, at the level of C, you store a pointer to the memory -somewhere else, then make sure you also keep the object alive for as -long as needed. (This also applies if you immediately cast the returned -pointer to a pointer of a different type: only the original object has -ownership, so you must keep it alive. As soon as you forget it, then -the casted pointer will point to garbage! In other words, the ownership -rules are attached to the *wrapper* cdata objects: they are not, and -cannot, be attached to the underlying raw memory.) Example: - -.. code-block:: python - - global_weakkeydict = weakref.WeakKeyDictionary() - - def make_foo(): - s1 = ffi.new("struct foo *") - fld1 = ffi.new("struct bar *") - fld2 = ffi.new("struct bar *") - s1.thefield1 = fld1 - s1.thefield2 = fld2 - # here the 'fld1' and 'fld2' object must not go away, - # otherwise 's1.thefield1/2' will point to garbage! - global_weakkeydict[s1] = (fld1, fld2) - # now 's1' keeps alive 'fld1' and 'fld2'. When 's1' goes - # away, then the weak dictionary entry will be removed. - return s1 - -Usually you don't need a weak dict: for example, to call a function with -a ``char * *`` argument that contains a pointer to a ``char *`` pointer, -it is enough to do this: - -.. code-block:: python - - p = ffi.new("char[]", "hello, world") # p is a 'char *' - q = ffi.new("char **", p) # q is a 'char **' - lib.myfunction(q) - # p is alive at least until here, so that's fine - -However, this is always wrong (usage of freed memory): - -.. code-block:: python - - p = ffi.new("char **", ffi.new("char[]", "hello, world")) - # WRONG! as soon as p is built, the inner ffi.new() gets freed! - -This is wrong too, for the same reason: - -.. code-block:: python - - p = ffi.new("struct my_stuff") - p.foo = ffi.new("char[]", "hello, world") - # WRONG! as soon as p.foo is set, the ffi.new() gets freed! - - -The cdata objects support mostly the same operations as in C: you can -read or write from pointers, arrays and structures. Dereferencing a -pointer is done usually in C with the syntax ``*p``, which is not valid -Python, so instead you have to use the alternative syntax ``p[0]`` -(which is also valid C). Additionally, the ``p.x`` and ``p->x`` -syntaxes in C both become ``p.x`` in Python. - -We have ``ffi.NULL`` to use in the same places as the C ``NULL``. -Like the latter, it is actually defined to be ``ffi.cast("void *", -0)``. For example, reading a NULL pointer returns a ``<cdata 'type *' -NULL>``, which you can check for e.g. by comparing it with -``ffi.NULL``. - -There is no general equivalent to the ``&`` operator in C (because it -would not fit nicely in the model, and it does not seem to be needed -here). There is `ffi.addressof()`__, but only for some cases. You -cannot take the "address" of a number in Python, for example; similarly, -you cannot take the address of a CFFI pointer. If you have this kind -of C code:: - - int x, y; - fetch_size(&x, &y); - - opaque_t *handle; // some opaque pointer - init_stuff(&handle); // initializes the variable 'handle' - more_stuff(handle); // pass the handle around to more functions - -then you need to rewrite it like this, replacing the variables in C -with what is logically pointers to the variables: - -.. code-block:: python - - px = ffi.new("int *") - py = ffi.new("int *") arr = ffi.new("int[2]") - lib.fetch_size(px, py) -OR- lib.fetch_size(arr, arr + 1) - x = px[0] x = arr[0] - y = py[0] y = arr[1] - - p_handle = ffi.new("opaque_t **") - lib.init_stuff(p_handle) # pass the pointer to the 'handle' pointer - handle = p_handle[0] # now we can read 'handle' out of 'p_handle' - lib.more_stuff(handle) - -.. __: ref.html#ffi-addressof - - -Any operation that would in C return a pointer or array or struct type -gives you a fresh cdata object. Unlike the "original" one, these fresh -cdata objects don't have ownership: they are merely references to -existing memory. - -As an exception to the above rule, dereferencing a pointer that owns a -*struct* or *union* object returns a cdata struct or union object -that "co-owns" the same memory. Thus in this case there are two -objects that can keep the same memory alive. This is done for cases where -you really want to have a struct object but don't have any convenient -place to keep alive the original pointer object (returned by -``ffi.new()``). - -Example: - -.. code-block:: python - - # void somefunction(int *); - - x = ffi.new("int *") # allocate one int, and return a pointer to it - x[0] = 42 # fill it - lib.somefunction(x) # call the C function - print x[0] # read the possibly-changed value - -The equivalent of C casts are provided with ``ffi.cast("type", value)``. -They should work in the same cases as they do in C. Additionally, this -is the only way to get cdata objects of integer or floating-point type:: - - >>> x = ffi.cast("int", 42) - >>> x - <cdata 'int' 42> - >>> int(x) - 42 - -To cast a pointer to an int, cast it to ``intptr_t`` or ``uintptr_t``, -which are defined by C to be large enough integer types (example on 32 -bits):: - - >>> int(ffi.cast("intptr_t", pointer_cdata)) # signed - -1340782304 - >>> int(ffi.cast("uintptr_t", pointer_cdata)) # unsigned - 2954184992L - -The initializer given as the optional second argument to ``ffi.new()`` -can be mostly anything that you would use as an initializer for C code, -with lists or tuples instead of using the C syntax ``{ .., .., .. }``. -Example:: - - typedef struct { int x, y; } foo_t; - - foo_t v = { 1, 2 }; // C syntax - v = ffi.new("foo_t *", [1, 2]) # CFFI equivalent - - foo_t v = { .y=1, .x=2 }; // C99 syntax - v = ffi.new("foo_t *", {'y': 1, 'x': 2}) # CFFI equivalent - -Like C, arrays of chars can also be initialized from a string, in -which case a terminating null character is appended implicitly:: - - >>> x = ffi.new("char[]", "hello") - >>> x - <cdata 'char[]' owning 6 bytes> - >>> len(x) # the actual size of the array - 6 - >>> x[5] # the last item in the array - '\x00' - >>> x[0] = 'H' # change the first item - >>> ffi.string(x) # interpret 'x' as a regular null-terminated string - 'Hello' - -Similarly, arrays of wchar_t or char16_t or char32_t can be initialized -from a unicode string, -and calling ``ffi.string()`` on the cdata object returns the current unicode -string stored in the source array (adding surrogates if necessary). -See the `Unicode character types`__ section for more details. - -.. __: ref.html#unichar - -Note that unlike Python lists or tuples, but like C, you *cannot* index in -a C array from the end using negative numbers. - -More generally, the C array types can have their length unspecified in C -types, as long as their length can be derived from the initializer, like -in C:: - - int array[] = { 1, 2, 3, 4 }; // C syntax - array = ffi.new("int[]", [1, 2, 3, 4]) # CFFI equivalent - -As an extension, the initializer can also be just a number, giving -the length (in case you just want zero-initialization):: - - int array[1000]; // C syntax - array = ffi.new("int[1000]") # CFFI 1st equivalent - array = ffi.new("int[]", 1000) # CFFI 2nd equivalent - -This is useful if the length is not actually a constant, to avoid things -like ``ffi.new("int[%d]" % x)``. Indeed, this is not recommended: -``ffi`` normally caches the string ``"int[]"`` to not need to re-parse -it all the time. - -The C99 variable-sized structures are supported too, as long as the -initializer says how long the array should be: - -.. code-block:: python - - # typedef struct { int x; int y[]; } foo_t; - - p = ffi.new("foo_t *", [5, [6, 7, 8]]) # length 3 - p = ffi.new("foo_t *", [5, 3]) # length 3 with 0 in the array - p = ffi.new("foo_t *", {'y': 3}) # length 3 with 0 everywhere - -Finally, note that any Python object used as initializer can also be -used directly without ``ffi.new()`` in assignments to array items or -struct fields. In fact, ``p = ffi.new("T*", initializer)`` is -equivalent to ``p = ffi.new("T*"); p[0] = initializer``. Examples: - -.. code-block:: python - - # if 'p' is a <cdata 'int[5][5]'> - p[2] = [10, 20] # writes to p[2][0] and p[2][1] - - # if 'p' is a <cdata 'foo_t *'>, and foo_t has fields x, y and z - p[0] = {'x': 10, 'z': 20} # writes to p.x and p.z; p.y unmodified - - # if, on the other hand, foo_t has a field 'char a[5]': - p.a = "abc" # writes 'a', 'b', 'c' and '\0'; p.a[4] unmodified - -In function calls, when passing arguments, these rules can be used too; -see `Function calls`_. - - -Python 3 support ----------------- - -Python 3 is supported, but the main point to note is that the ``char`` C -type corresponds to the ``bytes`` Python type, and not ``str``. It is -your responsibility to encode/decode all Python strings to bytes when -passing them to or receiving them from CFFI. - -This only concerns the ``char`` type and derivative types; other parts -of the API that accept strings in Python 2 continue to accept strings in -Python 3. - - -An example of calling a main-like thing ---------------------------------------- - -Imagine we have something like this: - -.. code-block:: python - - from cffi import FFI - ffi = FFI() - ffi.cdef(""" - int main_like(int argv, char *argv[]); - """) - lib = ffi.dlopen("some_library.so") - -Now, everything is simple, except, how do we create the ``char**`` argument -here? -The first idea: - -.. code-block:: python - - lib.main_like(2, ["arg0", "arg1"]) - -does not work, because the initializer receives two Python ``str`` objects -where it was expecting ``<cdata 'char *'>`` objects. You need to use -``ffi.new()`` explicitly to make these objects: - -.. code-block:: python - - lib.main_like(2, [ffi.new("char[]", "arg0"), - ffi.new("char[]", "arg1")]) - -Note that the two ``<cdata 'char[]'>`` objects are kept alive for the -duration of the call: they are only freed when the list itself is freed, -and the list is only freed when the call returns. - -If you want instead to build an "argv" variable that you want to reuse, -then more care is needed: - -.. code-block:: python - - # DOES NOT WORK! - argv = ffi.new("char *[]", [ffi.new("char[]", "arg0"), - ffi.new("char[]", "arg1")]) - -In the above example, the inner "arg0" string is deallocated as soon -as "argv" is built. You have to make sure that you keep a reference -to the inner "char[]" objects, either directly or by keeping the list -alive like this: - -.. code-block:: python - - argv_keepalive = [ffi.new("char[]", "arg0"), - ffi.new("char[]", "arg1")] - argv = ffi.new("char *[]", argv_keepalive) - - -Function calls --------------- - -When calling C functions, passing arguments follows mostly the same -rules as assigning to structure fields, and the return value follows the -same rules as reading a structure field. For example: - -.. code-block:: python - - # int foo(short a, int b); - - n = lib.foo(2, 3) # returns a normal integer - lib.foo(40000, 3) # raises OverflowError - -You can pass to ``char *`` arguments a normal Python string (but don't -pass a normal Python string to functions that take a ``char *`` -argument and may mutate it!): - -.. code-block:: python - - # size_t strlen(const char *); - - assert lib.strlen("hello") == 5 - -You can also pass unicode strings as ``wchar_t *`` or ``char16_t *`` or -``char32_t *`` arguments. Note that -the C language makes no difference between argument declarations that -use ``type *`` or ``type[]``. For example, ``int *`` is fully -equivalent to ``int[]`` (or even ``int[5]``; the 5 is ignored). For CFFI, -this means that you can always pass arguments that can be converted to -either ``int *`` or ``int[]``. For example: - -.. code-block:: python - - # void do_something_with_array(int *array); - - lib.do_something_with_array([1, 2, 3, 4, 5]) # works for int[] - -See `Reference: conversions`__ for a similar way to pass ``struct foo_s -*`` arguments---but in general, it is clearer in this case to pass -``ffi.new('struct foo_s *', initializer)``. - -__ ref.html#conversions - -CFFI supports passing and returning structs and unions to functions and -callbacks. Example: - -.. code-block:: python - - # struct foo_s { int a, b; }; - # struct foo_s function_returning_a_struct(void); - - myfoo = lib.function_returning_a_struct() - # `myfoo`: <cdata 'struct foo_s' owning 8 bytes> - -For performance, non-variadic API-level functions that you get by -writing ``lib.some_function`` are not ``<cdata>`` -objects, but an object of a different type (on CPython, ``<built-in -function>``). This means you cannot pass them directly to some other C -function expecting a function pointer argument. Only ``ffi.typeof()`` -works on them. To get a cdata containing a regular function pointer, -use ``ffi.addressof(lib, "name")``. - -There are a few (obscure) limitations to the supported argument and -return types. These limitations come from libffi and apply only to -calling ``<cdata>`` function pointers; in other words, they don't -apply to non-variadic ``cdef()``-declared functions if you are using -the API mode. The limitations are that you cannot pass directly as -argument or return type: - -* a union (but a *pointer* to a union is fine); - -* a struct which uses bitfields (but a *pointer* to such a struct is - fine); - -* a struct that was declared with "``...``" in the ``cdef()``. - -In API mode, you can work around these limitations: for example, if you -need to call such a function pointer from Python, you can instead write -a custom C function that accepts the function pointer and the real -arguments and that does the call from C. Then declare that custom C -function in the ``cdef()`` and use it from Python. - - -Variadic function calls ------------------------ - -Variadic functions in C (which end with "``...``" as their last -argument) can be declared and called normally, with the exception that -all the arguments passed in the variable part *must* be cdata objects. -This is because it would not be possible to guess, if you wrote this:: - - lib.printf("hello, %d\n", 42) # doesn't work! - -that you really meant the 42 to be passed as a C ``int``, and not a -``long`` or ``long long``. The same issue occurs with ``float`` versus -``double``. So you have to force cdata objects of the C type you want, -if necessary with ``ffi.cast()``: - -.. code-block:: python - - lib.printf("hello, %d\n", ffi.cast("int", 42)) - lib.printf("hello, %ld\n", ffi.cast("long", 42)) - lib.printf("hello, %f\n", ffi.cast("double", 42)) - -But of course: - -.. code-block:: python - - lib.printf("hello, %s\n", ffi.new("char[]", "world")) - -Note that if you are using ``dlopen()``, the function declaration in the -``cdef()`` must match the original one in C exactly, as usual --- in -particular, if this function is variadic in C, then its ``cdef()`` -declaration must also be variadic. You cannot declare it in the -``cdef()`` with fixed arguments instead, even if you plan to only call -it with these argument types. The reason is that some architectures -have a different calling convention depending on whether the function -signature is fixed or not. (On x86-64, the difference can sometimes be -seen in PyPy's JIT-generated code if some arguments are ``double``.) - -Note that the function signature ``int foo();`` is interpreted by CFFI -as equivalent to ``int foo(void);``. This differs from the C standard, -in which ``int foo();`` is really like ``int foo(...);`` and can be -called with any arguments. (This feature of C is a pre-C89 relic: the -arguments cannot be accessed at all in the body of ``foo()`` without -relying on compiler-specific extensions. Nowadays virtually all code -with ``int foo();`` really means ``int foo(void);``.) - - -Memory pressure (PyPy) ----------------------- - -This paragraph applies only to PyPy, because its garbage collector (GC) -is different from CPython's. It is very common in C code to have pairs -of functions, one which performs memory allocations or acquires other -resources, and the other which frees them again. Depending on how you -structure your Python code, the freeing function is only called when the -GC decides a particular (Python) object can be freed. This occurs -notably in these cases: - -* If you use a ``__del__()`` method to call the freeing function. - -* If you use ``ffi.gc()`` without also using ``ffi.release()``. - -* This does not occur if you call the freeing function at a - deterministic time, like in a regular ``try: finally:`` block. It - does however occur *inside a generator---* if the generator is not - explicitly exhausted but forgotten at a ``yield`` point, then the code - in the enclosing ``finally`` block is only invoked at the next GC. - -In these cases, you may have to use the built-in function -``__pypy__.add_memory_pressure(n)``. Its argument ``n`` is an estimate -of how much memory pressure to add. For example, if the pair of C -functions that we are talking about is ``malloc(n)`` and ``free()`` or -similar, you would call ``__pypy__.add_memory_pressure(n)`` after -``malloc(n)``. Doing so is not always a complete answer to the problem, -but it makes the next GC occur earlier, which is often enough. - -The same applies if the memory allocations are indirect, e.g. the C -function allocates some internal data structures. In that case, call -``__pypy__.add_memory_pressure(n)`` with an argument ``n`` that is an -rough estimation. Knowing the exact size is not important, and memory -pressure doesn't have to be manually brought down again after calling -the freeing function. If you are writing wrappers for the allocating / -freeing pair of functions, you should probably call -``__pypy__.add_memory_pressure()`` in the former even if the user may -invoke the latter at a known point with a ``finally:`` block. - -In case this solution is not sufficient, or if the acquired resource is -not memory but something else more limited (like file descriptors), then -there is no better way than restructuring your code to make sure the -freeing function is called at a known point and not indirectly by the -GC. - -Note that in PyPy <= 5.6 the discussion above also applies to -``ffi.new()``. In more recent versions of PyPy, both ``ffi.new()`` and -``ffi.new_allocator()()`` automatically account for the memory pressure -they create. (In case you need to support both older and newer PyPy's, -try calling ``__pypy__.add_memory_pressure()`` anyway; it is better to -overestimate than not account for the memory pressure.) - - -.. _extern-python: -.. _`extern "Python"`: - -Extern "Python" (new-style callbacks) -------------------------------------- - -When the C code needs a pointer to a function which invokes back a -Python function of your choice, here is how you do it in the -out-of-line API mode. The next section about Callbacks_ describes the -ABI-mode solution. - -This is *new in version 1.4.* Use old-style Callbacks_ if backward -compatibility is an issue. (The original callbacks are slower to -invoke and have the same issue as libffi's callbacks; notably, see the -warning__. The new style described in the present section does not -use libffi's callbacks at all.) - -.. __: Callbacks_ - -In the builder script, declare in the cdef a function prefixed with -``extern "Python"``:: - - ffibuilder.cdef(""" - extern "Python" int my_callback(int, int); - - void library_function(int(*callback)(int, int)); - """) - ffibuilder.set_source("_my_example", r""" - #include <some_library.h> - """) - -The function ``my_callback()`` is then implemented in Python inside -your application's code:: - - from _my_example import ffi, lib - - @ffi.def_extern() - def my_callback(x, y): - return 42 - -You obtain a ``<cdata>`` pointer-to-function object by getting -``lib.my_callback``. This ``<cdata>`` can be passed to C code and -then works like a callback: when the C code calls this function -pointer, the Python function ``my_callback`` is called. (You need -to pass ``lib.my_callback`` to C code, and not ``my_callback``: the -latter is just the Python function above, which cannot be passed to C.) - -CFFI implements this by defining ``my_callback`` as a static C -function, written after the ``set_source()`` code. The ``<cdata>`` -then points to this function. What this function does is invoke the -Python function object that is, at runtime, attached with -``@ffi.def_extern()``. - -The ``@ffi.def_extern()`` decorator should be applied to **global -functions,** one for each ``extern "Python"`` function of the same -name. - -To support some corner cases, it is possible to redefine the attached -Python function by calling ``@ffi.def_extern()`` again for the same -name---but this is not recommended! Better attach a single global -Python function for this name, and write it more flexibly in the first -place. This is because each ``extern "Python"`` function turns into -only one C function. Calling ``@ffi.def_extern()`` again changes this -function's C logic to call the new Python function; the old Python -function is not callable any more. The C function pointer you get -from ``lib.my_function`` is always this C function's address, i.e. it -remains the same. - -Extern "Python" and ``void *`` arguments -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -As described just before, you cannot use ``extern "Python"`` to make a -variable number of C function pointers. However, achieving that -result is not possible in pure C code either. For this reason, it is -usual for C to define callbacks with a ``void *data`` argument. You -can use ``ffi.new_handle()`` and ``ffi.from_handle()`` to pass a -Python object through this ``void *`` argument. For example, if the C -type of the callbacks is:: - - typedef void (*event_cb_t)(event_t *evt, void *userdata); - -and you register events by calling this function:: - - void event_cb_register(event_cb_t cb, void *userdata); - -Then you would write this in the build script:: - - ffibuilder.cdef(""" - typedef ... event_t; - typedef void (*event_cb_t)(event_t *evt, void *userdata); - void event_cb_register(event_cb_t cb, void *userdata); - - extern "Python" void my_event_callback(event_t *, void *); - """) - ffibuilder.set_source("_demo_cffi", r""" - #include <the_event_library.h> - """) - -and in your main application you register events like this:: - - from _demo_cffi import ffi, lib - - class Widget(object): - def __init__(self): - userdata = ffi.new_handle(self) - self._userdata = userdata # must keep this alive! - lib.event_cb_register(lib.my_event_callback, userdata) - - def process_event(self, evt): - print "got event!" - - @ffi.def_extern() - def my_event_callback(evt, userdata): - widget = ffi.from_handle(userdata) - widget.process_event(evt) - -Some other libraries don't have an explicit ``void *`` argument, but -let you attach the ``void *`` to an existing structure. For example, -the library might say that ``widget->userdata`` is a generic field -reserved for the application. If the event's signature is now this:: - - typedef void (*event_cb_t)(widget_t *w, event_t *evt); - -Then you can use the ``void *`` field in the low-level -``widget_t *`` like this:: - - from _demo_cffi import ffi, lib - - class Widget(object): - def __init__(self): - ll_widget = lib.new_widget(500, 500) - self.ll_widget = ll_widget # <cdata 'struct widget *'> - userdata = ffi.new_handle(self) - self._userdata = userdata # must still keep this alive! - ll_widget.userdata = userdata # this makes a copy of the "void *" - lib.event_cb_register(ll_widget, lib.my_event_callback) - - def process_event(self, evt): - print "got event!" - - @ffi.def_extern() - def my_event_callback(ll_widget, evt): - widget = ffi.from_handle(ll_widget.userdata) - widget.process_event(evt) - -Extern "Python" accessed from C directly -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -In case you want to access some ``extern "Python"`` function directly -from the C code written in ``set_source()``, you need to write a -forward declaration. (By default it needs to be static, but see -`next paragraph`__.) The real implementation of this function -is added by CFFI *after* the C code---this is needed because the -declaration might use types defined by ``set_source()`` -(e.g. ``event_t`` above, from the ``#include``), so it cannot be -generated before. - -.. __: `extern-python-c`_ - -:: - - ffibuilder.set_source("_demo_cffi", r""" - #include <the_event_library.h> - - static void my_event_callback(widget_t *, event_t *); - - /* here you can write C code which uses '&my_event_callback' */ - """) - -This can also be used to write custom C code which calls Python -directly. Here is an example (inefficient in this case, but might be -useful if the logic in ``my_algo()`` is much more complex):: - - ffibuilder.cdef(""" - extern "Python" int f(int); - int my_algo(int); - """) - ffibuilder.set_source("_example_cffi", r""" - static int f(int); /* the forward declaration */ - - static int my_algo(int n) { - int i, sum = 0; - for (i = 0; i < n; i++) - sum += f(i); /* call f() here */ - return sum; - } - """) - -.. _extern-python-c: - -Extern "Python+C" -~~~~~~~~~~~~~~~~~ - -Functions declared with ``extern "Python"`` are generated as -``static`` functions in the C source. However, in some cases it is -convenient to make them non-static, typically when you want to make -them directly callable from other C source files. To do that, you can -say ``extern "Python+C"`` instead of just ``extern "Python"``. *New -in version 1.6.* - -+------------------------------------+--------------------------------------+ -| if the cdef contains | then CFFI generates | -+------------------------------------+--------------------------------------+ -| ``extern "Python" int f(int);`` | ``static int f(int) { /* code */ }`` | -+------------------------------------+--------------------------------------+ -| ``extern "Python+C" int f(int);`` | ``int f(int) { /* code */ }`` | -+------------------------------------+--------------------------------------+ - -The name ``extern "Python+C"`` comes from the fact that we want an -extern function in both senses: as an ``extern "Python"``, and as a -C function that is not static. - -You cannot make CFFI generate additional macros or other -compiler-specific stuff like the GCC ``__attribute__``. You can only -control whether the function should be ``static`` or not. But often, -these attributes must be written alongside the function *header*, and -it is fine if the function *implementation* does not repeat them:: - - ffibuilder.cdef(""" - extern "Python+C" int f(int); /* not static */ - """) - ffibuilder.set_source("_example_cffi", r""" - /* the forward declaration, setting a gcc attribute - (this line could also be in some .h file, to be included - both here and in the other C files of the project) */ - int f(int) __attribute__((visibility("hidden"))); - """) - - -Extern "Python": reference -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -``extern "Python"`` must appear in the cdef(). Like the C++ ``extern -"C"`` syntax, it can also be used with braces around a group of -functions:: - - extern "Python" { - int foo(int); - int bar(int); - } - -The ``extern "Python"`` functions cannot be variadic for now. This -may be implemented in the future. (`This demo`__ shows how to do it -anyway, but it is a bit lengthy.) - -.. __: https://foss.heptapod.net/pypy/cffi/-/blob/branch/default/demo/extern_python_varargs.py - -Each corresponding Python callback function is defined with the -``@ffi.def_extern()`` decorator. Be careful when writing this -function: if it raises an exception, or tries to return an object of -the wrong type, then the exception cannot be propagated. Instead, the -exception is printed to stderr and the C-level callback is made to -return a default value. This can be controlled with ``error`` and -``onerror``, described below. - -.. _def-extern: - -The ``@ffi.def_extern()`` decorator takes these optional arguments: - -* ``name``: the name of the function as written in the cdef. By default - it is taken from the name of the Python function you decorate. - -.. _error_onerror: - -* ``error``: the returned value in case the Python function raises an - exception. It is 0 or null by default. The exception is still - printed to stderr, so this should be used only as a last-resort - solution. - -* ``onerror``: if you want to be sure to catch all exceptions, use - ``@ffi.def_extern(onerror=my_handler)``. If an exception occurs and - ``onerror`` is specified, then ``onerror(exception, exc_value, - traceback)`` is called. This is useful in some situations where you - cannot simply write ``try: except:`` in the main callback function, - because it might not catch exceptions raised by signal handlers: if - a signal occurs while in C, the Python signal handler is called as - soon as possible, which is after entering the callback function but - *before* executing even the ``try:``. If the signal handler raises, - we are not in the ``try: except:`` yet. - - If ``onerror`` is called and returns normally, then it is assumed - that it handled the exception on its own and nothing is printed to - stderr. If ``onerror`` raises, then both tracebacks are printed. - Finally, ``onerror`` can itself provide the result value of the - callback in C, but doesn't have to: if it simply returns None---or - if ``onerror`` itself fails---then the value of ``error`` will be - used, if any. - - Note the following hack: in ``onerror``, you can access the original - callback arguments as follows. First check if ``traceback`` is not - None (it is None e.g. if the whole function ran successfully but - there was an error converting the value returned: this occurs after - the call). If ``traceback`` is not None, then - ``traceback.tb_frame`` is the frame of the outermost function, - i.e. directly the frame of the function decorated with - ``@ffi.def_extern()``. So you can get the value of ``argname`` in - that frame by reading ``traceback.tb_frame.f_locals['argname']``. - - -.. _Callbacks: - -Callbacks (old style) ---------------------- - -Here is how to make a new ``<cdata>`` object that contains a pointer -to a function, where that function invokes back a Python function of -your choice:: - - >>> @ffi.callback("int(int, int)") - >>> def myfunc(x, y): - ... return x + y - ... - >>> myfunc - <cdata 'int(*)(int, int)' calling <function myfunc at 0xf757bbc4>> - -Note that ``"int(*)(int, int)"`` is a C *function pointer* type, whereas -``"int(int, int)"`` is a C *function* type. Either can be specified to -ffi.callback() and the result is the same. - -.. warning:: - - Callbacks are provided for the ABI mode or for backward - compatibility. If you are using the out-of-line API mode, it is - recommended to use the `extern "Python"`_ mechanism instead of - callbacks: it gives faster and cleaner code. It also avoids several - issues with old-style callbacks: - - - On less common architecture, libffi is more likely to crash on - callbacks (`e.g. on NetBSD`__); - - - On hardened systems like PAX and SELinux, the extra memory - protections can interfere (for example, on SELinux you need to - run with ``deny_execmem`` set to ``off``). - - - `On Mac OS X,`__ you need to give your application the entitlement - ``com.apple.security.cs.allow-unsigned-executable-memory``. - - Note also that a cffi fix for this issue was attempted---see - the ``ffi_closure_alloc`` branch---but was not merged because it - creates potential `memory corruption`__ with ``fork()``. - - In other words: yes, it is dangerous to allow write+execute memory in your - program; that's why the various "hardening" options above exist. But at - the same time, these options open wide the door to another attack: if the - program forks and then attempts to call any of the ``ffi.callback()``, then - this immediately results in a crash---or, with a minimal amount of work - from an attacker, arbitrary code execution. To me it sounds even more - dangerous than the original problem, and that's why cffi is not playing - along. - - To fix the issue once and for all on the affected platforms, you need - to refactor the involved code so that it no longer uses ``ffi.callback()``. - -.. __: https://github.com/pyca/pyopenssl/issues/596 -.. __: https://foss.heptapod.net/pypy/cffi/-/issues/391 -.. __: https://bugzilla.redhat.com/show_bug.cgi?id=1249685 - -Warning: like ffi.new(), ffi.callback() returns a cdata that has -ownership of its C data. (In this case, the necessary C data contains -the libffi data structures to do a callback.) This means that the -callback can only be invoked as long as this cdata object is alive. -If you store the function pointer into C code, then make sure you also -keep this object alive for as long as the callback may be invoked. -The easiest way to do that is to always use ``@ffi.callback()`` at -module-level only, and to pass "context" information around with -`ffi.new_handle()`__, if possible. Example: - -.. __: ref.html#new-handle - -.. code-block:: python - - # a good way to use this decorator is once at global level - @ffi.callback("int(int, void *)") - def my_global_callback(x, handle): - return ffi.from_handle(handle).some_method(x) - - - class Foo(object): - - def __init__(self): - handle = ffi.new_handle(self) - self._handle = handle # must be kept alive - lib.register_stuff_with_callback_and_voidp_arg(my_global_callback, handle) - - def some_method(self, x): - print "method called!" - -(See also the section about `extern "Python"`_ above, where the same -general style is used.) - -Note that callbacks of a variadic function type are not supported. A -workaround is to add custom C code. In the following example, a -callback gets a first argument that counts how many extra ``int`` -arguments are passed: - -.. code-block:: python - - # file "example_build.py" - - import cffi - - ffibuilder = cffi.FFI() - ffibuilder.cdef(""" - int (*python_callback)(int how_many, int *values); - void *const c_callback; /* pass this const ptr to C routines */ - """) - ffibuilder.set_source("_example", r""" - #include <stdarg.h> - #include <alloca.h> - static int (*python_callback)(int how_many, int *values); - static int c_callback(int how_many, ...) { - va_list ap; - /* collect the "..." arguments into the values[] array */ - int i, *values = alloca(how_many * sizeof(int)); - va_start(ap, how_many); - for (i=0; i<how_many; i++) - values[i] = va_arg(ap, int); - va_end(ap); - return python_callback(how_many, values); - } - """) - ffibuilder.compile(verbose=True) - -.. code-block:: python - - # file "example.py" - - from _example import ffi, lib - - @ffi.callback("int(int, int *)") - def python_callback(how_many, values): - print ffi.unpack(values, how_many) - return 0 - lib.python_callback = python_callback - -Deprecated: you can also use ``ffi.callback()`` not as a decorator but -directly as ``ffi.callback("int(int, int)", myfunc)``. This is -discouraged: using this a style, we are more likely to forget the -callback object too early, when it is still in use. - -The ``ffi.callback()`` decorator also accepts the optional argument -``error``, and from CFFI version 1.2 the optional argument ``onerror``. -These two work in the same way as `described above for extern "Python".`__ - -.. __: error_onerror_ - - - -Windows: calling conventions ----------------------------- - -On Win32, functions can have two main calling conventions: either -"cdecl" (the default), or "stdcall" (also known as "WINAPI"). There -are also other rare calling conventions, but these are not supported. -*New in version 1.3.* - -When you issue calls from Python to C, the implementation is such that -it works with any of these two main calling conventions; you don't -have to specify it. However, if you manipulate variables of type -"function pointer" or declare callbacks, then the calling convention -must be correct. This is done by writing ``__cdecl`` or ``__stdcall`` -in the type, like in C:: - - @ffi.callback("int __stdcall(int, int)") - def AddNumbers(x, y): - return x + y - -or:: - - ffibuilder.cdef(""" - struct foo_s { - int (__stdcall *MyFuncPtr)(int, int); - }; - """) - -``__cdecl`` is supported but is always the default so it can be left -out. In the ``cdef()``, you can also use ``WINAPI`` as equivalent to -``__stdcall``. As mentioned above, it is mostly not needed (but doesn't -hurt) to say ``WINAPI`` or ``__stdcall`` when declaring a plain -function in the ``cdef()``. (The difference can still be seen if you -take explicitly a pointer to this function with ``ffi.addressof()``, -or if the function is ``extern "Python"``.) - -These calling convention specifiers are accepted but ignored on any -platform other than 32-bit Windows. - -In CFFI versions before 1.3, the calling convention specifiers are not -recognized. In API mode, you could work around it by using an -indirection, like in the example in the section about Callbacks_ -(``"example_build.py"``). There was no way to use stdcall callbacks -in ABI mode. - - -FFI Interface -------------- - -(The reference for the FFI interface has been moved to the `next page`__.) - -.. __: ref.html diff --git a/doc/source/whatsnew.rst b/doc/source/whatsnew.rst deleted file mode 100644 index aa7f2fe..0000000 --- a/doc/source/whatsnew.rst +++ /dev/null @@ -1,887 +0,0 @@ -====================== -What's New -====================== - -v1.15.0 -======= - -* Fixed MANIFEST.in to include missing file for Windows arm64 support - -* Fixed Linux wheel build to use gcc default ISA for libffi - -* Updated setup.py Python trove specifiers to currently-tested Python versions - -* CPython 3.10 support (including wheels) - -* MacOS arm64 support (including wheels) - -* Initial Windows arm64 support - -* Misc. doc and test updates - -v1.14.6 -======= - -* Test fixes for CPython 3.10.0b3 - -* Support for `sys.unraisablehook()` on Python >= 3.8 - -* Fix two minor memory leaks (thanks Sebastian!) - -* Like many projects that had an IRC channel on freenode, we moved it to - ``irc.libera.chat``. - -v1.14.5 -======= - -* Source fix for old gcc versions - -* This and future releases should include wheels on more platforms, - thanks to our new release managers Matt and Matt! - -v1.14.4 -======= - -Release done for pip reasons. - -v1.14.3 -======= - -Release done for pip reasons. - -v1.14.2 -======= - -* CPython 3 on Windows: we again try to compile with ``Py_LIMITED_API`` - by default. This flag is not added if you run the compilation with - CPython 3.4, as it only works with CPython >= 3.5, but by now this - version of Python is quite old (and we no longer distribute cffi - wheels for it). - - This may require that you upgrade ``virtualenv`` (requires version 16 - or newer) or at least copy manually ``python3.dll`` into your existing - virtualenvs. For distributing wheels with your cffi modules, you may - also need to upgrade ``wheel`` to the just-released version 0.35. - - You can manually disable ``Py_LIMITED_API`` by calling - ``ffi.set_source(..., py_limited_api=False)``. - - -v1.14.1 -======= - -* CFFI source code is now `hosted on Heptapod`_. - -* Improved support for ``typedef int my_array_t[...];`` with an explicit - dot-dot-dot in API mode (`issue #453`_) - -* Windows (32 and 64 bits): multiple fixes for ABI-mode call to functions - that return a structure. - -* Experimental support for MacOS 11 on aarch64. - -* and a few other minor changes and bug fixes. - -.. _`hosted on Heptapod`: https://foss.heptapod.net/pypy/cffi/ -.. _`issue #453`: https://foss.heptapod.net/pypy/cffi/issues/453 - - -v1.14 -===== - -* ``ffi.dlopen()`` can now be called with a handle (as a ``void *``) to an - already-opened C library. - -* CPython only: fixed a stack overflow issue for calls like - ``lib.myfunc([large list])``. If the function is declared as taking a - ``float *`` argument, for example, then the array is temporarily converted - into a C array of floats---however, the code used to use ``alloca()`` for - this temporary storage, no matter how large. This is now fixed. - - The fix concerns all modes: in-line/out-of-line API/ABI. Also note that your - API-mode C extension modules need to be regenerated with cffi 1.14 in order - to get the fix; i.e. for API mode, the fix is in the generated C sources. - (The C sources generated from cffi 1.14 should also work when running in - a different environment in which we have an older version of cffi. Also, - this change makes no difference on PyPy.) - - As a workaround that works on all versions of cffi, you can write - ``lib.myfunc(ffi.new("float[]", [large list]))``, which is - equivalent but explicity builds the intermediate array as a regular - Python object on the heap. - -* fixed a memory leak inside ``ffi.getwinerror()`` on CPython 3.x. - - -v1.13.2 -======= - -* re-release because the Linux wheels came with an attached version of libffi - that was very old and buggy (`issue #432`_). - -.. _`issue #432`: https://foss.heptapod.net/pypy/cffi/-/issues/432 - - - -v1.13.1 -======= - -* deprecate the way to declare in ``cdef()`` a global variable with only - ``void *foo;``. You should always use a storage class, like ``extern void - *foo;`` or maybe ``static void *foo;``. These are all equivalent for - the purposes of ``cdef()``, but the reason for deprecating the bare version - is that (as far as I know) it would always be mistake in a real C header. - -* fix the regression ``RuntimeError: found a situation in which we try - to build a type recursively`` (`issue #429`_). - -* fixed `issue #427`_ where a multithreading mistake in the embedding logic - initialization code would cause deadlocks on CPython 3.7. - -.. _`issue #429`: https://foss.heptapod.net/pypy/cffi/-/issues/429 -.. _`issue #427`: https://foss.heptapod.net/pypy/cffi/-/issues/427 - - -v1.13 -===== - -* ``ffi.from_buffer("type *", ..)`` is now supported, in addition to - ``"type[]"``. You can then write ``p.field`` to access the items, instead - of only ``p[0].field``. Be careful that no bounds checking is performed, so - ``p[n]`` might access data out of bounds. - -* fix for structs containing unnamed bitfields like ``int : 1;``. - -* when calling cdata of "function pointer" type, give a RuntimeError instead - of a crash if the pointer happens to be NULL - -* support some more binary operations between constants in enum definitions - (PR #96) - -* silence a warning incorrectly emitted if you use a quote in a preprocessor - line - -* detect a corner case that would throw the C code into an infinite - recursion, with ``ffi.cdef("""struct X { void(*fnptr)(struct X); };""")`` - - -Older Versions -============== - -v1.12.3 -------- - -* Fix for nested struct types that end in a var-sized array (#405). - -* Add support for using ``U`` and ``L`` characters at the end of integer - constants in ``ffi.cdef()`` (thanks Guillaume). - -* More 3.8 fixes. - - -v1.12.2 -------- - -* Added temporary workaround to compile on CPython 3.8.0a2. - - -v1.12.1 -------- - -* CPython 3 on Windows: we again no longer compile with ``Py_LIMITED_API`` - by default because such modules *still* cannot be used with virtualenv. - The problem is that it doesn't work in CPython <= 3.4, and for - technical reason we can't enable this flag automatically based on the - version of Python. - - Like before, `Issue #350`_ mentions a workaround if you still want - the ``Py_LIMITED_API`` flag and *either* you are not concerned about - virtualenv *or* you are sure your module will not be used on CPython - <= 3.4: pass ``define_macros=[("Py_LIMITED_API", None)]`` as a keyword to the - ``ffibuilder.set_source()`` call. - - -v1.12 ------ - -* `Direct support for pkg-config`__. - -* ``ffi.from_buffer()`` takes a new optional *first* argument that gives - the array type of the result. It also takes an optional keyword argument - ``require_writable`` to refuse read-only Python buffers. - -* ``ffi.new()``, ``ffi.gc()`` or ``ffi.from_buffer()`` cdata objects - can now be released at known times, either by using the ``with`` - keyword or by calling the new ``ffi.release()``. - -* Windows, CPython 3.x: cffi modules are linked with ``python3.dll`` - again. This makes them independant on the exact CPython version, - like they are on other platforms. **It requires virtualenv 16.0.0.** - -* Accept an expression like ``ffi.new("int[4]", p)`` if ``p`` is itself - another cdata ``int[4]``. - -* CPython 2.x: ``ffi.dlopen()`` failed with non-ascii file names on Posix - -* CPython: if a thread is started from C and then runs Python code (with - callbacks or with the embedding solution), then previous versions of - cffi would contain possible crashes and/or memory leaks. Hopefully, - this has been fixed (see `issue #362`_). - -* Support for ``ffi.cdef(..., pack=N)`` where N is a power of two. - Means to emulate ``#pragma pack(N)`` on MSVC. Also, the default on - Windows is now ``pack=8``, like on MSVC. This might make a difference - in corner cases, although I can't think of one in the context of CFFI. - The old way ``ffi.cdef(..., packed=True)`` remains and is equivalent - to ``pack=1`` (saying e.g. that fields like ``int`` should be aligned - to 1 byte instead of 4). - -.. __: cdef.html#pkgconfig -.. _`issue #362`: https://foss.heptapod.net/pypy/cffi/-/issues/362 - - -v1.11.5 -------- - -* `Issue #357`_: fix ``ffi.emit_python_code()`` which generated a buggy - Python file if you are using a ``struct`` with an anonymous ``union`` - field or vice-versa. - -* Windows: ``ffi.dlopen()`` should now handle unicode filenames. - -* ABI mode: implemented ``ffi.dlclose()`` for the in-line case (it used - to be present only in the out-of-line case). - -* Fixed a corner case for ``setup.py install --record=xx --root=yy`` - with an out-of-line ABI module. Also fixed `Issue #345`_. - -* More hacks on Windows for running CFFI's own ``setup.py``. - -* `Issue #358`_: in embedding, to protect against (the rare case of) - Python initialization from several threads in parallel, we have to use - a spin-lock. On CPython 3 it is worse because it might spin-lock for - a long time (execution of ``Py_InitializeEx()``). Sadly, recent - changes to CPython make that solution needed on CPython 2 too. - -* CPython 3 on Windows: we no longer compile with ``Py_LIMITED_API`` - by default because such modules cannot be used with virtualenv. - `Issue #350`_ mentions a workaround if you still want that and are not - concerned about virtualenv: pass ``define_macros=[("Py_LIMITED_API", - None)]`` as a keyword to the ``ffibuilder.set_source()`` call. - -.. _`Issue #345`: https://foss.heptapod.net/pypy/cffi/-/issues/345 -.. _`Issue #350`: https://foss.heptapod.net/pypy/cffi/-/issues/350 -.. _`Issue #358`: https://foss.heptapod.net/pypy/cffi/-/issues/358 -.. _`Issue #357`: https://foss.heptapod.net/pypy/cffi/-/issues/357 - - -v1.11.4 -------- - -* Windows: reverted linking with ``python3.dll``, because - virtualenv does not make this DLL available to virtual environments - for now. See `Issue #355`_. On Windows only, the C extension - modules created by cffi follow for now the standard naming scheme - ``foo.cp36-win32.pyd``, to make it clear that they are regular - CPython modules depending on ``python36.dll``. - -.. _`Issue #355`: https://foss.heptapod.net/pypy/cffi/-/issues/355 - - -v1.11.3 -------- - -* Fix on CPython 3.x: reading the attributes ``__loader__`` or - ``__spec__`` from the cffi-generated lib modules gave a buggy - SystemError. (These attributes are always None, and provided only to - help compatibility with tools that expect them in all modules.) - -* More Windows fixes: workaround for MSVC not supporting large - literal strings in C code (from - ``ffi.embedding_init_code(large_string)``); and an issue with - ``Py_LIMITED_API`` linking with ``python35.dll/python36.dll`` instead - of ``python3.dll``. - -* Small documentation improvements. - - -v1.11.2 -------- - -* Fix Windows issue with managing the thread-state on CPython 3.0 to 3.5 - - -v1.11.1 -------- - -* Fix tests, remove deprecated C API usage - -* Fix (hack) for 3.6.0/3.6.1/3.6.2 giving incompatible binary extensions - (cpython issue `#29943`_) - -* Fix for 3.7.0a1+ - -.. _`#29943`: https://bugs.python.org/issue29943 - - -v1.11 ------ - -* Support the modern standard types ``char16_t`` and ``char32_t``. - These work like ``wchar_t``: they represent one unicode character, or - when used as ``charN_t *`` or ``charN_t[]`` they represent a unicode - string. The difference with ``wchar_t`` is that they have a known, - fixed size. They should work at all places that used to work with - ``wchar_t`` (please report an issue if I missed something). Note - that with ``set_source()``, you need to make sure that these types are - actually defined by the C source you provide (if used in ``cdef()``). - -* Support the C99 types ``float _Complex`` and ``double _Complex``. - Note that libffi doesn't support them, which means that in the ABI - mode you still cannot call C functions that take complex numbers - directly as arguments or return type. - -* Fixed a rare race condition when creating multiple ``FFI`` instances - from multiple threads. (Note that you aren't meant to create many - ``FFI`` instances: in inline mode, you should write ``ffi = - cffi.FFI()`` at module level just after ``import cffi``; and in - out-of-line mode you don't instantiate ``FFI`` explicitly at all.) - -* Windows: using callbacks can be messy because the CFFI internal error - messages show up to stderr---but stderr goes nowhere in many - applications. This makes it particularly hard to get started with the - embedding mode. (Once you get started, you can at least use - ``@ffi.def_extern(onerror=...)`` and send the error logs where it - makes sense for your application, or record them in log files, and so - on.) So what is new in CFFI is that now, on Windows CFFI will try to - open a non-modal MessageBox (in addition to sending raw messages to - stderr). The MessageBox is only visible if the process stays alive: - typically, console applications that crash close immediately, but that - is also the situation where stderr should be visible anyway. - -* Progress on support for `callbacks in NetBSD`__. - -* Functions returning booleans would in some case still return 0 or 1 - instead of False or True. Fixed. - -* `ffi.gc()`__ now takes an optional third parameter, which gives an - estimate of the size (in bytes) of the object. So far, this is only - used by PyPy, to make the next GC occur more quickly (`issue #320`__). - In the future, this might have an effect on CPython too (provided - the CPython `issue 31105`__ is addressed). - -* Add a note to the documentation: the ABI mode gives function objects - that are *slower* to call than the API mode does. For some reason it - is often thought to be faster. It is not! - -.. __: https://foss.heptapod.net/pypy/cffi/-/issues/321 -.. __: ref.html#ffi-gc -.. __: https://foss.heptapod.net/pypy/cffi/-/issues/320 -.. __: http://bugs.python.org/issue31105 - - -v1.10.1 -------- - -(only released inside PyPy 5.8.0) - -* Fixed the line numbers reported in case of ``cdef()`` errors. - Also, I just noticed, but pycparser always supported the preprocessor - directive ``# 42 "foo.h"`` to mean "from the next line, we're in file - foo.h starting from line 42", which it puts in the error messages. - - -v1.10 ------ - -* Issue #295: use calloc() directly instead of - PyObject_Malloc()+memset() to handle ffi.new() with a default - allocator. Speeds up ``ffi.new(large-array)`` where most of the time - you never touch most of the array. - -* Some OS/X build fixes ("only with Xcode but without CLT"). - -* Improve a couple of error messages: when getting mismatched versions - of cffi and its backend; and when calling functions which cannot be - called with libffi because an argument is a struct that is "too - complicated" (and not a struct *pointer*, which always works). - -* Add support for some unusual compilers (non-msvc, non-gcc, non-icc, - non-clang) - -* Implemented the remaining cases for ``ffi.from_buffer``. Now all - buffer/memoryview objects can be passed. The one remaining check is - against passing unicode strings in Python 2. (They support the buffer - interface, but that gives the raw bytes behind the UTF16/UCS4 storage, - which is most of the times not what you expect. In Python 3 this has - been fixed and the unicode strings don't support the memoryview - interface any more.) - -* The C type ``_Bool`` or ``bool`` now converts to a Python boolean - when reading, instead of the content of the byte as an integer. The - potential incompatibility here is what occurs if the byte contains a - value different from 0 and 1. Previously, it would just return it; - with this change, CFFI raises an exception in this case. But this - case means "undefined behavior" in C; if you really have to interface - with a library relying on this, don't use ``bool`` in the CFFI side. - Also, it is still valid to use a byte string as initializer for a - ``bool[]``, but now it must only contain ``\x00`` or ``\x01``. As an - aside, ``ffi.string()`` no longer works on ``bool[]`` (but it never - made much sense, as this function stops at the first zero). - -* ``ffi.buffer`` is now the name of cffi's buffer type, and - ``ffi.buffer()`` works like before but is the constructor of that type. - -* ``ffi.addressof(lib, "name")`` now works also in in-line mode, not - only in out-of-line mode. This is useful for taking the address of - global variables. - -* Issue #255: ``cdata`` objects of a primitive type (integers, floats, - char) are now compared and ordered by value. For example, ``<cdata - 'int' 42>`` compares equal to ``42`` and ``<cdata 'char' b'A'>`` - compares equal to ``b'A'``. Unlike C, ``<cdata 'int' -1>`` does not - compare equal to ``ffi.cast("unsigned int", -1)``: it compares - smaller, because ``-1 < 4294967295``. - -* PyPy: ``ffi.new()`` and ``ffi.new_allocator()()`` did not record - "memory pressure", causing the GC to run too infrequently if you call - ``ffi.new()`` very often and/or with large arrays. Fixed in PyPy 5.7. - -* Support in ``ffi.cdef()`` for numeric expressions with ``+`` or - ``-``. Assumes that there is no overflow; it should be fixed first - before we add more general support for arbitrary arithmetic on - constants. - - -v1.9 ----- - -* Structs with variable-sized arrays as their last field: now we track - the length of the array after ``ffi.new()`` is called, just like we - always tracked the length of ``ffi.new("int[]", 42)``. This lets us - detect out-of-range accesses to array items. This also lets us - display a better ``repr()``, and have the total size returned by - ``ffi.sizeof()`` and ``ffi.buffer()``. Previously both functions - would return a result based on the size of the declared structure - type, with an assumed empty array. (Thanks andrew for starting this - refactoring.) - -* Add support in ``cdef()/set_source()`` for unspecified-length arrays - in typedefs: ``typedef int foo_t[...];``. It was already supported - for global variables or structure fields. - -* I turned in v1.8 a warning from ``cffi/model.py`` into an error: - ``'enum xxx' has no values explicitly defined: refusing to guess which - integer type it is meant to be (unsigned/signed, int/long)``. Now I'm - turning it back to a warning again; it seems that guessing that the - enum has size ``int`` is a 99%-safe bet. (But not 100%, so it stays - as a warning.) - -* Fix leaks in the code handling ``FILE *`` arguments. In CPython 3 - there is a remaining issue that is hard to fix: if you pass a Python - file object to a ``FILE *`` argument, then ``os.dup()`` is used and - the new file descriptor is only closed when the GC reclaims the Python - file object---and not at the earlier time when you call ``close()``, - which only closes the original file descriptor. If this is an issue, - you should avoid this automatic convertion of Python file objects: - instead, explicitly manipulate file descriptors and call ``fdopen()`` - from C (...via cffi). - - -v1.8.3 ------- - -* When passing a ``void *`` argument to a function with a different - pointer type, or vice-versa, the cast occurs automatically, like in C. - The same occurs for initialization with ``ffi.new()`` and a few other - places. However, I thought that ``char *`` had the same - property---but I was mistaken. In C you get the usual warning if you - try to give a ``char *`` to a ``char **`` argument, for example. - Sorry about the confusion. This has been fixed in CFFI by giving for - now a warning, too. It will turn into an error in a future version. - - -v1.8.2 ------- - -* Issue #283: fixed ``ffi.new()`` on structures/unions with nested - anonymous structures/unions, when there is at least one union in - the mix. When initialized with a list or a dict, it should now - behave more closely like the ``{ }`` syntax does in GCC. - - -v1.8.1 ------- - -* CPython 3.x: experimental: the generated C extension modules now use - the "limited API", which means that, as a compiled .so/.dll, it should - work directly on any version of CPython >= 3.2. The name produced by - distutils is still version-specific. To get the version-independent - name, you can rename it manually to ``NAME.abi3.so``, or use the very - recent setuptools 26. - -* Added ``ffi.compile(debug=...)``, similar to ``python setup.py build - --debug`` but defaulting to True if we are running a debugging - version of Python itself. - - -v1.8 ----- - -* Removed the restriction that ``ffi.from_buffer()`` cannot be used on - byte strings. Now you can get a ``char *`` out of a byte string, - which is valid as long as the string object is kept alive. (But - don't use it to *modify* the string object! If you need this, use - ``bytearray`` or other official techniques.) - -* PyPy 5.4 can now pass a byte string directly to a ``char *`` - argument (in older versions, a copy would be made). This used to be - a CPython-only optimization. - - -v1.7 ----- - -* ``ffi.gc(p, None)`` removes the destructor on an object previously - created by another call to ``ffi.gc()`` - -* ``bool(ffi.cast("primitive type", x))`` now returns False if the - value is zero (including ``-0.0``), and True otherwise. Previously - this would only return False for cdata objects of a pointer type when - the pointer is NULL. - -* bytearrays: ``ffi.from_buffer(bytearray-object)`` is now supported. - (The reason it was not supported was that it was hard to do in PyPy, - but it works since PyPy 5.3.) To call a C function with a ``char *`` - argument from a buffer object---now including bytearrays---you write - ``lib.foo(ffi.from_buffer(x))``. Additionally, this is now supported: - ``p[0:length] = bytearray-object``. The problem with this was that a - iterating over bytearrays gives *numbers* instead of *characters*. - (Now it is implemented with just a memcpy, of course, not actually - iterating over the characters.) - -* C++: compiling the generated C code with C++ was supposed to work, - but failed if you make use the ``bool`` type (because that is rendered - as the C ``_Bool`` type, which doesn't exist in C++). - -* ``help(lib)`` and ``help(lib.myfunc)`` now give useful information, - as well as ``dir(p)`` where ``p`` is a struct or pointer-to-struct. - - -v1.6 ----- - -* `ffi.list_types()`_ - -* `ffi.unpack()`_ - -* `extern "Python+C"`_ - -* in API mode, ``lib.foo.__doc__`` contains the C signature now. On - CPython you can say ``help(lib.foo)``, but for some reason - ``help(lib)`` (or ``help(lib.foo)`` on PyPy) is still useless; I - haven't yet figured out the hacks needed to convince ``pydoc`` to - show more. (You can use ``dir(lib)`` but it is not most helpful.) - -* Yet another attempt at robustness of ``ffi.def_extern()`` against - CPython's interpreter shutdown logic. - -.. _`ffi.list_types()`: ref.html#ffi-list-types -.. _`ffi.unpack()`: ref.html#ffi-unpack -.. _`extern "Python+C"`: using.html#extern-python-c - - -v1.5.2 ------- - -* Fix 1.5.1 for Python 2.6. - - -v1.5.1 ------- - -* A few installation-time tweaks (thanks Stefano!) - -* Issue #245: Win32: ``__stdcall`` was never generated for - ``extern "Python"`` functions - -* Issue #246: trying to be more robust against CPython's fragile - interpreter shutdown logic - - -v1.5.0 ------- - -* Support for `using CFFI for embedding`__. - -.. __: embedding.html - - -v1.4.2 ------- - -Nothing changed from v1.4.1. - - -v1.4.1 ------- - -* Fix the compilation failure of cffi on CPython 3.5.0. (3.5.1 works; - some detail changed that makes some underscore-starting macros - disappear from view of extension modules, and I worked around it, - thinking it changed in all 3.5 versions---but no: it was only in - 3.5.1.) - - -v1.4.0 ------- - -* A `better way to do callbacks`__ has been added (faster and more - portable, and usually cleaner). It is a mechanism for the - out-of-line API mode that replaces the dynamic creation of callback - objects (i.e. C functions that invoke Python) with the static - declaration in ``cdef()`` of which callbacks are needed. This is - more C-like, in that you have to structure your code around the idea - that you get a fixed number of function pointers, instead of - creating them on-the-fly. - -* ``ffi.compile()`` now takes an optional ``verbose`` argument. When - ``True``, distutils prints the calls to the compiler. - -* ``ffi.compile()`` used to fail if given ``sources`` with a path that - includes ``".."``. Fixed. - -* ``ffi.init_once()`` added. See docs__. - -* ``dir(lib)`` now works on libs returned by ``ffi.dlopen()`` too. - -* Cleaned up and modernized the content of the ``demo`` subdirectory - in the sources (thanks matti!). - -* ``ffi.new_handle()`` is now guaranteed to return unique ``void *`` - values, even if called twice on the same object. Previously, in - that case, CPython would return two ``cdata`` objects with the same - ``void *`` value. This change is useful to add and remove handles - from a global dict (or set) without worrying about duplicates. - It already used to work like that on PyPy. - *This change can break code that used to work on CPython by relying - on the object to be kept alive by other means than keeping the - result of ffi.new_handle() alive.* (The corresponding `warning in - the docs`__ of ``ffi.new_handle()`` has been here since v0.8!) - -.. __: using.html#extern-python -.. __: ref.html#ffi-init-once -.. __: ref.html#ffi-new-handle - - -v1.3.1 ------- - -* The optional typedefs (``bool``, ``FILE`` and all Windows types) were - not always available from out-of-line FFI objects. - -* Opaque enums are phased out from the cdefs: they now give a warning, - instead of (possibly wrongly) being assumed equal to ``unsigned int``. - Please report if you get a reasonable use case for them. - -* Some parsing details, notably ``volatile`` is passed along like - ``const`` and ``restrict``. Also, older versions of pycparser - mis-parse some pointer-to-pointer types like ``char * const *``: the - "const" ends up at the wrong place. Added a workaround. - - -v1.3.0 ------- - -* Added `ffi.memmove()`_. - -* Pull request #64: out-of-line API mode: we can now declare - floating-point types with ``typedef float... foo_t;``. This only - works if ``foo_t`` is a float or a double, not ``long double``. - -* Issue #217: fix possible unaligned pointer manipulation, which crashes - on some architectures (64-bit, non-x86). - -* Issues #64 and #126: when using ``set_source()`` or ``verify()``, - the ``const`` and ``restrict`` keywords are copied from the cdef - to the generated C code; this fixes warnings by the C compiler. - It also fixes corner cases like ``typedef const int T; T a;`` - which would previously not consider ``a`` as a constant. (The - cdata objects themselves are never ``const``.) - -* Win32: support for ``__stdcall``. For callbacks and function - pointers; regular C functions still don't need to have their `calling - convention`_ declared. - -* Windows: CPython 2.7 distutils doesn't work with Microsoft's official - Visual Studio for Python, and I'm told this is `not a bug`__. For - ffi.compile(), we `removed a workaround`__ that was inside cffi but - which had unwanted side-effects. Try saying ``import setuptools`` - first, which patches distutils... - -.. _`ffi.memmove()`: ref.html#ffi-memmove -.. __: https://bugs.python.org/issue23246 -.. __: https://bitbucket.org/cffi/cffi/pull-requests/65/remove-_hack_at_distutils-which-imports/diff -.. _`calling convention`: using.html#windows-calling-conventions - - -v1.2.1 ------- - -Nothing changed from v1.2.0. - - -v1.2.0 ------- - -* Out-of-line mode: ``int a[][...];`` can be used to declare a structure - field or global variable which is, simultaneously, of total length - unknown to the C compiler (the ``a[]`` part) and each element is - itself an array of N integers, where the value of N *is* known to the - C compiler (the ``int`` and ``[...]`` parts around it). Similarly, - ``int a[5][...];`` is supported (but probably less useful: remember - that in C it means ``int (a[5])[...];``). - -* PyPy: the ``lib.some_function`` objects were missing the attributes - ``__name__``, ``__module__`` and ``__doc__`` that are expected e.g. by - some decorators-management functions from ``functools``. - -* Out-of-line API mode: you can now do ``from _example.lib import x`` - to import the name ``x`` from ``_example.lib``, even though the - ``lib`` object is not a standard module object. (Also works in ``from - _example.lib import *``, but this is even more of a hack and will fail - if ``lib`` happens to declare a name called ``__all__``. Note that - ``*`` excludes the global variables; only the functions and constants - make sense to import like this.) - -* ``lib.__dict__`` works again and gives you a copy of the - dict---assuming that ``lib`` has got no symbol called precisely - ``__dict__``. (In general, it is safer to use ``dir(lib)``.) - -* Out-of-line API mode: global variables are now fetched on demand at - every access. It fixes issue #212 (Windows DLL variables), and also - allows variables that are defined as dynamic macros (like ``errno``) - or ``__thread`` -local variables. (This change might also tighten - the C compiler's check on the variables' type.) - -* Issue #209: dereferencing NULL pointers now raises RuntimeError - instead of segfaulting. Meant as a debugging aid. The check is - only for NULL: if you dereference random or dead pointers you might - still get segfaults. - -* Issue #152: callbacks__: added an argument ``ffi.callback(..., - onerror=...)``. If the main callback function raises an exception - and ``onerror`` is provided, then ``onerror(exception, exc_value, - traceback)`` is called. This is similar to writing a ``try: - except:`` in the main callback function, but in some cases (e.g. a - signal) an exception can occur at the very start of the callback - function---before it had time to enter the ``try: except:`` block. - -* Issue #115: added ``ffi.new_allocator()``, which officializes - support for `alternative allocators`__. - -.. __: using.html#callbacks -.. __: ref.html#ffi-new-allocator - - -v1.1.2 ------- - -* ``ffi.gc()``: fixed a race condition in multithreaded programs - introduced in 1.1.1 - - -v1.1.1 ------- - -* Out-of-line mode: ``ffi.string()``, ``ffi.buffer()`` and - ``ffi.getwinerror()`` didn't accept their arguments as keyword - arguments, unlike their in-line mode equivalent. (It worked in PyPy.) - -* Out-of-line ABI mode: documented a restriction__ of ``ffi.dlopen()`` - when compared to the in-line mode. - -* ``ffi.gc()``: when called several times with equal pointers, it was - accidentally registering only the last destructor, or even none at - all depending on details. (It was correctly registering all of them - only in PyPy, and only with the out-of-line FFIs.) - -.. __: cdef.html#dlopen-note - - -v1.1.0 ------- - -* Out-of-line API mode: we can now declare integer types with - ``typedef int... foo_t;``. The exact size and signedness of ``foo_t`` - is figured out by the compiler. - -* Out-of-line API mode: we can now declare multidimensional arrays - (as fields or as globals) with ``int n[...][...]``. Before, only the - outermost dimension would support the ``...`` syntax. - -* Out-of-line ABI mode: we now support any constant declaration, - instead of only integers whose value is given in the cdef. Such "new" - constants, i.e. either non-integers or without a value given in the - cdef, must correspond to actual symbols in the lib. At runtime they - are looked up the first time we access them. This is useful if the - library defines ``extern const sometype somename;``. - -* ``ffi.addressof(lib, "func_name")`` now returns a regular cdata object - of type "pointer to function". You can use it on any function from a - library in API mode (in ABI mode, all functions are already regular - cdata objects). To support this, you need to recompile your cffi - modules. - -* Issue #198: in API mode, if you declare constants of a ``struct`` - type, what you saw from lib.CONSTANT was corrupted. - -* Issue #196: ``ffi.set_source("package._ffi", None)`` would - incorrectly generate the Python source to ``package._ffi.py`` instead - of ``package/_ffi.py``. Also fixed: in some cases, if the C file was - in ``build/foo.c``, the .o file would be put in ``build/build/foo.o``. - - -v1.0.3 ------- - -* Same as 1.0.2, apart from doc and test fixes on some platforms. - - -v1.0.2 ------- - -* Variadic C functions (ending in a "..." argument) were not supported - in the out-of-line ABI mode. This was a bug---there was even a - (non-working) example__ doing exactly that! - -.. __: overview.html#out-of-line-abi-level - - -v1.0.1 ------- - -* ``ffi.set_source()`` crashed if passed a ``sources=[..]`` argument. - Fixed by chrippa on pull request #60. - -* Issue #193: if we use a struct between the first cdef() where it is - declared and another cdef() where its fields are defined, then this - definition was ignored. - -* Enums were buggy if you used too many "..." in their definition. - - -v1.0.0 ------- - -* The main news item is out-of-line module generation: - - * `for ABI level`_, with ``ffi.dlopen()`` - - * `for API level`_, which used to be with ``ffi.verify()``, now deprecated - -* (this page will list what is new from all versions from 1.0.0 - forward.) - -.. _`for ABI level`: overview.html#out-of-line-abi-level -.. _`for API level`: overview.html#out-of-line-api-level |