diff options
Diffstat (limited to 'docs')
68 files changed, 6500 insertions, 2216 deletions
diff --git a/docs/AUTHORS b/docs/AUTHORS deleted file mode 100644 index 2843b56..0000000 --- a/docs/AUTHORS +++ /dev/null @@ -1,44 +0,0 @@ -Primary Authors -=============== - -Yuri Takteyev <http://freewisdom.org/>, who has written much of the current code -while procrastingating his Ph.D. - -Waylan Limberg <http://achinghead.com/>, who has written most of the available -extensions and later was asked to join Yuri, fixing nummrious bugs, adding -documentation and making general improvements to the existing codebase, -included a complete refactor of the core. - -Artem Yunusov, who as part of a 2008 GSoC project, has refactored inline -patterns, replaced the NanoDOM with ElementTree support and made various other -improvements. - -Manfed Stienstra <http://www.dwerg.net/>, who wrote the original version of -the script and is responsible for various parts of the existing codebase. - -David Wolever, who refactored the extension API and made other improvements -as he helped to integrate Markdown into Dr.Project. - -Other Contributors -================== - -The incomplete list of individuals below have provided patches or otherwise -contributed to the project in various ways. We would like to thank everyone -who has contributed to the progect in any way. - -Eric Abrahamsen -Jeff Balogh -Sergej Chodarev -Chris Clark -Tiago Cogumbreiro -Kjell Magne Fauske -G. Clark Haynes -Daniel Krech -Steward Midwinter -Jack Miller -Neale Pickett -Paul Stansifer -John Szakmeister -Malcolm Tredinnick -Ben Wilson -and many others who helped by reporting bugs diff --git a/docs/CHANGE_LOG b/docs/CHANGE_LOG deleted file mode 100644 index e005ff8..0000000 --- a/docs/CHANGE_LOG +++ /dev/null @@ -1,180 +0,0 @@ -PYTHON MARKDOWN CHANGELOG -========================= - -Sept 28, 2009: Released version 2.0.2-Final. - -May 20, 2009: Released version 2.0.1-Final. - -Mar 30, 2009: Released version 2.0-Final. - -Mar 8, 2009: Release Candidate 2.0-rc-1. - -Feb 2009: Added support for multi-level lists to new Blockprocessors. - -Jan 2009: Added HTML 4 output as an option (thanks Eric Abrahamsen) - -Nov 2008: Added Definistion List ext. Replaced old core with BlockProcessors. -Broken up into multiple files. - -Oct 2008: Changed logging behavior to work better with other systems. -Refactored tree tarversing. Added treap implementation, then replaced with -OrderedDEict. Renamed various processors to better reflect what they actually -do. Refactored footnote ext to match php Extra's output. - -Sept 2008: Moved prettifyTree to a Postprocessor, replaced wikilink ext -with wikilinks (note the s) ext (uses bracketed links instead of CamelCase) -and various bug fixes. - -August 18 2008: Reorganized directory structure. Added a 'docs' dir -and moved all extensions into a 'markdown-extensions' package. -Added additional documentation and a few bug fixes. (v2.0-beta) - -August 4 2008: Updated included extensions to ElementTree. Added a -seperate commanline script. (v2.0-alpha) - -July 2008: Switched from home-grown NanoDOM to ElementTree and -various related bugs (thanks Artem Yunusov). - -June 2008: Fixed issues with nested inline patterns and cleaned -up testing framework (thanks Artem Yunusov). - -May 2008: Added a number of additional extensions to the -distribution and other minor changes. Moved repo to git from svn. - -Mar 2008: Refactored extension api to accept either an -extension name (as a string) or an instance of an extension -(Thanks David Wolever). Fixed various bugs and added doc strings. - -Feb 2008: Various bugfixes mostly regarding extensions. - -Feb 18, 2008: Version 1.7. - -Feb 13, 2008: A little code cleanup and better documentation -and inheritance for pre/post proccessors. - -Feb 9, 2008: Doublequotes no longer html escaped and rawhtml -honors <?foo>, <@foo>, and <%foo> for those who run markdown on -template syntax. - -Dec 12, 2007: Updated docs. Removed encoding arg from Markdown -and markdown as per list discussion. Clean up in prep for 1.7. - -Nov 29, 2007: Added support for images inside links. Also fixed -a few bugs in the footnote extension. - -Nov 19, 2007: `message` now uses python's logging module. Also removed -limit imposed by recursion in _process_section(). You can now parse as -long of a document as your memory can handle. - -Nov 5, 2007: Moved safe_mode code to a textPostprocessor and added -escaping option. - -Nov 3, 2007: Fixed convert method to accept empty strings. - -Oct 30, 2007: Fixed BOM removal (thanks Malcolm Tredinnick). Fixed -infinite loop in bracket regex for inline links. - -Oct 11, 2007: LineBreaks is now an inlinePattern. Fixed HR in -blockquotes. Refactored _processSection method (see tracker #1793419). - -Oct 9, 2007: Added textPreprocessor (from 1.6b). - -Oct 8, 2008: Fixed Lazy Blockquote. Fixed code block on first line. -Fixed empty inline image link. - -Oct 7, 2007: Limit recursion on inlinePatterns. Added a 'safe' tag -to htmlStash. - -March 18, 2007: Fixed or merged a bunch of minor bugs, including -multi-line comments and markup inside links. (Tracker #s: 1683066, -1671153, 1661751, 1627935, 1544371, 1458139.) -> v. 1.6b - -Oct 10, 2006: Fixed a bug that caused some text to be lost after -comments. Added "safe mode" (user's html tags are removed). - -Sept 6, 2006: Added exception for PHP tags when handling html blocks. - -August 7, 2006: Incorporated Sergej Chodarev's patch to fix a problem -with ampersand normalization and html blocks. - -July 10, 2006: Switched to using optparse. Added proper support for -unicode. - -July 9, 2006: Fixed the <!--@address.com> problem (Tracker #1501354). - -May 18, 2006: Stopped catching unquoted titles in reference links. -Stopped creating blank headers. - -May 15, 2006: A bug with lists, recursion on block-level elements, -run-in headers, spaces before headers, unicode input (thanks to Aaron -Swartz). Sourceforge tracker #s: 1489313, 1489312, 1489311, 1488370, -1485178, 1485176. (v. 1.5) - -Mar. 24, 2006: Switched to a not-so-recursive algorithm with -_handleInline. (Version 1.4) - -Mar. 15, 2006: Replaced some instance variables with class variables -(a patch from Stelios Xanthakis). Chris Clark's new regexps that do -not trigger midword underlining. - -Feb. 28, 2006: Clean-up and command-line handling by Stewart -Midwinter. (Version 1.3) - -Feb. 24, 2006: Fixed a bug with the last line of the list appearing -again as a separate paragraph. Incorporated Chris Clark's "mailto" -patch. Added support for <br /> at the end of lines ending in two or -more spaces. Fixed a crashing bug when using ImageReferencePattern. -Added several utility methods to Nanodom. (Version 1.2) - -Jan. 31, 2006: Added "hr" and "hr/" to BLOCK_LEVEL_ELEMENTS and -changed <hr/> to <hr />. (Thanks to Sergej Chodarev.) - -Nov. 26, 2005: Fixed a bug with certain tabbed lines inside lists -getting wrapped in <pre><code>. (v. 1.1) - -Nov. 19, 2005: Made "<!...", "<?...", etc. behave like block-level -HTML tags. - -Nov. 14, 2005: Added entity code and email autolink fix by Tiago -Cogumbreiro. Fixed some small issues with backticks to get 100% -compliance with John's test suite. (v. 1.0) - -Nov. 7, 2005: Added an unlink method for documents to aid with memory -collection (per Doug Sauder's suggestion). - -Oct. 29, 2005: Restricted a set of html tags that get treated as -block-level elements. - -Sept. 18, 2005: Refactored the whole script to make it easier to -customize it and made footnote functionality into an extension. -(v. 0.9) - -Sept. 5, 2005: Fixed a bug with multi-paragraph footnotes. Added -attribute support. - -Sept. 1, 2005: Changed the way headers are handled to allow inline -syntax in headers (e.g. links) and got the lists to use p-tags -correctly (v. 0.8) - -Aug. 29, 2005: Added flexible tabs, fixed a few small issues, added -basic support for footnotes. Got rid of xml.dom.minidom and added -pretty-printing. (v. 0.7) - -Aug. 13, 2005: Fixed a number of small bugs in order to conform to the -test suite. (v. 0.6) - -Aug. 11, 2005: Added support for inline html and entities, inline -images, autolinks, underscore emphasis. Cleaned up and refactored the -code, added some more comments. - -Feb. 19, 2005: Rewrote the handling of high-level elements to allow -multi-line list items and all sorts of nesting. - -Feb. 3, 2005: Reference-style links, single-line lists, backticks, -escape, emphasis in the beginning of the paragraph. - -Nov. 2004: Added links, blockquotes, html blocks to Manfred -Stienstra's code - -Apr. 2004: Manfred's version at http://www.dwerg.net/projects/markdown/ - diff --git a/docs/INSTALL b/docs/INSTALL deleted file mode 100644 index d8feade..0000000 --- a/docs/INSTALL +++ /dev/null @@ -1,73 +0,0 @@ -Installing Python-Markdown -========================== - -Checking Dependencies ---------------------- - -Python-Markdown requires the ElementTree module to be installed. In Python2.5+ -ElementTree is included as part of the standard library. For earlier versions -of Python, open a Python shell and type the following: - - >>> import cElementTree - >>> import ElementTree - -If at least one of those does not generate any errors, then you have a working -copy of ElementTree installed on your system. As cElementTree is faster, you -may want to install that if you don't already have it and it's available for -your system. - -See <http://effbot.org/zone/element-index.htm> for more information or to -download the latest version of ElementTree. - -The East Way ------------- - -The simplest way to install Python-Markdown is by using SetupTools. As and -Admin/Root user on your system do: - - easy_install ElementTree - easy_install Markdown - -That's it, your done. - -Installing on Windows ---------------------- - -Download the Windows installer (.exe) from PyPI: -<http://pypi.python.org/pypi/Markdown> - -Double-click the file and follow the instructions. - -If you prefer to manually install Python-Markdown in Windows, download the -Zip file, unzip it, and on the command line in the directory you unzipped to: - - python setup.py install - -If you plan to use the provided command line script, you need to make sure your -script directory is on your system path. On a typical Python install of Windows -the Scripts directory is `C:\Python25\Scripts\`. Adjust according to your -system and add that to your system path. - -Installing on *nix Systems --------------------------- - -From the command line do the following: - - wget http://pypi.python.org/packages/source/M/Markdown/Markdown-2.0.tar.gz - tar xvzf Markdown-2.0.tar.gz - cd markdown-2.0/ - sudo python setup.py install - -Using the Git Repository ------------------------- - -If your the type that like to live on the edge, you may want to keep up with -the latest additions and bug fixes in the repository between releases. -Python-Markdown is maintained in a Git repository on Gitorious.org. To -get a copy of Python-Markdown from the repository do the following from the -command line: - - git clone git://gitorious.org/python-markdown/mainline.git python-markdown - cd python-markdown - python setup.py install - diff --git a/docs/LICENSE b/docs/LICENSE deleted file mode 100644 index 4cd8b14..0000000 --- a/docs/LICENSE +++ /dev/null @@ -1,30 +0,0 @@ -Copyright 2007, 2008 The Python Markdown Project (v. 1.7 and later) -Copyright 2004, 2005, 2006 Yuri Takhteyev (v. 0.2-1.6b) -Copyright 2004 Manfred Stienstra (the original version) - -All rights reserved. - -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions are met: - -* Redistributions of source code must retain the above copyright - notice, this list of conditions and the following disclaimer. -* Redistributions in binary form must reproduce the above copyright - notice, this list of conditions and the following disclaimer in the - documentation and/or other materials provided with the distribution. -* Neither the name of the <organization> nor the - names of its contributors may be used to endorse or promote products - derived from this software without specific prior written permission. - -THIS SOFTWARE IS PROVIDED BY THE PYTHON MARKDOWN PROJECT ''AS IS'' AND ANY -EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -DISCLAIMED. IN NO EVENT SHALL ANY CONTRIBUTORS TO THE PYTHON MARKDOWN PROJECT -BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR -CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF -SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS -INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN -CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) -ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE -POSSIBILITY OF SUCH DAMAGE. - diff --git a/docs/README b/docs/README deleted file mode 100644 index d19a1ea..0000000 --- a/docs/README +++ /dev/null @@ -1,30 +0,0 @@ -[Python-Markdown][] -=================== - -This is a Python implementation of John Gruber's [Markdown][]. -It is almost completely compliant with the reference implementation, -though there are a few known issues. See [Features][] for information -on what exactly is supported and what is not. Additional features are -supported by the [Available Extensions][]. - -[Python-Markdown]: http://freewisdom.org/projects/python-markdown -[Markdown]: http://daringfireball.net/projects/markdown/ -[Features]: http://www.freewisdom.org/projects/python-markdown/Features -[Available Extensions]: http://www.freewisdom.org/projects/python-markdown/Available_Extensions - - -Documentation -------------- - -Installation and usage documentation is available in the `docs/` directory -of the distribution and on the project website at -<http://freewisdom.org/projects/python-markdown>. - -Support -------- - -You may ask for help and discuss various other issues on the [mailing list][] and report bugs on the [bug tracker][]. - -[mailing list]: http://lists.sourceforge.net/lists/listinfo/python-markdown-discuss -[bug tracker]: http://www.freewisdom.org/projects/python-markdown/Tickets - diff --git a/docs/README.html b/docs/README.html deleted file mode 100644 index 49e3b07..0000000 --- a/docs/README.html +++ /dev/null @@ -1,12 +0,0 @@ -<h1><a href="http://freewisdom.org/projects/python-markdown">Python-Markdown</a></h1> -<p>This is a Python implementation of John Gruber's <a href="http://daringfireball.net/projects/markdown/">Markdown</a>. -It is almost completely compliant with the reference implementation, -though there are a few known issues. See <a href="http://www.freewisdom.org/projects/python-markdown/Features">Features</a> for information -on what exactly is supported and what is not. Additional features are -supported by the <a href="http://www.freewisdom.org/projects/python-markdown/Available_Extensions">Available Extensions</a>.</p> -<h2>Documentation</h2> -<p>Installation and usage documentation is available in the <code>docs/</code> directory -of the distribution and on the project website at -<a href="http://freewisdom.org/projects/python-markdown">http://freewisdom.org/projects/python-markdown</a>.</p> -<h2>Support</h2> -<p>You may ask for help and discuss various other issues on the <a href="http://lists.sourceforge.net/lists/listinfo/python-markdown-discuss">mailing list</a> and report bugs on the <a href="http://www.freewisdom.org/projects/python-markdown/Tickets">bug tracker</a>.</p>
\ No newline at end of file diff --git a/docs/authors.md b/docs/authors.md new file mode 100644 index 0000000..114227d --- /dev/null +++ b/docs/authors.md @@ -0,0 +1,70 @@ +title: Authors + +Primary Authors +=============== + +* __[Waylan Limberg](https://github.com/waylan)__ + + @waylan is the current maintainer of the code and has written much of the + current code base, including a complete refactor of the core for version 2.0. + He started out by authoring many of the available extensions and later was + asked to join Yuri, where he began fixing numerous bugs, adding + documentation and making general improvements to the existing code base. + +* __[Dmitry Shachnev](https://github.com/mitya57)__ + + @mitya57 joined the team after providing a number of helpful patches and has + been assisting with maintenance, reviewing pull requests and ticket triage. + +* __[Isaac Muse](https://github.com/facelessuser)__ + + @facelessuser joined the team after providing a number of helpful patches + and has been assisting with maintenance, reviewing pull requests and ticket + triage. + +* __[Yuri Takteyev](https://github.com/yuri)__ + + Yuri wrote most of the code found in version 1.x while procrastinating his + Ph.D. Various pieces of his code still exist, most notably the basic + structure. + +* __Manfed Stienstra__ + + Manfed wrote the original version of the script and is responsible for + various parts of the existing code base. + +* __Artem Yunusov__ + + Artem, who as part of a 2008 GSoC project, refactored inline patterns, + replaced the NanoDOM with ElementTree support and made various other + improvements. + +* __David Wolever__ + + David refactored the extension API and made other improvements + as he helped to integrate Markdown into Dr.Project. + +Other Contributors +================== + +The incomplete list of individuals below have provided patches or otherwise +contributed to the project prior to the project being hosted on GitHub. See the +GitHub commit log for a list of recent contributors. We would like to thank +everyone who has contributed to the project in any way. + +* Eric Abrahamsen +* Jeff Balogh +* Sergej Chodarev +* Chris Clark +* Tiago Cogumbreiro +* Kjell Magne Fauske +* G. Clark Haynes +* Daniel Krech +* Steward Midwinter +* Jack Miller +* Neale Pickett +* Paul Stansifer +* John Szakmeister +* Malcolm Tredinnick +* Ben Wilson +* and many others who helped by reporting bugs diff --git a/docs/change_log/index.md b/docs/change_log/index.md new file mode 100644 index 0000000..9626900 --- /dev/null +++ b/docs/change_log/index.md @@ -0,0 +1,321 @@ +title: Change Log + +Python-Markdown Change Log +========================= + +July 15, 2022: version 3.4.1 (a bug-fix release). + +* Fix an import issue with `importlib.util` (#1274). + +July 15, 2022: version 3.4 ([Notes](release-3.4.md)). + +May 5, 2022: version 3.3.7 (a bug-fix release). + +* Disallow square brackets in reference link ids (#1209). +* Retain configured `pygments_style` after first code block (#1240). +* Ensure fenced code attributes are properly escaped (#1247). + +Nov 17, 2021: version 3.3.6 (a bug-fix release). + +* Fix a dependency issue (#1195, #1196). + +Nov 16, 2021: version 3.3.5 (a bug-fix release). + +* Make the `slugify_unicode` function not remove diacritical marks (#1118). +* Fix `[toc]` detection when used with `nl2br` extension (#1160). +* Re-use compiled regex for block level checks (#1169). +* Don't process shebangs in fenced code blocks when using CodeHilite (#1156). +* Improve email address validation for Automatic Links (#1165). +* Ensure `<summary>` tags are parsed correctly (#1079). +* Support Python 3.10 (#1124). + +Feb 24, 2021: version 3.3.4 (a bug-fix release). + +* Properly parse unclosed tags in code spans (#1066). +* Properly parse processing instructions in md_in_html (#1070). +* Properly parse code spans in md_in_html (#1069). +* Preserve text immediately before an admonition (#1092). +* Simplified regex for HTML placeholders (#928) addressing (#932). +* Ensure `permalinks` and `anchorlinks` are not restricted by `toc_depth` (#1107). +* Fix corner cases with lists under admonitions (#1102). + +Oct 25, 2020: version 3.3.3 (a bug-fix release). + +* Unify all block-level tags (#1047). +* Fix issue where some empty elements would have text rendered as `None` when using `md_in_html` (#1049). +* Avoid catastrophic backtracking in `hr` regex (#1055). +* Fix `hr` HTML handling (#1053). + +Oct 19, 2020: version 3.3.2 (a bug-fix release). + +* Properly parse inline HTML in md_in_html (#1040 & #1045). +* Avoid crashing when md_in_html fails (#1040). + +Oct 12, 2020: version 3.3.1 (a bug-fix release). + +* Correctly parse raw `script` and `style` tags (#1036). +* Ensure consistent class handling by `fenced_code` and `codehilite` (#1032). + +Oct 6, 2020: version 3.3 ([Notes](release-3.3.md)). + +May 8, 2020: version 3.2.2 (a bug-fix release). + +* Add `checklinks` tox environment to ensure all links in documentation are good. +* Refactor extension API documentation (#729). +* Load entry_points (for extensions) only once using `importlib.metadata`. +* Do not double escape entities in TOC. +* Correctly report if an extension raises a `TypeError` (#939). +* Raise a `KeyError` when attempting to delete a nonexistent key from the + extension registry (#939). +* Remove import of `packaging` (or `pkg_resources` fallback) entirely. +* Remove `setuptools` as a run-time dependency (`install_required`). + +Feb 12, 2020: Released version 3.2.1 (a bug-fix release). + +* The `name` property in `toc_tokens` from the TOC extension now + escapes HTML special characters (`<`, `>`, and `&`). + +Feb 7, 2020: Released version 3.2 ([Notes](release-3.2.md)). + +May 20, 2019: Released version 3.1.1 (a bug-fix release). + +* Fixed import failure in `setup.py` when the source directory is not + on `sys.path` (#823). +* Prefer public `packaging` module to pkg_resources' private copy of + it (#825). + +Mar 25, 2019: Released version 3.1 ([Notes](release-3.1.md)). + +Sept 28, 2018: Released version 3.0.1 (a bug-fix release). + +* Brought back the `version` and `version_info` variables (#709). +* Added support for hexadecimal HTML entities (#712). + +Sept 21, 2018: Released version 3.0 ([Notes](release-3.0.md)). + +Jan 4, 2018: Released version 2.6.11 (a bug-fix release). Added a new +`BACKLINK-TITLE` option to the footnote extension so that non-English +users can provide a custom title to back links (see #610). + +Dec 7, 2017: Released version 2.6.10 (a documentation update). + +Aug 17, 2017: Released version 2.6.9 (a bug-fix release). + +Jan 25, 2017: Released version 2.6.8 (a bug-fix release). + +Sept 23, 2016: Released version 2.6.7 (a bug-fix release). + +Mar 20, 2016: Released version 2.6.6 (a bug-fix release). + +Nov 24, 2015: Released version 2.6.5 (a bug-fix release). + +Nov 6, 2015: Released version 2.6.4 (a bug-fix release). + +Oct 26, 2015: Released version 2.6.3 (a bug-fix release). + +Apr 20, 2015: Released version 2.6.2 (a bug-fix release). + +Mar 8, 2015: Released version 2.6.1 (a bug-fix release). The (new) +`yaml` option has been removed from the Meta-Data Extension as it was buggy +(see [#390](https://github.com/Python-Markdown/markdown/issues/390)). + +Feb 19, 2015: Released version 2.6 ([Notes](release-2.6.md)). + +Nov 19, 2014: Released version 2.5.2 (a bug-fix release). + +Sept 26, 2014: Released version 2.5.1 (a bug-fix release). + +Sept 12, 2014: Released version 2.5.0 ([Notes](release-2.5.md)). + +Feb 16, 2014: Released version 2.4.0 ([Notes](release-2.4.md)). + +Mar 22, 2013: Released version 2.3.1 (a bug-fix release). + +Mar 14, 2013: Released version 2.3.0 ([Notes](release-2.3.md)) + +Nov 4, 2012: Released version 2.2.1 (a bug-fix release). + +Jul 5, 2012: Released version 2.2.0 ([Notes](release-2.2.md)). + +Jan 22, 2012: Released version 2.1.1 (a bug-fix release). + +Nov 24, 2011: Released version 2.1.0 ([Notes](release-2.1.md)). + +Oct 7, 2009: Released version 2.0.3. (a bug-fix release). + +Sept 28, 2009: Released version 2.0.2 (a bug-fix release). + +May 20, 2009: Released version 2.0.1 (a bug-fix release). + +Mar 30, 2009: Released version 2.0 ([Notes](release-2.0.md)). + +Mar 8, 2009: Release Candidate 2.0-rc-1. + +Feb 2009: Added support for multi-level lists to new Blockprocessors. + +Jan 2009: Added HTML 4 output as an option (thanks Eric Abrahamsen) + +Nov 2008: Added Definition List ext. Replaced old core with Blockprocessors. +Broken up into multiple files. + +Oct 2008: Changed logging behavior to work better with other systems. +Refactored tree traversing. Added `treap` implementation, then replaced with +OrderedDict. Renamed various processors to better reflect what they actually +do. Refactored footnote ext to match PHP Extra's output. + +Sept 2008: Moved `prettifyTree` to a Postprocessor, replaced WikiLink ext +with WikiLinks (note the s) ext (uses bracketed links instead of CamelCase) +and various bug fixes. + +August 18 2008: Reorganized directory structure. Added a 'docs' directory +and moved all extensions into a 'markdown-extensions' package. +Added additional documentation and a few bug fixes. (v2.0-beta) + +August 4 2008: Updated included extensions to ElementTree. Added a +separate command line script. (v2.0-alpha) + +July 2008: Switched from home-grown NanoDOM to ElementTree and +various related bugs (thanks Artem Yunusov). + +June 2008: Fixed issues with nested inline patterns and cleaned +up testing framework (thanks Artem Yunusov). + +May 2008: Added a number of additional extensions to the +distribution and other minor changes. Moved repository to git from svn. + +Mar 2008: Refactored extension API to accept either an +extension name (as a string) or an instance of an extension +(Thanks David Wolever). Fixed various bugs and added doc strings. + +Feb 2008: Various bug-fixes mostly regarding extensions. + +Feb 18, 2008: Version 1.7. + +Feb 13, 2008: A little code cleanup and better documentation +and inheritance for Preprocessors/Postprocessors. + +Feb 9, 2008: Double-quotes no longer HTML escaped and raw HTML +honors `<?foo>`, `<@foo>`, and `<%foo>` for those who run markdown on +template syntax. + +Dec 12, 2007: Updated docs. Removed encoding argument from Markdown +and markdown as per list discussion. Clean up in prep for 1.7. + +Nov 29, 2007: Added support for images inside links. Also fixed +a few bugs in the footnote extension. + +Nov 19, 2007: `message` now uses python's logging module. Also removed +limit imposed by recursion in `_process_section()`. You can now parse as +long of a document as your memory can handle. + +Nov 5, 2007: Moved `safe_mode` code to a `textPostprocessor` and added +escaping option. + +Nov 3, 2007: Fixed convert method to accept empty strings. + +Oct 30, 2007: Fixed `BOM` removal (thanks Malcolm Tredinnick). Fixed +infinite loop in bracket regular expression for inline links. + +Oct 11, 2007: `LineBreaks` is now an `inlinePattern`. Fixed `HR` in +blockquotes. Refactored `_processSection` method (see tracker #1793419). + +Oct 9, 2007: Added `textPreprocessor` (from 1.6b). + +Oct 8, 2008: Fixed Lazy Blockquote. Fixed code block on first line. +Fixed empty inline image link. + +Oct 7, 2007: Limit recursion on inline patterns. Added a 'safe' tag +to `htmlStash`. + +March 18, 2007: Fixed or merged a bunch of minor bugs, including +multi-line comments and markup inside links. (Tracker #s: 1683066, +1671153, 1661751, 1627935, 1544371, 1458139.) -> v. 1.6b + +Oct 10, 2006: Fixed a bug that caused some text to be lost after +comments. Added "safe mode" (user's HTML tags are removed). + +Sept 6, 2006: Added exception for PHP tags when handling HTML blocks. + +August 7, 2006: Incorporated Sergej Chodarev's patch to fix a problem +with ampersand normalization and HTML blocks. + +July 10, 2006: Switched to using `optparse`. Added proper support for +Unicode. + +July 9, 2006: Fixed the `<!--@address.com>` problem (Tracker #1501354). + +May 18, 2006: Stopped catching unquoted titles in reference links. +Stopped creating blank headers. + +May 15, 2006: A bug with lists, recursion on block-level elements, +run-in headers, spaces before headers, Unicode input (thanks to Aaron +Swartz). Sourceforge tracker #s: 1489313, 1489312, 1489311, 1488370, +1485178, 1485176. (v. 1.5) + +Mar. 24, 2006: Switched to a not-so-recursive algorithm with +`_handleInline`. (Version 1.4) + +Mar. 15, 2006: Replaced some instance variables with class variables +(a patch from Stelios Xanthakis). Chris Clark's new regexps that do +not trigger mid-word underlining. + +Feb. 28, 2006: Clean-up and command-line handling by Stewart +Midwinter. (Version 1.3) + +Feb. 24, 2006: Fixed a bug with the last line of the list appearing +again as a separate paragraph. Incorporated Chris Clark's "mail-to" +patch. Added support for `<br />` at the end of lines ending in two or +more spaces. Fixed a crashing bug when using `ImageReferencePattern`. +Added several utility methods to `Nanodom`. (Version 1.2) + +Jan. 31, 2006: Added `hr` and `hr/` to BLOCK_LEVEL_ELEMENTS and +changed `<hr/>` to `<hr />`. (Thanks to Sergej Chodarev.) + +Nov. 26, 2005: Fixed a bug with certain tabbed lines inside lists +getting wrapped in `<pre><code>`. (v. 1.1) + +Nov. 19, 2005: Made `<!...`, `<?...`, etc. behave like block-level +HTML tags. + +Nov. 14, 2005: Added entity code and email auto-link fix by Tiago +Cogumbreiro. Fixed some small issues with backticks to get 100% +compliance with John's test suite. (v. 1.0) + +Nov. 7, 2005: Added an `unlink` method for documents to aid with memory +collection (per Doug Sauder's suggestion). + +Oct. 29, 2005: Restricted a set of HTML tags that get treated as +block-level elements. + +Sept. 18, 2005: Refactored the whole script to make it easier to +customize it and made footnote functionality into an extension. +(v. 0.9) + +Sept. 5, 2005: Fixed a bug with multi-paragraph footnotes. Added +attribute support. + +Sept. 1, 2005: Changed the way headers are handled to allow inline +syntax in headers (e.g. links) and got the lists to use p-tags +correctly (v. 0.8) + +Aug. 29, 2005: Added flexible tabs, fixed a few small issues, added +basic support for footnotes. Got rid of xml.dom.minidom and added +pretty-printing. (v. 0.7) + +Aug. 13, 2005: Fixed a number of small bugs in order to conform to the +test suite. (v. 0.6) + +Aug. 11, 2005: Added support for inline HTML and entities, inline +images, auto-links, underscore emphasis. Cleaned up and refactored the +code, added some more comments. + +Feb. 19, 2005: Rewrote the handling of high-level elements to allow +multi-line list items and all sorts of nesting. + +Feb. 3, 2005: Reference-style links, single-line lists, backticks, +escape, emphasis in the beginning of the paragraph. + +Nov. 2004: Added links, blockquotes, HTML blocks to Manfred +Stienstra's code + +Apr. 2004: Manfred's version at `http://www.dwerg.net/projects/markdown/` diff --git a/docs/release-2.0.txt b/docs/change_log/release-2.0.md index b1f71ad..abd8d29 100644 --- a/docs/release-2.0.txt +++ b/docs/change_log/release-2.0.md @@ -1,34 +1,36 @@ +title: Release Notes for v2.0 + Python-Markdown 2.0 Release Notes ================================= -We are happy to release Python-Markdown 2.0, which has been over a year in the -making. We have rewritten significant portions of the code, dramatically -extending the extension API, increased performance, and added numerous +We are happy to release Python-Markdown 2.0, which has been over a year in the +making. We have rewritten significant portions of the code, dramatically +extending the extension API, increased performance, and added numerous extensions to the distribution (including an extension that mimics PHP Markdown Extra), all while maintaining backward compatibility with the end user API in version 1.7. -Python-Markdown supports Python versions 2.3, 2.4, 2.5, and 2.6. We've even +Python-Markdown supports Python versions 2.3, 2.4, 2.5, and 2.6. We have even released a version converted to Python 3.0! Backwards-incompatible Changes ------------------------------ -While Python-Markdown has experienced numerous internal changes, those changes -should only affect extension authors. If you have not written your own -extensions, then you should not need to make any changes to your code. +While Python-Markdown has experienced numerous internal changes, those changes +should only affect extension authors. If you have not written your own +extensions, then you should not need to make any changes to your code. However, you may want to ensure that any third party extensions you are using are compatible with the new API. -The new extension API is fully documented in [[writing_extensions]]. Below is a -summary of the significant changes: +The new extension API is fully [documented](../extensions/api.md) in the docs. +Below is a summary of the significant changes: * The old home-grown NanoDOM has been replaced with ElementTree. Therefore all extensions must use ElementTree rather than the old NanoDOM. -* The various processors and patterns are now stored with OrderedDicts rather - than lists. Any code adding processors and/or patterns into Python-Markdown +* The various processors and patterns are now stored with OrderedDicts rather + than lists. Any code adding processors and/or patterns into Python-Markdown will need to be adjusted to use the new API using OrderedDicts. -* The various types of processors available have been either combined, added, +* The various types of processors available have been either combined, added, or removed. Ensure that your processors match the currently supported types. What's New in Python-Markdown 2.0 @@ -39,20 +41,20 @@ ElementTree internally to build the (X)HTML document from markdown source text. This has resolved various issues with the older home-grown NanoDOM and made notable increases in performance. -Artem also refactored the Inline Patterns to better support nested patterns -which has resolved many inconsistencies in Python-Markdown's parsing of the +Artem also refactored the Inline Patterns to better support nested patterns +which has resolved many inconsistencies in Python-Markdown's parsing of the markdown syntax. -The core parser had been completely rewritten, increasing performance and, for +The core parser had been completely rewritten, increasing performance and, for the first time, making it possible to override/add/change the way block level content is parsed. -Python-Markdown now parses markdown source text more closely to the other +Python-Markdown now parses markdown source text more closely to the other popular implementations (Perl, PHP, etc.) than it ever has before. With the exception of a few minor insignificant differences, any difference should be considered a bug, rather than a limitation of the parser. -The option to return HTML4 output as apposed to XHTML has been added. In +The option to return HTML4 output as apposed to XHTML has been added. In addition, extensions should be able to easily add additional output formats. As part of implementing markdown in the Dr. Project project (a Trac fork), among @@ -61,7 +63,7 @@ accepts either the extension names as strings or instances of extensions. This makes it possible to include multiple extensions in a single module. Numerous extensions are included in the distribution by default. See -[[available_extensions]] for a complete list. +[available_extensions](../extensions/index.md) for a complete list. -See the [[change_log]] for a full list of changes. +See the [Change Log](index.md) for a full list of changes. diff --git a/docs/change_log/release-2.1.md b/docs/change_log/release-2.1.md new file mode 100644 index 0000000..4f7ba75 --- /dev/null +++ b/docs/change_log/release-2.1.md @@ -0,0 +1,118 @@ +title: Release Notes for v2.1 + +Python-Markdown 2.1 Release Notes +================================= + +We are pleased to release Python-Markdown 2.1 which makes many +improvements on 2.0. In fact, we consider 2.1 to be what 2.0 should have been. +While 2.1 consists mostly of bug fixes, bringing Python-Markdown more inline +with other implementations, some internal improvements were made to the parser, +a few new built-in extensions were added, and HTML5 support was added. + +Python-Markdown supports Python versions 2.4, 2.5, 2.6, 2.7, 3.1, and 3.2 out +of the box. In fact, the same code base installs on Python 3.1 and 3.2 with no +extra work by the end user. + +Backwards-incompatible Changes +------------------------------ + +While Python-Markdown has received only minor internal changes since the last +release, there are a few backward-incompatible changes to note: + +* Support had been dropped for Python 2.3. No guarantees are made that the + library will work in any version of Python lower than 2.4. Additionally, while + the library had been tested with Python 2.4, consider Python 2.4 support to be + depreciated. It is not likely that any future versions will continue to + support any version of Python less than 2.5. Note that Python 3.0 is not + supported due to a bug in its 2to3 tool. If you must use Python-Markdown with + Python 3.0, it is suggested you manually use Python 3.1's 2to3 tool to do a + conversion. + +* Python-Markdown previously accepted positional arguments on its class and + wrapper methods. It now expects keyword arguments. Currently, the positional + arguments should continue to work, but the solution feels hacky and may be + removed in a future version. All users are encouraged to use keyword arguments + as documented in the [Library Reference](../reference.md). + +* Past versions of Python-Markdown provided module level Global variables which + controlled the behavior of a few different aspects of the parser. Those global + variables have been replaced with attributes on the Markdown class. + Additionally, those attributes are settable as keyword arguments when + initializing a class instance. Therefore, if you were editing the global + variables (either by editing the source or by overriding them in your code), + you should now set them on the class. See the [Library + Reference](../reference.md) for the options available. + +* If you have been using the HeaderId extension to + define custom ids on headers, you will want to switch to using the new + [Attribute List](../extensions/attr_list.md) extension. The HeaderId extension + now only auto-generates ids on headers which have not already had ids defined. + Note that the [Extra](../extensions/extra.md) extension has been switched to + use Attribute Lists instead of HeaderId as it did previously. + +* Some code was moved into the `markdown.util` namespace which was previously in + the `markdown` namespace. Extension authors may need to adjust a few import + statements in their extensions to work with the changes. + +* The command line script name was changed to `markdown_py`. The previous name + (`markdown`) was conflicting with people (and Linux package systems) who also + had markdown.pl installed on there system as markdown.pl's command line script + was also named `markdown`. Be aware that installing Python-Markdown 2.1 will + not remove the old versions of the script with different names. You may want + to remove them yourself as they are unlikely to work properly. + +What's New in Python-Markdown 2.1 +--------------------------------- + +Three new extensions were added. [Attribute Lists](../extensions/attr_list.md), +which was inspired by Maruku's feature of the same name, [Newline to +Break](../extensions/nl2br.md), which was inspired by GitHub Flavored Markdown, +and Smart Strong, which fills a hole in the Extra extension. + +HTML5 is now supported. All this really means is that new block level elements +introduced in the HTML5 spec are now properly recognized as raw HTML. As +valid HTML5 can consist of either HTML4 or XHTML1, there is no need to add a +new HTML5 serializers. That said, `html5` and `xhtml5` have been added as +aliases of the `html4` and `xhtml1` serializers respectively. + +An XHTML serializer has been added. Previously, ElementTree's XML serializer +was being used for XHTML output. With the new serializer we are able to avoid +more invalid output like empty elements (i.e., `<p />`) which can choke +browsers. + +Improved support for Python 3.x. Now when running `setupy.py install` in +Python 3.1 or greater the 2to3 tool is run automatically. Note that Python 3.0 +is not supported due to a bug in its 2to3 tool. If you must use Python-Markdown +with Python 3.0, it is suggested you manually use Python 3.1's 2to3 tool to +do a conversion. + +Methods on instances of the Markdown class that do not return results can now +be changed allowing one to do `md.reset().convert(moretext)`. + +The Markdown class was refactored so that a subclass could define its own +`build_parser` method which would build a completely different parser. In +other words, one could use the basic machinery in the markdown library to +build a parser of a different markup language without the overhead of building +the markdown parser and throwing it away. + +Import statements within markdown have been improved so that third party +libraries can embed the markdown library if they desire (licensing permitting). + +Added support for Python's `-m` command line option. You can run the markdown +package as a command line script. Do `python -m markdown [options] [args]`. +Note that this is only fully supported in Python 2.7+. Python 2.5 & 2.6 +require you to call the module directly (`markdown.__main__`) rather than +the package (`markdown`). This does not work in Python 2.4. + +The command line script has been renamed to `markdown_py` which avoids all the +various problems we had with previous names. Also improved the command line +script to accept input on `stdin`. + +The testing framework has been completely rebuilt using the Nose testing +framework. This provides a number of benefits including the ability to better +test the built-in extensions and other options available to change the parsing +behavior. See the Test Suite documentation for details. + +Various bug fixes have been made, which are too numerous to list here. See the +[commit log](https://github.com/Python-Markdown/markdown/commits/master) for a +complete history of the changes. diff --git a/docs/change_log/release-2.2.md b/docs/change_log/release-2.2.md new file mode 100644 index 0000000..75f47fa --- /dev/null +++ b/docs/change_log/release-2.2.md @@ -0,0 +1,64 @@ +title: Release Notes for v2.2 + +Python-Markdown 2.2 Release Notes +================================= + +We are pleased to release Python-Markdown 2.2 which makes improvements on 2.1. +While 2.2 is primarily a bug fix release, some internal improvements were made +to the parser, and a few security issues were resolved. + +Python-Markdown supports Python versions 2.5, 2.6, 2.7, 3.1, and 3.2 out +of the box. + +Backwards-incompatible Changes +------------------------------ + +While Python-Markdown has received only minor internal changes since the last +release, there are a few backward-incompatible changes to note: + +* Support had been dropped for Python 2.4. No guarantees are made that the + library will work in any version of Python lower than 2.5. Additionally, while + the library had been tested with Python 2.5, consider Python 2.5 support to be + depreciated. It is not likely that any future versions will continue to + support any version of Python less than 2.6. + +* For many years Python-Markdown has identified `<ins>` and `<del>` tags in raw + HTML input as block level tags. As they are actually inline level tags, this + behavior has been changed. This may result in slightly different output. While + in most cases, the new output is more correct, there may be a few edge cases + where a document author has relied on the previous incorrect behavior. It is + likely that a few adjustments may need to be made to those documents. + +* The behavior of the `enable_attributes` keyword has been slightly altered. If + authors have been using attributes in documents with `safe_mode` on, those + attributes will no longer be parsed unless `enable_attributes` is explicitly + set to `True`. This change was made to prevent untrusted authors from + injecting potentially harmful JavaScript in documents. This change had no + effect when not in `safe_mode`. + +What's New in Python-Markdown 2.2 +--------------------------------- + +The docs were refactored and can now be found at +`http://packages.python.org/Markdown/`. The docs are now maintained in the +Repository and are generated by the `setup.py build_docs` command. + +The [Sane_Lists](../extensions/sane_lists.md) +extension was added. The Sane Lists Extension alters the behavior of the +Markdown List syntax to be less surprising by not allowing the mixing of list +types. In other words, an ordered list will not continue when an unordered list +item is encountered and vice versa. + +Markdown now excepts a full path to an extension module. In other words, your +extensions no longer need to be in the primary namespace (and start with `mdx_`) +for Markdown to find them. Just do `Markdown(extension=['path.to.some.module'])`. +As long as the provided module contains a compatible extension, the extension +will be loaded. + +The BlockParser API was slightly altered to allow `blockprocessor.run` to return +`True` or `False` which provides more control to the block processor loop from +within any Blockprocessor instance. + +Various bug fixes have been made. See the +[commit log](https://github.com/Python-Markdown/markdown/commits/master) +for a complete history of the changes. diff --git a/docs/change_log/release-2.3.md b/docs/change_log/release-2.3.md new file mode 100644 index 0000000..f60e426 --- /dev/null +++ b/docs/change_log/release-2.3.md @@ -0,0 +1,85 @@ +title: Release Notes for v2.3 + +Python-Markdown 2.3 Release Notes +================================= + +We are pleased to release Python-Markdown 2.3 which adds one new extension, +removes a few old (obsolete) extensions, and now runs on both Python 2 and +Python 3 without running the 2to3 conversion tool. See the list of changes +below for details. + +Python-Markdown supports Python versions 2.6, 2.7, 3.1, 3.2, and 3.3. + +Backwards-incompatible Changes +------------------------------ + +* Support has been dropped for Python 2.5. No guarantees are made that the + library will work in any version of Python lower than 2.6. As all supported + Python versions include the ElementTree library, Python-Markdown will no + longer try to import a third-party installation of ElementTree. + +* All classes are now "new-style" classes. In other words, all classes subclass + from 'object'. While this is not likely to affect most users, extension + authors may need to make a few minor adjustments to their code. + +* "safe_mode" has been further restricted. Markdown formatted links must be of a + known white-listed scheme when in "safe_mode" or the URL is discarded. The + white-listed schemes are: 'HTTP', 'HTTPS', 'FTP', 'FTPS', 'MAILTO', and + 'news'. Schemeless URLs are also permitted, but are checked in other ways - as + they have been for some time. + +* The ids assigned to footnotes now contain a dash (`-`) rather than a colon + (`:`) when `output_format` it set to `"html5"` or `"xhtml5"`. If you are + making reference to those ids in your JavaScript or CSS and using the HTML5 + output, you will need to update your code accordingly. No changes are + necessary if you are outputting XHTML (the default) or HTML4. + +* The `force_linenos` configuration setting of the CodeHilite extension has been + marked as **Pending Deprecation** and a new setting `linenums` has been added + to replace it. See documentation for the [CodeHilite Extension] for an + explanation of the new `linenums` setting. The new setting will honor the old + `force_linenos` if it is set, but it will raise a `PendingDeprecationWarning` + and will likely be removed in a future version of Python-Markdown. + +[CodeHilite Extension]: ../extensions/code_hilite.md + +* The "RSS" extension has been removed and no longer ships with Python-Markdown. + If you would like to continue using the extension (not recommended), it is + archived on [GitHub](https://gist.github.com/waylan/4773365). + +* The "HTML Tidy" Extension has been removed and no longer ships with + Python-Markdown. If you would like to continue using the extension (not + recommended), it is archived on + [GitHub](https://gist.github.com/waylan/5152650). Note that the underlying + library, uTidylib, is not Python 3 compatible. Instead, it is recommended that + the newer [PyTidyLib] (version 0.2.2+ for Python 3 comparability - install + from GitHub not PyPI) be used. As the API for that library is rather simple, + it is recommended that the output of Markdown be wrapped in a call to + PyTidyLib rather than using an extension (for example: + `tidylib.tidy_fragment(markdown.markdown(source), options={...})`). + +[PyTidyLib]: http://countergram.github.io/pytidylib/ + +What's New in Python-Markdown 2.3 +--------------------------------- + +* The entire code base now universally runs in Python 2 and Python 3 without any + need for running the 2to3 conversion tool. This not only simplifies testing, + but by using Unicode_literals, results in more consistent behavior across + Python versions. Additionally, the relative imports (made possible in Python 2 + via absolute_import) allows the entire library to more easily be embedded in a + sub-directory of another project. The various files within the library will + still import each other properly even though 'markdown' may not be in Python's + root namespace. + +* The [Admonition Extension] has been added, which implements [rST-style][rST] + admonitions in the Markdown syntax. However, be warned that this extension is + experimental and the syntax and behavior is still subject to change. Please + try it out and report bugs and/or improvements. + +[Admonition Extension]: ../extensions/admonition.md +[rST]: http://docutils.sourceforge.net/docs/ref/rst/directives.html#specific-admonitions + +* Various bug fixes have been made. See the [commit + log](https://github.com/Python-Markdown/markdown/commits/master) for a + complete history of the changes. diff --git a/docs/change_log/release-2.4.md b/docs/change_log/release-2.4.md new file mode 100644 index 0000000..1f4ab23 --- /dev/null +++ b/docs/change_log/release-2.4.md @@ -0,0 +1,73 @@ +title: Release Notes for v2.4 + +Python-Markdown 2.4 Release Notes +================================= + +We are pleased to release Python-Markdown 2.4 which adds one new extension +and fixes various bugs. See the list of changes below for details. + +Python-Markdown supports Python versions 2.6, 2.7, 3.1, 3.2, and 3.3. + +Backwards-incompatible Changes +------------------------------ + +* The `force_linenos` configuration setting of the CodeHilite extension has been + marked as **Deprecated**. It had previously been marked as "Pending + Deprecation" in version 2.3 when a new setting `linenums` was added to replace + it. See documentation for the [CodeHilite Extension] for an explanation of the + new `linenums` setting. The new setting will honor the old `force_linenos` if + it is set, but `force_linenos` will raise a `DeprecationWarning` and will + likely be removed in a future version of Python-Markdown. + +[CodeHilite Extension]: ../extensions/code_hilite.md + +* URLs are no longer percent-encoded. This improves compatibility with the + original (written in Perl) Markdown implementation. Please percent-encode your + URLs manually when needed. + +What's New in Python-Markdown 2.4 +--------------------------------- + +* Thanks to the hard work of [Dmitry Shachnev] the [Smarty Extension] has been + added, which implements [SmartyPants] using Python-Markdown's Extension API. + This offers a few benefits over a third party script. The HTML does not need + to be "tokenized" twice, no hacks are required to combine SmartyPants and code + highlighting, and we get markdown's escaping feature for free. Please try it + out and report bugs and/or improvements. + +[Dmitry Shachnev]: https://github.com/mitya57 +[Smarty Extension]: ../extensions/smarty.md +[SmartyPants]: https://daringfireball.net/projects/smartypants/ + +* The [Table of Contents Extension] now supports new `permalink` option for + creating [Sphinx]-style anchor links. + +[Table of Contents Extension]: ../extensions/toc.md +[Sphinx]: http://sphinx-doc.org/ + +* It is now possible to enable Markdown formatting inside HTML blocks by + appending `markdown=1` to opening tag attributes. See [Markdown Inside HTML + Blocks] section for details. Thanks to [ryneeverett] for implementing this + feature. + +[Markdown Inside HTML Blocks]: ../extensions/extra.md#nested-markdown-inside-html-blocks +[ryneeverett]: https://github.com/ryneeverett + +* The code blocks now support emphasizing some of the code lines. To use this + feature, specify `hl_lines` option after language name, for example (using the + [Fenced Code Extension]): + + ```.python hl_lines="1 3" + # This line will be emphasized. + # This one won't. + # This one will be also emphasized. + ``` + + Thanks to [A. Jesse Jiryu Davis] for implementing this feature. + +[Fenced Code Extension]: ../extensions/fenced_code_blocks.md +[A. Jesse Jiryu Davis]: https://github.com/ajdavis + +* Various bug fixes have been made. See the [commit + log](https://github.com/Python-Markdown/markdown/commits/master) for a + complete history of the changes. diff --git a/docs/change_log/release-2.5.md b/docs/change_log/release-2.5.md new file mode 100644 index 0000000..519f013 --- /dev/null +++ b/docs/change_log/release-2.5.md @@ -0,0 +1,189 @@ +title: Release Notes for v2.5 + +Python-Markdown 2.5 Release Notes +================================= + +We are pleased to release Python-Markdown 2.5 which adds a few new features +and fixes various bugs. See the list of changes below for details. + +Python-Markdown version 2.5 supports Python versions 2.7, 3.2, 3.3, and 3.4. + +Backwards-incompatible Changes +------------------------------ + +* Python-Markdown no longer supports Python version 2.6. You must be using Python + versions 2.7, 3.2, 3.3, or 3.4. + +[importlib]: https://pypi.org/project/importlib/ + +* The `force_linenos` configuration key on the [CodeHilite Extension] has been **deprecated** + and will raise a `KeyError` if provided. In the previous release (2.4), it was + issuing a `DeprecationWarning`. The [`linenums`][linenums] keyword should be used + instead, which provides more control of the output. + +[CodeHilite Extension]: ../extensions/code_hilite.md +[linenums]: ../extensions/code_hilite.md#usage + +* Both `safe_mode` and the associated `html_replacement_text` keywords will be + deprecated in version 2.6 and will raise a **`PendingDeprecationWarning`** in + 2.5. The so-called "safe mode" was never actually "safe" which has resulted in + many people having a false sense of security when using it. As an alternative, + the developers of Python-Markdown recommend that any untrusted content be + passed through an HTML sanitizer (like [Bleach]) after being converted to HTML + by markdown. + + If your code previously looked like this: + + html = markdown.markdown(text, same_mode=True) + + Then it is recommended that you change your code to read something like this: + + import bleach + html = bleach.clean(markdown.markdown(text)) + + If you are not interested in sanitizing untrusted text, but simply desire to + escape raw HTML, then that can be accomplished through an extension which + removes HTML parsing: + + from markdown.extensions import Extension + + class EscapeHtml(Extension): + def extendMarkdown(self, md, md_globals): + del md.preprocessors['html_block'] + del md.inlinePatterns['html'] + + html = markdown.markdown(text, extensions=[EscapeHtml()]) + + As the HTML would not be parsed with the above Extension, then the + serializer will escape the raw HTML, which is exactly what happens now when + `safe_mode="escape"`. + +[Bleach]: https://bleach.readthedocs.io/ + +* Positional arguments on the `markdown.Markdown()` are pending deprecation as are + all except the `text` argument on the `markdown.markdown()` wrapper function. + Only keyword arguments should be used. For example, if your code previously + looked like this: + + html = markdown.markdown(text, ['extra']) + + Then it is recommended that you change it to read something like this: + + html = markdown.markdown(text, extensions=['extra']) + + !!! Note + This change is being made as a result of deprecating `"safe_mode"` as the + `safe_mode` argument was one of the positional arguments. When that argument + is removed, the two arguments following it will no longer be at the correct + position. It is recommended that you always use keywords when they are supported + for this reason. + +* In previous versions of Python-Markdown, the built-in extensions received + special status and did not require the full path to be provided. Additionally, + third party extensions whose name started with `"mdx_"` received the same + special treatment. This behavior will be deprecated in version 2.6 and will + raise a **`PendingDeprecationWarning`** in 2.5. Ensure that you always use the + full path to your extensions. For example, if you previously did the + following: + + markdown.markdown(text, extensions=['extra']) + + You should change your code to the following: + + markdown.markdown(text, extensions=['markdown.extensions.extra']) + + The same applies to the command line: + + $ python -m markdown -x markdown.extensions.extra input.txt + + See the [documentation](../reference.md#extensions) for a full explanation + of the current behavior. + +* The previously documented method of appending the extension configuration as + a string to the extension name will be deprecated in Python-Markdown + version 2.6 and will raise a **`PendingDeprecationWarning`** in 2.5. The + [`extension_configs`](../reference.md#extension_configs) keyword should + be used instead. See the [documentation](../reference.md#extension-configs) + for a full explanation of the current behavior. + +What's New in Python-Markdown 2.5 +--------------------------------- + +* The [Smarty Extension] has had a number of additional configuration settings + added, which allows one to define their own substitutions to better support + languages other than English. Thanks to [Martin Altmayer] for implementing this + feature. + +[Smarty Extension]: ../extensions/smarty.md +[Martin Altmayer]:https://github.com/MartinAltmayer + +* Named Extensions (strings passed to the [`extensions`][ex] keyword of + `markdown.Markdown`) can now point to any module and/or Class on your + PYTHONPATH. While dot notation was previously supported, a module could not + be at the root of your PYTHONPATH. The name had to contain at least one dot + (requiring it to be a sub-module). This restriction no longer exists. + + Additionally, a Class may be specified in the name. The class must be at the + end of the name (which uses dot notation from PYTHONPATH) and be separated + by a colon from the module. + + Therefore, if you were to import the class like this: + + from path.to.module import SomeExtensionClass + + Then the named extension would comprise this string: + + "path.to.module:SomeExtensionClass" + + This allows multiple extensions to be implemented within the same module and + still accessible when the user is not able to import the extension directly + (perhaps from a template filter or the command line). + + This also means that extension modules are no longer required to include the + `makeExtension` function which returns an instance of the extension class. + However, if the user does not specify the class name (she only provides + `"path.to.module"`) the extension will fail to load without the + `makeExtension` function included in the module. Extension authors will want + to document carefully what is required to load their extensions. + +[ex]: ../reference.md#extensions + +* The Extension Configuration code has been refactored to make it a little + easier for extension authors to work with configuration settings. As a + result, the [`extension_configs`][ec] keyword now accepts a dictionary + rather than requiring a list of tuples. A list of tuples is still supported + so no one needs to change their existing code. This should also simplify the + learning curve for new users. + + Extension authors are encouraged to review the new methods available on the + `markdown.extnesions.Extension` class for handling configuration and adjust + their code going forward. The included extensions provide a model for best + practices. See the [API] documentation for a full explanation. + +[ec]: ../reference.md#extension_configs +[API]: ../extensions/api.md#configsettings + +* The [Command Line Interface][cli] now accepts a `--extensions_config` (or + `-c`) option which accepts a file name and passes the parsed content of a + [YAML] or [JSON] file to the [`extension_configs`][ec] keyword of the + `markdown.Markdown` class. The contents of the YAML or JSON must map to a + Python Dictionary which matches the format required by the + `extension_configs` keyword. Note that [PyYAML] is required to parse YAML + files. + +[cli]: ../cli.md#using-extensions +[YAML]: https://yaml.org/ +[JSON]: https://json.org/ +[PyYAML]: https://pyyaml.org/ + +* The [Admonition Extension][ae] is no longer considered "experimental." + +[ae]: ../extensions/admonition.md + +* There have been various refactors of the testing framework. While those + changes will not directly effect end users, the code is being better tested + which will benefit everyone. + +* Various bug fixes have been made. See the [commit + log](https://github.com/Python-Markdown/markdown/commits/master) for a + complete history of the changes. diff --git a/docs/change_log/release-2.6.md b/docs/change_log/release-2.6.md new file mode 100644 index 0000000..f117eb8 --- /dev/null +++ b/docs/change_log/release-2.6.md @@ -0,0 +1,304 @@ +title: Release Notes for v2.6 + +# Python-Markdown 2.6 Release Notes + +We are pleased to release Python-Markdown 2.6 which adds a few new features +and fixes various bugs. See the list of changes below for details. + +Python-Markdown version 2.6 supports Python versions 2.7, 3.2, 3.3, and 3.4 as +well as PyPy. + +## Backwards-incompatible Changes + +### `safe_mode` Deprecated + +Both `safe_mode` and the associated `html_replacement_text` keywords are +deprecated in version 2.6 and will raise a **`DeprecationWarning`**. The +`safe_mode` and `html_replacement_text` keywords will be ignored in the next +release. The so-called "safe mode" was never actually "safe" which has resulted +in many people having a false sense of security when using it. As an +alternative, the developers of Python-Markdown recommend that any untrusted +content be passed through an HTML sanitizer (like [Bleach]) after being +converted to HTML by markdown. In fact, [Bleach Whitelist] provides a curated +list of tags, attributes, and styles suitable for filtering user-provided HTML +using bleach. + +If your code previously looked like this: + +```python +html = markdown.markdown(text, safe_mode=True) +``` + +Then it is recommended that you change your code to read something like this: + +```python +import bleach +from bleach_whitelist import markdown_tags, markdown_attrs +html = bleach.clean(markdown.markdown(text), markdown_tags, markdown_attrs) +``` + +If you are not interested in sanitizing untrusted text, but simply desire to +escape raw HTML, then that can be accomplished through an extension which +removes HTML parsing: + +```python +from markdown.extensions import Extension + +class EscapeHtml(Extension): + def extendMarkdown(self, md, md_globals): + del md.preprocessors['html_block'] + del md.inlinePatterns['html'] + +html = markdown.markdown(text, extensions=[EscapeHtml()]) +``` + +As the HTML would not be parsed with the above Extension, then the serializer +will escape the raw HTML, which is exactly what happens now when +`safe_mode="escape"`. + +[Bleach]: https://bleach.readthedocs.io/ +[Bleach Whitelist]: https://github.com/yourcelf/bleach-whitelist + +### Positional Arguments Deprecated + +Positional arguments on the `markdown.Markdown()` class are deprecated as are +all except the `text` argument on the `markdown.markdown()` wrapper function. +Using positional arguments will raise a **`DeprecationWarning`** in 2.6 and an +error in the next release. Only keyword arguments should be used. For example, +if your code previously looked like this: + +```python +html = markdown.markdown(text, [SomeExtension()]) +``` + +Then it is recommended that you change it to read something like this: + +```python +html = markdown.markdown(text, extensions=[SomeExtension()]) +``` + +!!! Note + This change is being made as a result of deprecating `"safe_mode"` as the + `safe_mode` argument was one of the positional arguments. When that argument + is removed, the two arguments following it will no longer be at the correct + position. It is recommended that you always use keywords when they are + supported for this reason. + +### "Shortened" Extension Names Deprecated + +In previous versions of Python-Markdown, the built-in extensions received +special status and did not require the full path to be provided. Additionally, +third party extensions whose name started with `"mdx_"` received the same +special treatment. This behavior is deprecated and will raise a +**`DeprecationWarning`** in version 2.6 and an error in the next release. Ensure +that you always use the full path to your extensions. For example, if you +previously did the following: + +```python +markdown.markdown(text, extensions=['extra']) +``` + +You should change your code to the following: + +```python +markdown.markdown(text, extensions=['markdown.extensions.extra']) +``` + +The same applies to the command line: + +```python +python -m markdown -x markdown.extensions.extra input.txt +``` + +Similarly, if you have used a third party extension (for example `mdx_math`), +previously you might have called it like this: + +```python +markdown.markdown(text, extensions=['math']) +``` + +As the `"mdx"` prefix will no longer be appended, you will need to change your +code as follows (assuming the file `mdx_math.py` is installed at the root of +your PYTHONPATH): + +```python +markdown.markdown(text, extensions=['mdx_math']) +``` + +Extension authors will want to update their documentation to reflect the new +behavior. + +See the [documentation](../reference.md#extensions) for a full explanation +of the current behavior. + +### Extension Configuration as Part of Extension Name Deprecated + +The previously documented method of appending the extension configuration +options as a string to the extension name is deprecated and will raise a +**`DeprecationWarning`** in version 2.6 and an error in 2.7. The +[`extension_configs`](../reference.md#extension_configs) keyword should be used +instead. See the [documentation](../reference.md#extension-configs) for a full +explanation of the current behavior. + +### HeaderId Extension Pending Deprecation + +The HeaderId Extension is pending deprecation and will raise a +**`PendingDeprecationWarning`** in version 2.6. The extension will be deprecated +in the next release and raise an error in the release after that. Use the [Table +of Contents][TOC] Extension instead, which offers most of the features of the +HeaderId Extension and more (support for meta data is missing). + +Extension authors who have been using the `slugify` and `unique` functions +defined in the HeaderId Extension should note that those functions are now +defined in the Table of Contents extension and should adjust their import +statements accordingly (`from markdown.extensions.toc import slugify, unique`). + +### The `configs` Keyword is Deprecated + +Positional arguments and the `configs` keyword on the +`markdown.extension.Extension` class (and its subclasses) are deprecated. Each +individual configuration option should be passed to the class as a keyword/value +pair. For example. one might have previously initiated an extension subclass +like this: + +```python +ext = SomeExtension(configs={'somekey': 'somevalue'}) +``` + +That code should be updated to pass in the options directly: + +```python +ext = SomeExtension(somekey='somevalue') +``` + +Extension authors will want to note that this affects the `makeExtension` +function as well. Previously it was common for the function to be defined as +follows: + +```python +def makeExtension(configs=None): + return SomeExtension(configs=configs) +``` + +Extension authors will want to update their code to the following instead: + +```python +def makeExtension(**kwargs): + return SomeExtension(**kwargs) +``` + +Failing to do so will result in a **`DeprecationWarning`** and will raise an +error in the next release. See the [Extension API][mext] documentation for more +information. + +In the event that an `markdown.extension.Extension` subclass overrides the +`__init__` method and implements its own configuration handling, then the above +may not apply. However, it is recommended that the subclass still calls the +parent `__init__` method to handle configuration options like so: + +```python +class SomeExtension(markdown.extension.Extension): + def __init__(**kwargs): + # Do pre-config stuff here + # Set config defaults + self.config = { + 'option1' : ['value1', 'description1'], + 'option2' : ['value2', 'description2'] + } + # Set user defined configs + super(MyExtension, self).__init__(**kwargs) + # Do post-config stuff here +``` + +Note the call to `super` to get the benefits of configuration handling from the +parent class. See the [documentation][config] for more information. + +[config]: ../extensions/api.md#configsettings +[mext]: ../extensions/api.md#makeextension + +## What's New in Python-Markdown 2.6 + +### Official Support for PyPy + +Official support for [PyPy] has been added. While Python-Markdown has most +likely worked on PyPy for some time, it is now officially supported and tested +on PyPy. + +[PyPy]: https://pypy.org/ + +### YAML Style Meta-Data + +<del>The [Meta-Data] Extension now includes optional support for [YAML] style +meta-data.</del> By default, the YAML deliminators are recognized, however, the +actual data is parsed as previously. This follows the syntax of [MultiMarkdown], +which inspired this extension. + +<del>Alternatively, if the `yaml` option is set, then the data is parsed as +YAML.</del> <ins>As the `yaml` option was buggy, it was removed in 2.6.1. It is +suggested that a preprocessor (like [docdata]) or a third party extension be +used if you want true YAML support. See [Issue #390][#390] for a full +explanation.</ins> + +[MultiMarkdown]: https://fletcherpenney.net/multimarkdown/#metadata +[Meta-Data]: ../extensions/meta_data.md +[YAML]: https://yaml.org/ +[#390]: https://github.com/Python-Markdown/markdown/issues/390 +[docdata]: https://github.com/waylan/docdata + +### Table of Contents Extension Refactored + +The [Table of Contents][TOC] Extension has been refactored and some new features +have been added. See the documentation for a full explanation of each feature +listed below: + +* The extension now assigns the Table of Contents to the `toc` attribute of + the Markdown class regardless of whether a "marker" was found in the + document. Third party frameworks no longer need to insert a "marker," run + the document through Markdown, then extract the Table of Contents from the + document. + +* The Table of Contents Extension is now a "registered extension." Therefore, + when the `reset` method of the Markdown class is called, the `toc` attribute + on the Markdown class is cleared (set to an empty string). + +* When the `marker` configuration option is set to an empty string, the parser + completely skips the process of searching the document for markers. This + should save parsing time when the Table of Contents Extension is being used + only to assign ids to headers. + +* A `separator` configuration option has been added allowing users to override + the separator character used by the slugify function. + +* A `baselevel` configuration option has been added allowing users to set the + base level of headers in their documents (h1-h6). This allows the header + levels to be automatically adjusted to fit within the hierarchy of an HTML + template. + +[TOC]: ../extensions/toc.md + +### Pygments can now be disabled + +The [CodeHilite][ch] Extension has gained a new configuration option: +`use_pygments`. The option is `True` by default, however, it allows one to turn +off Pygments code highlighting (set to `False`) while preserving the language +detection features of the extension. Note that Pygments language guessing is not +used as that would 'use Pygments'. If a language is defined for a code block, it +will be assigned to the `<code>` tag as a class in the manner suggested by the +[HTML5 spec][spec] (alternate output will not be entertained) and could +potentially be used by a JavaScript library in the browser to highlight the code +block. + +[ch]: ../extensions/code_hilite.md +[spec]: https://www.w3.org/TR/html5/text-level-semantics.html#the-code-element + +### Miscellaneous + +Test coverage has been improved including running [flake8]. While those changes +will not directly effect end users, the code is being better tested which will +benefit everyone. + +[flake8]: https://flake8.readthedocs.io/en/latest/ + +Various bug fixes have been made. See the +[commit log](https://github.com/Python-Markdown/markdown/commits/master) +for a complete history of the changes. diff --git a/docs/change_log/release-3.0.md b/docs/change_log/release-3.0.md new file mode 100644 index 0000000..29cdc4d --- /dev/null +++ b/docs/change_log/release-3.0.md @@ -0,0 +1,228 @@ +title: Release Notes for v3.0 + +# Python-Markdown 3.0 Release Notes + +We are pleased to release Python-Markdown 3.0 which adds a few new features and +fixes various bugs and deprecates various old features. See the list of changes +below for details. + +Python-Markdown version 3.0 supports Python versions 2.7, 3.4, 3.5, 3.6, 3.7, +PyPy and PyPy3. + +## Backwards-incompatible changes + +### `enable_attributes` keyword deprecated + +The `enable_attributes` keyword is deprecated in version 3.0 and will be +ignored. Previously the keyword was `True` by default and enabled an +undocumented way to define attributes on document elements. The feature has been +removed from version 3.0. As most users did not use the undocumented feature, it +should not affect most users. For the few who did use the feature, it can be +enabled by using the [Legacy Attributes](../extensions/legacy_attrs.md) +extension. + +### `smart_emphasis` keyword and `smart_strong` extension deprecated + +The `smart_emphasis` keyword is deprecated in version 3.0 and will be ignored. +Previously the keyword was `True` by default and caused the parser to ignore +middle-word emphasis. Additionally, the optional `smart_strong` extension +provided the same behavior for strong emphasis. Both of those features are now +part of the default behavior, and the [Legacy +Emphasis](../extensions/legacy_em.md) extension is available to disable that +behavior. + +### `output_formats` simplified to `html` and `xhtml`. + +The `output_formats` keyword now only accepts two options: `html` and `xhtml` +Note that if `(x)html1`, `(x)html4` or `(x)html5` are passed in, the number is +stripped and ignored. + +### `safe_mode` and `html_replacement_text` keywords deprecated + +Both `safe_mode` and the associated `html_replacement_text` keywords are +deprecated in version 3.0 and will be ignored. The so-called "safe mode" was +never actually "safe" which has resulted in many people having a false sense of +security when using it. As an alternative, the developers of Python-Markdown +recommend that any untrusted content be passed through an HTML sanitizer (like +[Bleach]) after being converted to HTML by markdown. In fact, [Bleach Whitelist] +provides a curated list of tags, attributes, and styles suitable for filtering +user-provided HTML using bleach. + +If your code previously looked like this: + +```python +html = markdown.markdown(text, safe_mode=True) +``` + +Then it is recommended that you change your code to read something like this: + +```python +import bleach +from bleach_whitelist import markdown_tags, markdown_attrs +html = bleach.clean(markdown.markdown(text), markdown_tags, markdown_attrs) +``` + +If you are not interested in sanitizing untrusted text, but simply desire to +escape raw HTML, then that can be accomplished through an extension which +removes HTML parsing: + +```python +from markdown.extensions import Extension + +class EscapeHtml(Extension): + def extendMarkdown(self, md): + md.preprocessors.deregister('html_block') + md.inlinePatterns.deregister('html') + +html = markdown.markdown(text, extensions=[EscapeHtml()]) +``` + +As the HTML would not be parsed with the above Extension, then the serializer +will escape the raw HTML, which is exactly what happened in previous versions +with `safe_mode="escape"`. + +[Bleach]: https://bleach.readthedocs.io/ +[Bleach Whitelist]: https://github.com/yourcelf/bleach-whitelist + +### Positional arguments deprecated + +Positional arguments on the `markdown.Markdown()` class are deprecated as are +all except the `text` argument on the `markdown.markdown()` wrapper function. +Using positional arguments will raise an error. Only keyword arguments should be +used. For example, if your code previously looked like this: + +```python +html = markdown.markdown(text, [SomeExtension()]) +``` + +Then it is recommended that you change it to read something like this: + +```python +html = markdown.markdown(text, extensions=[SomeExtension()]) +``` + +!!! Note + This change is being made as a result of deprecating `"safe_mode"` as the + `safe_mode` argument was one of the positional arguments. When that argument + is removed, the two arguments following it will no longer be at the correct + position. It is recommended that you always use keywords when they are + supported for this reason. + +### Extension name behavior has changed + +In previous versions of Python-Markdown, the built-in extensions received +special status and did not require the full path to be provided. Additionally, +third party extensions whose name started with `"mdx_"` received the same +special treatment. This is no longer the case. + +Support has been added for extensions to define an [entry +point](../extensions/api.md#entry_point). An entry point is a string name which +can be used to point to an `Extension` class. The built-in extensions now have +entry points which match the old short names. And any third-party extensions +which define entry points can now get the same behavior. See the documentation +for each specific extension to find the assigned name. + +If an extension does not define an entry point, then the full path to the +extension must be used. See the [documentation](../reference.md#extensions) for +a full explanation of the current behavior. + +### Extension configuration as part of extension name deprecated + +The previously documented method of appending the extension configuration +options as a string to the extension name is deprecated and will raise an error. +The [`extension_configs`](../reference.md#extension_configs) keyword should be +used instead. See the [documentation](../reference.md#extension_configs) for a +full explanation of the current behavior. + +### HeaderId extension deprecated + +The HeaderId Extension is deprecated and will raise an error if specified. Use +the [Table of Contents](../extensions/toc.md) Extension instead, which offers +most of the features of the HeaderId Extension and more (support for meta data +is missing). + +Extension authors who have been using the `slugify` and `unique` functions +defined in the HeaderId Extension should note that those functions are now +defined in the Table of Contents extension and should adjust their import +statements accordingly (`from markdown.extensions.toc import slugify, unique`). + +### Homegrown `OrderedDict` has been replaced with a purpose-built `Registry` + +All processors and patterns now get "registered" to a +[Registry](../extensions/api.md#registry). A backwards compatible shim is +included so that existing simple extensions should continue to work. +A `DeprecationWarning` will be raised for any code which calls the old API. + +### Markdown class instance references. + +Previously, instances of the `Markdown` class were represented as any one of +`md`, `md_instance`, or `markdown`. This inconsistency made it difficult when +developing extensions, or just maintaining the existing code. Now, all instances +are consistently represented as `md`. + +The old attributes on class instances still exist, but raise a +`DeprecationWarning` when accessed. Also on classes where the instance was +optional, the attribute always exists now and is simply `None` if no instance +was provided (previously the attribute would not exist). + +### `markdown.util.isBlockLevel` deprecated + +The `markdown.util.isBlockLevel` function is deprecated and will raise a +`DeprecationWarning`. Instead, extensions should use the `isBlockLevel` method +of the `Markdown` class instance. Additionally, a list of block level elements +is defined in the `block_level_elements` attribute of the `Markdown` class which +extensions can access to alter the list of elements which are treated as block +level elements. + +### `md_globals` keyword deprecated from extension API + +Previously, the `extendMarkdown` method of a `markdown.extensions.Extension` +subclasses accepted an `md_globals` keyword, which contained the value returned +by Python's `globals()` built-in function. As all of the configuration is now +held within the `Markdown` class instance, access to the globals is no longer +necessary and any extensions which expect the keyword will raise a +`DeprecationWarning`. A future release will raise an error. + +### `markdown.version` and `markdown.version_info` deprecated + +Historically, version numbers were acquired via the attributes +`markdown.version` and `markdown.version_info`. Moving forward, a more +standardized approach is being followed and versions are acquired via the +`markdown.__version__` and `markdown.__version_info__` attributes. The legacy +attributes are still available to allow distinguishing versions between the +legacy Markdown 2.0 series and the Markdown 3.0 series, but in the future the +legacy attributes will be removed. + +### Added new, more flexible `InlineProcessor` class + +A new `InlineProcessor` class handles inline processing much better and allows +for more flexibility. The new `InlineProcessor` classes no longer utilize +unnecessary pretext and post-text captures. New class can accept the buffer that +is being worked on and manually process the text without regular expressions and +return new replacement bounds. This helps us to handle links in a better way and +handle nested brackets and logic that is too much for regular expression. + +## New features + +The following new features have been included in the release: + +* A new [testing framework](../test_tools.md) is included as a part of the + Markdown library, which can also be used by third party extensions. + +* A new `toc_depth` parameter has been added to the + [Table of Contents Extension](../extensions/toc.md). + +* A new `toc_tokens` attribute has been added to the Markdown class by the + [Table of Contents Extension](../extensions/toc.md), which contains the raw + tokens used to build the Table of Contents. Users can use this to build their + own custom Table of Contents rather than needing to parse the HTML available + on the `toc` attribute of the Markdown class. + +* When the [Table of Contents Extension](../extensions/toc.md) is used in + conjunction with the [Attribute Lists Extension](../extensions/attr_list.md) + and a `data-toc-label` attribute is defined on a header, the content of the + `data-toc-label` attribute is now used as the content of the Table of Contents + item for that header. + +* Additional CSS class names can be appended to + [Admonitions](../extensions/admonition.md). diff --git a/docs/change_log/release-3.1.md b/docs/change_log/release-3.1.md new file mode 100644 index 0000000..b05cd23 --- /dev/null +++ b/docs/change_log/release-3.1.md @@ -0,0 +1,48 @@ +title: Release Notes for v3.1 + +# Python-Markdown 3.1 Release Notes + +Python-Markdown version 3.1 supports Python versions 2.7, 3.5, 3.6, 3.7, +PyPy and PyPy3. + +## Backwards-incompatible changes + +### `markdown.version` and `markdown.version_info` deprecated + +Historically, version numbers were acquired via the attributes +`markdown.version` and `markdown.version_info`. As of 3.0, a more standardized +approach is being followed and versions are acquired via the +`markdown.__version__` and `markdown.__version_info__` attributes. As of 3.1 +the legacy attributes will raise a `DeprecationWarning` if they are accessed. In +a future release the legacy attributes will be removed. + +## New features + +The following new features have been included in the release: + +* A [Contributing Guide](../contributing.md) has been added (#732). + +* A new configuration option to set the footnote separator has been added. Also, + the `rel` and `rev` attributes have been removed from footnotes as they are + not valid in HTML5. The `refs` and `backrefs` classes already exist and + serve the same purpose (#723). + +* A new option for `toc_depth` to set not only the bottom section level, + but also the top section level. A string consisting of two digits + separated by a hyphen in between (`"2-5"`), defines the top (`t`) and the + bottom (`b`) (`<ht>..<hb>`). A single integer still defines the bottom + section level (`<h1>..<hb>`) only. (#787). + +## Bug fixes + +The following bug fixes are included in the 3.1 release: + +* Update CLI to support PyYAML 5.1. +* Overlapping raw HTML matches no longer leave placeholders behind (#458). +* Emphasis patterns now recognize newline characters as whitespace (#783). +* Version format had been updated to be PEP 440 compliant (#736). +* Block level elements are defined per instance, not as class attributes + (#731). +* Double escaping of block code has been eliminated (#725). +* Problems with newlines in references has been fixed (#742). +* Escaped `#` are now handled in header syntax (#762). diff --git a/docs/change_log/release-3.2.md b/docs/change_log/release-3.2.md new file mode 100644 index 0000000..f9452cc --- /dev/null +++ b/docs/change_log/release-3.2.md @@ -0,0 +1,96 @@ +title: Release Notes for v3.2 + +# Python-Markdown 3.2 Release Notes + +Python-Markdown version 3.2 supports Python versions 3.5, 3.6, 3.7, 3.8, and +PyPy3. + +## Backwards-incompatible changes + +### Drop support for Python 2.7 + +Python 2.7 reaches end-of-life on 2020-01-01 and Python-Markdown 3.2 has dropped +support for it. Please upgrade to Python 3, or use Python-Markdown 3.1. + +### `em` and `strong` inline processor changes + +In order to fix issue #792, `em`/`strong` inline processors were refactored. This +translated into removing many of the existing inline processors that handled this +logic: + +* `em_strong` +* `strong` +* `emphasis` +* `strong2` +* `emphasis` + +These processors were replaced with two new ones: + +* `em_strong` +* `em_strong2` + +The [`legacy_em`](../extensions/legacy_em.md) extension was also modified with new, +refactored logic and simply overrides the `em_strong2` inline processor. + +### CodeHilite now always wraps with `<code>` tags + +Before, the HTML generated by CodeHilite looked like: +- `<pre><code>foo = 'bar'</code></pre>` if you **were not** using Pygments. +- `<pre>foo = 'bar'</pre>` if you **were** using Pygments. + +To make the cases more consistent (and adhere to many Markdown specifications and +HTML code block markup suggestions), CodeHilite will now always additionally wrap +code with `<code>` tags. See #862 for more details. + +This change does not alter the Python-Markdown API, but users relying on the old +markup will find their output now changed. + +Internally, this change relies on the Pygments 2.4, so you must be using at least +that version to see this effect. Users with earlier Pygments versions will +continue to see the old behavior. + +### `markdown.util.etree` deprecated + +Previously, Python-Markdown was using either the `xml.etree.cElementTree` module +or the `xml.etree.ElementTree` module, based on their availability. In modern +Python versions, the former is a deprecated alias for the latter. Thus, the +compatibility layer is deprecated and extensions are advised to use +`xml.etree.ElementTree` directly. Importing `markdown.util.etree` will raise +a `DeprecationWarning` beginning in version 3.2 and may be removed in a future +release. + +Therefore, extension developers are encouraged to replace +`from markdown.util import etree` with +`import xml.etree.ElementTree as etree` in their code. + +## New features + +The following new features have been included in the release: + +* Some new configuration options have been added to the [toc](../extensions/toc.md) + extension: + + * The `anchorlink_class` and `permalink_class` options allow class(es) to be + assigned to the `anchorlink` and `permalink` respectively. This allows using + icon fonts from CSS for the links. Therefore, an empty string passed to + `permalink` now generates an empty `permalink`. Previously no `permalink` + would have been generated. (#776) + + * The `permalink_title` option allows the title attribute of a `permalink` to be + set to something other than the default English string `Permanent link`. (#877) + +* Document thread safety (#812). + +* Markdown parsing in HTML has been exposed via a separate extension called + [`md_in_html`](../extensions/md_in_html.md). + +* Add support for Python 3.8. + +## Bug fixes + +The following bug fixes are included in the 3.2 release: + +* HTML tag placeholders are no longer included in `.toc_tokens` (#899). +* Unescape backslash-escaped characters in TOC ids (#864). +* Refactor bold and italic logic in order to solve complex nesting issues (#792). +* Always wrap CodeHilite code in `code` tags (#862). diff --git a/docs/change_log/release-3.3.md b/docs/change_log/release-3.3.md new file mode 100644 index 0000000..79e22b2 --- /dev/null +++ b/docs/change_log/release-3.3.md @@ -0,0 +1,109 @@ +title: Release Notes for v3.3 + +# Python-Markdown 3.3 Release Notes + +Python-Markdown version 3.3 supports Python versions 3.6, 3.7, 3.8, 3.9 and PyPy3. + +## Backwards-incompatible changes + +### The prefix `language-` is now prepended to all language classes by default on code blocks. + +The [HTML5 spec][spec] recommends that the class defining the language of a code block be prefixed with `language-`. +Therefore, by default, both the [fenced_code] and [codehilite] extensions now prepend the prefix when code +highlighting is disabled. + +If you have previously been including the prefix manually in your fenced code blocks, then you will not want a second +instance of the prefix. Similarly, if you are using a third party syntax highlighting tool which does not recognize +the prefix, or requires a different prefix, then you will want to redefine the prefix globally using the `lang_prefix` +configuration option of either the `fenced_code` or `codehilite` extensions. + +For example, to configure `fenced_code` to not apply any prefix (the previous behavior), set the option to an empty string: + +```python +from markdown.extensions.fenced_code import FencedCodeExtension + +markdown.markdown(src, extensions=[FencedCodeExtension(lang_prefix='')]) +``` + +!!! note + When code highlighting is [enabled], the output from Pygments is used unaltered. Currently, Pygments does not + provide an option to include the language class in the output, let alone prefix it. Therefore, any language prefix + is only applied when syntax highlighting is disabled. + +### Attribute Lists are more strict (#898). + +Empty curly braces are now completely ignored by the [Attribute List] extension. Previously, the extension would +recognize them as attribute lists and remove them from the document. Therefore, it is no longer necessary to backslash +escape a set of curly braces which are empty or only contain whitespace. + +Despite not being documented, previously an attribute list could be defined anywhere within a table cell and get +applied to the cell (`<td>` element). Now the attribute list must be defined at the end of the cell content and must +be separated from the rest of the content by at least one space. This makes it easy to differentiate between attribute +lists defined on inline elements within a cell and the attribute list for the cell itself. It is also more consistent +with how attribute lists are defined on other types of elements. + +The extension has also added support for defining attribute lists on table header cells (`<th>` elements) in the same +manner as data cells (`<td>` elements). + +In addition, the documentation for the extensions received an overhaul. The features (#987) and limitations (#965) of the extension are now fully documented. + +## New features + +The following new features have been included in the 3.3 release: + +* All Pygments' options are now available for syntax highlighting (#816). + - The [Codehilite](../extensions/code_hilite.md) extension now accepts any options + which Pygments supports as global configuration settings on the extension. + - [Fenced Code Blocks](../extensions/fenced_code_blocks.md) will accept any of the + same options on individual code blocks. + - Any of the previously supported aliases to Pygments' options continue to be + supported at this time. However, it is recommended that the Pygments option names + be used directly to ensure continued compatibility in the future. + +* [Fenced Code Blocks](../extensions/fenced_code_blocks.md) now work with + [Attribute Lists](../extensions/attr_list.md) when syntax highlighting is disabled. + Any random HTML attribute can be defined and set on the `<code>` tag of fenced code + blocks when the `attr_list` extension is enabled (#816). + +* The HTML parser has been completely replaced. The new HTML parser is built on Python's + [html.parser.HTMLParser](https://docs.python.org/3/library/html.parser.html), which + alleviates various bugs and simplify maintenance of the code (#803, #830). + +* The [Markdown in HTML](../extensions/md_in_html.md) extension has been rebuilt on the + new HTML Parser, which drastically simplifies it. Note that raw HTML elements with a + `markdown` attribute defined are now converted to ElementTree Elements and are rendered + by the serializer. Various bugs have been fixed (#803, #595, #780, and #1012). + +* Link reference parsing, abbreviation reference parsing and footnote reference parsing + has all been moved from `preprocessors` to `blockprocessors`, which allows them to be + nested within other block level elements. Specifically, this change was necessary to + maintain the current behavior in the rebuilt Markdown in HTML extension. A few random + edge-case bugs (see the included tests) were resolved in the process (#803). + +* An alternate function `markdown.extensions.headerid.slugify_unicode` has been included + with the [Table of Contents](../extensions/toc.md) extension which supports Unicode + characters in table of contents slugs. The old `markdown.extensions.headerid.slugify` + method which removes non-ASCII characters remains the default. Import and pass + `markdown.extensions.headerid.slugify_unicode` to the `slugify` configuration option + to use the new behavior. + +* Support was added for Python 3.9 and dropped for Python 3.5. + +## Bug fixes + +The following bug fixes are included in the 3.3 release: + +* Document how to pass configuration options to Extra (#1019). +* Fix HR which follows strong em (#897). +* Support short reference image links (#894). +* Avoid a `RecursionError` from deeply nested blockquotes (#799). +* Fix issues with complex emphasis (#979). +* Fix unescaping of HTML characters `<>` in CodeHilite (#990). +* Fix complex scenarios involving lists and admonitions (#1004). +* Fix complex scenarios with nested ordered and unordered lists in a definition list (#918). + +[spec]: https://www.w3.org/TR/html5/text-level-semantics.html#the-code-element +[fenced_code]: ../extensions/fenced_code_blocks.md +[codehilite]: ../extensions/code_hilite.md +[enabled]: ../extensions/fenced_code_blocks.md#enabling-syntax-highlighting +[Attribute List]: ../extensions/attr_list.md diff --git a/docs/change_log/release-3.4.md b/docs/change_log/release-3.4.md new file mode 100644 index 0000000..bfa8075 --- /dev/null +++ b/docs/change_log/release-3.4.md @@ -0,0 +1,113 @@ +title: Release Notes for v3.4 + +# Python-Markdown 3.4 Release Notes + +Python-Markdown version 3.4 supports Python versions 3.7, 3.8, 3.9, 3.10 and +PyPy3. + +## Backwards-incompatible changes + +### The `tables` extension now uses a `style` attribute instead of an `align` attribute for alignment. + +The [HTML4 spec][spec4] specifically deprecates the use of the `align` attribute +and it does not appear at all in the [HTML5 spec][spec5]. Therefore, by default, +the [tables] extension will now use the `style` attribute (setting just the +`text-align` property) in `td` and `th` blocks. + +[spec4]: https://www.w3.org/TR/html4/present/graphics.html#h-15.1.2 +[spec5]: https://www.w3.org/TR/html53/tabular-data.html#attributes-common-to-td-and-th-elements +[tables]: ../extensions/tables.md + +The former behavior is available by setting the `use_align_attribute` +configuration option to `True` when enabling the extension. + +For example, to configure the old `align` behavior: + +```python +from markdown.extensions.tables import TableExtension + +markdown.markdown(src, extensions=[TableExtension(use_align_attribute=True)]) +``` + +### Backslash unescaping moved to Treeprocessor (#1131). + +Unescaping backslash escapes has been moved to a Treeprocessor, which enables +proper HTML escaping during serialization. However, it is recognized that +various third-party extensions may be calling the old class at +`postprocessors.UnescapePostprocessor`. Therefore, the old class remains in the +code base, but has been deprecated and will be removed in a future release. The +new class `treeprocessors.UnescapeTreeprocessor` should be used instead. + +### Previously deprecated objects have been removed + +Various objects were deprecated in version 3.0 and began raising deprecation +warnings (see the [version 3.0 release notes] for details). Any of those objects +which remained in version 3.3 have been removed from the code base in version 3.4 +and will now raise errors. The relevant objects are listed below. + +[version 3.0 release notes]: release-3.0.md + +| Deprecated Object | Replacement Object | +|----------------------------------------|-------------------------------------| +| `markdown.version` | `markdown.__version__` | +| `markdown.version_info` | `markdown.__version_info__` | +| `markdown.util.etree` | `xml.etree.ElementTree` | +| `markdown.util.string_type` | `str` | +| `markdown.util.text_type` | `str` | +| `markdown.util.int2str` | `chr` | +| `markdown.util.iterrange` | `range` | +| `markdown.util.isBlockLevel` | `markdown.Markdown().is_block_level`| +| `markdown.util.Processor().markdown` | `markdown.util.Processor().md` | +| `markdown.util.Registry().__setitem__` | `markdown.util.Registry().register` | +| `markdown.util.Registry().__delitem__` |`markdown.util.Registry().deregister`| +| `markdown.util.Registry().add` | `markdown.util.Registry().register` | + +In addition, the `md_globals` parameter of +`Markdown.extensions.Extension.extendMarkdown()` is no longer recognized as a +valid parameter and will raise an error if provided. + +## New features + +The following new features have been included in the 3.4 release: + + +* Some new configuration options have been added to the + [footnotes](../extensions/footnotes.md) extension (#1218): + + * Small refactor of the `BACKLINK_TITLE` option; The use of `format()` + instead of "old" `%d` formatter allows one to specify text without the + need to have the number of the footnote in it (like footnotes on + Wikipedia for example). The modification is backward compatible so no + configuration change is required. + + * Addition of a new option `SUPERSCRIPT_TEXT` that allows one to specify a + custom placeholder for the footnote itself in the text. + Ex: `[{}]` will give `<sup>[1]</sup>`, `({})` will give `<sup>(1)</sup>`, + or by default, the current behavior: `<sup>1</sup>`. + +* The [Table of Contents](../extensions/toc.md) extension now accepts a + `toc_class` parameter which can be used to set the CSS class(es) on the + `<div>` that contains the Table of Contents (#1224). + +* The CodeHilite extension now supports a `pygments_formatter` option that can + be set to a custom formatter class (#1187). + + - If `pygments_formatter` is set to a string (ex: `'html'`), Pygments' + default formatter by that name is used. + - If `pygments_formatter` is set to a formatter class (or any callable + which returns a formatter instance), then an instance of that class is + used. + + The formatter class is now passed an additional option, `lang_str`, to + denote the language of the code block (#1258). While Pygments' built-in + formatters will ignore the option, a custom formatter assigned to the + `pygments_formatter` option can make use of the `lang_str` to include the + code block's language in the output. + +## Bug fixes + +The following bug fixes are included in the 3.4 release: + +* Extension entry-points are only loaded if needed (#1216). +* Added additional checks to the `<pre><code>` handling of + `PrettifyTreeprocessor` (#1261, #1263). diff --git a/docs/cli.md b/docs/cli.md new file mode 100644 index 0000000..50e9ec2 --- /dev/null +++ b/docs/cli.md @@ -0,0 +1,189 @@ +title: Command Line + +Using Python-Markdown on the Command Line +========================================= + +While Python-Markdown is primarily a python library, a command line script is +included as well. While there are many other command line implementations +of Markdown, you may not have them installed, or you may prefer to use +Python-Markdown's various extensions. + +Generally, you will want to have the Markdown library fully installed on your +system to run the command line script. See the +[Installation instructions](install.md) for details. + +Python-Markdown's command line script takes advantage of Python's `-m` flag. +Therefore, assuming the python executable is on your system path, use the +following format: + +```bash +python -m markdown [options] [args] +``` + +That will run the module as a script with the options and arguments provided. + +At its most basic usage, one would simply pass in a file name as the only argument: + +```bash +python -m markdown input_file.txt +``` + +Piping input and output (on `STDIN` and `STDOUT`) is fully supported as well. +For example: + +```bash +echo "Some **Markdown** text." | python -m markdown > output.html +``` + +Use the `--help` option for a list all available options and arguments: + +```bash +python -m markdown --help +``` + +If you don't want to call the python executable directly (using the `-m` flag), +follow the instructions below to use a wrapper script: + +Setup +----- + +Upon installation, the `markdown_py` script will have been copied to +your Python "Scripts" directory. Different systems require different methods to +ensure that any files in the Python "Scripts" directory are on your system +path. + +* **Windows**: + + Assuming a default install of Python on Windows, your "Scripts" directory + is most likely something like `C:\\Python37\Scripts`. Verify the location + of your "Scripts" directory and add it to you system path. + + Calling `markdown_py` from the command line will call the wrapper batch + file `markdown_py.bat` in the `"Scripts"` directory created during install. + +* __*nix__ (Linux, OSX, BSD, Unix, etc.): + + As each \*nix distribution is different and we can't possibly document all + of them here, we'll provide a few helpful pointers: + + * Some systems will automatically install the script on your path. Try it + and see if it works. Just run `markdown_py` from the command line. + + * Other systems may maintain a separate "Scripts" ("bin") directory which + you need to add to your path. Find it (check with your distribution) and + either add it to your path or make a symbolic link to it from your path. + + * If you are sure `markdown_py` is on your path, but it still is not being + found, check the permissions of the file and make sure it is executable. + + As an alternative, you could just `cd` into the directory which contains + the source distribution, and run it from there. However, remember that your + markdown text files will not likely be in that directory, so it is much + more convenient to have `markdown_py` on your path. + +!!!Note + Python-Markdown uses `"markdown_py"` as a script name because the Perl + implementation has already taken the more obvious name "markdown". + Additionally, the default Python configuration on some systems would cause a + script named `"markdown.py"` to fail by importing itself rather than the + markdown library. Therefore, the script has been named `"markdown_py"` as a + compromise. If you prefer a different name for the script on your system, it + is suggested that you create a symbolic link to `markdown_py` with your + preferred name. + +Usage +----- + +To use `markdown_py` from the command line, run it as + +```bash +markdown_py input_file.txt +``` + +or + +```bash +markdown_py input_file.txt > output_file.html +``` + +For a complete list of options, run + +```bash +markdown_py --help +``` + +Using Extensions +---------------- + +To load a Python-Markdown extension from the command line use the `-x` +(or `--extension`) option. The extension module must be on your `PYTHONPATH` +(see the [Extension API](extensions/api.md) for details). The extension can +then be invoked by the name assigned to an entry point or using Python's dot +notation to point to an extension + +For example, to load an extension with the assigned entry point name `myext`, +run the following command: + +```bash +python -m markdown -x myext input.txt +``` + +And to load an extension with Python's dot notation: + +```bash +python -m markdown -x path.to.module:MyExtClass input.txt +``` + +To load multiple extensions, specify an `-x` option for each extension: + +```bash +python -m markdown -x myext -x path.to.module:MyExtClass input.txt +``` + +If the extension supports configuration options (see the documentation for the +extension you are using to determine what settings it supports, if any), you +can pass them in as well: + +```bash +python -m markdown -x myext -c config.yml input.txt +``` + +The `-c` (or `--extension_configs`) option accepts a file name. The file must be +in either the [YAML] or [JSON] format and contain YAML or JSON data that would +map to a Python Dictionary in the format required by the +[`extension_configs`][ec] keyword of the `markdown.Markdown` class. Therefore, +the file `config.yaml` referenced in the above example might look like this: + +```yaml +myext: + option1: 'value1' + option2: True +``` + +Similarly, a JSON configuration file might look like this: + +```json +{ + "myext": + { + "option1": "value1", + "option2": "value2" + } +} +``` + +Note that while the `--extension_configs` option does specify the +`myext` extension, you still need to load the extension with the `-x` option, +or the configuration for that extension will be ignored. Further, if an +extension requires a value that cannot be parsed in JSON (for example a +reference to a function), one has to use a YAML configuration file. + +The `--extension_configs` option will only support YAML configuration files if +[PyYAML] is installed on your system. JSON should work with no additional +dependencies. The format of your configuration file is automatically detected. + +[ec]: reference.md#extension_configs +[YAML]: https://yaml.org/ +[JSON]: https://json.org/ +[PyYAML]: https://pyyaml.org/ +[2.5 release notes]: change_log/release-2.5.md diff --git a/docs/command_line.txt b/docs/command_line.txt deleted file mode 100644 index d0134ea..0000000 --- a/docs/command_line.txt +++ /dev/null @@ -1,98 +0,0 @@ -Using Python-Markdown on the Command Line -========================================= - -While Python-Markdown is primarily a python library, a command line script is -included as well. While there are many other command line implementations -of Markdown, you may not have them installed, or you may prefer to use -Python-Markdown's various extensions. - -Setup ------ - -Generally, you may simply call the ``markdown`` file from the command -line. However, if you have fully installed Markdown (``setup.py install`` or -``easy_install``), then the ``markdown`` script will have been copied to -you Python "Scripts" directory. Different systems require different methods to -ensure that any files in the Python "Scripts" directory are on your system -path. - -* **Windows**: - - Assuming a default install on Windows, your "Scripts" directory is most - likely something like ``C:\\Python25\Scripts``. Verify the location of - your "Scripts" directory and add it to you system path. - - Calling ``markdown`` from th ecommand line will call the wrapper batch file - ``markdown.bat`` in the "Scripts" directory created during install. - -* **Linux**: - - As each Linux distribution is different and we can't possibly document all - of them here, we'll provide a few helpful pointers: - - * Some systems will automatically install the script on your path. Try it - and see if it works. Just run ``markdown`` from the command line. - - * Other systems may maintain a separate "Scripts" directory which you - need to add to your path. Find it (check with your distribution) and - either add it to your path or make a symbolic link to it from your path. - - * If you are sure ``markdown`` is on your path, but it still isn't being - found, check the permissions of the file and make sure it is executable. - - As an alternative, you could just ``cd`` into the directory which contains - the source distribution, and run it from there. However, remember that your - markdown text files will not likely be in that directory, so it is much more - convenient to have ``markdown`` on your path. - -The Basics ----------- - -To use ``markdown`` from the command line, run it as - - $ markdown input_file.txt - -or - - $ markdown input_file.txt > output_file.html - -More Options ------------- - -If you are using Python 2.3 or higher, you can also use advanced -command line options to specify encoding or to run extensions. - - $ markdown --help - Usage: markdown INPUTFILE [options] - - Options: - -h, --help show this help message and exit - -f OUTPUT_FILE, --file=OUTPUT_FILE - write output to OUTPUT_FILE - -e ENCODING, --encoding=ENCODING - encoding for input and output files - -q, --quiet suppress all messages - -v, --verbose print info messages - -s SAFE_MODE, --safe=SAFE_MODE - safe mode ('replace', 'remove' or 'escape' user's - HTML tag) - -o OUTPUT_FORMAT, --output_format=OUTPUT_FORMAT - Format of output. One of 'xhtml1' (default) or - 'html4'. - --noisy print debug messages - -x EXTENSION, --extension=EXTENSION - load extension EXTENSION - -Using Extensions ----------------- - -For an extension to be ran this way it must be provided in a module -which should be in your python path (see [[writing_extensions]] for details). -It can then be invoked by the name of that module: - - $ markdown -x footnotes text_with_footnotes.txt > output.html - -If the extension supports config options, you can pass them in as well: - - $ markdown -x "footnotes(PLACE_MARKER=~~~~~~~~)" input.txt - diff --git a/docs/contributing.md b/docs/contributing.md new file mode 100644 index 0000000..5022e45 --- /dev/null +++ b/docs/contributing.md @@ -0,0 +1,561 @@ +# Contributing to Python-Markdown + +The following is a set of guidelines for contributing to Python-Markdown and its +extensions, which are hosted in the [Python-Markdown Organization] on GitHub. +These are mostly guidelines, not rules. Use your best judgment, and feel free to +propose changes to this document in a pull request. + +## Code of Conduct + +This project and everyone participating in it is governed by the +[Python-Markdown Code of Conduct]. By participating, you are expected to uphold +this code. Please report unacceptable behavior to <python.markdown@gmail.com>. + +## Project Organization + +The core Python-Markdown code base and any built-in extensions are hosted in the +[Python-Markdown/markdown] project on GitHub. Other extensions maintained by the +Python-Markdown project may be hosted as separate repositories in the +[Python-Markdown Organization] on GitHub and must follow best practices for +third-party extensions. + +The [Python-Markdown/markdown] project is organized as follows: + +* Branch `master` should generally be stable and release-ready at all times. +* Version branches should be used for bug-fixes back-ported to the most recent + PATCH release. +* No other branches should be created. Any other branches which exist are + preserved for historical reasons only. + +## Issues + +Feature requests, bug reports, usage questions, and other issues can all be +raised on the GitHub [issue tracker]. + +When describing issues try to phrase your ticket in terms of the behavior you +think needs to change rather than the code you think needs to change. + +Make sure you're running the latest version of Python-Markdown before reporting +an issue. + +Search the issue list first for related items. Be sure to check closed issues +and pull requests. GitHub's search only checks open issues by default. + +You may want to check the [syntax rules] and/or [Babelmark] to confirm that your +expectations align with the rules and/or other implementations of Markdown. + +If reporting a syntax bug, you must provide the minimal input which exhibits the +behavior, the actual output and the output you expected. All three items must be +provided as textual code blocks (screen-shots are not helpful). It may also be +helpful to point to the [syntax rules] which specifically address the area of +concern. + +Feature requests will often be closed with a recommendation that they be +implemented as third party extensions outside of the core Python-Markdown +library. Keeping new feature requests implemented as third party extensions +allows us to keep the maintenance overhead of Python-Markdown to a minimum, so +that the focus can be on continued stability, bug fixes, and documentation. + +Closing an issue does not necessarily mean the end of a discussion. If you +believe your issue has been closed incorrectly, explain why and we'll consider +if it needs to be reopened. + +## Pull Requests + +A pull request often represents the start of a discussion, and does not +necessarily need to be the final, finished submission. In fact, if you discover +an issue and intend to provide a fix for it, there is no need to open an issue +first. You can report the issue and provide the fix together in a pull request. + +All pull requests should be made from your personal fork of the library hosted +in your personal GitHub account. Do not create branches on the +[Python-Markdown/markdown] project for pull requests. All pull requests should +be implemented in a new branch with a unique name. Remember that if you have an +outstanding pull request, pushing new commits to the related branch of your +GitHub repository will also automatically update the pull request. It may help +to review GitHub's documentation on [Creating a pull request from a fork]. + +If you are providing a fix for a previously reported issue, you must reference +the issue in your commit message. Be sure to prefix the reference with one of +GitHub's [action words] which will automatically close the issue when the pull +request is merged. For example, `fixes #42` and `closes #42` would be +acceptable, whereas `ref #42` would not. Of course, if merging a pull request +should not cause an issue to be closed, then the action word should not be +included when referencing that issue. + +Before being accepted, each pull request must include the applicable code, new +tests of all new features, updated tests for any changed features, documentation +updates, and an appropriate update to the release notes. All changes must follow +the applicable style guides. Failure to meet any one of the requirements is +likely to delay any serious consideration of your pull request and may even +cause it to be closed. Of course, if you are in the early stages of development, +you may include a note in the pull request acknowledging that it is incomplete +along with a request for feedback. + +Pull requests will generally not be accepted if any tests are failing. +Therefore, it is recommended that you run the tests before submitting your pull +request. After making a pull request, check the build status in the +GitHub interface to ensure that all tests are running as expected. If any checks +fail, you may push additional commits to your branch. GitHub will add those +commits to the pull request and rerun the checks. + +## Style Guides + +In an effort to maintain consistency, Python-Markdown adheres to the following +style guides in its code and documentation. A pull request may be rejected if it +fails to match the relevant style guides. + +### Code Style Guide + +Except as noted below, all pull requests should follow Python's standard [PEP8 +Style Guide] and are run through [Flake8] to ensure that the style guide is +followed. + +Legacy code which does not follow the guidelines should only be updated if and +when other changes (bug fix, feature addition, etc.) are being made to that +section of code. While new features should be given names that follow modern +Python naming conventions, existing names should be preserved to avoid backward +incompatible changes. + +Line length is limited to a maximum of 119 characters. + +When a line of code does not fit within the line length limit, continuation +lines should align elements wrapped inside parentheses, brackets and braces +using a *hanging indent*. When using a hanging indent there should be no +arguments on the first line and further indentation should be used to clearly +distinguish itself as a continuation line. The closing parenthesis, bracket or +brace should be on a line by itself and should line up under the first character +of the line that starts the multi-line construct. + +```python +my_list = [ + 1, 2, 3, + 4, 5, 6, +] +result = some_function_that_takes_arguments( + 'a', 'b', 'c', + 'd', 'e', 'f', +) +``` + +When the conditional part of an `if`-statement is long enough to require that it +be written across multiple lines, extra indentation should be included on the +conditional continuation line. + +```python +if (this_is_one_thing + and that_is_another_thing): + do_something() +``` + +### Documentation Style Guide + +Documentation should be in American English. The tone of the documentation +should be simple, plain, objective and well-balanced where possible. + +Keep paragraphs reasonably short. + +With the exception of code blocks, limit line length to 79 characters. You may +want to use your editor's tools to automatically hard wrap lines of text. + +Don't use abbreviations such as 'e.g.' but instead use the long form, such as +'For example'. + +The documentation is built from the [Markdown] source files in the [`docs` +directory][docs directory] by the [MkDocs] static site generator. In addition to +the basic Markdown syntax, the following extensions are supported: [extra], +[admonition], [smarty], [codehilite], and [toc]. + +There are a few conventions you should follow when working on the +documentation. + +#### Headers + +Headers should use the hash style. For example: + +```md +## Some important topic +``` + +The underline style should not be used. Don't do this: + +```md +Some important topic +==================== +``` + +#### Links + +Links should always use the reference style, with the referenced hyperlinks kept +at the end of the document. + +```md +Here is a link to [some other thing][other-thing]. + +More text... + +[other-thing]: http://example.com/other/thing +``` + +This style helps keep the documentation source consistent and readable. + +If you are linking to another document within Python-Markdown's documentation, +you should use a relative link, and link to the `.md` suffix. If applicable, it +is preferred that the link includes a hash fragment pointing to the specific +section of the page. For example: + +```md +[authentication]: reference.md#Markdown +``` + +Linking in this style ensures that the links work when browsing the +documentation on GitHub. If your Markdown editor makes links clickable, they +will work there as well. When the documentation is built, these links will be +converted into regular links which point to the built HTML pages. + +#### Notes and Warnings + +If you want to draw attention to a note or warning, use the syntax defined in +Python-Markdown's [Admonition Extension]: + +```md +!!! note + + This is the content of the note. +``` + +### Commit Message Style Guide + +Use the present tense ("Add feature" not "Added feature"). + +Use the imperative mood ("Move item to..." not "Moves item to..."). + +Limit the first line to 72 characters or less. + +Reference issues and pull requests liberally after the first line. Include a +summary of the changes/additions made without replicating the content of the +documentation or release notes. This is where an explanation of the choices made +should be found. References to issues and pull requests should only provide the +context in which a choice was made. However, the commit should be able to stand +on its own. + +## Development Environment + +To start developing on Python-Markdown is it best to create a [fork] of the +project on GitHub. After [cloning your fork] to your local system, you will want +to [configure a remote] that points to the upstream repository so that you can +[sync changes] made in the original repository with your fork. + +It is recommended that all development be done from within a Python [virtual +environment], which isolates any experimental code from the general system. To +create a virtual environment, use the following command from the root of the +local working copy of your GitHub fork: + +```sh +virtualenv venv +``` + +That creates a virtual environment which is contained in the `venv` directory +within your local working copy. Note that the repository is configured so that +git will ignore any files within a directory named `venv` or `ENV` for this +very reason. + +On Posix systems (Linux, BSD, MacOS, etc.), use the following command to +activate the environment: + +```sh +source venv/bin/activate +``` + +On Windows, use this command instead: + +```sh +venv/Scripts/activate +``` + +See the [User Guide] for more information on using virtual environments. + +To be able to run the Markdown library directly while working on it, install the +working copy into the environment in [Development Mode] after activating the +virtual environment for the first time: + +```sh +pip install --editable . +``` + +Now any saved changes will immediately be available within the virtual +environment. + +You can run the command line script with the following command: + +```sh +python -m markdown +``` + +And you can directly run the tests with: + +```sh +python -m unittest discover tests +``` + +!!! note + + Some tests require the [PyTidyLib] library, which depends on the [HTML Tidy] + library. If you do not have PyTidyLib installed, the tests which depend upon + it will be skipped. Given the difficulty in installing the HTML Tidy library + on many systems, you may choose to leave both libraries uninstalled and + depend on the Travis server to run those tests when you submit a pull + request. + +The above setup will only run tests against the code in one version of Python. +However, Python-Markdown supports multiple versions of Python. Therefore, a +[tox] configuration is included in the repository, which includes test +environments for all supported Python versions, a [Flake8] test environment, and +a spellchecker for the documentation. While it is generally fine to leave those +tests for the Travis server to run when a pull request is submitted, for more +advanced changes, you may want to run those tests locally. To do so, simply +install tox: + +```sh +pip install tox +``` + +Then, to run all configured test environments, simply call the command `tox` +with no arguments. See help (`tox -h`) for more options. + +!!! note + + The tox environments expect that some dependencies are already installed on + your system. For example, by default, any Python version specific + environment will fail if that version of Python is not installed. + Additionally, the tox environments assume that the [HTML Tidy] library is + installed and may fail when attempting to install [PyTidyLib] if it is not. + Finally, the `spellchecker` environment requires [aspell] and the + `aspell-en` dictionary to be installed. Unfortunately, installing those + dependencies may differ significantly from system to system and is outside + the scope of this guide. + +!!! seealso "See Also" + + Python-Markdown provides [test tools] which simply testing Markdown syntax. + Understanding those tools will often help in understanding why a test may be + failing. + +## Versions + +Python-Markdown follows [Semantic Versioning] and uses the +`MAJOR.MINOR.PATCH[.dev#|a#|b#|rc#]` format for identifying releases. The status +of the `master` branch should always be identified in the `__version_info__` +tuple defined in [`markdown/__meta__.py`][markdown/__meta__.py]. The contents of +that tuple will automatically be converted into a normalized version which +conforms to [PEP 440]. An invalid `__version_info__` tuple will raise an error, +preventing the library from running and the package from building. + +### Version Status + +A MAJOR version is in development status when the MINOR version is `0`, the +PATCH version is `0`, and the version includes a `dev` segment. + +A MINOR version is in development status when the MINOR version is not `0`, the +PATCH version is `0`, and the version includes a `dev` segment. + +At all other times, the code is considered stable and release-ready. + +MAJOR and MINOR releases may or may not get pre-releases (alpha, beta, release +candidate, etc.) at the discretion of the project maintainers. + +### Version Workflow + +Bug fixes may be merged from a pull request to the `master` branch at any time +so long as all tests pass, including one or more new tests which would have +failed prior to the change. + +New features and backward incompatible changes may only be merged to the +`master` branch when the MAJOR and/or MINOR version is in development status +pursuant to [Semantic Versioning]. + +A separate commit to the `master` branch should be made to bump up the MAJOR +and/or MINOR version and set development status. Only then will any pull +requests implementing new features or backward incompatible changes be accepted. + +If a bug fix is deemed to be important and the `master` branch is in development +status, a back-port of the fix should be committed to a version branch. If the +appropriate version branch does not exist, then it should be created and a pull +request back-porting the fix made against that branch. The version branch should +be named with the most recently released MINOR version. For example, if the +`master` branch is at `3.1.dev0` and the most recent MINOR release was `3.0.4`, +then the version branch would be named `3.0` and any releases from that branch +would increment the PATCH version only (`3.0.5`, `3.0.6`...). + +## Release Process + +When a new release is being prepared, the release manager should follow the +following steps: + +1. Verify that all outstanding issues and pull requests related to the release + have been resolved. + +2. Confirm that the release notes and change log have been updated and indicate + the date of the new release. + +3. Update the version defined in [`markdown/__meta__.py`][markdown/__meta__.py]. + +4. Build a local copy of the documentation, browse through the pages and + confirm that no obvious issues exist with the documentation. + +5. Create a pull request with a commit message in the following format: + + Bump version to X.X.X + +6. After all checks have passed, merge the pull request. + +7. Create a git tag with the new version as the tag name and push to the + [Python-Markdown/markdown] repository. The new tag should trigger a GitHub + workflow which will automatically deploy the release to PyPI and update the + documentation. + + In the event that the deployment fails, the following steps can be taken to + deploy manually: + + - Deploy the release to [PyPI] with the command `make deploy`. + + - Deploy an update to the documentation using [MkDocs]. The following example + assumes that local clones of the [Python-Markdown/markdown] and + [Python-Markdown/Python-Markdown.github.io] repositories are in sibling + directories named `markdown` and `Python-Markdown.github.io` respectively. + + cd Python-Markdown.github.io + mkdocs gh-deploy --config-file ../markdown/mkdocs.yml --remote-branch master + +## Issue and Pull Request Labels + +Below are the labels used to track and manages issues and pull requests. The +labels are loosely grouped by their purpose, but it is not necessary for every +issue to have a label from every group, and an issue may have more than one +label from the same group. + +### Type of Issue or Pull Request + +| Label name | Description | +| ---------------------------- | ---------------- | +| `bug`{ .label .bug } | Bug report. | +| `feature`{ .label .feature } | Feature request. | +| `support`{ .label .support } | Support request. | +| `process`{ .label .process } | Discussions regarding policies and development process. | + +### Category of Issue or Pull Request + +| Label name | Description | +| -------------------------------- | ---------------------------------------- | +| `core`{ .label .core } | Related to the core parser code. | +| `extension`{ .label .extension } | Related to one or more of the included extensions. | +| `docs`{ .label .docs } | Related to the project documentation. | + +### Status of Issue + +| Label name | Description | +| --------------------------------------- | --------------------------------- | +| `more-info-needed`{ .label .pending } | More information needs to be provided. | +| `needs-confirmation`{ .label .pending } | The alleged behavior needs to be confirmed. | +| `needs-decision`{ .label .pending } | A decision needs to be made regarding request. | +| `confirmed`{ .label .approved } | Confirmed bug report or approved feature request. | +| `someday-maybe`{ .label .low } | Approved **low priority** request. | +| `duplicate`{ .label .rejected } | The issue has been previously reported. | +| `wontfix`{ .label .rejected } | The issue will not be fixed for the stated reasons. | +| `invalid`{ .label .rejected } | Invalid report (user error, upstream issue, etc). | +| `3rd-party`{ .label .rejected } | Should be implemented as a third party extension. | + + +### Status of Pull Request + +| Label name | Description | +| ------------------------------------- | ----------------------------------- | +| `work-in-progress`{ .label .pending } | A partial solution. More changes will be coming. | +| `needs-review`{ .label .pending } | Needs to be reviewed and/or approved. | +| `requires-changes`{ .label .pending } | Awaiting updates after a review. | +| `approved`{ .label .approved } | The pull request is ready to be merged. | +| `rejected`{ .label .rejected } | The pull request is rejected for the stated reasons. | + +[Python-Markdown Organization]: https://github.com/Python-Markdown +[Python-Markdown Code of Conduct]: https://github.com/Python-Markdown/markdown/blob/master/CODE_OF_CONDUCT.md +[Python-Markdown/markdown]: https://github.com/Python-Markdown/markdown +[issue tracker]: https://github.com/Python-Markdown/markdown/issues +[syntax rules]: https://daringfireball.net/projects/markdown/syntax +[Babelmark]: https://johnmacfarlane.net/babelmark2/ +[Creating a pull request from a fork]: https://help.github.com/articles/creating-a-pull-request-from-a-fork/ +[action words]: https://help.github.com/articles/closing-issues-using-keywords/ +[PEP8 Style Guide]: https://www.python.org/dev/peps/pep-0008/ +[Flake8]: http://flake8.pycqa.org/en/latest/index.html +[Markdown]: https://daringfireball.net/projects/markdown/basics +[docs directory]: https://github.com/Python-Markdown/markdown/tree/master/docs +[MkDocs]: https://www.mkdocs.org/ +[extra]: extensions/extra.md +[admonition]: extensions/admonition.md +[smarty]: extensions/smarty.md +[codehilite]: extensions/code_hilite.md +[toc]: extensions/toc.md +[Admonition Extension]: extensions/admonition.md#syntax +[fork]: https://help.github.com/articles/about-forks +[cloning your fork]: https://help.github.com/articles/cloning-a-repository/ +[configure a remote]: https://help.github.com/articles/configuring-a-remote-for-a-fork +[sync changes]: https://help.github.com/articles/syncing-a-fork +[virtual environment]: https://virtualenv.pypa.io/en/stable/ +[User Guide]: https://virtualenv.pypa.io/en/stable/user_guide.html +[Development Mode]: https://setuptools.readthedocs.io/en/latest/setuptools.html#development-mode +[PyTidyLib]: https://countergram.github.io/pytidylib/ +[HTML Tidy]: https://www.html-tidy.org/ +[tox]: https://tox.readthedocs.io/en/latest/ +[aspell]: http://aspell.net/ +[test tools]: test_tools.md +[Semantic Versioning]: https://semver.org/ +[markdown/__meta__.py]: https://github.com/Python-Markdown/markdown/blob/master/markdown/__meta__.py#L29 +[PEP 440]: https://www.python.org/dev/peps/pep-0440/ +[PyPI]: https://pypi.org/project/Markdown/ +[Python-Markdown/Python-Markdown.github.io]: https://github.com/Python-Markdown/Python-Markdown.github.io + +<style type="text/css"> + /* GitHub Label Styles */ + + code.label { + color: #000000; + font-weight: 600; + line-height: 15px; + display: inline-block; + padding: 4px 6px; + } + code.bug { + background-color: #c45b46; + } + code.feature { + background-color: #7b17d8; + color: #ffffff; + } + code.support { + background-color: #efbe62; + } + code.process { + background-color: #eec9ff; + } + code.core { + background-color: #0b02e1; + color: #ffffff; + } + code.extension { + background-color: #709ad8; + } + code.docs { + background-color: #b2ffeb; + } + code.approved { + background-color: #beed6d; + } + code.low { + background-color: #dddddd; + } + code.pending { + background-color: #f0f49a; + } + code.rejected { + background-color: #f7c7be; + } +</style> diff --git a/docs/extensions/Abbreviations.txt b/docs/extensions/Abbreviations.txt deleted file mode 100644 index fa54d3c..0000000 --- a/docs/extensions/Abbreviations.txt +++ /dev/null @@ -1,53 +0,0 @@ -Abbreviations -------------- - -Summary -------- - -The Markdown Abbreviation Extension adds the ability to define abbreviations. -Specifically, any defined abbreviation is wrapped in an `<abbr>` tag. - -The Abbreviation extension is included in the standard Markdown library. - -Syntax ------- - -Abbreviations are defined using the syntax established in -[PHP Markdown Extra][php]. - -[php]: http://www.michelf.com/projects/php-markdown/extra/#abbr - -Thus, the following text (taken from the above referenced PHP documentation): - - The HTML specification - is maintained by the W3C. - - *[HTML]: Hyper Text Markup Language - *[W3C]: World Wide Web Consortium - -will be rendered like so: - - <p>The <abbr title="Hyper Text Markup Language">HTML</abbr> specification - is maintained by the <abbr title="World Wide Web Consortium">W3C</abbr>.</p> - -Usage ------ - -From the Python interpreter: - - >>> import markdown - >>> text = """ - ... Some text with an ABBR. - ... - ... *[ABBR]: Abbreviation - ... """ - >>> html = markdown.markdown(text, ['abbr']) - -To use with other extensions, just add them to the list, like this: - - >>> html = markdown.markdown(text, ['abbr', 'footnotes']) - -Abbreviations can also be called from the command line using Markdown's `-x` -parameter, like so: - - markdown.py -x abbr source.txt > output.html diff --git a/docs/extensions/CodeHilite.txt b/docs/extensions/CodeHilite.txt deleted file mode 100644 index 482ad60..0000000 --- a/docs/extensions/CodeHilite.txt +++ /dev/null @@ -1,113 +0,0 @@ -CodeHilite -========== - -Summary -------- - -The CodeHilite Extension adds code/syntax highlighting to standard -Python-Markdown code blocks using [Pygments][]. - -[Python-Markdown]: http://www.freewisdom.org/projects/python-markdown/ -[Pygments]: http://pygments.org/ - -This extension is included in the Markdown library. - -Setup ------ - -You will also need to [download][dl] and install the Pygments package on your -`PYTHONPATH`. You will need to determine the appropriate CSS classes and create -appropriate rules for them, which are either defined in or linked from the -header of your HTML templates. See the excellent [documentation][] for more -details. If no language is defined, Pygments will attempt to guess the -language. When that fails, the code block will display as un-highlighted code. - -[dl]: http://pygments.org/download/ -[documentation]: http://pygments.org/docs - -**Note:** The css and/or javascript is not included as part of this extension -but shall always be provided by the end user. - -Syntax ------- - -The CodeHilite Extension follows the same [syntax][] as regular Markdown code -blocks, with one exception. The hiliter needs to know what language to use for -the code block. There are three ways to tell the hiliter what language the code -block contains and each one has a different result. - -[syntax]: http://daringfireball.net/projects/markdown/syntax#precode - -###SheBang (with path) - -If the first line of the codeblock contains a shebang, the language is derived -from that and line numbers are used. - - #!/usr/bin/python - # Code goes here ... - -Will result in: - - #!/usr/bin/python - # Code goes here ... - - -###SheBang (no path) - -If the first line contains a shebang, but the shebang line does not contain a -path (a single `/` or even a space), then that line is removed from the code -block before processing. Line numbers are used. - - #!python - # Code goes here ... - -Will result in: - - # Code goes here ... - -####Colons - -If the first line begins with three or more colons, the text following the -colons identifies the language. The first line is removed from the code block -before processing and line numbers are not used. - - :::python - # Code goes here ... - -Will result in: - - # Code goes here ... - -###When No Language is Defined - -CodeHilite is completely backward compatible so that if a code block is -encountered that does not define a language, the block is simple wrapped in -`<pre>` tags and output. Note: one exception would be that the Pygments -highlighting engine will try to guess the language. Upon failure, the same -behavior will happen as described here. - - # Code goes here ... - -Will result in: - - # Code goes here ... - -Lets see the source for that: - - <div class="codehilite" ><pre><code># Code goes here ... - </code></pre></div> - -Usage ------ - -From the Python interpreter: - - >>> html = markdown.markdown(text, ['codehilite']) - -If you want every code block to have line numbers, even when using colons -(`:::`) for language identification, the setting `force_linenos` is available -to do so. - - >>> html = markdown.markdown(text, - ... ['codehilite(force_linenos=True)'] - ... ) diff --git a/docs/extensions/Definition_Lists.txt b/docs/extensions/Definition_Lists.txt deleted file mode 100644 index 983070d..0000000 --- a/docs/extensions/Definition_Lists.txt +++ /dev/null @@ -1,55 +0,0 @@ -Definition Lists ----------------- - -Summary -------- - -The Definition List Extension adds the ability to create definition list in -Markdown documents. - -This extension is included in the standard Markdown library. - -Syntax ------- - -Definition lists are defined using the syntax established in -[PHP Markdown Extra][php]. - -[php]: http://www.michelf.com/projects/php-markdown/extra/#def-list - -Thus, the following text (taken from the above referenced PHP documentation): - - Apple - : Pomaceous fruit of plants of the genus Malus in - the family Rosaceae. - - Orange - : The fruit of an evergreen tree of the genus Citrus. - -will be rendered like so: - - <dl> - <dt>Apple</dt> - <dd>Pomaceous fruit of plants of the genus Malus in - the family Rosaceae.</dd> - - <dt>Orange</dt> - <dd>The fruit of an evergreen tree of the genus Citrus.</dd> - </dl> - - -Usage ------ - -From the Python interpreter: - - >>> html = markdown.markdown(text, ['def_list']) - -To use with other extensions, just add them to the list, like this: - - >>> html = markdown.markdown(text, ['def_list', 'footnotes']) - -The extension can also be called from the command line using Markdown's `-x` -parameter: - - markdown.py -x def_list source.txt > output.html diff --git a/docs/extensions/Fenced_Code_Blocks.txt b/docs/extensions/Fenced_Code_Blocks.txt deleted file mode 100644 index 6b1ba76..0000000 --- a/docs/extensions/Fenced_Code_Blocks.txt +++ /dev/null @@ -1,63 +0,0 @@ -Fenced Code Blocks -================== - -Summary -------- - -This extension adds a secondary way to define code blocks which overcomes a few -limitations of the indented code blocks. - -This extension is included in the standard Markdown library. - -Syntax ------- - -Fenced Code Blocks are defined using the syntax established in -[PHP Markdown Extra][php]. - -[php]: http://www.michelf.com/projects/php-markdown/extra/#fenced-code-blocks - -Thus, the following text (taken from the above referenced PHP documentation): - - This is a paragraph introducing: - - ~~~~~~~~~~~~~~~~~~~~ - a one-line code block - ~~~~~~~~~~~~~~~~~~~~ - -Fenced code blocks can have a blank line as the first and/or last line of a -code block and they can also come immediately after a list item without becoming -part of the list. - -In addition to PHP Extra's syntax, you can define the language of the code -block for use by syntax highlighters etc. The language will be assigned as a -class attribute of the ``<code>`` element in the output. Therefore, you should -define the language as you would a css class - ``.language``. For consistency -with other markdown syntax, the language can *optionally* be wrapped in curly -brackets: - - ~~~~{.python} - # python code - ~~~~ - - ~~~~.html - <p>HTML Document</p> - ~~~~ - -The above will output: - - <pre><code class="python"># python code - </code></pre> - - <pre><code class="html"><p>HTML Document</p> - </code></pre> - -Usage ------ - -From the Python interpreter: - - >>> html = markdown.markdown(text, ['fenced_code']) - - - diff --git a/docs/extensions/HTML_Tidy.txt b/docs/extensions/HTML_Tidy.txt deleted file mode 100644 index 52f991f..0000000 --- a/docs/extensions/HTML_Tidy.txt +++ /dev/null @@ -1,27 +0,0 @@ -HTML Tidy -========= - -Runs [HTML Tidy][] on the output of Python-Markdown using the [uTidylib][] -Python wrapper. Both libtidy and uTidylib must be installed on your system. - -This extension is available in the standard Markdown library since version 2.0. - -[HTML Tidy]: http://tidy.sourceforge.net/ -[uTidylib]: http://utidylib.berlios.de/ - -Note than any Tidy [options][] can be passed in as extension configs. So, -for example, to output HTML rather than XHTML, set ``output_xhtml=0``. To -indent the output, set ``indent=auto`` and to have Tidy wrap the output in -``<html>`` and ``<body>`` tags, set ``show_body_only=0``. See Tidy's -[options][] for a full list of the available options. The defaults are set to -most closely match Markdowns defaults with the exception that you get much -better pretty-printing. - -[options]: http://tidy.sourceforge.net/docs/quickref.html - -Note that options set in this extension will override most any other settings -passed on to Markdown (such as "output_format"). Unlike Markdown, this extension -will also treat raw HTML no different than that output by Markdown. In other -words, it may munge a document authors carefully crafted HTML. Of course, it -may also transform poorly formed raw HTML into nice, valid HTML. Take these -things into consideration when electing to use this extension. diff --git a/docs/extensions/HeaderId.txt b/docs/extensions/HeaderId.txt deleted file mode 100644 index efd1eb8..0000000 --- a/docs/extensions/HeaderId.txt +++ /dev/null @@ -1,104 +0,0 @@ -HeaderId -======== - -Summary -------- - -An extension to Python-Markdown that adds an 'id' attribute to HTML header -elements (h1-h6) in markdown's output. - -This extension is included in the standard Markdown library. - -Syntax ------- - -The basic syntax follows [PHP Markdown Extra][]'s implementation: - -[PHP Markdown Extra]: http://michelf.com/projects/php-markdown/extra/#header-id - - Header 1 {#header1} - ======== - - ## Header 2 ## {#header2} - -will result in the following HTML: - - <h1 id="header1">Header 1</h1> - - <h2 id="header2">Header 2</h2> - -However, there is much more that this extension does. - -By default, all headers will automatically have unique "id" attributes -generated based upon the text of the header (See below to turn this off). -Note this example in which all three headers would have the same "id": - - #Header - #Another Header {#header} - #Header - -Results in: - - <h1 id="header">Header</h1> - <h1 id="header_1">Another Header</h1> - <h1 id="header_2">Third Header</h1> - -Configuring the Output ----------------------- - -The HeaderId extension has two configuration settings: - -* **level**: Base level for headers. - - Default: `1` - -* **forceid**: Force all headers to have an id. - - Default: `True` - -The `level` setting allows you to automatically adjust the header levels to fit -within the hierarchy of your html templates. For example, the markdown text for -this page should not contain any headers higher than level 3 (`<h3>`). -Therefore, do the following: - - >>> text = ''' - ... #Some Header - ... ## Next Level''' - >>> html = markdown.markdown(text, ['headerid(level=3)']) - >>> print html - <h3 id="some_header">Some Header</h3> - <h4 id="next_level">Next Level</h4>' - -The `forceid` setting turns on or off the automatically generated ids for -headers that do not have one explicitly defined. - - >>> text = ''' - ... # Some Header - ... # Header with ID # { #foo }''' - >>> html = markdown.markdown(text, ['headerid(forceid=False)']) - >>> print html - <h1>Some Header</h1> - <h1 id="foo">Header with ID</h1> - -Using with Meta-Data --------------------- - -The HeaderId Extension also supports the [[Meta-Data]] Extension. Please see the documentation for that extension for specifics. The supported meta-data keywords are: - -* `header_level` -* `header_forceid` - -When used, the meta-data will override the settings provided through the -`extension_configs` interface. - -This document: - - header_level: 2 - header_forceid: Off - - # A Header - - -Will result in the following output: - - <h2>A Header</h2> diff --git a/docs/extensions/ImageLinks.txt b/docs/extensions/ImageLinks.txt deleted file mode 100644 index db4f99f..0000000 --- a/docs/extensions/ImageLinks.txt +++ /dev/null @@ -1,27 +0,0 @@ -ImageLinks -========== - -Summary -------- - -ImageLinks is a Python-Markdown extension that provides a mechanism for -defining mini-photo galleries within a markdown document. - -This extension is part of the Markdown library since 2.0. - -Syntax ------- - -Turns paragraphs like - - <~~~~~~~~~~~~~~~~~~~~~~~~ - dir/subdir - dir/subdir - dir/subdir - ~~~~~~~~~~~~~~ - dir/subdir - dir/subdir - dir/subdir - ~~~~~~~~~~~~~~~~~~~> - -Into mini-photo galleries. diff --git a/docs/extensions/Meta-Data.txt b/docs/extensions/Meta-Data.txt deleted file mode 100644 index 982ea67..0000000 --- a/docs/extensions/Meta-Data.txt +++ /dev/null @@ -1,88 +0,0 @@ -Meta-Data -========= - -Summary -------- - -An extension to Python-Markdown that adds a syntax for defining meta-data about -a document. The Meta-Data extension is inspired by and follows the syntax of -[MultiMarkdown][]. Currently, this extension does not use the meta-data in any -way, but simply provides it as a `Meta` attribute of a markdown instance for -use by other extensions or directly by your python code. - -[MultiMarkdown]: http://fletcherpenney.net/MultiMarkdown_Syntax_Guide#metadata - -This extension has been a part of the Markdown library since 2.0. - -Syntax ------- - -Meta-data consists of a series of keywords and values defined at the beginning -of a markdown document like this: - - Title: My Document - Summary: A brief description of my document. - Authors: Waylan Limberg - John Doe - Date: October 2, 2007 - blank-value: - base_url: http://example.com - - This is the first paragraph of the document. - -The keywords are case-insensitive and may consist of letters, numbers, -underscores and dashes and must end with a colon. The values consist of -anything following the colon on the line and may even be blank. If a line is -indented 4 or more spaces, that line is assumed to be an additional line of the -value for the previous keyword. A keyword may have as many lines as desired. -The first blank line ends all meta-data for the document. Therefore, the first -line of a document must not be blank. All meta-data is stripped from the -document prior to any further processing by markdown. - -Accessing the Meta-Data ------------------------ - -The meta-data is made available as a python Dict in the `Meta` attribute of an -instance of the Markdown class. For example, using the above document: - - >>> md = markdown.Markdown(extensions = ['meta']) - >>> html = md.convert(text) - >>> # Meta-data has been stripped from output - >>> print html - <p>This is the first paragraph of the document.</p> - - >>> # View meta-data - >>> print md.Meta - { - 'title' : ['My Document'], - 'summary' : ['A brief description of my document.'], - 'authors' : ['Waylan Limberg', 'John Doe'], - 'date' : ['October 2, 2007'], - 'blank-value' : [''], - 'base_url' : ['http://example.com'] - } - -Note that the keys are all lowercase and the values consist of a list of -strings where each item is one line for that key. This way, one could preserve -line breaks if desired. Or the items could be joined where appropriate. No -assumptions are made regarding the data. It is simply passed as found to the -`Meta` attribute. - -Perhaps the meta-data could be passed into a template system, or used by -various markdown extensions. The possibilities are left to the imagination of -the developer. - -Compatible Extensions ---------------------- - -The following are extensions currently known to work with the Meta-Data -Extension and the keywords they are known to support: - -* [[HeaderId]] - * `header_level` - * `header_forceid` -* [[WikiLinks]] - * `wiki_base_url` - * `wiki_end_url` - * `wiki_html_class` - diff --git a/docs/extensions/RSS.txt b/docs/extensions/RSS.txt deleted file mode 100644 index f2ecf0c..0000000 --- a/docs/extensions/RSS.txt +++ /dev/null @@ -1,35 +0,0 @@ -RSS -=== - -Summary -------- - -An extension to Python-Markdown that outputs a markdown document as RSS. This -extension has been included with Python-Markdown since 1.7 and should be -available to anyone who has a typical install of Python-Markdown. - -Usage ------ - -From the Python interpreter: - - >>> import markdown - >>> text = "Some markdown document." - >>> rss = markdown.markdown(text, ['rss']) - -Configuring the Output ----------------------- - -An RSS document includes some data about the document (URI, author, title) that -will likely need to be configured for your needs. Therefore, three configuration -options are available: - -* **URL** : The Main URL for the document. -* **CREATOR** : The Feed creator's name. -* **TITLE** : The title for the feed. - -An example: - - >>> rss = markdown.markdown(text, extensions = \ - ... ['rss(URL=http://example.com,CREATOR=JOHN DOE,TITLE=My Document)'] - ... ) diff --git a/docs/extensions/Tables.txt b/docs/extensions/Tables.txt deleted file mode 100644 index 63d6849..0000000 --- a/docs/extensions/Tables.txt +++ /dev/null @@ -1,53 +0,0 @@ -Tables ----------------- - -Summary -------- - -The Table Extension adds the ability to create tables in Markdown documents. - -This extension is included in the standard Markdown library. - -Syntax ------- - -Tables are defined using the syntax established in [PHP Markdown Extra][php]. - -[php]: http://www.michelf.com/projects/php-markdown/extra/#table - -Thus, the following text (taken from the above referenced PHP documentation): - -First Header | Second Header -------------- | ------------- -Content Cell | Content Cell -Content Cell | Content Cell - -will be rendered as: - -<table> -<thead> -<tr> -<th>First Header</th> -<th>Second Header</th> -</tr> -</thead> -<tbody> -<tr> -<td>Content Cell</td> -<td>Content Cell</td> - -</tr> -<tr> -<td>Content Cell</td> -<td>Content Cell</td> -</tr> -</tbody> -</table> - -Usage ------ - -From the Python interpreter: - - >>> html = markdown.markdown(text, ['tables']) - diff --git a/docs/extensions/Tables_of_Contents.txt b/docs/extensions/Tables_of_Contents.txt deleted file mode 100644 index 032c25c..0000000 --- a/docs/extensions/Tables_of_Contents.txt +++ /dev/null @@ -1,50 +0,0 @@ -Table of Contents -================= - -Summary -------- - -Adds a Table of Contents to a Markdown document. - -This extension is included with the Markdown library since version 2.0. - -Syntax ------- - -Place a marker in the document where you would like the table of contents to -appear. Then, a nested list of all the headers in the document will replace the -marker. The marker defaults to ``[TOC]`` so the following document: - - [TOC] - - # Header 1 - - ## Header 2 - -would generate the following output: - - <div class="toc"> - <ul> - <li><a href="#header-1">Header 1</a></li> - <ul> - <li><a href="#header-2">Header 2</a></li> - </ul> - </ul> - </div> - <h1 id="header-1">Header 1</h1> - <h1 id="header-2">Header 2</h1> - -Configuration Options ---------------------- - -The following options are provided to configure the output: - -* **marker**: Text to find and replace with the Table of Contents. Defaults - to ``[TOC]``. -* **slugify**: Callable to generate anchors based on header text. Defaults to a - built in ``slugify`` method. The callable must accept one argument which - contains the text content of the header and return a string which will be - used as the anchor text. -* **title**: Title to insert in TOC ``<div>``. Defaults to ``None``. -* **anchorlink**: Set to ``True`` to have the headers link to themselves. - Default is ``False``. diff --git a/docs/extensions/WikiLinks.txt b/docs/extensions/WikiLinks.txt deleted file mode 100644 index 8bbead5..0000000 --- a/docs/extensions/WikiLinks.txt +++ /dev/null @@ -1,144 +0,0 @@ -WikiLinks -========= - -Summary -------- - -An extension to Python-Markdown that adds [WikiLinks][]. Specifically, any -``[[bracketed]]`` word is converted to a link. - -[WikiLinks]: http://en.wikipedia.org/wiki/Wikilink - -This extension has been included in the Markdown library since 2.0. - -Syntax ------- - -A ``[[bracketed]]`` word is any combination of upper or lower case letters, -number, dashes, underscores and spaces surrounded by double brackets. Therefore - - [[Bracketed]] - -Would produce the following html: - - <a href="/Bracketed/" class="wikilink">Bracketed</a> - -Note that wikilinks are automatically assigned `class="wikilink"` making it -easy to style wikilinks differently from other links on a page if one so -desires. See below for ways to alter the class. - -You should also note that when a space is used, the space is converted to an -underscore in the link but left as-is in the label. Perhaps an example -would illustrate this best: - - [[Wiki Link]] - -Becomes - - <a href="/Wiki_Link/" class="wikilink">Wiki Link</a> - -Usage ------ - -From the Python interpreter: - - >>> text = "Some text with a [[WikiLink]]." - >>> html = markdown.markdown(text, ['wikilink']) - -The default behavior is to point each link to the document root of the current -domain and close with a trailing slash. Additionally, each link is assigned to -the html class `wikilink`. This may not always be desirable. Therefore, one can -customize that behavior within Python code. Three settings are provided to -change the default behavior: - -1. **base_url**: String to append to beginning of URL. - - Default: `'/'` - -2. **end_url**: String to append to end of URL. - - Default: `'/'` - -3. **html_class**: CSS hook. Leave blank for none. - - Default: `'wikilink'` - -4. **build_url**: Callable which formats the URL from it's parts. - -For an example, let us suppose links should always point to the subdirectory -`/wiki/` and end with `.html` - - >>> html = markdown.markdown(text, - ... ['wikilink(base_url=/wiki/,end_url=.html)'] - ... ) - -The above would result in the following link for `[[WikiLink]]`. - - <a href="/wiki/WikiLink.html" class="wikilink">WikiLink</a> - -If you want to do more that just alter the base and/or end of the URL, you -could also pass in a callable which must accept three arguments (``label``, -``base``, and ``end``). The callable must return the URL in it's entirety. - - def my_url_builder(label, base, end): - # do stuff - return url - - md = markdown.Markdown( - extensions=['wikilinks], - extension_configs={'wikilinks' : [('build_url', my_url_builder)]} - ) - - -The option is also provided to change or remove the class attribute. - - >>> html = markdown.markdown(text, - ... ['wikilink(base_url=myclass)'] - ... ) - -Would cause all wikilinks to be assigned to the class `myclass`. - - <a href="/WikiLink/" class="myclass">WikiLink</a> - -The same options can be used on the command line as well: - - python markdown.py -x wikilink(base_url=http://example.com/,end_url=.html,html_class=foo) src.txt - -Some may prefer the more complex format when calling the `Markdown` class directly: - - >>> md = markdown.Markdown( - ... extensions = ['wikilink'], - ... extension_configs = {'wikilink': [ - ... ('base_url', 'http://example.com/'), - ... ('end_url', '.html'), - ... ('html_class', '') ]}, - ... safe_mode = True - ... ) - >>> html = md.convert(text) - -Using with Meta-Data --------------------- - -The WikiLink Extension also supports the [[Meta-Data]] Extension. Please see -the documentation for that extension for specifics. The supported meta-data -keywords are: - -* `wiki_base_url` -* `wiki_end_url` -* `wiki_html_class` - -When used, the meta-data will override the settings provided through the -`extension_configs` interface. - -This document: - - wiki_base_url: http://example.com/ - wiki_end_url: .html - wiki_html_class: - - A [[WikiLink]] in the first paragraph. - -would result in the following output (notice the blank `wiki_html_class`): - - <p>A <a href="http://example.com/WikiLink.html">WikiLink</a> in the first paragraph.</p> - diff --git a/docs/extensions/abbreviations.md b/docs/extensions/abbreviations.md new file mode 100644 index 0000000..d03651f --- /dev/null +++ b/docs/extensions/abbreviations.md @@ -0,0 +1,51 @@ +title: Abbreviations Extension + +Abbreviations +============= + +Summary +------- + +The Abbreviations extension adds the ability to define abbreviations. +Specifically, any defined abbreviation is wrapped in an `<abbr>` tag. + +The Abbreviations extension is included in the standard Markdown library. + +Syntax +------ + +Abbreviations are defined using the syntax established in +[PHP Markdown Extra][php]. + +[php]: http://www.michelf.com/projects/php-markdown/extra/#abbr + +Thus, the following text (taken from the above referenced PHP documentation): + +```md +The HTML specification +is maintained by the W3C. + +*[HTML]: Hyper Text Markup Language +*[W3C]: World Wide Web Consortium +``` + +will be rendered as: + +```html +<p>The <abbr title="Hyper Text Markup Language">HTML</abbr> specification +is maintained by the <abbr title="World Wide Web Consortium">W3C</abbr>.</p> +``` + +Usage +----- + +See [Extensions](index.md) for general extension usage. Use `abbr` as the name +of the extension. + +This extension does not accept any special configuration options. + +A trivial example: + +```python +markdown.markdown(some_text, extensions=['abbr']) +``` diff --git a/docs/extensions/admonition.md b/docs/extensions/admonition.md new file mode 100644 index 0000000..4f79d48 --- /dev/null +++ b/docs/extensions/admonition.md @@ -0,0 +1,115 @@ +title: Admonition Extension + +Admonition +========== + +Summary +------- + +The Admonition extension adds [rST-style][rST] admonitions to Markdown documents. + +This extension is included in the standard Markdown library. + +[rST]: http://docutils.sourceforge.net/docs/ref/rst/directives.html#specific-admonitions + +Syntax +------ + +Admonitions are created using the following syntax: + +```md +!!! type "optional explicit title within double quotes" + Any number of other indented markdown elements. + + This is the second paragraph. +``` + +`type` will be used as the CSS class name and as default title. It must be a +single word. So, for instance: + +```md +!!! note + You should note that the title will be automatically capitalized. +``` + +will render: + +```html +<div class="admonition note"> +<p class="admonition-title">Note</p> +<p>You should note that the title will be automatically capitalized.</p> +</div> +``` + +Optionally, you can use custom titles. For instance: + +```md +!!! danger "Don't try this at home" + ... +``` + +will render: + +```html +<div class="admonition danger"> +<p class="admonition-title">Don't try this at home</p> +<p>...</p> +</div> +``` + +If you don't want a title, use a blank string `""`: + +```md +!!! important "" + This is an admonition box without a title. +``` + +results in: + +```html +<div class="admonition important"> +<p>This is an admonition box without a title.</p> +</div> +``` + +You can also provide additional CSS class names separated by spaces. The first +class should be the "type." For example: + +```md +!!! danger highlight blink "Don't try this at home" + ... +``` + +will render: + +```html +<div class="admonition danger highlight blink"> +<p class="admonition-title">Don't try this at home</p> +<p>...</p> +</div> +``` + +rST suggests the following "types": `attention`, `caution`, `danger`, `error`, +`hint`, `important`, `note`, `tip`, and `warning`; however, you're free to use +whatever you want. + +Styling +------- + +There is no CSS included as part of this extension. Check out the default +[Sphinx][sphinx] theme for inspiration. + +[sphinx]: https://www.sphinx-doc.org/en/master/ + +## Usage + +See [Extensions](index.md) for general extension usage. Use `admonition` as the +name of the extension. + +This extension does not accept any special configuration options. + +A trivial example: + +```python +markdown.markdown(some_text, extensions=['admonition']) +``` diff --git a/docs/extensions/api.md b/docs/extensions/api.md new file mode 100644 index 0000000..d7eb689 --- /dev/null +++ b/docs/extensions/api.md @@ -0,0 +1,886 @@ +title: Extensions API + +# Writing Extensions for Python-Markdown + +Python-Markdown includes an API for extension writers to plug their own custom functionality and syntax into the +parser. An extension will patch into one or more stages of the parser: + +* [*Preprocessors*](#preprocessors) alter the source before it is passed to the parser. +* [*Block Processors*](#blockprocessors) work with blocks of text separated by blank lines. +* [*Tree Processors*](#treeprocessors) modify the constructed ElementTree +* [*Inline Processors*](#inlineprocessors) are common tree processors for inline elements, such as `*strong*`. +* [*Postprocessors*](#postprocessors) munge of the output of the parser just before it is returned. + +The parser loads text, applies the preprocessors, creates and builds an [ElementTree][ElementTree] object from the +block processors and inline processors, renders the ElementTree object as Unicode text, and then then applies the +postprocessors. + +There are classes and helpers provided to ease writing your extension. Each part of the API is discussed in its +respective section below. Additionally, you can walk through the [Tutorial on Writing Extensions][tutorial]; look at +some of the [Available Extensions][] and their [source code][extension source]. As always, you may report bugs, ask +for help, and discuss various other issues on the [bug tracker]. + +## Phases of processing {: #stages } + +### Preprocessors {: #preprocessors } + +Preprocessors munge the source text before it is passed to the Markdown parser. This is an excellent place to clean up +bad characters or to extract portions for later processing that the parser may otherwise choke on. + +Preprocessors inherit from `markdown.preprocessors.Preprocessor` and implement a `run` method, which takes a single +parameter `lines`. This parameter is the entire source text stored as a list of Unicode strings, one per line. `run` +should return its processed list of Unicode strings, one per line. + +#### Example + +This simple example removes any lines with 'NO RENDER' before processing: + +```python +from markdown.preprocessors import Preprocessor +import re + +class NoRender(Preprocessor): + """ Skip any line with words 'NO RENDER' in it. """ + def run(self, lines): + new_lines = [] + for line in lines: + m = re.search("NO RENDER", line) + if not m: + # any line without NO RENDER is passed through + new_lines.append(line) + return new_lines +``` + +#### Usages + +Some preprocessors in the Markdown source tree include: + +| Class | Kind | Description | +| ------------------------------|-----------|------------------------------------------------- | +| [`NormalizeWhiteSpace`][c1] | built-in | Normalizes whitespace by expanding tabs, fixing `\r` line endings, etc. | +| [`HtmlBlockPreprocessor`][c2] | built-in | Removes html blocks from the text and stores them for later processing | +| [`ReferencePreprocessor`][c3] | built-in | Removes reference definitions from text and stores for later processing | +| [`MetaPreprocessor`][c4] | extension | Strips and records meta data at top of documents | +| [`FootnotesPreprocessor`][c5] | extension | Removes footnote blocks from the text and stores them for later processing | + +[c1]: https://github.com/Python-Markdown/markdown/blob/master/markdown/preprocessors.py +[c2]: https://github.com/Python-Markdown/markdown/blob/master/markdown/preprocessors.py +[c3]: https://github.com/Python-Markdown/markdown/blob/master/markdown/preprocessors.py +[c4]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/meta.py +[c5]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/footnotes.py + +### Block Processors {: #blockprocessors } + +A block processor parses blocks of text and adds new elements to the `ElementTree`. Blocks of text, separated from +other text by blank lines, may have a different syntax and produce a differently structured tree than other Markdown. +Block processors excel at code formatting, equation layouts, and tables. + +Block processors inherit from `markdown.blockprocessors.BlockProcessor`, are passed `md.parser` on initialization, and +implement both the `test` and `run` methods: + +* `test(self, parent, block)` takes two parameters: `parent` is the parent `ElementTree` element and `block` is a + single, multi-line, Unicode string of the current block. `test`, often a regular expression match, returns a true + value if the block processor's `run` method should be called to process starting at that block. +* `run(self, parent, blocks)` has the same `parent` parameter as `test`; and `blocks` is the list of all remaining + blocks in the document, starting with the `block` passed to `test`. `run` may return `False` (not `None`) to signal + failure, meaning that it did not process the blocks after all. On success, `run` is expected to `pop` one or more + blocks from the front of `blocks` and attach new nodes to `parent`. + +Crafting block processors is more involved and flexible than the other processors, involving controlling recursive +parsing of the block's contents and managing state across invocations. For example, a blank line is allowed in +indented code, so the second invocation of the inline code processor appends to the element tree generated by the +previous call. Other block processors may insert new text into the `blocks` list, signal to future calls of itself, +and more. + +To make writing these complex beasts more tractable, three convenience functions have been provided by the +`BlockProcessor` parent class: + +* `lastChild(parent)` returns the last child of the given element or `None` if it has no children. +* `detab(text)` removes one level of indent (four spaces by default) from the front of each line of the given + multi-line, text string, until a non-blank line is indented less. +* `looseDetab(text, level)` removes multiple levels + of indent from the front of each line of `text` but does not affect lines indented less. + +Also, `BlockProcessor` provides the fields `self.tab_length`, the tab length (default 4), and `self.parser`, the +current `BlockParser` instance. + +#### BlockParser + +`BlockParser`, not to be confused with `BlockProcessor`, is the class used by Markdown to cycle through all the +registered block processors. You should never need to create your own instance; use `self.parser` instead. + +The `BlockParser` instance provides a stack of strings for its current state, which your processor can push with +`self.parser.set(state)`, pop with `self.parser.reset()`, or check the the top state with +`self.parser.isstate(state)`. Be sure your code pops the states it pushes. + +The `BlockParser` instance can also be called recursively, that is, to process blocks from within your block +processor. There are three methods: + +* `parseDocument(lines)` parses a list of lines, each a single-line Unicode string, returning a complete + `ElementTree`. +* `parseChunk(parent, text)` parses a single, multi-line, possibly multi-block, Unicode string `text` and attaches the + resulting tree to `parent`. +* `parseBlocks(parent, blocks)` takes a list of `blocks`, each a multi-line Unicode string without blank lines, and + attaches the resulting tree to `parent`. + +For perspective, Markdown calls `parseDocument` which calls `parseChunk` which calls `parseBlocks` which calls your +block processor, which, in turn, might call one of these routines. + +#### Example + +This example calls out important paragraphs by giving them a border. It looks for a fence line of exclamation points +before and after and renders the fenced blocks into a new, styled `div`. If it does not find the ending fence line, +it does nothing. + +Our code, like most block processors, is longer than other examples: + +```python +def test_block_processor(): + class BoxBlockProcessor(BlockProcessor): + RE_FENCE_START = r'^ *!{3,} *\n' # start line, e.g., ` !!!! ` + RE_FENCE_END = r'\n *!{3,}\s*$' # last non-blank line, e.g, '!!!\n \n\n' + + def test(self, parent, block): + return re.match(self.RE_FENCE_START, block) + + def run(self, parent, blocks): + original_block = blocks[0] + blocks[0] = re.sub(self.RE_FENCE_START, '', blocks[0]) + + # Find block with ending fence + for block_num, block in enumerate(blocks): + if re.search(self.RE_FENCE_END, block): + # remove fence + blocks[block_num] = re.sub(self.RE_FENCE_END, '', block) + # render fenced area inside a new div + e = etree.SubElement(parent, 'div') + e.set('style', 'display: inline-block; border: 1px solid red;') + self.parser.parseBlocks(e, blocks[0:block_num + 1]) + # remove used blocks + for i in range(0, block_num + 1): + blocks.pop(0) + return True # or could have had no return statement + # No closing marker! Restore and do nothing + blocks[0] = original_block + return False # equivalent to our test() routine returning False + + class BoxExtension(Extension): + def extendMarkdown(self, md): + md.parser.blockprocessors.register(BoxBlockProcessor(md.parser), 'box', 175) +``` + +Start with this example input: + +``` text +A regular paragraph of text. + +!!!!! +First paragraph of wrapped text. + +Second Paragraph of **wrapped** text. +!!!!! + +Another regular paragraph of text. +``` + +The fenced text adds one node with two children to the tree: + +* `div`, with a `style` attribute. It renders as + `<div style="display: inline-block; border: 1px solid red;">...</div>` + * `p` with text `First paragraph of wrapped text.` + * `p` with text `Second Paragraph of **wrapped** text`. The conversion to a `<strong>` tag will happen when + running the inline processors, which will happen after all of the block processors have completed. + +The example output might display as follows: + +!!! note "" + <p>A regular paragraph of text.</p> + <div style="display: inline-block; border: 1px solid red;"> + <p>First paragraph of wrapped text.</p> + <p>Second Paragraph of **wrapped** text.</p> + </div> + <p>Another regular paragraph of text.</p> + +#### Usages + +Some block processors in the Markdown source tree include: + +| Class | Kind | Description | +| ----------------------------|-----------|---------------------------------------------| +| [`HashHeaderProcessor`][b1] | built-in | Title hashes (`#`), which may split blocks | +| [`HRProcessor`][b2] | built-in | Horizontal lines, e.g., `---` | +| [`OListProcessor`][b3] | built-in | Ordered lists; complex and using `state` | +| [`Admonition`][b4] | extension | Render each [Admonition][] in a new `div` | + +[b1]: https://github.com/Python-Markdown/markdown/blob/master/markdown/blockprocessors.py +[b2]: https://github.com/Python-Markdown/markdown/blob/master/markdown/blockprocessors.py +[b3]: https://github.com/Python-Markdown/markdown/blob/master/markdown/blockprocessors.py +[Admonition]: https://python-markdown.github.io/extensions/admonition/ +[b4]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/admonition.py + +### Tree processors {: #treeprocessors } + +Tree processors manipulate the tree created by block processors. They can even create an entirely new ElementTree +object. This is an excellent place for creating summaries, adding collected references, or last minute adjustments. + +A tree processor must inherit from `markdown.treeprocessors.Treeprocessor` (note the capitalization). A tree processor +must implement a `run` method which takes a single argument `root`. In most cases `root` would be an +`xml.etree.ElementTree.Element` instance; however, in rare cases it could be some other type of ElementTree object. +The `run` method may return `None`, in which case the (possibly modified) original `root` object is used, or it may +return an entirely new `Element` object, which will replace the existing `root` object and all of its children. It is +generally preferred to modify `root` in place and return `None`, which avoids creating multiple copies of the entire +document tree in memory. + +For specifics on manipulating the ElementTree, see [Working with the ElementTree][workingwithetree] below. + +#### Example + +A pseudo example: + +```python +from markdown.treeprocessors import Treeprocessor + +class MyTreeprocessor(Treeprocessor): + def run(self, root): + root.text = 'modified content' + # No return statement is same as `return None` +``` + +#### Usages + +The core `InlineProcessor` class is a tree processor. It walks the tree, matches patterns, and splits and creates +nodes on matches. + +Additional tree processors in the Markdown source tree include: + +| Class | Kind | Description | +| ----------------------------------|-----------|---------------------------------------------------------------| +| [`PrettifyTreeprocessor`][e1] | built-in | Add line breaks to the html document | +| [`TocTreeprocessor`][e2] | extension | Builds a [table of contents][] from the finished tree | +| [`FootnoteTreeprocessor`][e3] | extension | Create [footnote][] div at end of document | +| [`FootnotePostTreeprocessor`][e4] | extension | Amend div created by `FootnoteTreeprocessor` with duplicates | + +[e1]: https://github.com/Python-Markdown/markdown/blob/master/markdown/treeprocessors.py +[e2]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/toc.py +[e3]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/footnotes.py +[e4]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/footnotes.py +[table of contents]: https://python-markdown.github.io/extensions/toc/ +[footnote]: https://python-markdown.github.io/extensions/footnotes/ + +### Inline Processors {: #inlineprocessors } + +Inline processors, previously called inline patterns, are used to add formatting, such as `**emphasis**`, by replacing +a matched pattern with a new element tree node. It is an excellent for adding new syntax for inline tags. Inline +processor code is often quite short. + +Inline processors inherit from `InlineProcessor`, are initialized, and implement `handleMatch`: + +* `__init__(self, pattern, md=None)` is the inherited constructor. You do not need to implement your own. + * `pattern` is the regular expression string that must match the code block in order for the `handleMatch` method + to be called. + * `md`, an optional parameter, is a pointer to the instance of `markdown.Markdown` and is available as `self.md` + on the `InlineProcessor` instance. + +* `handleMatch(self, m, data)` must be implemented in all `InlineProcessor` subclasses. + * `m` is the regular expression [match object][] found by the `pattern` passed to `__init__`. + * `data` is a single, multi-line, Unicode string containing the entire block of text around the pattern. A block + is text set apart by blank lines. + * Returns either `(None, None, None)`, indicating the provided match was rejected or `(el, start, end)`, if the + match was successfully processed. On success, `el` is the element being added the tree, `start` and `end` are + indexes in `data` that were "consumed" by the pattern. The "consumed" span will be replaced by a placeholder. + The same inline processor may be called several times on the same block. + +Inline Processors can define the property `ANCESTOR_EXCLUDES` which is either a list or tuple of undesirable ancestors. +The processor will be skipped if it would cause the content to be a descendant of one of the listed tag names. + +##### Convenience Classes + +Convenience subclasses of `InlineProcessor` are provide for common operations: + +* [`SimpleTextInlineProcessor`][i1] returns the text of `group(1)` of the match. +* [`SubstituteTagInlineProcessor`][i4] is initialized as `SubstituteTagInlineProcessor(pattern, tag)`. It returns a + new element `tag` whenever `pattern` is matched. +* [`SimpleTagInlineProcessor`][i3] is initialized as `SimpleTagInlineProcessor(pattern, tag)`. It returns an element + `tag` with a text field of `group(2)` of the match. + +##### Example + +This example changes `--strike--` to `<del>strike</del>`. + +```python +from markdown.inlinepatterns import InlineProcessor +from markdown.extensions import Extension +import xml.etree.ElementTree as etree + + +class DelInlineProcessor(InlineProcessor): + def handleMatch(self, m, data): + el = etree.Element('del') + el.text = m.group(1) + return el, m.start(0), m.end(0) + +class DelExtension(Extension): + def extendMarkdown(self, md): + DEL_PATTERN = r'--(.*?)--' # like --del-- + md.inlinePatterns.register(DelInlineProcessor(DEL_PATTERN, md), 'del', 175) +``` + +Use this input example: + +``` text +First line of the block. +This is --strike one--. +This is --strike two--. +End of the block. +``` + +The example output might display as follows: + +!!! note "" + <p>First line of the block. + This is <del>strike one</del>. + This is <del>strike two</del>. + End of the block.</p> + +* On the first call to `handleMatch` + * `m` will be the match for `--strike one--` + * `data` will be the string: + `First line of the block.\nThis is --strike one--.\nThis is --strike two--.\nEnd of the block.` + + Because the match was successful, the region between the returned `start` and `end` are replaced with a + placeholder token and the new element is added to the tree. + +* On the second call to `handleMatch` + * `m` will be the match for `--strike two--` + * `data` will be the string + `First line of the block.\nThis is klzzwxh:0000.\nThis is --strike two--.\nEnd of the block.` + +Note the placeholder token `klzzwxh:0000`. This allows the regular expression to be run against the entire block, +not just the the text contained in an individual element. The placeholders will later be swapped back out for the +actual elements by the parser. + +Actually it would not be necessary to create the above inline processor. The fact is, that example is not very DRY +(Don't Repeat Yourself). A pattern for `**strong**` text would be almost identical, with the exception that it would +create a `strong` element. Therefore, Markdown provides a number of generic `InlineProcessor` subclasses that can +provide some common functionality. For example, strike could be implemented with an instance of the +`SimpleTagInlineProcessor` class as demonstrated below. Feel free to use or extend any of the `InlineProcessor` +subclasses found at `markdown.inlinepatterns`. + +```python +from markdown.inlinepatterns import SimpleTagInlineProcessor +from markdown.extensions import Extension + +class DelExtension(Extension): + def extendMarkdown(self, md): + md.inlinePatterns.register(SimpleTagInlineProcessor(r'()--(.*?)--', 'del'), 'del', 175) +``` + + +##### Usages + +Here are some convenience functions and other examples: + +| Class | Kind | Description | +| ---------------------------------|-----------|---------------------------------------------------------------| +| [`AsteriskProcessor`][i5] | built-in | Emphasis processor for handling strong and em matches inside asterisks | +| [`AbbrInlineProcessor`][i6] | extension | Apply tag to abbreviation registered by preprocessor | +| [`WikiLinksInlineProcessor`][i7] | extension | Link `[[article names]]` to wiki given in metadata | +| [`FootnoteInlineProcessor`][i8] | extension | Replaces footnote in text with link to footnote div at bottom | + +[i1]: https://github.com/Python-Markdown/markdown/blob/master/markdown/inlinepatterns.py +[i2]: https://github.com/Python-Markdown/markdown/blob/master/markdown/inlinepatterns.py +[i3]: https://github.com/Python-Markdown/markdown/blob/master/markdown/inlinepatterns.py +[i4]: https://github.com/Python-Markdown/markdown/blob/master/markdown/inlinepatterns.py +[i5]: https://github.com/Python-Markdown/markdown/blob/master/markdown/inlinepatterns.py +[i6]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/abbr.py +[i7]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/wikilinks.py +[i8]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/footnotes.py + +### Patterns + +In version 3.0, a new, more flexible inline processor was added, `markdown.inlinepatterns.InlineProcessor`. The +original inline patterns, which inherit from `markdown.inlinepatterns.Pattern` or one of its children are still +supported, though users are encouraged to migrate. + +#### Comparison with new `InlineProcessor` + +The new `InlineProcessor` provides two major enhancements to `Patterns`: + +1. Inline Processors no longer need to match the entire block, so regular expressions no longer need to start with + `r'^(.*?)'` and end with `r'(.*?)%'`. This runs faster. The returned [match object][] will only contain what is + explicitly matched in the pattern, and extension pattern groups now start with `m.group(1)`. + +2. The `handleMatch` method now takes an additional input called `data`, which is the entire block under analysis, + not just what is matched with the specified pattern. The method now returns the element *and* the indexes relative + to `data` that the return element is replacing (usually `m.start(0)` and `m.end(0)`). If the boundaries are + returned as `None`, it is assumed that the match did not take place, and nothing will be altered in `data`. + + This allows handling of more complex constructs than regular expressions can handle, e.g., matching nested + brackets, and explicit control of the span "consumed" by the processor. + +#### Inline Patterns + +Inline Patterns can implement inline HTML element syntax for Markdown such as `*emphasis*` or +`[links](http://example.com)`. Pattern objects should be instances of classes that inherit from +`markdown.inlinepatterns.Pattern` or one of its children. Each pattern object uses a single regular expression and +must have the following methods: + +* **`getCompiledRegExp()`**: + + Returns a compiled regular expression. + +* **`handleMatch(m)`**: + + Accepts a match object and returns an ElementTree element of a plain Unicode string. + +Inline Patterns can define the property `ANCESTOR_EXCLUDES` with is either a list or tuple of undesirable ancestors. +The pattern will be skipped if it would cause the content to be a descendant of one of the listed tag names. + +Note that any regular expression returned by `getCompiledRegExp` must capture the whole block. Therefore, they should +all start with `r'^(.*?)'` and end with `r'(.*?)!'`. When using the default `getCompiledRegExp()` method provided in +the `Pattern` you can pass in a regular expression without that and `getCompiledRegExp` will wrap your expression for +you and set the `re.DOTALL` and `re.UNICODE` flags. This means that the first group of your match will be `m.group(2)` +as `m.group(1)` will match everything before the pattern. + +For an example, consider this simplified emphasis pattern: + +```python +from markdown.inlinepatterns import Pattern +import xml.etree.ElementTree as etree + +class EmphasisPattern(Pattern): + def handleMatch(self, m): + el = etree.Element('em') + el.text = m.group(2) + return el +``` + +As discussed in [Integrating Your Code Into Markdown][], an instance of this class will need to be provided to +Markdown. That instance would be created like so: + +```python +# an oversimplified regex +MYPATTERN = r'\*([^*]+)\*' +# pass in pattern and create instance +emphasis = EmphasisPattern(MYPATTERN) +``` + +### Postprocessors {: #postprocessors } + +Postprocessors munge the document after the ElementTree has been serialized into a string. Postprocessors should be +used to work with the text just before output. Usually, they are used add back sections that were extracted in a +preprocessor, fix up outgoing encodings, or wrap the whole document. + +Postprocessors inherit from `markdown.postprocessors.Postprocessor` and implement a `run` method which takes a single +parameter `text`, the entire HTML document as a single Unicode string. `run` should return a single Unicode string +ready for output. Note that preprocessors use a list of lines while postprocessors use a single multi-line string. + +#### Example + +Here is a simple example that changes the output to one big page showing the raw html. + +```python +from markdown.postprocessors import Postprocessor +import re + +class ShowActualHtmlPostprocesor(Postprocessor): + """ Wrap entire output in <pre> tags as a diagnostic. """ + def run(self, text): + return '<pre>\n' + re.sub('<', '<', text) + '</pre>\n' +``` + +#### Usages + +Some postprocessors in the Markdown source tree include: + +| Class | Kind | Description | +| ------------------------------|-----------|----------------------------------------------------| +| [`raw_html`][p1] | built-in | Restore raw html from `htmlStash`, stored by `HTMLBlockPreprocessor`, and code highlighters | +| [`amp_substitute`][p2] | built-in | Convert ampersand substitutes to `&`; used in links | +| [`unescape`][p3] | built-in | Convert some escaped characters back from integers; used in links | +| [`FootnotePostProcessor`][p4] | extension | Replace footnote placeholders with html entities; as set by other stages | + + [p1]: https://github.com/Python-Markdown/markdown/blob/master/markdown/postprocessors.py + [p2]: https://github.com/Python-Markdown/markdown/blob/master/markdown/postprocessors.py + [p3]: https://github.com/Python-Markdown/markdown/blob/master/markdown/postprocessors.py + [p4]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/footnotes.py + + +## Working with the ElementTree {: #working_with_et } + +As mentioned, the Markdown parser converts a source document to an [ElementTree][ElementTree] object before +serializing that back to Unicode text. Markdown has provided some helpers to ease that manipulation within the context +of the Markdown module. + +First, import the ElementTree module: + +```python +import xml.etree.ElementTree as etree +``` +Sometimes you may want text inserted into an element to be parsed by [Inline Patterns][]. In such a situation, simply +insert the text as you normally would and the text will be automatically run through the Inline Patterns. However, if +you do *not* want some text to be parsed by Inline Patterns, then insert the text as an `AtomicString`. + +```python +from markdown.util import AtomicString +some_element.text = AtomicString(some_text) +``` + +Here's a basic example which creates an HTML table (note that the contents of the second cell (`td2`) will be run +through Inline Patterns latter): + +```python +table = etree.Element("table") +table.set("cellpadding", "2") # Set cellpadding to 2 +tr = etree.SubElement(table, "tr") # Add child tr to table +td1 = etree.SubElement(tr, "td") # Add child td1 to tr +td1.text = markdown.util.AtomicString("Cell content") # Add plain text content +td2 = etree.SubElement(tr, "td") # Add second td to tr +td2.text = "*text* with **inline** formatting." # Add markup text +table.tail = "Text after table" # Add text after table +``` + +You can also manipulate an existing tree. Consider the following example which adds a `class` attribute to `<a>` +elements: + +```python +def set_link_class(self, element): + for child in element: + if child.tag == "a": + child.set("class", "myclass") #set the class attribute + set_link_class(child) # run recursively on children +``` + +For more information about working with ElementTree see the [ElementTree +Documentation][ElementTree]. + +## Working with Raw HTML {: #working_with_raw_html } + +Occasionally an extension may need to call out to a third party library which returns a pre-made string +of raw HTML that needs to be inserted into the document unmodified. Raw strings can be stashed for later +retrieval using an `htmlStash` instance, rather than converting them into `ElementTree` objects. A raw string +(which may or may not be raw HTML) passed to `self.md.htmlStash.store()` will be saved to the stash and a +placeholder string will be returned which should be inserted into the tree instead. After the tree is +serialized, a postprocessor will replace the placeholder with the raw string. This prevents subsequent +processing steps from modifying the HTML data. For example, + +```python +html = "<p>This is some <em>raw</em> HTML data</p>" +el = etree.Element("div") +el.text = self.md.htmlStash.store(html) +``` + +For the global `htmlStash` instance to be available from a processor, the `markdown.Markdown` instance must +be passed to the processor from [extendMarkdown](#extendmarkdown) and will be available as `self.md.htmlStash`. + +## Integrating Your Code Into Markdown {: #integrating_into_markdown } + +Once you have the various pieces of your extension built, you need to tell Markdown about them and ensure that they +are run in the proper sequence. Markdown accepts an `Extension` instance for each extension. Therefore, you will need +to define a class that extends `markdown.extensions.Extension` and over-rides the `extendMarkdown` method. Within this +class you will manage configuration options for your extension and attach the various processors and patterns to the +Markdown instance. + +It is important to note that the order of the various processors and patterns matters. For example, if we replace +`http://...` links with `<a>` elements, and *then* try to deal with inline HTML, we will end up with a mess. +Therefore, the various types of processors and patterns are stored within an instance of the `markdown.Markdown` class +in a [Registry][]. Your `Extension` class will need to manipulate those registries appropriately. You may `register` +instances of your processors and patterns with an appropriate priority, `deregister` built-in instances, or replace a +built-in instance with your own. + +### `extendMarkdown` {: #extendmarkdown } + +The `extendMarkdown` method of a `markdown.extensions.Extension` class accepts one argument: + +* **`md`**: + + A pointer to the instance of the `markdown.Markdown` class. You should use this to access the + [Registries][Registry] of processors and patterns. They are found under the following attributes: + + * `md.preprocessors` + * `md.inlinePatterns` + * `md.parser.blockprocessors` + * `md.treeprocessors` + * `md.postprocessors` + + Some other things you may want to access on the `markdown.Markdown` instance are: + + * `md.htmlStash` + * `md.output_formats` + * `md.set_output_format()` + * `md.output_format` + * `md.serializer` + * `md.registerExtension()` + * `md.tab_length` + * `md.block_level_elements` + * `md.isBlockLevel()` + +!!! Warning + With access to the above items, theoretically you have the option to change anything through various + [monkey_patching][] techniques. However, you should be aware that the various undocumented parts of Markdown may + change without notice and your monkey_patches may break with a new release. Therefore, what you really should be + doing is inserting processors and patterns into the Markdown pipeline. Consider yourself warned! + +[monkey_patching]: https://en.wikipedia.org/wiki/Monkey_patch + +A simple example: + +```python +from markdown.extensions import Extension + +class MyExtension(Extension): + def extendMarkdown(self, md): + # Register instance of 'mypattern' with a priority of 175 + md.inlinePatterns.register(MyPattern(md), 'mypattern', 175) +``` + +### registerExtension {: #registerextension } + +Some extensions may need to have their state reset between multiple runs of the `markdown.Markdown` class. For +example, consider the following use of the [Footnotes][] extension: + +```python +md = markdown.Markdown(extensions=['footnotes']) +html1 = md.convert(text_with_footnote) +md.reset() +html2 = md.convert(text_without_footnote) +``` + +Without calling `reset`, the footnote definitions from the first document will be inserted into the second document as +they are still stored within the class instance. Therefore the `Extension` class needs to define a `reset` method that +will reset the state of the extension (i.e.: `self.footnotes = {}`). However, as many extensions do not have a need +for `reset`, `reset` is only called on extensions that are registered. + +To register an extension, call `md.registerExtension` from within your `extendMarkdown` method: + +```python +def extendMarkdown(self, md): + md.registerExtension(self) + # insert processors and patterns here +``` + +Then, each time `reset` is called on the `markdown.Markdown` instance, the `reset` method of each registered extension +will be called as well. You should also note that `reset` will be called on each registered extension after it is +initialized the first time. Keep that in mind when over-riding the extension's `reset` method. + +### Configuration Settings {: #configsettings } + +If an extension uses any parameters that the user may want to change, those parameters should be stored in +`self.config` of your `markdown.extensions.Extension` class in the following format: + +```python +class MyExtension(markdown.extensions.Extension): + def __init__(self, **kwargs): + self.config = { + 'option1' : ['value1', 'description1'], + 'option2' : ['value2', 'description2'] + } + super(MyExtension, self).__init__(**kwargs) +``` + +When implemented this way the configuration parameters can be over-ridden at run time (thus the call to `super`). For +example: + +```python +markdown.Markdown(extensions=[MyExtension(option1='other value')]) +``` + +Note that if a keyword is passed in that is not already defined in `self.config`, then a `KeyError` is raised. + +The `markdown.extensions.Extension` class and its subclasses have the following methods available to assist in working +with configuration settings: + +* **`getConfig(key [, default])`**: + + Returns the stored value for the given `key` or `default` if the `key` does not exist. If not set, `default` + returns an empty string. + +* **`getConfigs()`**: + + Returns a dict of all key/value pairs. + +* **`getConfigInfo()`**: + + Returns all configuration descriptions as a list of tuples. + +* **`setConfig(key, value)`**: + + Sets a configuration setting for `key` with the given `value`. If `key` is unknown, a `KeyError` is raised. If the + previous value of `key` was a Boolean value, then `value` is converted to a Boolean value. If the previous value + of `key` is `None`, then `value` is converted to a Boolean value except when it is `None`. No conversion takes + place when the previous value of `key` is a string. + +* **`setConfigs(items)`**: + + Sets multiple configuration settings given a dict of key/value pairs. + +### Naming an Extension { #naming_an_extension } + +As noted in the [library reference] an instance of an extension can be passed directly to `markdown.Markdown`. In +fact, this is the preferred way to use third-party extensions. + +For example: + +```python +import markdown +from path.to.module import MyExtension +md = markdown.Markdown(extensions=[MyExtension(option='value')]) +``` + +However, Markdown also accepts "named" third party extensions for those occasions when it is impractical to import an +extension directly (from the command line or from within templates). A "name" can either be a registered [entry +point](#entry_point) or a string using Python's [dot notation](#dot_notation). + +#### Entry Point { #entry_point } + +[Entry points] are defined in a Python package's `setup.py` script. The script must use [setuptools] to support entry +points. Python-Markdown extensions must be assigned to the `markdown.extensions` group. An entry point definition +might look like this: + +```python +from setuptools import setup + +setup( + # ... + entry_points={ + 'markdown.extensions': ['myextension = path.to.module:MyExtension'] + } +) +``` + +After a user installs your extension using the above script, they could then call the extension using the +`myextension` string name like this: + +```python +markdown.markdown(text, extensions=['myextension']) +``` + +Note that if two or more entry points within the same group are assigned the same name, Python-Markdown will only ever +use the first one found and ignore all others. Therefore, be sure to give your extension a unique name. + +For more information on writing `setup.py` scripts, see the Python documentation on [Packaging and Distributing +Projects]. + +#### Dot Notation { #dot_notation } + +If an extension does not have a registered entry point, Python's dot notation may be used instead. The extension must +be installed as a Python module on your PYTHONPATH. Generally, a class should be specified in the name. The class must +be at the end of the name and be separated by a colon from the module. + +Therefore, if you were to import the class like this: + +```python +from path.to.module import MyExtension +``` + +Then the extension can be loaded as follows: + +```python +markdown.markdown(text, extensions=['path.to.module:MyExtension']) +``` + +You do not need to do anything special to support this feature. As long as your extension class is able to be +imported, a user can include it with the above syntax. + +The above two methods are especially useful if you need to implement a large number of extensions with more than one +residing in a module. However, if you do not want to require that your users include the class name in their string, +you must define only one extension per module and that module must contain a module-level function called +`makeExtension` that accepts `**kwargs` and returns an extension instance. + +For example: + +```python +class MyExtension(markdown.extensions.Extension) + # Define extension here... + +def makeExtension(**kwargs): + return MyExtension(**kwargs) +``` + +When `markdown.Markdown` is passed the "name" of your extension as a dot notation string that does not include a class +(for example `path.to.module`), it will import the module and call the `makeExtension` function to initiate your +extension. + +## Registries + +The `markdown.util.Registry` class is a priority sorted registry which Markdown uses internally to determine the +processing order of its various processors and patterns. + +A `Registry` instance provides two public methods to alter the data of the registry: `register` and `deregister`. Use +`register` to add items and `deregister` to remove items. See each method for specifics. + +When registering an item, a "name" and a "priority" must be provided. All items are automatically sorted by the value +of the "priority" parameter such that the item with the highest value will be processed first. The "name" is used to +remove (`deregister`) and get items. + +A `Registry` instance is like a list (which maintains order) when reading data. You may iterate over the items, get an +item and get a count (length) of all items. You may also check that the registry contains an item. + +When getting an item you may use either the index of the item or the string-based "name". For example: + +```python +registry = Registry() +registry.register(SomeItem(), 'itemname', 20) +# Get the item by index +item = registry[0] +# Get the item by name +item = registry['itemname'] +``` + +When checking that the registry contains an item, you may use either the string-based "name", or a reference to the +actual item. For example: + +```python +someitem = SomeItem() +registry.register(someitem, 'itemname', 20) +# Contains the name +assert 'itemname' in registry +# Contains the item instance +assert someitem in registry +``` + +`markdown.util.Registry` has the following methods: + +### `Registry.register(self, item, name, priority)` {: #registry.register data-toc-label='Registry.register'} + +: Add an item to the registry with the given name and priority. + + Parameters: + + * `item`: The item being registered. + * `name`: A string used to reference the item. + * `priority`: An integer or float used to sort against all items. + + If an item is registered with a "name" which already exists, the existing item is replaced with the new item. + Be careful as the old item is lost with no way to recover it. The new item will be sorted according to its + priority and will **not** retain the position of the old item. + +### `Registry.deregister(self, name, strict=True)` {: #registry.deregister data-toc-label='Registry.deregister'} + +: Remove an item from the registry. + + Set `strict=False` to fail silently. + +### `Registry.get_index_for_name(self, name)` {: #registry.get_index_for_name data-toc-label='Registry.get_index_for_name'} + +: Return the index of the given `name`. + +[match object]: https://docs.python.org/3/library/re.html#match-objects +[bug tracker]: https://github.com/Python-Markdown/markdown/issues +[extension source]: https://github.com/Python-Markdown/markdown/tree/master/markdown/extensions +[tutorial]: https://github.com/Python-Markdown/markdown/wiki/Tutorial-1---Writing-Extensions-for-Python-Markdown +[workingwithetree]: #working_with_et +[Integrating your code into Markdown]: #integrating_into_markdown +[extendMarkdown]: #extendmarkdown +[Registry]: #registry +[registerExtension]: #registerextension +[Config Settings]: #configsettings +[makeExtension]: #makeextension +[ElementTree]: https://docs.python.org/3/library/xml.etree.elementtree.html +[Available Extensions]: index.md +[Footnotes]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/footnotes.py +[Definition Lists]: https://github.com/Python-Markdown/markdown/blob/master/markdown/extensions/definition_lists +[library reference]: ../reference.md +[setuptools]: https://packaging.python.org/key_projects/#setuptools +[Entry points]: https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins +[Packaging and Distributing Projects]: https://packaging.python.org/tutorials/distributing-packages/ diff --git a/docs/extensions/attr_list.md b/docs/extensions/attr_list.md new file mode 100644 index 0000000..d3d5e12 --- /dev/null +++ b/docs/extensions/attr_list.md @@ -0,0 +1,202 @@ +title: Attribute Lists Extension + +# Attribute Lists + +## Summary + +The Attribute Lists extension adds a syntax to define attributes on the various +HTML elements in markdown's output. + +This extension is included in the standard Markdown library. + +## Syntax + +The basic syntax was inspired by Maruku's Attribute Lists feature (see [web archive][Maruku]). + +[Maruku]: https://web.archive.org/web/20170324172643/http://maruku.rubyforge.org/proposal.html + +### The List + +An example attribute list might look like this: + +```text +{: #someid .someclass somekey='some value' } +``` + +A word which starts with a hash (`#`) will set the id of an element. + +A word which starts with a dot (`.`) will be added to the list of classes +assigned to an element. + +A key/value pair (`somekey='some value'`) will assign that pair to the element. + +Be aware that while the dot syntax will add to a class, using key/value pairs +will always override the previously defined attribute. Consider the following: + +```text +{: #id1 .class1 id=id2 class="class2 class3" .class4 } +``` + +The above example would result in the following attributes being defined: + +```text +id="id2" class="class2 class3 class4" +``` + +HTML includes support for some attributes to be a single term, like `checked`, for example. Therefore, the attribute +list `{: checked }` would result in `checked` if the [output format](../reference.md#output_format) is `html` or +`checked="checked"` if the output format is `xhtml`. + +Curly braces can be backslash escaped to avoid being identified as an attribute list. + +```text +\{ not an attribute list } +``` + +Opening and closing curly braces which are empty or only contain whitespace are ignored whether they are escaped or +not. Additionally, any attribute lists which are not located in the specific locations documented below are ignored. + +The colon after the opening brace is optional, but is supported to maintain consistency with other implementations. +Therefore, the following is also a valid attribute list: + +```text +{ #someid .someclass somekey='some value' } +``` + +In addition, the spaces after the opening brace and before the closing brace are optional. They are recommended as +they improve readability, but they are not required. + +The Attribute List extension does not have any knowledge of which keys and/or values are valid in HTML. Therefore, it +is up to the document author to ensure that valid keys and values are used. However, the extension will escape any +characters in the key which are not valid by replacing them with an underscore. Multiple consecutive invalid +characters are reduced to a single underscore. + +### Block Level + +To define attributes for a block level element, the attribute list should +be defined on the last line of the block by itself. + +```text +This is a paragraph. +{: #an_id .a_class } +``` + +The above results in the following output: + +```html +<p id="an_id" class="a_class">This is a paragraph.</p> +``` + +An exception is headers, as they are only ever allowed on one line. + +```text +A setext style header {: #setext} +================================= + +### A hash style header ### {: #hash } +``` + +The above results in the following output: + +```html +<h1 id="setext">A setext style header</h1> +<h3 id="hash">A hash style header</h3> +``` + +!!! seealso "See Also" + By default, the [Fenced Code Blocks](./fenced_code_blocks.md#attributes) extension includes limited support for + attribute lists. To get [full support](./fenced_code_blocks.md#keyvalue-pairs), both extensions must be enabled. + +### Inline + +To define attributes on inline elements, the attribute list should be defined +immediately after the inline element with no white space. + +```text +[link](http://example.com){: class="foo bar" title="Some title!" } +``` + +The above results in the following output: + +```html +<p><a href="http://example.com" class="foo bar" title="Some title!">link</a></p> +``` + +If the [tables](./tables.md) extension is enabled, attribute lists can be defined on table cells. To differentiate +attributes for an inline element from attributes for the containing cell, the attribute list must be separated from +the content by at least one space and be defined at the end of the cell content. As table cells can only ever be on +a single line, the attribute list must remain on the same line as the content of the cell. + +```text +| set on td | set on em | +|--------------|-------------| +| *a* { .foo } | *b*{ .foo } | +``` + +The above example results in the following output: + +```html +<table> + <thead> + <tr> + <th>set on td</th> + <th>set on em</th> + </tr> + </thead> + <tbody> + <tr> + <td class="foo"><em>a</em></td> + <td><em class="foo">b</em></td> + </tr> + </tbody> +</table> +``` + +Note that in the first column, the attribute list is preceded by a space; therefore, it is assigned to the table cell +(`<td>` element). However, in the second column, the attribute list is not preceded by a space; therefore, it is +assigned to the inline element (`<em>`) which immediately preceded it. + +Attribute lists may also be defined on table header cells (`<th>` elements) in the same manner. + +### Limitations + +There are a few types of elements which attribute lists do not work with. As a reminder, Markdown is a subset of HTML +and anything which cannot be expressed in Markdown can always be expressed with raw HTML directly. + +__Code Blocks:__ + +: Code blocks are unique in that they must be able to display Markdown syntax. Therefore, there is no way to + determine if an attribute list is intended to be part of the code block or intended to define attributes on the + wrapping element. For that reason, the extension ignores code blocks. To define attributes on code blocks, the + [codehilite] and [fenced code blocks] extensions provide some options. + +[codehilite]: code_hilite.md +[fenced code blocks]: fenced_code_blocks.md + +__Nested Elements:__ + +: Markdown provides mechanisms for nesting various block level elements within other elements. However, attribute + lists only ever get applied to the immediate parent. There is no way to specify that an attribute list should be + applied some number of levels up the document tree. For example, when including an attribute list within a + blockquote, the attribute list is only ever applied to the paragraph the list is defined in. There is no way to + define attributes on the `blockquote` element itself. + +__Implied Elements:__ + +: There are various HTML elements which are not represented in Markdown text, but only implied. For example, the + `ul` and `ol` elements do not exist in Markdown. They are only implied by the presence of list items (`li`). There + is no way to use an attribute list to define attributes on implied elements, including but not limited to the + following: `ul`, `ol`, `dl`, `table`, `thead`, `tbody`, and `tr`. + +## Usage + +See [Extensions](index.md) for general extension usage. Use `attr_list` as the +name of the extension. + +This extension does not accept any special configuration options. + +A trivial example: + +```python +markdown.markdown(some_text, extensions=['attr_list']) +``` diff --git a/docs/extensions/code_hilite.md b/docs/extensions/code_hilite.md new file mode 100644 index 0000000..19b9d52 --- /dev/null +++ b/docs/extensions/code_hilite.md @@ -0,0 +1,308 @@ +title: CodeHilite Extension + +# CodeHilite + +## Summary + +The CodeHilite extension adds code/syntax highlighting to standard +Python-Markdown code blocks using [Pygments][]. + +[Pygments]: http://pygments.org/ + +This extension is included in the standard Markdown library. + +## Setup + +### Step 1: Download and Install Pygments + +You will also need to [download][dl] and install the Pygments package on your +`PYTHONPATH`. The CodeHilite extension will produce HTML output without +Pygments, but it won't highlight anything (same behavior as setting +`use_pygments` to `False`). + +[dl]: http://pygments.org/download/ + +### Step 2: Add CSS Classes + +You will need to define the appropriate CSS classes with appropriate rules. +The CSS rules either need to be defined in or linked from the header of your +HTML templates. Pygments can generate CSS rules for you. Just run the following +command from the command line: + +```bash +pygmentize -S default -f html -a .codehilite > styles.css +``` + +If you are using a different `css_class` (default: `.codehilite`), then +set the value of the `-a` option to that class name. The CSS rules will be +written to the `styles.css` file which you can copy to your site and link from +your HTML templates. + +If you would like to use a different theme, swap out `default` for the desired +theme. For a list of themes installed on your system (additional themes can be +installed via Pygments plugins), run the following command: + +```bash +pygmentize -L style +``` + +See Pygments' excellent [documentation] for more details. If no language is +defined, Pygments will attempt to guess the language. When that fails, the code +block will not be highlighted. + +!!! seealso "See Also" + + GitHub user [richeland] has provided a number of different [CSS style + sheets][rich] which work with Pygments along with a [preview] of each theme. + The `css_class` used is `.highlight`. Therefore, one would need to override the + [`css_class`](#css_class) option when using richeland's CSS styles. However, the + Python-Markdown project makes no guarantee that richeland's CSS styles will + work with the version of Pygments you are using. To ensure complete + compatibility, you should generate the CSS rules from your own installation + of Pygments. + +[richeland]: https://github.com/richleland +[rich]: https://github.com/richleland/pygments-css +[preview]: https://richleland.github.io/pygments-css/ +[documentation]: http://pygments.org/docs/ + +## Syntax + +The CodeHilite extension follows the same [syntax][] as regular Markdown code +blocks, with one exception. The highlighter needs to know what language to use for +the code block. There are three ways to tell the highlighter what language the +code block contains and each one has a different result. + +!!! Note + The format of the language identifier only effects the display of line numbers + if `linenums` is set to `None` (the default). If set to `True` or `False` + (see [Usage](#usage) below) the format of the identifier has no effect on the + display of line numbers -- it only serves as a means to define the language + of the code block. + +[syntax]: https://daringfireball.net/projects/markdown/syntax#precode + +### Shebang (with path) + +If the first line of the code block contains a shebang, the language is derived +from that and line numbers are used. + +```md + #!/usr/bin/python + # Code goes here ... +``` + +Will result in: + + #!/usr/bin/python + # Code goes here ... + +### Shebang (no path) + +If the first line contains a shebang, but the shebang line does not contain a +path (a single `/` or even a space), then that line is removed from the code +block before processing. Line numbers are used. + +```md + #!python + # Code goes here ... +``` + +Will result in: + + #!python + # Code goes here ... + +### Colons + +If the first line begins with three or more colons, the text following the +colons identifies the language. The first line is removed from the code block +before processing and line numbers are not used. + +```md + :::python + # Code goes here ... +``` + +Will result in: + + :::python + # Code goes here ... + +Certain lines can be selected for emphasis with the colon syntax. When +using Pygments' default CSS styles, emphasized lines have a yellow background. +This is useful to direct the reader's attention to specific lines. + +```md + :::python hl_lines="1 3" + # This line is emphasized + # This line isn't + # This line is emphasized +``` + +Will result in: + + :::python hl_lines="1 3" + # This line is emphasized + # This line isn't + # This line is emphasized + +!!! Note + `hl_lines` is named for Pygments' option meaning "highlighted lines". + +### When No Language is Defined + +CodeHilite is completely backwards compatible so that if a code block is +encountered that does not define a language, the block is simply wrapped in +`<pre>` tags and output. + +```md + # Code goes here ... +``` + +Will result in: + + # Code goes here ... + +Lets see the source for that: + +```html +<div class="codehilite"><pre><code># Code goes here ... +</code></pre></div> +``` + +!!! Note + When no language is defined, the Pygments highlighting engine will try to guess + the language (unless `guess_lang` is set to `False`). Upon failure, the same + behavior will happen as described above. + +## Usage + +See [Extensions](index.md) for general extension usage. Use `codehilite` as the +name of the extension. + +See the [Library Reference](../reference.md#extensions) for information about +configuring extensions. + +The following options are provided to configure the output: + +* **`linenums`**{ #linenums }: + An alias to Pygments' `linenos` formatter option. Possible values are `True` for yes, `False` for no and `None` + for auto. Defaults to `None`. + + Using `True` will force every code block to have line numbers, even when + using colons (`:::`) for language identification. + + Using `False` will turn off all line numbers, even when using shebangs + (`#!`) for language identification. + +* **`guess_lang`**{ #guess_lang }: + Automatic language detection. Defaults to `True`. + + Using `False` will prevent Pygments from guessing the language, and thus + highlighting blocks only when you explicitly set the language. + +* **`css_class`**{ #css_class }: + An alias to Pygments `cssclass` formatter option. Set CSS class name for the wrapper `<div>` tag. Defaults to + `codehilite`. + +* **`pygments_style`**{ #pygments_style }: + Pygments HTML Formatter Style (`ColorScheme`). Defaults to `default`. + + !!! Note + This is useful only when `noclasses` is set to `True`, otherwise the + CSS styles must be provided by the end user. + +* **`noclasses`**{ #noclasses }: + Use inline styles instead of CSS classes. Defaults to `False`. + +* **`use_pygments`**{ #use_pygments }: + Specifies the use of Pygments in generating the output. + + If `True` (the default) and Pygments is available, CodeHilite will use + Pygments to analyze and format the output. Additionally, if using Pygments + >= 2.4, the output will be wrapped in `<code>` tags, whereas earlier + versions will not. + + Otherwise, Pygments will not be used. If a language is defined for a code block, it will be assigned to the + `<code>` tag as a class in the manner suggested by the [HTML5 spec][spec] and may be used by a JavaScript library + in the browser to highlight the code block. See the [`lang_prefix`](#lang_prefix) option to customize the prefix. + +* **`lang_prefix`**{ #lang_prefix }: + The prefix prepended to the language class assigned to the HTML `<code>` tag. Default: `language-`. + +* **`pygments_formatter`**{ #pygments_formatter }: + This option can be used to change the Pygments formatter used for highlighting code blocks. By default, this + is set to the string `'html'`, which means it'll use the default `HtmlFormatter` provided by Pygments. + + This can be set to a string representing any of the other default formatters, or set to a formatter class (or + any callable). + + The code's language is always passed to the formatter as an extra option `lang_str`, with the value formatted as + `{lang_prefix}{lang}`. If the language is unspecified, the language guessed by Pygments will be used. While + this option has no effect to the Pygments's builtin formatters, a user can make use of the language in their custom + formatter. See an example below. + + To see what formatters are available and how to subclass an existing formatter, please visit [Pygments + documentation on this topic][pygments formatters]. + +* Any other Pygments' options: + + All other options are accepted and passed on to Pygments' lexer and formatter. Therefore, + valid options include any options which are accepted by the [html formatter] or + whichever [lexer] the code's language uses. Invalid options are ignored without error. + +A trivial example: + +```python +markdown.markdown(some_text, extensions=['codehilite']) +``` + +To keep the code block's language in the Pygments generated HTML output, one can provide a custom Pygments formatter +that takes the `lang_str` option. For example, + +```python +from pygments.formatters import HtmlFormatter +from markdown.extensions.codehilite import CodeHiliteExtension + + +class CustomHtmlFormatter(HtmlFormatter): + def __init__(self, lang_str='', **options): + super().__init__(**options) + # lang_str has the value {lang_prefix}{lang} + # specified by the CodeHilite's options + self.lang_str = lang_str + + def _wrap_code(self, source): + yield 0, f'<code class="{self.lang_str}">' + yield from source + yield 0, '</code>' + + +some_text = '''\ + :::python + print('hellow world') +''' + +markdown.markdown( + some_text, + extensions=[CodeHiliteExtension(pygments_formatter=CustomHtmlFormatter)], +) +``` + +The formatter above will output the following HTML structure for a code block: + +```html +<div class="codehilite"> + <pre> + <code class="language-python"> + ... + </code> + </pre> +</div> +``` + +[html formatter]: https://pygments.org/docs/formatters/#HtmlFormatter +[lexer]: https://pygments.org/docs/lexers/ +[spec]: https://www.w3.org/TR/html5/text-level-semantics.html#the-code-element +[pygments formatters]: https://pygments.org/docs/formatters/ diff --git a/docs/extensions/definition_lists.md b/docs/extensions/definition_lists.md new file mode 100644 index 0000000..047996a --- /dev/null +++ b/docs/extensions/definition_lists.md @@ -0,0 +1,58 @@ +title: Definition Lists Extension + +Definition Lists +================ + +Summary +------- + +The Definition Lists extension adds the ability to create definition lists in +Markdown documents. + +This extension is included in the standard Markdown library. + +Syntax +------ + +Definition lists are defined using the syntax established in +[PHP Markdown Extra][php]. + +[php]: http://www.michelf.com/projects/php-markdown/extra/#def-list + +Thus, the following text (taken from the above referenced PHP documentation): + +```md +Apple +: Pomaceous fruit of plants of the genus Malus in + the family Rosaceae. + +Orange +: The fruit of an evergreen tree of the genus Citrus. +``` + +will be rendered as: + +```html +<dl> +<dt>Apple</dt> +<dd>Pomaceous fruit of plants of the genus Malus in +the family Rosaceae.</dd> + +<dt>Orange</dt> +<dd>The fruit of an evergreen tree of the genus Citrus.</dd> +</dl> +``` + +Usage +----- + +See [Extensions](index.md) for general extension usage. Use `def_list` as the +name of the extension. + +This extension does not accept any special configuration options. + +A trivial example: + +```python +markdown.markdown(some_text, extensions=['def_list']) +``` diff --git a/docs/extensions/extra.md b/docs/extensions/extra.md new file mode 100644 index 0000000..77f5de0 --- /dev/null +++ b/docs/extensions/extra.md @@ -0,0 +1,62 @@ +title: Extra Extension + +# Python-Markdown Extra + +## Summary + +A compilation of various Python-Markdown extensions that (mostly) imitates +[PHP Markdown Extra](http://michelf.com/projects/php-markdown/extra/). + +The supported extensions include: + +* [Abbreviations](abbreviations.md) +* [Attribute Lists](attr_list.md) +* [Definition Lists](definition_lists.md) +* [Fenced Code Blocks](fenced_code_blocks.md) +* [Footnotes](footnotes.md) +* [Tables](tables.md) +* [Markdown in HTML](md_in_html.md) + +See each individual extension for syntax documentation. Extra and all its +supported extensions are included in the standard Markdown library. + +## Usage + +From the Python interpreter: + +```pycon +>>> import markdown +>>> html = markdown.markdown(text, extensions=['extra']) +``` + +To pass configuration options to the extensions included with Extra, they must be passed to Extra, with the +underlying extension identified as well. In that way Extra will have access to the options and can pass them on to +the appropriate underlying extension. + +```python +config = { + 'extra': { + 'footnotes': { + 'UNIQUE_IDS': True + }, + 'fenced_code': { + 'lang_prefix': 'lang-' + } + }, + 'toc': { + 'permalink': True + } +} + +html = markdown.markdown(text, extensions=['extra', 'toc'], extension_configs=config) +``` + +Note that in the above example, `footnotes` and `fenced_code` are both nested under the `extra` key as those +extensions are included with Extra. However, the `toc` extension is not included with `extra` and therefore its +configuration options are not nested under the `extra` key. + +See each individual extension for a list of supported configuration options. + +There are many other [extensions](index.md) which are distributed with Python-Markdown that are not included here in +Extra. The features of those extensions are not part of PHP Markdown Extra, and therefore, not part of Python-Markdown +Extra. diff --git a/docs/extensions/extra.txt b/docs/extensions/extra.txt deleted file mode 100644 index 817d58f..0000000 --- a/docs/extensions/extra.txt +++ /dev/null @@ -1,43 +0,0 @@ -Python-Markdown Extra -===================== - -Summary -------- - -A compilation of various Python-Markdown extensions that (mostly) imitates -[PHP Markdown Extra](http://michelf.com/projects/php-markdown/extra/). - -The supported extensions include: - -* [[Abbreviations]] -* [[Definition_Lists]] -* [[Fenced_Code_Blocks]] -* [[Footnotes]] -* [[HeaderId]] -* [[Tables]] - -See each individual extension for syntax documentation. Extra and all it's -supported extensions are included in the standard Markdown library. - -Usage ------ - -From the Python interpreter: - - >>> import markdown - >>> html = markdown.markdown(text, ['extra']) - -In the unlikely event that one or more of the supported extensions are not -available for import, Markdown will simply continue without that -extension. If you would like to be notified of such failures, -you may set Python-Markdown's logger level to "WARN". - -There may be additional extensions that are distributed with -Python-Markdown that are not included here in Extra. Those extensions -are not part of PHP Markdown Extra, and therefore, not part of -Python-Markdown Extra. If you really would like Extra to include -additional extensions, we suggest creating your own clone of Extra -under a different name (see [[Writing Extensions]]). You could also -edit the `extensions` global variable defined in the source, but be -aware that such changes may be lost when you upgrade to any future -version of Python-Markdown. diff --git a/docs/extensions/fenced_code_blocks.md b/docs/extensions/fenced_code_blocks.md new file mode 100644 index 0000000..fc5c9ca --- /dev/null +++ b/docs/extensions/fenced_code_blocks.md @@ -0,0 +1,269 @@ +title: Fenced Code Blocks Extension + +# Fenced Code Blocks + +## Summary + +The Fenced Code Blocks extension adds a secondary way to define code blocks, which overcomes a few limitations of +indented code blocks. + +This extension is included in the standard Markdown library. + +## Syntax + +Fenced Code Blocks are defined using the syntax originally established in [PHP Markdown Extra][php] and popularized by +[GitHub Flavored Markdown][gfm]. + +Fenced code blocks begin with three or more backticks (` ``` `) or tildes (`~~~`) on a line by themselves and end with +a matching set of backticks or tildes on a line by themselves. The closing set must contain the same number and type +of characters as the opening set. It is recommended that a blank line be placed before and after the code block. + +````md +A paragraph before the code block. + +``` +a one-line code block +``` + +A paragraph after the code block. +```` + +While backticks seem to be more popular among users, tildes may be used as well. + +````md +~~~ +a one-line code block +~~~ +```` + +To include a set of backticks (or tildes) within a code block, use a different number of backticks for the +deliminators. + +`````md +```` +``` +```` +````` + +Fenced code blocks can have a blank line as the first and/or last line of the code block and those lines will be +preserved. + +````md +``` + +a three-line code block + +``` +```` + +Unlike indented code blocks, a fenced code block can immediately follow a list item without becoming +part of the list. + +````md +* A list item. + +``` +not part of the list +``` +```` + +!!! warning + + Fenced Code Blocks are only supported at the document root level. Therefore, they cannot be nested inside lists or + blockquotes. If you need to nest fenced code blocks, you may want to try the third party extension [SuperFences] + instead. + +### Attributes + +Various attributes may be defined on a per-code-block basis by defining them immediately following the opening +deliminator. The attributes should be wrapped in curly braces `{}` and be on the same line as the deliminator. It is +generally best to separate the attribute list from the deliminator with a space. Attributes within the list must be +separated by a space. + +````md +``` { attributes go here } +a code block with attributes +``` +```` + +How those attributes will affect the output will depend on various factors as described below. + +#### Language + +The language of the code within a code block can be specified for use by syntax highlighters, etc. The language should +be prefixed with a dot and not contain any whitespace (`.language-name`). + +````md +``` { .html } +<p>HTML Document</p> +``` +```` + +So long as the language is the only option specified, the curly brackets and/or the dot may be excluded: + +````md +``` html +<p>HTML Document</p> +``` +```` + +Either of the above examples will output the following HTML: + +```html +<pre><code class="language-html"><p>HTML Document</p> +</code></pre> +``` + +Note that the language name has been prefixed with `language-` and it has been assigned to the `class` attribute on +the `<code>` tag, which is the format suggested by the [HTML 5 Specification][html5] (see the second "example" in the +Specification). While `language` is the default prefix, the prefix may be overridden using the +[`lang_prefix`](#lang_prefix) configuration option. + +#### Classes + +In addition to the language, additional classes may be defined by prefixing them with a dot, just like the language. + +````md +``` { .html .foo .bar } +<p>HTML Document</p> +``` +```` + +When defining multiple classes, only the first class will be used as the "language" for the code block. All others are +assigned to the `<pre>` tag unaltered. Additionally, the curly braces and dot are required for all classes, including +the language class if more than one class is defined. + +The above example will output the following HTML: + +```html +<pre class="foo bar"><code class="language-html"><p>HTML Document</p> +</code></pre> +``` + +#### ID + +An `id` can be defined for a code block, which would allow a link to point directly to the code block using a URL +hash. IDs must be prefixed with a hash character (`#`) and only contain characters permitted in HTML `id` attributes. + +````md +``` { #example } +A linkable code block +``` +```` + +The `id` attribute is assigned to the `<pre>` tag of the output. The above example will output the following HTML: + +```html +<pre id="example"><code>A linkable code block +</code></pre> +``` + +From elsewhere within the same document, one could link to the code block with `[link](#example)`. + +IDs may be defined along with the language, other classes, or any other supported attributes. The order of items does +not matter. + +````md +``` { #example .lang .foo .bar } +A linkable code block +``` +```` + +#### Key/Value Pairs + +If the `fenced_code` and [`attr_list`][attr_list] extensions are both enabled, then key/value pairs can be defined in +the attribute list. So long as code highlighting is not enabled (see below), the key/value pairs will be assigned as +attributes on the `<code>` tag in the output. Key/value pairs must be defined using the syntax documented for the +`attr_list` extension (for example, values with whitespace must be wrapped in quotes). + +````md +``` { .lang #example style="color: #333; background: #f8f8f8;" } +A code block with inline styles. Fancy! +``` +```` + +The above example will output the following HTML: + +```html +<pre id="example"><code class="language-lang" style="color: #333; background: #f8f8f8;">A code block with inline styles. Fancy! +</code></pre> +``` + +If the `attr_list` extension is not enabled, then the key/value pairs will be ignored. + +#### Syntax Highlighting + +If the `fenced_code` extension and syntax highlighting are both enabled, then the [`codehilite`][codehilite] +extension will be used for syntax highlighting the contents of the code block. The language defined in the attribute +list will be passed to `codehilite` to ensure that the correct language is used. If no language is specified and +language guessing is not disabled for the `codehilite` extension, then the language will be guessed. + +The `codehilite` extension uses the [Pygments] engine to do syntax highlighting. Any valid Pygments options can be +defined as key/value pairs in the attribute list and will be passed on to Pygments. + +````md +``` { .lang linenos=true linenostart=42 hl_lines="43-44 50" title="An Example Code Block" }` +A truncated code block... +``` +```` + +Valid options include any option accepted by Pygments' [`HTMLFormatter`][HTMLFormatter] except for the `full` option, +as well as any options accepted by the relevant [lexer][lexer] (each language has its own lexer). While most lexers +don't have options that are all that useful in this context, there are a few important exceptions. For example, the +[PHP lexer's] `startinline` option eliminates the need to start each code fragment with `<?php`. + +!!! note + + The `fenced_code` extension does not alter the output provided by Pygments. Therefore, only options which Pygments + provides can be utilized. As Pygments does not currently provide a way to define an ID, any ID defined in an + attribute list will be ignored when syntax highlighting is enabled. Additionally, any key/value pairs which are not Pygments options will be ignored, regardless of whether the `attr_list` extension is enabled. + +##### Enabling Syntax Highlighting + +To enable syntax highlighting, the [`codehilite`][codehilite] extension must be enabled and the `codehilite` +extension's `use_pygments` option must be set to `True` (the default). + +Alternatively, so long as the `codehilite` extension is enabled, you can override a global `use_pygments=False` +option for an individual code block by including `use_pygments=true` in the attribute list. While the `use_pygments` +key/value pair will not be included in the output, all other attributes will behave as they would if syntax +highlighting was enabled only for that code block. + +Conversely, to disable syntax highlighting on an individual code block, include `use_pygments=false` in the attribute +list. While the `use_pygments` key/value pair will not be included in the output, all other attributes will behave as +they would if syntax highlighting was disabled for that code block regardless of any global setting. + +!!! seealso "See Also" + + You will need to properly install and setup Pygments for syntax highlighting to work. See the `codehilite` + extension's [documentation][setup] for details. + +## Usage + +See [Extensions] for general extension usage. Use `fenced_code` as the name of the extension. + +See the [Library Reference] for information about configuring extensions. + +The following option is provided to configure the output: + +* **`lang_prefix`**{#lang_prefix}: + The prefix prepended to the language class assigned to the HTML `<code>` tag. Default: `language-`. + +A trivial example: + +```python +markdown.markdown(some_text, extensions=['fenced_code']) +``` + +[php]: http://www.michelf.com/projects/php-markdown/extra/#fenced-code-blocks +[gfm]: https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks +[SuperFences]: https://facelessuser.github.io/pymdown-extensions/extensions/superfences/ +[html5]: https://html.spec.whatwg.org/#the-code-element +[attr_list]: ./attr_list.md +[codehilite]: ./code_hilite.md +[Pygments]: http://pygments.org/ +[HTMLFormatter]: https://pygments.org/docs/formatters/#HtmlFormatter +[lexer]: https://pygments.org/docs/lexers/ +[PHP lexer's]: https://pygments.org/docs/lexers/#lexers-for-php-and-related-languages +[setup]: ./code_hilite.md#setup +[Extensions]: ./index.md +[Library Reference]: ../reference.md#extensions diff --git a/docs/extensions/footnotes.md b/docs/extensions/footnotes.md new file mode 100644 index 0000000..e841a32 --- /dev/null +++ b/docs/extensions/footnotes.md @@ -0,0 +1,133 @@ +title: Footnotes Extension + +Footnotes +========= + +Summary +------- + +The Footnotes extension adds syntax for defining footnotes in Markdown +documents. + +This extension is included in the standard Markdown library. + +Syntax +------ + +Python-Markdown's Footnote syntax follows the generally accepted syntax of the +Markdown community at large and almost exactly matches [PHP Markdown Extra][]'s +implementation of footnotes. The only differences involve a few subtleties in +the output. + +[PHP Markdown Extra]: http://michelf.com/projects/php-markdown/extra/#footnotes + +Example: + +```md +Footnotes[^1] have a label[^@#$%] and the footnote's content. + +[^1]: This is a footnote content. +[^@#$%]: A footnote on the label: "@#$%". +``` + +A footnote label must start with a caret `^` and may contain any inline text +(including spaces) between a set of square brackets `[]`. Only the first +caret has any special meaning. + +A footnote content must start with the label followed by a colon and at least +one space. The label used to define the content must exactly match the label used +in the body (including capitalization and white space). The content would then +follow the label either on the same line or on the next line. The content may +contain multiple lines, paragraphs, code blocks, blockquotes and most any other +markdown syntax. The additional lines must be indented one level (four spaces or +one tab). + +When working with multiple blocks, it may be helpful to start the content on a +separate line from the label which defines the content. This way the entire block +is indented consistently and any errors are more easily discernible by the author. + +```md +[^1]: + The first paragraph of the definition. + + Paragraph two of the definition. + + > A blockquote with + > multiple lines. + + a code block + + A final paragraph. +``` + +Usage +----- + +See [Extensions](index.md) for general extension usage. Use `footnotes` as the +name of the extension. + +See the [Library Reference](../reference.md#extensions) for information about +configuring extensions. + +The following options are provided to configure the output: + +* **`PLACE_MARKER`**: + A text string used to mark the position where the footnotes are rendered. + Defaults to `///Footnotes Go Here///`. + + If the place marker text is not found in the document, the footnote + definitions are placed at the end of the resulting HTML document. + +* **`UNIQUE_IDS`**: + Whether to avoid collisions across multiple calls to `reset()`. Defaults to + `False`. + +* **`BACKLINK_TEXT`**: + The text string that links from the footnote definition back to the position + in the document. Defaults to `↩`. + +* **`SUPERSCRIPT_TEXT`**: + The text string that links from the position in the document to the footnote + definition. Defaults to `{}`, i.e. only the footnote's number. + +* **`BACKLINK_TITLE`**: + The text string for the `title` HTML attribute of the footnote definition link. + The placeholder `{}` will be replaced by the footnote number. Defaults to + `Jump back to footnote {} in the text`. + +* **`SEPARATOR`**: + The text string used to set the footnote separator. Defaults to `:`. + +A trivial example: + +```python +markdown.markdown(some_text, extensions=['footnotes']) +``` + +Resetting Instance State +----- + +Footnote definitions are stored within the `markdown.Markdown` class instance between +multiple runs of the class. This allows footnotes from all runs to be included in +output, with links and references that are unique, even though the class has been +called multiple times. + +However, if needed, the definitions can be cleared between runs by calling `reset`. + +For instance, the home page of a blog might include the content from multiple documents. +By not calling `reset`, all of the footnotes will be rendered, and they will all have +unique links and references. + +On the other hand, individual blog post pages might need the content from only one +document, and should have footnotes pertaining only to that page. By calling `reset` +between runs, the footnote definitions from the first document will be cleared before +the second document is rendered. + +An example of calling `reset`: + +```python +md = markdown.Markdown(extensions=['footnotes']) +html1 = md.convert(text_with_footnote) +md.reset() +html2 = md.convert(text_without_footnote) +``` diff --git a/docs/extensions/footnotes.txt b/docs/extensions/footnotes.txt deleted file mode 100644 index 7188f44..0000000 --- a/docs/extensions/footnotes.txt +++ /dev/null @@ -1,62 +0,0 @@ -Footnotes -========= - -Summary -------- - -An extension to Python-Markdown that adds footnote syntax. This extension has -been included with Python-Markdown since 1.7 and should be available to anyone -who has a typical install of Python-Markdown. - -Syntax ------- - -Python-Markdown's Footnote syntax follows the generally accepted syntax of the -Markdown community at large and almost exactly matches [PHP Markdown Extra][]'s -implementation of footnotes. The only differences involve a few subtleties in -the output. - -[PHP Markdown Extra]: http://michelf.com/projects/php-markdown/extra/#footnotes - -Example: - - Footnotes[^1] have a label[^label] and a definition[^!DEF]. - - [^1]: This is a footnote - [^label]: A footnote on "label" - [^!DEF]: The definition of a footnote. - -A footnote definition may contain multiple lines, paragraphs, code blocks, -blockquotes and most any other markdown syntax. The additional line simply -must be indented at least an additional four spaces. - - [^1]: The first paragraph of the definition. - - Paragraph two of the definition. - - > A blockquote with - > multiple lines. - - a code block - - A final paragraph. - -By default, the footnote definitions are placed at the end of the resulting -HTML document. However, you may want the footnotes in another location within -the document. Simply place the following text at that location within your -markdown document (See how to configure this text below): - - ///Footnotes Go Here/// - -Usage ------ - -From the Python interpreter: - - >>> html = markdown.markdown(text, ['footnotes']) - -To configure the place marker for footnote definitions (just be sure not to -use any existing markdown syntax): - - >>> html = markdown.markdown(text, ['footnotes(PLACE_MARKER=+++my marker+++)']) - diff --git a/docs/extensions/index.md b/docs/extensions/index.md new file mode 100644 index 0000000..6481caa --- /dev/null +++ b/docs/extensions/index.md @@ -0,0 +1,87 @@ +title: Extensions + +# Extensions + +Python Markdown offers a flexible extension mechanism, which makes it possible +to change and/or extend the behavior of the parser without having to edit the +actual source files. + +To use an extension, pass it to markdown with the `extensions` keyword. + +```python +markdown.markdown(some_text, extensions=[MyExtClass(), 'myext', 'path.to.my.ext:MyExtClass']) +``` + +See the [Library Reference](../reference.md#extensions) for more details. + +From the command line, specify an extension with the `-x` option. + +```bash +python -m markdown -x myext -x path.to.module:MyExtClass input.txt > output.html +``` + +See the [Command Line docs](../cli.md) or use the `--help` option for more details. + +!!! seealso "See Also" + If you would like to write your own extensions, see the + [Extension API](api.md) for details. + +Officially Supported Extensions +------------------------------- + +The extensions listed below are included with (at least) the most recent release +and are officially supported by Python-Markdown. Any documentation is +maintained here and all bug reports should be made to the project. If you +have a typical install of Python-Markdown, these extensions are already +available to you using the "Entry Point" name listed in the second column below. + +Extension | Entry Point | Dot Notation +---------------------------------- | -------------- | ------------ +[Extra] | `extra` | `markdown.extensions.extra` + [Abbreviations] | `abbr` | `markdown.extensions.abbr` + [Attribute Lists] | `attr_list` | `markdown.extensions.attr_list` + [Definition Lists] | `def_list` | `markdown.extensions.def_list` + [Fenced Code Blocks] | `fenced_code` | `markdown.extensions.fenced_code` + [Footnotes] | `footnotes` | `markdown.extensions.footnotes` + [Markdown in HTML] | `md_in_html` | `markdown.extensions.md_in_html` + [Tables] | `tables` | `markdown.extensions.tables` +[Admonition] | `admonition` | `markdown.extensions.admonition` +[CodeHilite] | `codehilite` | `markdown.extensions.codehilite` +[Legacy Attributes] | `legacy_attrs` | `markdown.extensions.legacy_attrs` +[Legacy Emphasis] | `legacy_em` | `markdown.extensions.legacy_em` +[Meta-Data] | `meta` | `markdown.extensions.meta` +[New Line to Break] | `nl2br` | `markdown.extensions.nl2br` +[Sane Lists] | `sane_lists` | `markdown.extensions.sane_lists` +[SmartyPants] | `smarty` | `markdown.extensions.smarty` +[Table of Contents] | `toc` | `markdown.extensions.toc` +[WikiLinks] | `wikilinks` | `markdown.extensions.wikilinks` + +[Extra]: extra.md +[Abbreviations]: abbreviations.md +[Attribute Lists]: attr_list.md +[Definition Lists]: definition_lists.md +[Fenced Code Blocks]: fenced_code_blocks.md +[Footnotes]: footnotes.md +[Tables]: tables.md +[Admonition]: admonition.md +[CodeHilite]: code_hilite.md +[Legacy Attributes]: legacy_attrs.md +[Legacy Emphasis]: legacy_em.md +[Meta-Data]: meta_data.md +[New Line to Break]: nl2br.md +[Markdown in HTML]: md_in_html.md +[Sane Lists]: sane_lists.md +[SmartyPants]: smarty.md +[Table of Contents]: toc.md +[WikiLinks]: wikilinks.md + +Third Party Extensions +---------------------- + +Various individuals and/or organizations have developed extensions which they +have made available to the public. A [list of third party extensions][list] +is maintained on the wiki for your convenience. The Python-Markdown team +offers no official support for these extensions. Please see the developer of +each extension for support. + +[list]: https://github.com/Python-Markdown/markdown/wiki/Third-Party-Extensions diff --git a/docs/extensions/index.txt b/docs/extensions/index.txt deleted file mode 100644 index 71d857c..0000000 --- a/docs/extensions/index.txt +++ /dev/null @@ -1,44 +0,0 @@ -Available Extensions -==================== - -Officially Supported Extensions -------------------------------- - -These extensions are included with (at least) the most recent release and are -officially supported by the Python-Markdown developers. Any documentation is -maintained here and all bug reports should be made to the project. If you -have a typical install of Python-Markdown, these extensions are already -available to you. - -* [[Extra]] - * [[Abbreviations]] - * [[Definition_Lists]] - * [[Fenced_Code_Blocks]] - * [[Footnotes]] - * [[HeaderId]] - * [[Tables]] -* [[CodeHilite]] -* [[HTML_Tidy]] -* [[ImageLinks]] -* [[Meta-Data]] -* [[RSS]] -* [[Table_of_Contents]] -* [[WikiLinks]] - -Unofficially Supported Extensions ---------------------------------- - -These extensions have not yet been included in any official Python-Markdown -release. However, the code is maintained in the projects -[mainline git repository](http://gitorious.org/projects/python-markdown/repos/mainline) -by the Python-Markdown developers and the official documentation is maintained -here. All bug reports should be made to the project. It is anticipated that -these extensions will be included with some future official release, at which -time they will be moved to the above list of official extensions. - -* [[Legacy]] - - - - - diff --git a/docs/extensions/legacy_attrs.md b/docs/extensions/legacy_attrs.md new file mode 100644 index 0000000..daa566d --- /dev/null +++ b/docs/extensions/legacy_attrs.md @@ -0,0 +1,66 @@ +title: Legacy Attributes Extension + +# Legacy Attributes + +## Summary + +The Legacy Attributes extension restores Python-Markdown's original attribute +setting syntax. Older versions of Python Markdown (prior to 3.0) included +built-in and undocumented support for defining attributes on elements. Most +users have never made use of the syntax and it has been deprecated in favor of +[Attribute Lists](attr_list.md). This extension restores the legacy behavior for +users who have existing documents which use the syntax. + +## Syntax + +Attributes are defined by including the following within the element you wish to +assign the attributes to: + +```md +{@key=value} +``` + +For example, to define a class to a paragraph: + +```md +A paragraph with the attribute defined {@class=foo}anywhere within. +``` + +Which results in the following output: + +```html +<p class="foo">A paragraph with the attribute defined anywhere within.</p> +``` + +The same applies for inline elements: + +```md +Some *emphasized{@id=bar}* text. +``` + +```html +<p>Some <em id="bar">emphasized</em> text.</p> +``` + +You can also define attributes in images: + +```md + +``` + +```html +<p><img alt="Alt text" id="baz" src="path/to/image.jpg" /></p> +``` + +## Usage + +See [Extensions](index.md) for general extension usage. Use `legacy_attrs` as the +name of the extension. + +This extension does not accept any special configuration options. + +A trivial example: + +```python +markdown.markdown(some_text, extensions=['legacy_attrs']) +``` diff --git a/docs/extensions/legacy_em.md b/docs/extensions/legacy_em.md new file mode 100644 index 0000000..67c6cd0 --- /dev/null +++ b/docs/extensions/legacy_em.md @@ -0,0 +1,31 @@ +title: Legacy EM Extension + +# Legacy EM + +## Summary + +The Legacy EM extension restores Markdown's original behavior for emphasis and +strong syntax when using underscores. + +By default Python-Markdown treats `_connected_words_` intelligently by +recognizing that mid-word underscores should not be used for emphasis. In other +words, by default, that input would result in this output: +`<em>connected_words</em>`. + +However, that behavior is not consistent with the original rules or the behavior +of the reference implementation. Therefore, this extension can be used to better +match the reference implementation. With the extension enabled, the above input +would result in this output: `<em>connected</em>words_`. + +## Usage + +See [Extensions](index.md) for general extension usage. Use `legacy_em` as the +name of the extension. + +This extension does not accept any special configuration options. + +A trivial example: + +```python +markdown.markdown(some_text, extensions=['legacy_em']) +``` diff --git a/docs/extensions/md_in_html.md b/docs/extensions/md_in_html.md new file mode 100644 index 0000000..978f5c3 --- /dev/null +++ b/docs/extensions/md_in_html.md @@ -0,0 +1,237 @@ +title: Markdown in HTML Extension + +# Markdown in HTML + +## Summary + +An extension that parses Markdown inside of HTML tags. + +## Syntax + +By default, Markdown ignores any content within a raw HTML block-level element. With the `md-in-html` extension +enabled, the content of a raw HTML block-level element can be parsed as Markdown by including a `markdown` attribute +on the opening tag. The `markdown` attribute will be stripped from the output, while all other attributes will be +preserved. + +The `markdown` attribute can be assigned one of three values: [`"1"`](#1), [`"block"`](#block), or [`"span"`](#span). + +!!! note + + The expressions "block-level" and "span-level" as used in this document refer to an element's designation + according to the HTML specification. Whereas the `"span"` and `"block"` values assigned to the `markdown` + attribute refer to the Markdown parser's behavior. + +### `markdown="1"` { #1 } + +When the `markdown` attribute is set to `"1"`, then the parser will use the default behavior for that specific tag. + +The following tags have the `block` behavior by default: `article`, `aside`, `blockquote`, `body`, `colgroup`, +`details`, `div`, `dl`, `fieldset`, `figcaption`, `figure`, `footer`, `form`, `group`, `header`, `hgroup`, `hr`, +`iframe`, `main`, `map`, `menu`, `nav`, `noscript`, `object`, `ol`, `output`, `progress`, `section`, `table`, +`tbody`, `tfoot`, `thead`, `tr`, `ul` and `video`. + +For example, the following: + +``` +<div markdown="1"> +This is a *Markdown* Paragraph. +</div> +``` + +... is rendered as: + +``` html +<div> +<p>This is a <em>Markdown</em> Paragraph.</p> +</div> +``` + +The following tags have the `span` behavior by default: `address`, `dd`, `dt`, `h[1-6]`, `legend`, `li`, `p`, `td`, +and `th`. + +For example, the following: + +``` +<p markdown="1"> +This is not a *Markdown* Paragraph. +</p> +``` + +... is rendered as: + +``` html +<p> +This is not a <em>Markdown</em> Paragraph. +</p> +``` + +### `markdown="block"` { #block } + +When the `markdown` attribute is set to `"block"`, then the parser will force the `block` behavior on the contents of +the element so long as it is one of the `block` or `span` tags. + +The content of a `block` element is parsed into block-level content. In other words, the text is rendered as +paragraphs, headers, lists, blockquotes, etc. Any inline syntax within those elements is processed as well. + +For example, the following: + +``` +<section markdown="block"> +# A header. + +A *Markdown* paragraph. + +* A list item. +* A second list item. + +</section> +``` + +... is rendered as: + +``` html +<section> +<h1>A header.</h1> +<p>A <em>Markdown</em> paragraph.</p> +<ul> +<li>A list item.</li> +<li>A second list item.</li> +</ul> +</section> +``` + +!!! warning + + Forcing elements to be parsed as `block` elements when they are not by default could result in invalid HTML. + For example, one could force a `<p>` element to be nested within another `<p>` element. In most cases it is + recommended to use the default behavior of `markdown="1"`. Explicitly setting `markdown="block"` should be + reserved for advanced users who understand the HTML specification and how browsers parse and render HTML. + +### `markdown="span"` { #span } + +When the `markdown` attribute is set to `"span"`, then the parser will force the `span` behavior on the contents +of the element so long as it is one of the `block` or `span` tags. + +The content of a `span` element is not parsed into block-level content. In other words, the content will not be +rendered as paragraphs, headers, etc. Only inline syntax will be rendered, such as links, strong, emphasis, etc. + +For example, the following: + +``` +<div markdown="span"> +# *Not* a header +</div> +``` + +... is rendered as: + +``` html +<div> +# <em>Not</em> a header +</div> +``` + +### Ignored Elements + +The following tags are always ignored, regardless of any `markdown` attribute: `canvas`, `math`, `option`, `pre`, +`script`, `style`, and `textarea`. All other raw HTML tags are treated as span-level tags and are not affected by this +extension. + +### Nesting + +When nesting multiple levels of raw HTML elements, a `markdown` attribute must be defined for each block-level +element. For any block-level element which does not have a `markdown` attribute, everything inside that element is +ignored, including child elements with `markdown` attributes. + +For example, the following: + +``` +<article id="my-article" markdown="1"> +# Article Title + +A Markdown paragraph. + +<section id="section-1" markdown="1"> +## Section 1 Title + +<p>Custom raw **HTML** which gets ignored.</p> + +</section> + +<section id="section-2" markdown="1"> +## Section 2 Title + +<p markdown="1">**Markdown** content.</p> + +</section> + +</article> +``` + +... is rendered as: + +```html +<article id="my-article"> +<h1>Article Title</h1> +<p>A Markdown paragraph.</p> +<section id="section-1"> +<h2>Section 1 Title</h2> +<p>Custom raw **HTML** which gets ignored.</p> +</section> +<section id="section-2"> +<h2>Section 2 Title</h2> +<p><strong>Markdown</strong> content.</p> +</section> +</article> +``` + +When the value of an element's `markdown` attribute is more permissive that its parent, then the parent's stricter +behavior is enforced. For example, a `block` element nested within a `span` element will be parsed using the `span` +behavior. However, if the value of an element's `markdown` attribute is the same as, or more restrictive than, its +parent, the the child element's behavior is observed. For example, a `block` element may contain either `block` +elements or `span` elements as children and each element will be parsed using the specified behavior. + +### Tag Normalization + +While the default behavior is for Markdown to not alter raw HTML, as this extension is parsing the content of raw HTML elements, it will do some normalization of the tags of block-level elements. For example, the following raw HTML: + +``` +<div markdown="1"> +<p markdown="1">A Markdown paragraph with *no* closing tag. +<p>A raw paragraph with *no* closing tag. +</div> +``` + +... is rendered as: + +``` html +<div> +<p>A Markdown paragraph with <em>no</em> closing tag. +</p> +<p>A raw paragraph with *no* closing tag. +</p> +</div> +``` + +Notice that the parser properly recognizes that an unclosed `<p>` tag ends when another `<p>` tag begins or when the +parent element ends. In both cases, a closing `</p>` was added to the end of the element, regardless of whether a +`markdown` attribute was assigned to the element. + +To avoid any normalization, an element must not be a descendant of any block-level element which has a `markdown` +attribute defined. + +!!! warning + + The normalization behavior is only documented here so that document authors are not surprised when their carefully + crafted raw HTML is altered by Markdown. This extension should not be relied on to normalize and generate valid + HTML. For the best results, always include valid raw HTML (with both opening and closing tags) in your Markdown + documents. + +## Usage + +From the Python interpreter: + +``` pycon +>>> import markdown +>>> html = markdown.markdown(text, extensions=['md_in_html']) +``` diff --git a/docs/extensions/meta_data.md b/docs/extensions/meta_data.md new file mode 100644 index 0000000..d5a4c87 --- /dev/null +++ b/docs/extensions/meta_data.md @@ -0,0 +1,113 @@ +title: Meta-Data Extension + +Meta-Data +========= + +Summary +------- + +The Meta-Data extension adds a syntax for defining meta-data about a document. +It is inspired by and follows the syntax of [MultiMarkdown][]. Currently, +this extension does not use the meta-data in any way, but simply provides it as +a `Meta` attribute of a Markdown instance for use by other extensions or +directly by your python code. + +This extension is included in the standard Markdown library. + +[MultiMarkdown]: https://fletcherpenney.net/multimarkdown/#metadata + +Syntax +------ + +Meta-data consists of a series of keywords and values defined at the beginning +of a markdown document like this: + +```md +Title: My Document +Summary: A brief description of my document. +Authors: Waylan Limberg + John Doe +Date: October 2, 2007 +blank-value: +base_url: http://example.com + +This is the first paragraph of the document. +``` + +The keywords are case-insensitive and may consist of letters, numbers, +underscores and dashes and must end with a colon. The values consist of +anything following the colon on the line and may even be blank. + +If a line is indented by 4 or more spaces, that line is assumed to be an +additional line of the value for the previous keyword. A keyword may have as +many lines as desired. + +The first blank line ends all meta-data for the document. Therefore, the first +line of a document must not be blank. + +Alternatively, You may use YAML style deliminators to mark the start and/or end +of your meta-data. When doing so, the first line of your document must be `---`. +The meta-data ends at the first blank line or the first line containing an end +deliminator (either `---` or `...`), whichever comes first. Even though YAML +deliminators are supported, meta-data is not parsed as YAML. + +All meta-data is stripped from the document prior to any further processing +by Markdown. + +Usage +----- + +See [Extensions](index.md) for general extension usage. Use `meta` as the name +of the extension. + +A trivial example: + +```python +markdown.markdown(some_text, extensions=['meta']) +``` + +Accessing the Meta-Data +----------------------- + +The meta-data is made available as a python Dict in the `Meta` attribute of an +instance of the Markdown class. For example, using the above document: + +```pycon +>>> md = markdown.Markdown(extensions = ['meta']) +>>> html = md.convert(text) +>>> # Meta-data has been stripped from output +>>> print html +<p>This is the first paragraph of the document.</p> + +>>> # View meta-data +>>> print md.Meta +{ +'title' : ['My Document'], +'summary' : ['A brief description of my document.'], +'authors' : ['Waylan Limberg', 'John Doe'], +'date' : ['October 2, 2007'], +'blank-value' : [''], +'base_url' : ['http://example.com'] +} +``` + +Note that the keys are all lowercase and the values consist of a list of +strings where each item is one line for that key. This way, one could preserve +line breaks if desired. Or the items could be joined where appropriate. No +assumptions are made regarding the data. It is simply passed as found to the +`Meta` attribute. + +Perhaps the meta-data could be passed into a template system, or used by +various Markdown extensions. The possibilities are left to the imagination of +the developer. + +Compatible Extensions +--------------------- + +The following extensions are currently known to work with the Meta-Data +extension. The keywords they are known to support are also listed. + +* [WikiLinks](wikilinks.md) + * `wiki_base_url` + * `wiki_end_url` + * `wiki_html_class` diff --git a/docs/extensions/nl2br.md b/docs/extensions/nl2br.md new file mode 100644 index 0000000..2c7a1a5 --- /dev/null +++ b/docs/extensions/nl2br.md @@ -0,0 +1,41 @@ +title: New Line to Break Extension + +New-Line-to-Break Extension +=========================== + +Summary +------- + +The New-Line-to-Break (`nl2br`) Extension will cause newlines to be treated as +hard breaks; like StackOverflow and [GitHub][] flavored Markdown do. + +[Github]: https://github.github.com/github-flavored-markdown/ + +Example +------- + +```pycon +>>> import markdown +>>> text = """ +... Line 1 +... Line 2 +... """ +>>> html = markdown.markdown(text, extensions=['nl2br']) +>>> print html +<p>Line 1<br /> +Line 2</p> +``` + +Usage +----- + +See [Extensions](index.md) for general extension usage. Use `nl2br` as the name +of the extension. + +This extension does not accept any special configuration options. + +A trivial example: + +```python +markdown.markdown(some_text, extensions=['nl2br']) +``` diff --git a/docs/extensions/sane_lists.md b/docs/extensions/sane_lists.md new file mode 100644 index 0000000..bb43833 --- /dev/null +++ b/docs/extensions/sane_lists.md @@ -0,0 +1,104 @@ +title: Sane Lists Extension + +Sane Lists +========== + +Summary +------- + +The Sane Lists extension alters the behavior of the Markdown List syntax +to be less surprising. + +This extension is included in the standard Markdown library. + +Syntax +------ + +Sane Lists do not allow the mixing of list types. In other words, an ordered +list will not continue when an unordered list item is encountered and +vice versa. For example: + +```md +1. Ordered item 1 +2. Ordered item 2 + +* Unordered item 1 +* Unordered item 2 +``` + +will result in the following output: + +```html +<ol> + <li>Ordered item 1</li> + <li>Ordered item 2</li> +</ol> + +<ul> + <li>Unordered item 1</li> + <li>Unordered item 2</li> +</ul> +``` + +Whereas the default Markdown behavior would be to generate an unordered list. + +Note that, unlike the default Markdown behavior, if a blank line is not +included between list items, the different list type is ignored completely. +This corresponds to the behavior of paragraphs. For example: + +```md +A Paragraph. +* Not a list item. + +1. Ordered list item. +* Not a separate list item. +``` + +With this extension the above will result in the following output: + +```html +<p>A Paragraph. +* Not a list item.</p> + +<ol> + <li>Ordered list item. + * Not a separate list item.</li> +</ol> +``` + +Sane lists also recognize the number used in ordered lists. Given the following +list: + +```md +4. Apples +5. Oranges +6. Pears +``` + +By default markdown will ignore the fact that the first line started +with item number "4" and the HTML list will start with a number "1". +This extension will result in the following HTML output: + +```html +<ol start="4"> + <li>Apples</li> + <li>Oranges</li> + <li>Pears</li> +</ol> +``` + +In all other ways, Sane Lists should behave as normal Markdown lists. + +Usage +----- + +See [Extensions](index.md) for general extension usage. Use `sane_lists` as the +name of the extension. + +This extension does not accept any special configuration options. + +A trivial example: + +```python +markdown.markdown(some_text, extensions=['sane_lists']) +``` diff --git a/docs/extensions/smarty.md b/docs/extensions/smarty.md new file mode 100644 index 0000000..d7211c0 --- /dev/null +++ b/docs/extensions/smarty.md @@ -0,0 +1,84 @@ +title: SmartyPants Extension + +SmartyPants +=========== + +Summary +------- + +The SmartyPants extension converts ASCII dashes, quotes and ellipses to +their HTML entity equivalents. + +ASCII symbol | Replacements | HTML Entities | Substitution Keys +------------ | --------------- | ------------------- | ---------------------------------------- +`'` | ‘ ’ | `‘` `’` | `'left-single-quote'`, `'right-single-quote'` +`"` | “ ” | `“` `”` | `'left-double-quote'`, `'right-double-quote'` +`<< >>` | « » | `«` `»` | `'left-angle-quote'`, `'right-angle-quote'` +`...` | … | `…` | `'ellipsis'` +`--` | – | `–` | `'ndash'` +`---` | — | `—` | `'mdash'` + +Using the configuration option 'substitutions' you can overwrite the +default substitutions. Just pass a dict mapping (a subset of) the +keys to the substitution strings. + +For example, one might use the following configuration to get correct quotes for +the German language: + +```python +extension_configs = { + 'smarty': { + 'substitutions': { + 'left-single-quote': '‚', # sb is not a typo! + 'right-single-quote': '‘', + 'left-double-quote': '„', + 'right-double-quote': '“' + } + } +} +``` + +!!! note + This extension re-implements the Python [SmartyPants] + library by integrating it into the markdown parser. + While this does not provide any additional features, + it does offer a few advantages. Notably, it will not + try to work on highlighted code blocks (using the + [CodeHilite] Extension) like the third party library + has been known to do. + +[SmartyPants]: https://pythonhosted.org/smartypants/ +[CodeHilite]: code_hilite.md + +Usage +----- + +See [Extensions](index.md) for general extension usage. Use `smarty` as the +name of the extension. + +See the [Library Reference](../reference.md#extensions) for information about +configuring extensions. + +The following options are provided to configure the output: + +Option | Default value | Description +------ | ------------- | ----------- +`smart_dashes` | `True` | whether to convert dashes +`smart_quotes` | `True` | whether to convert straight quotes +`smart_angled_quotes` | `False` | whether to convert angled quotes +`smart_ellipses` | `True` | whether to convert ellipses +`substitutions` | `{}` | overwrite default substitutions + +A trivial example: + +```python +markdown.markdown(some_text, extensions=['smarty']) +``` + +Further reading +--------------- + +SmartyPants extension is based on the original SmartyPants implementation +by John Gruber. Please read its [documentation][1] for details. + +[1]: https://daringfireball.net/projects/smartypants/ diff --git a/docs/extensions/tables.md b/docs/extensions/tables.md new file mode 100644 index 0000000..aaffc09 --- /dev/null +++ b/docs/extensions/tables.md @@ -0,0 +1,87 @@ +title: Tables Extension + +Tables +====== + +Summary +------- + +The Tables extension adds the ability to create tables in Markdown documents. + +This extension is included in the standard Markdown library. + +Syntax +------ + +Tables are defined using the syntax established in [PHP Markdown Extra][php]. + +[php]: http://www.michelf.com/projects/php-markdown/extra/#table + +Thus, the following text (taken from the above referenced PHP documentation): + +```md +First Header | Second Header +------------- | ------------- +Content Cell | Content Cell +Content Cell | Content Cell +``` + +will be rendered as: + +```html +<table> + <thead> + <tr> + <th>First Header</th> + <th>Second Header</th> + </tr> + </thead> + <tbody> + <tr> + <td>Content Cell</td> + <td>Content Cell</td> + </tr> + <tr> + <td>Content Cell</td> + <td>Content Cell</td> + </tr> + </tbody> +</table> +``` + +!!! seealso "See Also" + The [Attribute Lists](./attr_list.md) extension includes support for defining attributes on table cells. + +Usage +----- + +See [Extensions](index.md) for general extension usage. Use `tables` as the +name of the extension. + +See the [Library Reference](../reference.md#extensions) for information about +configuring extensions. + +The following options are provided to change the default behavior: + +* **`use_align_attribute`**: Set to `True` to use `align` instead of an appropriate `style` attribute + + Default: `'False'` + + +A trivial example: + +```python +markdown.markdown(some_text, extensions=['tables']) +``` + +### Examples + +For an example, let us suppose that alignment should be controlled by the legacy `align` +attribute. + +```pycon +>>> from markdown.extensions.tables import TableExtension +>>> html = markdown.markdown(text, +... extensions=[TableExtension(use_align_attribute=True)] +... ) +``` diff --git a/docs/extensions/toc.md b/docs/extensions/toc.md new file mode 100644 index 0000000..8d7bd35 --- /dev/null +++ b/docs/extensions/toc.md @@ -0,0 +1,232 @@ +title: Table of Contents Extension + +Table of Contents +================= + +Summary +------- + +The Table of Contents extension generates a Table of Contents from a Markdown +document and adds it into the resulting HTML document. + +This extension is included in the standard Markdown library. + +Syntax +------ + +By default, all headers will automatically have unique `id` attributes +generated based upon the text of the header. Note this example, in which all +three headers would have the same `id`: + +```md +#Header +#Header +#Header +``` + +Results in: + +```html +<h1 id="header">Header</h1> +<h1 id="header_1">Header</h1> +<h1 id="header_2">Header</h1> +``` + +Place a marker in the document where you would like the Table of Contents to +appear. Then, a nested list of all the headers in the document will replace the +marker. The marker defaults to `[TOC]` so the following document: + +```md +[TOC] + +# Header 1 + +## Header 2 +``` + +would generate the following output: + +```html +<div class="toc"> + <ul> + <li><a href="#header-1">Header 1</a></li> + <ul> + <li><a href="#header-2">Header 2</a></li> + </ul> + </ul> +</div> +<h1 id="header-1">Header 1</h1> +<h2 id="header-2">Header 2</h2> +``` + +Regardless of whether a `marker` is found in the document (or disabled), the +Table of Contents is available as an attribute (`toc`) on the Markdown class. +This allows one to insert the Table of Contents elsewhere in their page +template. For example: + +```pycon +>>> md = markdown.Markdown(extensions=['toc']) +>>> html = md.convert(text) +>>> page = render_some_template(context={'body': html, 'toc': md.toc}) +``` + +The `toc_tokens` attribute is also available on the Markdown class and contains +a nested list of dict objects. For example, the above document would result in +the following object at `md.toc_tokens`: + +```python +[ + { + 'level': 1, + 'id': 'header-1', + 'name': 'Header 1', + 'children': [ + {'level': 2, 'id': 'header-2', 'name': 'Header 2', 'children':[]} + ] + } +] +``` + +Note that the `level` refers to the `hn` level. In other words, `<h1>` is level +`1` and `<h2>` is level `2`, etc. Be aware that improperly nested levels in the +input may result in odd nesting of the output. + +### Custom Labels + +In most cases, the text label in the Table of Contents should match the text of +the header. However, occasionally that is not desirable. In that case, if this +extension is used in conjunction with the [Attribute Lists Extension] and a +`data-toc-label` attribute is defined on the header, then the contents of that +attribute will be used as the text label for the item in the Table of Contents. +For example, the following Markdown: + +[Attribute Lists Extension]: attr_list.md + +```md +[TOC] + +# Functions + +## `markdown.markdown(text [, **kwargs])` { #markdown data-toc-label='markdown.markdown' } +``` +would generate the following output: + +```html +<div class="toc"> + <ul> + <li><a href="#functions">Functions</a></li> + <ul> + <li><a href="#markdown">markdown.markdown</a></li> + </ul> + </ul> +</div> +<h1 id="functions">Functions</h1> +<h2 id="markdown"><code>markdown.markdown(text [, **kwargs])</code></h2> +``` + +Notice that the text in the Table of Contents is much cleaner and easier to read +in the context of a Table of Contents. The `data-toc-label` is not included in +the HTML header element. Also note that the ID was manually defined in the +attribute list to provide a cleaner URL when linking to the header. If the ID is +not manually defined, it is always derived from the text of the header, never +from the `data-toc-label` attribute. + +Usage +----- + +See [Extensions](index.md) for general extension usage. Use `toc` as the name +of the extension. + +See the [Library Reference](../reference.md#extensions) for information about +configuring extensions. + +The following options are provided to configure the output: + +* **`marker`**: + Text to find and replace with the Table of Contents. Defaults to `[TOC]`. + + Set to an empty string to disable searching for a marker, which may save + some time, especially on long documents. + +* **`title`**: + Title to insert in the Table of Contents' `<div>`. Defaults to `None`. + +* **`toc_class`**: + CSS class(es) used for the `<div>` containing the Table of Contents. Defaults to `toc`. + +* **`anchorlink`**: + Set to `True` to cause all headers to link to themselves. Default is `False`. + +* **`anchorlink_class`**: + CSS class(es) used for the link. Defaults to `toclink`. + +* **`permalink`**: + Set to `True` or a string to generate permanent links at the end of each header. + Useful with Sphinx style sheets. + + When set to `True` the paragraph symbol (¶ or "`¶`") is used as + the link text. When set to a string, the provided string is used as the link + text. + +* **`permalink_class`**: + CSS class(es) used for the link. Defaults to `headerlink`. + +* **`permalink_title`**: + Title attribute of the permanent link. Defaults to `Permanent link`. + +* **`baselevel`**: + Base level for headers. Defaults to `1`. + + The `baselevel` setting allows the header levels to be automatically + adjusted to fit within the hierarchy of your HTML templates. For example, + suppose the Markdown text for a page should not contain any headers higher + than level 3 (`<h3>`). The following will accomplish that: + + :::pycon + >>> text = ''' + ... #Some Header + ... ## Next Level''' + >>> from markdown.extensions.toc import TocExtension + >>> html = markdown.markdown(text, extensions=[TocExtension(baselevel=3)]) + >>> print html + <h3 id="some_header">Some Header</h3> + <h4 id="next_level">Next Level</h4>' + +* **`slugify`**: + Callable to generate anchors. + + Default: `markdown.extensions.headerid.slugify` + + In order to use a different algorithm to define the id attributes, define and + pass in a callable which takes the following two arguments: + + * `value`: The string to slugify. + * `separator`: The Word Separator. + + The callable must return a string appropriate for use in HTML `id` attributes. + + An alternate version of the default callable supporting Unicode strings is also + provided as `markdown.extensions.headerid.slugify_unicode`. + +* **`separator`**: + Word separator. Character which replaces white space in id. Defaults to "`-`". + +* **`toc_depth`** + Define the range of section levels to include in the Table of Contents. + A single integer (`b`) defines the bottom section level (`<h1>..<hb>`) only. + A string consisting of two digits separated by a hyphen in between (`"2-5"`), + define the top (`t`) and the bottom (`b`) (`<ht>..<hb>`). Defaults to `6` (bottom). + + When used with conjunction with `baselevel`, this parameter will not + take the fitted hierarchy from `baselevel` into account. That is, if + both `toc_depth` and `baselevel` are `3`, then only the highest level + will be present in the table. If you set `baselevel` to `3` and + `toc_depth` to `"2-6"`, the *first* headline will be `<h3>` and so still + included in the Table of Contents. To exclude this first level, you + have to set `toc_depth` to `"4-6"`. + +A trivial example: + +```python +markdown.markdown(some_text, extensions=['toc']) +``` diff --git a/docs/extensions/wikilinks.md b/docs/extensions/wikilinks.md new file mode 100644 index 0000000..cd355f7 --- /dev/null +++ b/docs/extensions/wikilinks.md @@ -0,0 +1,154 @@ +title: WikiLinks Extension + +# WikiLinks + +## Summary + +The WikiLinks extension adds support for [WikiLinks][]. Specifically, any +``[[bracketed]]`` word is converted to a link. + +This extension is included in the standard Markdown library. + +[WikiLinks]: https://en.wikipedia.org/wiki/Wikilink + +## Syntax + +A ``[[bracketed]]`` word is any combination of upper or lower case letters, +number, dashes, underscores and spaces surrounded by double brackets. Therefore + +```md +[[Bracketed]] +``` + +would produce the following HTML: + +```html +<a href="/Bracketed/" class="wikilink">Bracketed</a> +``` + +Note that WikiLinks are automatically assigned `class="wikilink"` making it +easy to style WikiLinks differently from other links on a page if one so +desires. See below for ways to alter the class. + +Also note that when a space is used, the space is converted to an underscore in +the link but left as-is in the label. Perhaps an example would illustrate this +best: + +```md +[[Wiki Link]] +``` + +becomes + +```html +<a href="/Wiki_Link/" class="wikilink">Wiki Link</a> +``` + +## Usage + +See [Extensions](index.md) for general extension usage. Use `wikilinks` as the +name of the extension. + +See the [Library Reference](../reference.md#extensions) for information about +configuring extensions. + +The default behavior is to point each link to the document root of the current +domain and close with a trailing slash. Additionally, each link is assigned to +the HTML class `wikilink`. + +The following options are provided to change the default behavior: + +* **`base_url`**: String to append to beginning of URL. + + Default: `'/'` + +* **`end_url`**: String to append to end of URL. + + Default: `'/'` + +* **`html_class`**: CSS class. Leave blank for none. + + Default: `'wikilink'` + +* **`build_url`**: Callable which formats the URL from its parts. + +A trivial example: + +```python +markdown.markdown(some_text, extensions=['wikilinks']) +``` + +### Examples + +For an example, let us suppose links should always point to the sub-directory +`/wiki/` and end with `.html` + +```pycon +>>> from markdown.extensions.wikilinks import WikiLinkExtension +>>> html = markdown.markdown(text, +... extensions=[WikiLinkExtension(base_url='/wiki/', end_url='.html')] +... ) +``` + +The above would result in the following link for `[[WikiLink]]`. + +```html +<a href="/wiki/WikiLink.html" class="wikilink">WikiLink</a> +``` + +If you want to do more that just alter the base and/or end of the URL, you +could also pass in a callable which must accept three arguments (``label``, +``base``, and ``end``). The callable must return the URL in it's entirety. + +```pycon +>>> def my_url_builder(label, base, end): +... # do stuff +... return url +... +>>> html = markdown.markdown(text, +... extensions=[WikiLinkExtension(build_url=my_url_builder)], +... ) +``` + +The option is also provided to change or remove the class attribute. + +```pycon +>>> html = markdown.markdown(text, +... extensions=[WikiLinkExtension(html_class='myclass')] +... ) +``` + +Would cause all WikiLinks to be assigned to the class `myclass`. + +```html +<a href="/WikiLink/" class="myclass">WikiLink</a> +``` + +## Using with Meta-Data extension + +The WikiLink extension also supports the [Meta-Data](meta_data.md) extension. +Please see the documentation for that extension for specifics. The supported +meta-data keywords are: + +* `wiki_base_url` +* `wiki_end_url` +* `wiki_html_class` + +When used, the meta-data will override the settings provided through the +`extension_configs` interface. + +This document: + +```md +wiki_base_url: http://example.com/ +wiki_end_url: .html +wiki_html_class: + +A [[WikiLink]] in the first paragraph. +``` + +would result in the following output (notice the blank `wiki_html_class`): + +```html +<p>A <a href="http://example.com/WikiLink.html">WikiLink</a> in the first paragraph.</p> +``` diff --git a/docs/favicon.ico b/docs/favicon.ico Binary files differnew file mode 100644 index 0000000..c9efc58 --- /dev/null +++ b/docs/favicon.ico diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..0388070 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,112 @@ +title: Python-Markdown + +Python-Markdown +=============== + +This is a Python implementation of John Gruber's +[Markdown](https://daringfireball.net/projects/markdown/). +It is almost completely compliant with the reference implementation, +though there are a few very minor [differences](#differences). See John's +[Syntax Documentation](https://daringfireball.net/projects/markdown/syntax) +for the syntax rules. + +To get started, see the [installation instructions](install.md), the [library +reference](reference.md), and the [command line interface](cli.md). + +Goals +----- + +The Python-Markdown project is developed with the following goals in mind: + +* Maintain a Python library (with an optional CLI wrapper) suited to use in web + server environments (never raise an exception, never write to stdout, etc.) as + an implementation of the markdown parser that follows the + [syntax rules](https://daringfireball.net/projects/markdown/syntax) and the + behavior of the original (markdown.pl) implementation as reasonably as + possible (see [differences](#differences) for a few exceptions). + +* Provide an [Extension API](extensions/api.md) which makes it possible + to change and/or extend the behavior of the parser. + +Features +-------- + +In addition to the basic markdown syntax, Python-Markdown supports the following +features: + +* __International Input__ + + Python-Markdown will accept [input](reference.md#text) in any language + supported by Unicode including bi-directional text. In fact the test suite + includes documents written in Russian and Arabic. + +* __Extensions__ + + Various [extensions](extensions/index.md) are provided (including + [extra](extensions/extra.md)) to change and/or extend the base syntax. + Additionally, a public [Extension API](extensions/api.md) is available + to write your own extensions. + +* __Output Formats__ + + Python-Markdown can output documents with either HTML or XHTML style tags. + See the [Library Reference](reference.md#output_format) for details. + +* __Command Line Interface__ + + In addition to being a Python Library, a + [command line script](cli.md) is available for your convenience. + +Differences +----------- + +While Python-Markdown strives to fully implement markdown as described in the +[syntax rules](https://daringfireball.net/projects/markdown/syntax), the rules +can be interpreted in different ways and different implementations +occasionally vary in their behavior (see the +[Babelmark FAQ](https://johnmacfarlane.net/babelmark2/faq.html#what-are-some-examples-of-interesting-divergences-between-implementations) +for some examples). Known and intentional differences found in Python-Markdown +are summarized below: + +* __Middle-Word Emphasis__ + + Python-Markdown defaults to ignoring middle-word emphasis (and strong + emphasis). In other words, `some_long_filename.txt` will not become + `some<em>long</em>filename.txt`. This can be switched off if desired. See + the [Legacy EM Extension](extensions/legacy_em.md) for details. + +* __Indentation/Tab Length__ + + The [syntax rules](https://daringfireball.net/projects/markdown/syntax#list) + clearly state that when a list item consists of multiple paragraphs, "each + subsequent paragraph in a list item **must** be indented by either 4 spaces + or one tab" (emphasis added). However, many implementations do not enforce + this rule and allow less than 4 spaces of indentation. The implementers of + Python-Markdown consider it a bug to not enforce this rule. + + This applies to any block level elements nested in a list, including + paragraphs, sub-lists, blockquotes, code blocks, etc. They **must** always + be indented by at least four spaces (or one tab) for each level of nesting. + + In the event that one would prefer different behavior, + [tab_length](reference.md#tab_length) can be set to whatever length is + desired. Be warned however, as this will affect indentation for all aspects + of the syntax (including root level code blocks). Alternatively, a + [third party extension] may offer a solution that meets your needs. + +* __Consecutive Lists__ + + While the syntax rules are not clear on this, many implementations (including + the original) do not end one list and start a second list when the list marker + (asterisks, pluses, hyphens, and numbers) changes. For consistency, + Python-Markdown maintains the same behavior with no plans to change in the + foreseeable future. That said, the [Sane List Extension](extensions/sane_lists.md) + is available to provide a less surprising behavior. + +Support +------- + +You may report bugs, ask for help, and discuss various other issues on the [bug tracker][]. + +[third party extension]: https://github.com/Python-Markdown/markdown/wiki/Third-Party-Extensions +[bug tracker]: https://github.com/Python-Markdown/markdown/issues diff --git a/docs/install.md b/docs/install.md new file mode 100644 index 0000000..25530d6 --- /dev/null +++ b/docs/install.md @@ -0,0 +1,32 @@ +title: Installation + +# Installing Python-Markdown + +## The Easy Way + +The easiest way to install Python-Markdown is simply to type the +following command from the command line: + +```bash +pip install markdown +``` + +That's it! You're ready to [use](reference.md) Python-Markdown. Enjoy! + +For more detailed instructions on installing Python packages, see the +[Installing Packages] tutorial in the [Python Packaging User Guide]. + +[Installing Packages]: https://packaging.python.org/tutorials/installing-packages/ +[Python Packaging User Guide]: https://packaging.python.org/ + +## Using the Git Repository {: #git } + +If you're the type that likes to live on the edge, you may want to keep up with +the latest additions and bug fixes in the repository between releases. +Python-Markdown is maintained in a Git repository on GitHub.com. To +get a copy of Python-Markdown from the repository do the following from the +command line: + +```bash +pip install git+https://github.com/Python-Markdown/markdown.git +``` diff --git a/docs/py.png b/docs/py.png Binary files differnew file mode 100644 index 0000000..93e4a02 --- /dev/null +++ b/docs/py.png diff --git a/docs/reference.md b/docs/reference.md new file mode 100644 index 0000000..8153ebe --- /dev/null +++ b/docs/reference.md @@ -0,0 +1,269 @@ +title: Library Reference + +# Using Markdown as a Python Library + +First and foremost, Python-Markdown is intended to be a python library module +used by various projects to convert Markdown syntax into HTML. + +## The Basics + +To use markdown as a module: + +```python +import markdown +html = markdown.markdown(your_text_string) +``` + +## The Details + +Python-Markdown provides two public functions ([`markdown.markdown`](#markdown) +and [`markdown.markdownFromFile`](#markdownFromFile)) both of which wrap the +public class [`markdown.Markdown`](#Markdown). If you're processing one +document at a time, these functions will serve your needs. However, if you need +to process multiple documents, it may be advantageous to create a single +instance of the `markdown.Markdown` class and pass multiple documents through +it. If you do use a single instance though, make sure to call the `reset` +method appropriately ([see below](#convert)). + +### markdown.markdown(text [, **kwargs]) {: #markdown data-toc-label='markdown.markdown' } + +The following options are available on the `markdown.markdown` function: + +__text__{: #text } + +: The source Unicode string. (required) + + !!! note "Important" + Python-Markdown expects a **Unicode** string as input (some simple ASCII binary strings *may* work only by + coincidence) and returns output as a Unicode string. Do not pass binary strings to it! If your input is + encoded, (e.g. as UTF-8), it is your responsibility to decode it. For example: + + :::python + with open("some_file.txt", "r", encoding="utf-8") as input_file: + text = input_file.read() + html = markdown.markdown(text) + + If you want to write the output to disk, you *must* encode it yourself: + + :::python + with open("some_file.html", "w", encoding="utf-8", errors="xmlcharrefreplace") as output_file: + output_file.write(html) + +__extensions__{: #extensions } + +: A list of extensions. + + Python-Markdown provides an [API](extensions/api.md) for third parties to + write extensions to the parser adding their own additions or changes to the + syntax. A few commonly used extensions are shipped with the markdown + library. See the [extension documentation](extensions/index.md) for a + list of available extensions. + + The list of extensions may contain instances of extensions and/or strings + of extension names. + + :::python + extensions=[MyExtClass(), 'myext', 'path.to.my.ext:MyExtClass'] + + !!! note + The preferred method is to pass in an instance of an extension. Strings + should only be used when it is impossible to import the Extension Class + directly (from the command line or in a template). + + When passing in extension instances, each class instance must be a subclass + of `markdown.extensions.Extension` and any configuration options should be + defined when initiating the class instance rather than using the + [`extension_configs`](#extension_configs) keyword. For example: + + :::python + from markdown.extensions import Extension + class MyExtClass(Extension): + # define your extension here... + + markdown.markdown(text, extensions=[MyExtClass(option='value')]) + + If an extension name is provided as a string, the string must either be the + registered entry point of any installed extension or the importable path + using Python's dot notation. + + See the documentation specific to an extension for the string name assigned + to an extension as an entry point. Simply include the defined name as + a string in the list of extensions. For example, if an extension has the + name `myext` assigned to it and the extension is properly installed, then + do the following: + + :::python + markdown.markdown(text, extensions=['myext']) + + If an extension does not have a registered entry point, Python's dot + notation may be used instead. The extension must be installed as a + Python module on your PYTHONPATH. Generally, a class should be specified in + the name. The class must be at the end of the name and be separated by a + colon from the module. + + Therefore, if you were to import the class like this: + + :::python + from path.to.module import MyExtClass + + Then load the extension as follows: + + :::python + markdown.markdown(text, extensions=['path.to.module:MyExtClass']) + + If only one extension is defined within a module and the module includes a + `makeExtension` function which returns an instance of the extension, then + the class name is not necessary. For example, in that case one could do + `extensions=['path.to.module']`. Check the documentation for a specific + extension to determine if it supports this feature. + + When loading an extension by name (as a string), you can only pass in + configuration settings to the extension by using the + [`extension_configs`](#extension_configs) keyword. + + !!! seealso "See Also" + See the documentation of the [Extension API](extensions/api.md) for + assistance in creating extensions. + +__extension_configs__{: #extension_configs } + +: A dictionary of configuration settings for extensions. + + Any configuration settings will only be passed to extensions loaded by name + (as a string). When loading extensions as class instances, pass the + configuration settings directly to the class when initializing it. + + !!! Note + The preferred method is to pass in an instance of an extension, which + does not require use of the `extension_configs` keyword at all. + See the [extensions](#extensions) keyword for details. + + The dictionary of configuration settings must be in the following format: + + :::python + extension_configs = { + 'extension_name_1': { + 'option_1': 'value_1', + 'option_2': 'value_2' + }, + 'extension_name_2': { + 'option_1': 'value_1' + } + } + + When specifying the extension name, be sure to use the exact same + string as is used in the [extensions](#extensions) keyword to load the + extension. Otherwise, the configuration settings will not be applied to + the extension. In other words, you cannot use the entry point in on + place and Python dot notation in the other. While both may be valid for + a given extension, they will not be recognized as being the same + extension by Markdown. + + See the documentation specific to the extension you are using for help in + specifying configuration settings for that extension. + +__output_format__{: #output_format }: + +: Format of output. + + Supported formats are: + + * `"xhtml"`: Outputs XHTML style tags. **Default**. + * `"html5"`: Outputs HTML style tags. + + The values can be in either lowercase or uppercase. + +__tab_length__{: #tab_length }: + +: Length of tabs in the source. Default: 4 + +### `markdown.markdownFromFile (**kwargs)` {: #markdownFromFile data-toc-label='markdown.markdownFromFile' } + +With a few exceptions, `markdown.markdownFromFile` accepts the same options as +`markdown.markdown`. It does **not** accept a `text` (or Unicode) string. +Instead, it accepts the following required options: + +__input__{: #input } (required) + +: The source text file. + + `input` may be set to one of three options: + + * a string which contains a path to a readable file on the file system, + * a readable file-like object, + * or `None` (default) which will read from `stdin`. + +__output__{: #output } + +: The target which output is written to. + + `output` may be set to one of three options: + + * a string which contains a path to a writable file on the file system, + * a writable file-like object, + * or `None` (default) which will write to `stdout`. + +__encoding__{: #encoding } + +: The encoding of the source text file. + + Defaults to `"utf-8"`. The same encoding will always be used for input and output. + The `xmlcharrefreplace` error handler is used when encoding the output. + + !!! Note + This is the only place that decoding and encoding of Unicode + takes place in Python-Markdown. If this rather naive solution does not + meet your specific needs, it is suggested that you write your own code + to handle your encoding/decoding needs. + +### markdown.Markdown([**kwargs]) {: #Markdown data-toc-label='markdown.Markdown' } + +The same options are available when initializing the `markdown.Markdown` class +as on the [`markdown.markdown`](#markdown) function, except that the class does +**not** accept a source text string on initialization. Rather, the source text +string must be passed to one of two instance methods. + +!!! warning + + Instances of the `markdown.Markdown` class are only thread safe within + the thread they were created in. A single instance should not be accessed + from multiple threads. + +#### Markdown.convert(source) {: #convert data-toc-label='Markdown.convert' } + +The `source` text must meet the same requirements as the [`text`](#text) +argument of the [`markdown.markdown`](#markdown) function. + +You should also use this method if you want to process multiple strings +without creating a new instance of the class for each string. + +```python +md = markdown.Markdown() +html1 = md.convert(text1) +html2 = md.convert(text2) +``` + +Depending on which options and/or extensions are being used, the parser may +need its state reset between each call to `convert`. + +```python +html1 = md.convert(text1) +md.reset() +html2 = md.convert(text2) +``` + +To make this easier, you can also chain calls to `reset` together: + +```python +html3 = md.reset().convert(text3) +``` + +#### Markdown.convertFile(**kwargs) {: #convertFile data-toc-label='Markdown.convertFile' } + +The arguments of this method are identical to the arguments of the same +name on the `markdown.markdownFromFile` function ([`input`](#input), +[`output`](#output), and [`encoding`](#encoding)). As with the +[`convert`](#convert) method, this method should be used to +process multiple files without creating a new instance of the class for +each document. State may need to be `reset` between each call to +`convertFile` as is the case with `convert`. diff --git a/docs/release-2.0.1.txt b/docs/release-2.0.1.txt deleted file mode 100644 index e5946b2..0000000 --- a/docs/release-2.0.1.txt +++ /dev/null @@ -1,16 +0,0 @@ -Python-Markdown 2.0.1 Release Notes -=================================== - -Python-Markdown 2.0.1 is a bug-fix release. No new features have been added. -Most notably, various issues with the command line script have been fixed. -There have also been a few fixes for minor parsing bugs in some edge cases. -For a full list of changes, see the git log. - -Backwards-incompatible Changes ------------------------------- - -Due to various complications in how Python handles command line scripts in -differance systems and with differant installation tools, we were forced to -rename the commandline script to ``markdown`` (no ".py"). A matching batch -script will get installed on Windows. Any shell scripts which call -``markdown.py`` will need to be altered to call ``markdown`` instead. diff --git a/docs/release-2.0.2.txt b/docs/release-2.0.2.txt deleted file mode 100644 index 8ae9a3d..0000000 --- a/docs/release-2.0.2.txt +++ /dev/null @@ -1,9 +0,0 @@ -Python-Markdown 2.0.2 Release Notes -=================================== - -Python-Markdown 2.0.2 is a bug-fix release. No new features have been added. -Most notably, the setup script has been updated to include a dependency on -ElementTree on older versions of Python (< 2.5). There have also been a few -fixes for minor parsing bugs in some edge cases. For a full list of changes, -see the git log. - diff --git a/docs/test_tools.md b/docs/test_tools.md new file mode 100644 index 0000000..f3ed4e5 --- /dev/null +++ b/docs/test_tools.md @@ -0,0 +1,178 @@ +title: Test Tools + +# Test Tools + +Python-Markdown provides some testing tools which simplify testing actual +Markdown output against expected output. The tools are built on the Python +standard library [`unittest`][unittest]. Therefore, no additional libraries are +required. While Python-Markdown uses the tools for its own tests, they were +designed and built so that third party extensions could use them as well. +Therefore, the tools are importable from `markdown.test_tools`. + +The test tools include two different `unittest.TestCase` subclasses: +`markdown.test_tools.TestCase` and `markdown.test_tools.LegacyTestCase`. + +## markdown.test_tools.TestCase + +The `markdown.test_tools.TestCase` class is a `unittest.TestCase` subclass with +a few additional helpers to make testing Markdown output easier. + +Properties +: `default_kwargs`: A `dict` of keywords to pass to Markdown for each +test. The defaults can be overridden on individual tests. + +Methods +: `assertMarkdownRenders`: accepts the source text, the expected output, an optional + dictionary of `expected_attrs`, and any keywords to pass to Markdown. The + `default_kwargs` defined on the class are used except where overridden by + keyword arguments. The output and expected output are passed to + `TestCase.assertMultiLineEqual`. An `AssertionError` is raised with a diff + if the actual output does not equal the expected output. The optional + keyword `expected_attrs` accepts a dictionary of attribute names as keys with + expected values. Each value is checked against the attribute of that + name on the instance of the `Markdown` class using `TestCase.assertEqual`. An + `AssertionError` is raised if any value does not match the expected value. + +: `dedent`: Dedent triple-quoted strings. + +In all other respects, `markdown.test_tools.TestCase` behaves as +`unittest.TestCase`. In fact, `assertMarkdownRenders` tests could be mixed with +other `unittest` style tests within the same test class. + +An example Markdown test might look like this: + +```python +from markdown.test_tools import TestCase + +class TestHr(TestCase): + def test_hr_before_paragraph(self): + self.assertMarkdownRenders( + # The Markdown source text used as input + self.dedent( + """ + *** + An HR followed by a paragraph with no blank line. + """ + ), + # The expected HTML output + self.dedent( + """ + <hr> + <p>An HR followed by a paragraph with no blank line.</p> + """ + ), + # Other keyword arguments to pass to `markdown.markdown` + output_format='html' + ) +``` + +## markdown.test_tools.LegacyTestCase + +In the past Python-Markdown exclusively used file-based tests. Many of those +tests still exist in Python-Markdown's test suite, including the test files from +the [reference implementation][perl] (`markdown.pl`) and [PHP Markdown][PHP]. +Each test consists of a matching pair of text and HTML files. The text file +contains a snippet of Markdown source text formatted for a specific syntax +feature and the HTML file contains the expected HTML output of that snippet. +When the test suite is run, each text file is run through Markdown and the +output is compared with the HTML file as a separate unit test. When a test +fails, the error report includes a diff of the expected output compared to the +actual output to easily identify any problems. + +A separate `markdown.test_tools.LegacyTestCase` subclass must be created for +each directory of test files. Various properties can be defined within the +subclass to point to a directory of text-based test files and define various +behaviors/defaults for those tests. The following properties are supported: + +* `location`: A path to the directory of test files. An absolute path is + preferred. +* `exclude`: A list of tests to skip. Each test name should comprise of a + file name without an extension. +* `normalize`: A boolean value indicating if the HTML should be normalized. + Default: `False`. Note: Normalization of HTML requires that [PyTidyLib] be + installed on the system. If PyTidyLib is not installed and `normalize` is set + to `True`, then the test will be skipped, regardless of any other settings. +* `input_ext`: A string containing the file extension of input files. + Default: `.txt`. +* `output_ext`: A string containing the file extension of expected output files. + Default: `html`. +* `default_kwargs`: A `markdown.test_tools.Kwargs` instance which stores the + default set of keyword arguments for all test files in the directory. + +In addition, properties can be defined for each individual set of test files +within the directory. The property should be given the name of the file without +the file extension. Any spaces and dashes in the file name should be replaced +with underscores. The value of the property should be a +`markdown.test_tools.Kwargs` instance which contains the keyword arguments that +should be passed to `markdown.markdown` for that test file. The keyword +arguments will "update" the `default_kwargs`. + +When the class instance is created during a test run, it will walk the given +directory and create a separate unit test for each set of test files using the +naming scheme: `test_filename`. One unit test will be run for each set of input +and output files. + +The definition of an example set of tests might look like this: + +```python +from markdown.test_tools import LegacyTestCase, Kwargs +import os + +# Get location of this file and use to find text file dirs. +parent_test_dir = os.path.abspath(os.path.dirname(__file__)) + + +class TestFoo(LegacyTestCase): + # Define location of text file directory. In this case, the directory is + # named "foo" and is in the same parent directory as this file. + location = os.path.join(parent_test_dir, 'foo') + # Define default keyword arguments. In this case, unless specified + # differently, all tests should use the output format "html". + default_kwargs = Kwargs(output_format='html') + + # The "xhtml" test should override the output format and use "xhtml". + xhtml = Kwargs(output_format='xhtml') + + # The "toc" test should use the "toc" extension with a custom permalink + # setting. + toc = Kwargs( + extensions=['markdown.extensions.toc'], + extension_configs={'markdown.extensions.toc': {'permalink': "[link]"}} + ) +``` + +Note that in the above example, the text file directory may contain many more +text-based test files than `xhtml` (`xhtml.txt` and `xhtml.html`) and `toc` +(`toc.txt` and `toc.html`). As long as each set of files exists as a pair, a +test will be created and run for each of them. Only the `xhtml` and `toc` tests +needed to be specifically identified as they had specific, non-default settings +which needed to be defined. + +## Running Python-Markdown's Tests + +As all of the tests for the `markdown` library are unit tests, standard +`unittest` methods of calling tests can be used. For example, to run all of +Python-Markdown's tests, from the root of the git repository, run the following +command: + +```sh +python -m unittest discover tests +``` + +That simple command will search everything in the `tests` directory and it's +sub-directories and run all `unittest` tests that it finds, including +`unittest.TestCase`, `markdown.test_tools.TestCase`, and +`markdown.test_tools.LegacyTestCase` subclasses. Normal [unittest] discovery +rules apply. + +!!! seealso "See Also" + + See the [Contributing Guide] for instructions on setting up a + [development environment] for running the tests. + +[unittest]: https://docs.python.org/3/library/unittest.html +[Perl]: https://daringfireball.net/projects/markdown/ +[PHP]: http://michelf.com/projects/php-markdown/ +[PyTidyLib]: http://countergram.github.io/pytidylib/ +[Contributing Guide]: contributing.md +[development environment]: contributing.md#development-environment diff --git a/docs/using_as_module.txt b/docs/using_as_module.txt deleted file mode 100644 index 130d0a7..0000000 --- a/docs/using_as_module.txt +++ /dev/null @@ -1,150 +0,0 @@ -Using Markdown as Python Library -================================ - -First and foremost, Python-Markdown is intended to be a python library module -used by various projects to convert Markdown syntax into HTML. - -The Basics ----------- - -To use markdown as a module: - - import markdown - html = markdown.markdown(your_text_string) - -Encoded Text ------------- - -Note that ``markdown()`` expects **Unicode** as input (although a simple ASCII -string should work) and returns output as Unicode. Do not pass encoded strings to it! -If your input is encoded, e.g. as UTF-8, it is your responsibility to decode -it. E.g.: - - input_file = codecs.open("some_file.txt", mode="r", encoding="utf-8") - text = input_file.read() - html = markdown.markdown(text, extensions) - -If you later want to write it to disk, you should encode it yourself: - - output_file = codecs.open("some_file.html", "w", encoding="utf-8") - output_file.write(html) - -More Options ------------- - -If you want to pass more options, you can create an instance of the ``Markdown`` -class yourself and then use ``convert()`` to generate HTML: - - import markdown - md = markdown.Markdown( - extensions=['footnotes'], - extension_configs= {'footnotes' : ('PLACE_MARKER','~~~~~~~~')}, - safe_mode=True, - output_format='html4' - ) - return md.convert(some_text) - -You should also use this method if you want to process multiple strings: - - md = markdown.Markdown() - html1 = md.convert(text1) - html2 = md.convert(text2) - -Working with Files ------------------- - -While the Markdown class is only intended to work with Unicode text, some -encoding/decoding is required for the command line features. These functions -and methods are only intended to fit the common use case. - -The ``Markdown`` class has the method ``convertFile`` which reads in a file and -writes out to a file-like-object: - - md = markdown.Markdown() - md.convertFile(input="in.txt", output="out.html", encoding="utf-8") - -The markdown module also includes a shortcut function ``markdownFromFile`` that -wraps the above method. - - markdown.markdownFromFile(input="in.txt", - output="out.html", - extensions=[], - encoding="utf-8", - safe=False) - -In either case, if the ``output`` keyword is passed a file name (i.e.: -``output="out.html"``), it will try to write to a file by that name. If -``output`` is passed a file-like-object (i.e. ``output=StringIO.StringIO()``), -it will attempt to write out to that object. Finally, if ``output`` is -set to ``None``, it will write to ``stdout``. - -Using Extensions ----------------- - -One of the parameters that you can pass is a list of Extensions. Extensions -must be available as python modules either within the ``markdown.extensions`` -package or on your PYTHONPATH with names starting with `mdx_`, followed by the -name of the extension. Thus, ``extensions=['footnotes']`` will first look for -the module ``markdown.extensions.footnotes``, then a module named -``mdx_footnotes``. See the documentation specific to the extension you are -using for help in specifying configuration settings for that extension. - -Note that some extensions may need their state reset between each call to -``convert``: - - html1 = md.convert(text1) - md.reset() - html2 = md.convert(text2) - -Safe Mode ---------- - -If you are using Markdown on a web system which will transform text provided -by untrusted users, you may want to use the "safe_mode" option which ensures -that the user's HTML tags are either replaced, removed or escaped. (They can -still create links using Markdown syntax.) - -* To replace HTML, set ``safe_mode="replace"`` (``safe_mode=True`` still works - for backward compatibility with older versions). The HTML will be replaced - with the text defined in ``markdown.HTML_REMOVED_TEXT`` which defaults to - ``[HTML_REMOVED]``. To replace the HTML with something else: - - markdown.HTML_REMOVED_TEXT = "--RAW HTML IS NOT ALLOWED--" - md = markdown.Markdown(safe_mode="replace") - - **Note**: You could edit the value of ``HTML_REMOVED_TEXT`` directly in - markdown/__init__.py but you will need to remember to do so every time you - upgrade to a newer version of Markdown. Therefore, this is not recommended. - -* To remove HTML, set ``safe_mode="remove"``. Any raw HTML will be completely - stripped from the text with no warning to the author. - -* To escape HTML, set ``safe_mode="escape"``. The HTML will be escaped and - included in the document. - -Output Formats --------------- - -If Markdown is outputing (X)HTML as part of a web page, most likely you will -want the output to match the (X)HTML version used by the rest of your page/site. -Currently, Markdown offers two output formats out of the box; "HTML4" and -"XHTML1" (the default) . Markdown will also accept the formats "HTML" and -"XHTML" which currently map to "HTML4" and "XHTML" respectively. However, -you should use the more explicit keys as the general keys may change in the -future if it makes sense at that time. The keys can either be lowercase or -uppercase. - -To set the output format do: - - html = markdown.markdown(text, output_format='html4') - -Or, when using the Markdown class: - - md = markdown.Markdown(output_format='html4') - html = md.convert(text) - -Note that the output format is only set once for the class and cannot be -specified each time ``convert()`` is called. If you really must change the -output format for the class, you can use the ``set_output_format`` method: - - md.set_output_format('xhtml1') diff --git a/docs/writing_extensions.txt b/docs/writing_extensions.txt deleted file mode 100644 index 3aad74a..0000000 --- a/docs/writing_extensions.txt +++ /dev/null @@ -1,594 +0,0 @@ -Writing Extensions for Python-Markdown -====================================== - -Overview --------- - -Python-Markdown includes an API for extension writers to plug their own -custom functionality and/or syntax into the parser. There are preprocessors -which allow you to alter the source before it is passed to the parser, -inline patterns which allow you to add, remove or override the syntax of -any inline elements, and postprocessors which allow munging of the -output of the parser before it is returned. If you really want to dive in, -there are also blockprocessors which are part of the core BlockParser. - -As the parser builds an [ElementTree][] object which is later rendered -as Unicode text, there are also some helpers provided to ease manipulation of -the tree. Each part of the API is discussed in its respective section below. -Additionaly, reading the source of some [[Available Extensions]] may be helpful. -For example, the [[Footnotes]] extension uses most of the features documented -here. - -* [Preprocessors][] -* [InlinePatterns][] -* [Treeprocessors][] -* [Postprocessors][] -* [BlockParser][] -* [Working with the ElementTree][] -* [Integrating your code into Markdown][] - * [extendMarkdown][] - * [OrderedDict][] - * [registerExtension][] - * [Config Settings][] - * [makeExtension][] - -<h3 id="preprocessors">Preprocessors</h3> - -Preprocessors munge the source text before it is passed into the Markdown -core. This is an excellent place to clean up bad syntax, extract things the -parser may otherwise choke on and perhaps even store it for later retrieval. - -Preprocessors should inherit from ``markdown.preprocessors.Preprocessor`` and -implement a ``run`` method with one argument ``lines``. The ``run`` method of -each Preprocessor will be passed the entire source text as a list of Unicode -strings. Each string will contain one line of text. The ``run`` method should -return either that list, or an altered list of Unicode strings. - -A pseudo example: - - class MyPreprocessor(markdown.preprocessors.Preprocessor): - def run(self, lines): - new_lines = [] - for line in lines: - m = MYREGEX.match(line) - if m: - # do stuff - else: - new_lines.append(line) - return new_lines - -<h3 id="inlinepatterns">Inline Patterns</h3> - -Inline Patterns implement the inline HTML element syntax for Markdown such as -``*emphasis*`` or ``[links](http://example.com)``. Pattern objects should be -instances of classes that inherit from ``markdown.inlinepatterns.Pattern`` or -one of its children. Each pattern object uses a single regular expression and -must have the following methods: - -* **``getCompiledRegExp()``**: - - Returns a compiled regular expression. - -* **``handleMatch(m)``**: - - Accepts a match object and returns an ElementTree element of a plain - Unicode string. - -Note that any regular expression returned by ``getCompiledRegExp`` must capture -the whole block. Therefore, they should all start with ``r'^(.*?)'`` and end -with ``r'(.*?)!'``. When using the default ``getCompiledRegExp()`` method -provided in the ``Pattern`` you can pass in a regular expression without that -and ``getCompiledRegExp`` will wrap your expression for you. This means that -the first group of your match will be ``m.group(2)`` as ``m.group(1)`` will -match everything before the pattern. - -For an example, consider this simplified emphasis pattern: - - class EmphasisPattern(markdown.inlinepatterns.Pattern): - def handleMatch(self, m): - el = markdown.etree.Element('em') - el.text = m.group(3) - return el - -As discussed in [Integrating Your Code Into Markdown][], an instance of this -class will need to be provided to Markdown. That instance would be created -like so: - - # an oversimplified regex - MYPATTERN = r'\*([^*]+)\*' - # pass in pattern and create instance - emphasis = EmphasisPattern(MYPATTERN) - -Actually it would not be necessary to create that pattern (and not just because -a more sophisticated emphasis pattern already exists in Markdown). The fact is, -that example pattern is not very DRY. A pattern for `**strong**` text would -be almost identical, with the exception that it would create a 'strong' element. -Therefore, Markdown provides a number of generic pattern classes that can -provide some common functionality. For example, both emphasis and strong are -implemented with separate instances of the ``SimpleTagPettern`` listed below. -Feel free to use or extend any of these Pattern classes. - -**Generic Pattern Classes** - -* **``SimpleTextPattern(pattern)``**: - - Returns simple text of ``group(2)`` of a ``pattern``. - -* **``SimpleTagPattern(pattern, tag)``**: - - Returns an element of type "`tag`" with a text attribute of ``group(3)`` - of a ``pattern``. ``tag`` should be a string of a HTML element (i.e.: 'em'). - -* **``SubstituteTagPattern(pattern, tag)``**: - - Returns an element of type "`tag`" with no children or text (i.e.: 'br'). - -There may be other Pattern classes in the Markdown source that you could extend -or use as well. Read through the source and see if there is anything you can -use. You might even get a few ideas for different approaches to your specific -situation. - -<h3 id="treeprocessors">Treeprocessors</h3> - -Treeprocessors manipulate an ElemenTree object after it has passed through the -core BlockParser. This is where additional manipulation of the tree takes -place. Additionally, the InlineProcessor is a Treeprocessor which steps through -the tree and runs the InlinePatterns on the text of each Element in the tree. - -A Treeprocessor should inherit from ``markdown.treeprocessors.Treeprocessor``, -over-ride the ``run`` method which takes one argument ``root`` (an Elementree -object) and returns either that root element or a modified root element. - -A pseudo example: - - class MyTreeprocessor(markdown.treeprocessors.Treeprocessor): - def run(self, root): - #do stuff - return my_modified_root - -For specifics on manipulating the ElementTree, see -[Working with the ElementTree][] below. - -<h3 id="postprocessors">Postprocessors</h3> - -Postprocessors manipulate the document after the ElementTree has been -serialized into a string. Postprocessors should be used to work with the -text just before output. - -A Postprocessor should inherit from ``markdown.postprocessors.Postprocessor`` -and over-ride the ``run`` method which takes one argument ``text`` and returns -a Unicode string. - -Postprocessors are run after the ElementTree has been serialized back into -Unicode text. For example, this may be an appropriate place to add a table of -contents to a document: - - class TocPostprocessor(markdown.postprocessors.Postprocessor): - def run(self, text): - return MYMARKERRE.sub(MyToc, text) - -<h3 id="blockparser">BlockParser</h3> - -Sometimes, pre/tree/postprocessors and Inline Patterns aren't going to do what -you need. Perhaps you want a new type of block type that needs to be integrated -into the core parsing. In such a situation, you can add/change/remove -functionality of the core ``BlockParser``. The BlockParser is composed of a -number of Blockproccessors. The BlockParser steps through each block of text -(split by blank lines) and passes each block to the appropriate Blockprocessor. -That Blockprocessor parses the block and adds it to the ElementTree. The -[[Definition Lists]] extension would be a good example of an extension that -adds/modifies Blockprocessors. - -A Blockprocessor should inherit from ``markdown.blockprocessors.BlockProcessor`` -and implement both the ``test`` and ``run`` methods. - -The ``test`` method is used by BlockParser to identify the type of block. -Therefore the ``test`` method must return a boolean value. If the test returns -``True``, then the BlockParser will call that Blockprocessor's ``run`` method. -If it returns ``False``, the BlockParser will move on to the next -BlockProcessor. - -The **``test``** method takes two arguments: - -* **``parent``**: The parent etree Element of the block. This can be useful as - the block may need to be treated differently if it is inside a list, for - example. - -* **``block``**: A string of the current block of text. The test may be a - simple string method (such as ``block.startswith(some_text)``) or a complex - regular expression. - -The **``run``** method takes two arguments: - -* **``parent``**: A pointer to the parent etree Element of the block. The run - method will most likely attach additional nodes to this parent. Note that - nothing is returned by the method. The Elementree object is altered in place. - -* **``blocks``**: A list of all remaining blocks of the document. Your run - method must remove (pop) the first block from the list (which it altered in - place - not returned) and parse that block. You may find that a block of text - legitimately contains multiple block types. Therefore, after processing the - first type, your processor can insert the remaining text into the beginning - of the ``blocks`` list for future parsing. - -Please be aware that a single block can span multiple text blocks. For example, -The official Markdown syntax rules state that a blank line does not end a -Code Block. If the next block of text is also indented, then it is part of -the previous block. Therefore, the BlockParser was specifically designed to -address these types of situations. If you notice the ``CodeBlockProcessor``, -in the core, you will note that it checks the last child of the ``parent``. -If the last child is a code block (``<pre><code>...</code></pre>``), then it -appends that block to the previous code block rather than creating a new -code block. - -Each BlockProcessor has the following utility methods available: - -* **``lastChild(parent)``**: - - Returns the last child of the given etree Element or ``None`` if it had no - children. - -* **``detab(text)``**: - - Removes one level of indent (four spaces by default) from the front of each - line of the given text string. - -* **``looseDetab(text, level)``**: - - Removes "level" levels of indent (defaults to 1) from the front of each line - of the given text string. However, this methods allows secondary lines to - not be indented as does some parts of the Markdown syntax. - -Each BlockProcessor also has a pointer to the containing BlockParser instance at -``self.parser``, which can be used to check or alter the state of the parser. -The BlockParser tracks it's state in a stack at ``parser.state``. The state -stack is an instance of the ``State`` class. - -**``State``** is a subclass of ``list`` and has the additional methods: - -* **``set(state)``**: - - Set a new state to string ``state``. The new state is appended to the end - of the stack. - -* **``reset()``**: - - Step back one step in the stack. The last state at the end is removed from - the stack. - -* **``isstate(state)``**: - - Test that the top (current) level of the stack is of the given string - ``state``. - -Note that to ensure that the state stack doesn't become corrupted, each time a -state is set for a block, that state *must* be reset when the parser finishes -parsing that block. - -An instance of the **``BlockParser``** is found at ``Markdown.parser``. -``BlockParser`` has the following methods: - -* **``parseDocument(lines)``**: - - Given a list of lines, an ElementTree object is returned. This should be - passed an entire document and is the only method the ``Markdown`` class - calls directly. - -* **``parseChunk(parent, text)``**: - - Parses a chunk of markdown text composed of multiple blocks and attaches - those blocks to the ``parent`` Element. The ``parent`` is altered in place - and nothing is returned. Extensions would most likely use this method for - block parsing. - -* **``parseBlocks(parent, blocks)``**: - - Parses a list of blocks of text and attaches those blocks to the ``parent`` - Element. The ``parent`` is altered in place and nothing is returned. This - method will generally only be used internally to recursively parse nested - blocks of text. - -While is is not recommended, an extension could subclass or completely replace -the ``BlockParser``. The new class would have to provide the same public API. -However, be aware that other extensions may expect the core parser provided -and will not work with such a drastically different parser. - -<h3 id="working_with_et">Working with the ElementTree</h3> - -As mentioned, the Markdown parser converts a source document to an -[ElementTree][] object before serializing that back to Unicode text. -Markdown has provided some helpers to ease that manipulation within the context -of the Markdown module. - -First, to get access to the ElementTree module import ElementTree from -``markdown`` rather than importing it directly. This will ensure you are using -the same version of ElementTree as markdown. The module is named ``etree`` -within Markdown. - - from markdown import etree - -``markdown.etree`` tries to import ElementTree from any known location, first -as a standard library module (from ``xml.etree`` in Python 2.5), then as a third -party package (``Elementree``). In each instance, ``cElementTree`` is tried -first, then ``ElementTree`` if the faster C implementation is not available on -your system. - -Sometimes you may want text inserted into an element to be parsed by -[InlinePatterns][]. In such a situation, simply insert the text as you normally -would and the text will be automatically run through the InlinePatterns. -However, if you do *not* want some text to be parsed by InlinePatterns, -then insert the text as an ``AtomicString``. - - some_element.text = markdown.AtomicString(some_text) - -Here's a basic example which creates an HTML table (note that the contents of -the second cell (``td2``) will be run through InlinePatterns latter): - - table = etree.Element("table") - table.set("cellpadding", "2") # Set cellpadding to 2 - tr = etree.SubElement(table, "tr") # Add child tr to table - td1 = etree.SubElement(tr, "td") # Add child td1 to tr - td1.text = markdown.AtomicString("Cell content") # Add plain text content - td2 = etree.SubElement(tr, "td") # Add second td to tr - td2.text = "*text* with **inline** formatting." # Add markup text - table.tail = "Text after table" # Add text after table - -You can also manipulate an existing tree. Consider the following example which -adds a ``class`` attribute to ``<a>`` elements: - - def set_link_class(self, element): - for child in element: - if child.tag == "a": - child.set("class", "myclass") #set the class attribute - set_link_class(child) # run recursively on children - -For more information about working with ElementTree see the ElementTree -[Documentation](http://effbot.org/zone/element-index.htm) -([Python Docs](http://docs.python.org/lib/module-xml.etree.ElementTree.html)). - -<h3 id="integrating_into_markdown">Integrating Your Code Into Markdown</h3> - -Once you have the various pieces of your extension built, you need to tell -Markdown about them and ensure that they are run in the proper sequence. -Markdown accepts a ``Extension`` instance for each extension. Therefore, you -will need to define a class that extends ``markdown.Extension`` and over-rides -the ``extendMarkdown`` method. Within this class you will manage configuration -options for your extension and attach the various processors and patterns to -the Markdown instance. - -It is important to note that the order of the various processors and patterns -matters. For example, if we replace ``http://...`` links with <a> elements, and -*then* try to deal with inline html, we will end up with a mess. Therefore, -the various types of processors and patterns are stored within an instance of -the Markdown class in [OrderedDict][]s. Your ``Extension`` class will need to -manipulate those OrderedDicts appropriately. You may insert instances of your -processors and patterns into the appropriate location in an OrderedDict, remove -a built-in instance, or replace a built-in instance with your own. - -<h4 id="extendmarkdown">extendMarkdown</h4> - -The ``extendMarkdown`` method of a ``markdown.Extension`` class accepts two -arguments: - -* **``md``**: - - A pointer to the instance of the Markdown class. You should use this to - access the [OrderedDict][]s of processors and patterns. They are found - under the following attributes: - - * ``md.preprocessors`` - * ``md.inlinePatterns`` - * ``md.parser.blockprocessors`` - * ``md.treepreprocessors`` - * ``md.postprocessors`` - - Some other things you may want to access in the markdown instance are: - - * ``md.htmlStash`` - * ``md.output_formats`` - * ``md.set_output_format()`` - * ``md.registerExtension()`` - -* **``md_globals``**: - - Contains all the various global variables within the markdown module. - -Of course, with access to those items, theoretically you have the option to -changing anything through various [monkey_patching][] techniques. However, you -should be aware that the various undocumented or private parts of markdown -may change without notice and your monkey_patches may break with a new release. -Therefore, what you really should be doing is inserting processors and patterns -into the markdown pipeline. Consider yourself warned. - -[monkey_patching]: http://en.wikipedia.org/wiki/Monkey_patch - -A simple example: - - class MyExtension(markdown.Extension): - def extendMarkdown(self, md, md_globals): - # Insert instance of 'mypattern' before 'references' pattern - md.inlinePatterns.add('mypattern', MyPattern(md), '<references') - -<h4 id="ordereddict">OrderedDict</h4> - -An OrderedDict is a dictionary like object that retains the order of it's -items. The items are ordered in the order in which they were appended to -the OrderedDict. However, an item can also be inserted into the OrderedDict -in a specific location in relation to the existing items. - -Think of OrderedDict as a combination of a list and a dictionary as it has -methods common to both. For example, you can get and set items using the -``od[key] = value`` syntax and the methods ``keys()``, ``values()``, and -``items()`` work as expected with the keys, values and items returned in the -proper order. At the same time, you can use ``insert()``, ``append()``, and -``index()`` as you would with a list. - -Generally speaking, within Markdown extensions you will be using the special -helper method ``add()`` to add additional items to an existing OrderedDict. - -The ``add()`` method accepts three arguments: - -* **``key``**: A string. The key is used for later reference to the item. - -* **``value``**: The object instance stored in this item. - -* **``location``**: Optional. The items location in relation to other items. - - Note that the location can consist of a few different values: - - * The special strings ``"_begin"`` and ``"_end"`` insert that item at the - beginning or end of the OrderedDict respectively. - - * A less-than sign (``<``) followed by an existing key (i.e.: - ``"<somekey"``) inserts that item before the existing key. - - * A greater-than sign (``>``) followed by an existing key (i.e.: - ``">somekey"``) inserts that item after the existing key. - -Consider the following example: - - >>> import markdown - >>> od = markdown.OrderedDict() - >>> od['one'] = 1 # The same as: od.add('one', 1, '_begin') - >>> od['three'] = 3 # The same as: od.add('three', 3, '>one') - >>> od['four'] = 4 # The same as: od.add('four', 4, '_end') - >>> od.items() - [("one", 1), ("three", 3), ("four", 4)] - -Note that when building an OrderedDict in order, the extra features of the -``add`` method offer no real value and are not necessary. However, when -manipulating an existing OrderedDict, ``add`` can be very helpful. So let's -insert another item into the OrderedDict. - - >>> od.add('two', 2, '>one') # Insert after 'one' - >>> od.values() - [1, 2, 3, 4] - -Now let's insert another item. - - >>> od.add('twohalf', 2.5, '<three') # Insert before 'three' - >>> od.keys() - ["one", "two", "twohalf", "three", "four"] - -Note that we also could have set the location of "twohalf" to be 'after two' -(i.e.: ``'>two'``). However, it's unlikely that you will have control over the -order in which extensions will be loaded, and this could affect the final -sorted order of an OrderedDict. For example, suppose an extension adding -'twohalf' in the above examples was loaded before a separate extension which -adds 'two'. You may need to take this into consideration when adding your -extension components to the various markdown OrderedDicts. - -Once an OrderedDict is created, the items are available via key: - - MyNode = od['somekey'] - -Therefore, to delete an existing item: - - del od['somekey'] - -To change the value of an existing item (leaving location unchanged): - - od['somekey'] = MyNewObject() - -To change the location of an existing item: - - t.link('somekey', '<otherkey') - -<h4 id="registerextension">registerExtension</h4> - -Some extensions may need to have their state reset between multiple runs of the -Markdown class. For example, consider the following use of the [[Footnotes]] -extension: - - md = markdown.Markdown(extensions=['footnotes']) - html1 = md.convert(text_with_footnote) - md.reset() - html2 = md.convert(text_without_footnote) - -Without calling ``reset``, the footnote definitions from the first document will -be inserted into the second document as they are still stored within the class -instance. Therefore the ``Extension`` class needs to define a ``reset`` method -that will reset the state of the extension (i.e.: ``self.footnotes = {}``). -However, as many extensions do not have a need for ``reset``, ``reset`` is only -called on extensions that are registered. - -To register an extension, call ``md.registerExtension`` from within your -``extendMarkdown`` method: - - - def extendMarkdown(self, md, md_globals): - md.registerExtension(self) - # insert processors and patterns here - -Then, each time ``reset`` is called on the Markdown instance, the ``reset`` -method of each registered extension will be called as well. You should also -note that ``reset`` will be called on each registered extension after it is -initialized the first time. Keep that in mind when over-riding the extension's -``reset`` method. - -<h4 id="configsettings">Config Settings</h4> - -If an extension uses any parameters that the user may want to change, -those parameters should be stored in ``self.config`` of your -``markdown.Extension`` class in the following format: - - self.config = {parameter_1_name : [value1, description1], - parameter_2_name : [value2, description2] } - -When stored this way the config parameters can be over-ridden from the -command line or at the time Markdown is initiated: - - markdown.py -x myextension(SOME_PARAM=2) inputfile.txt > output.txt - -Note that parameters should always be assumed to be set to string -values, and should be converted at run time. For example: - - i = int(self.getConfig("SOME_PARAM")) - -<h4 id="makeextension">makeExtension</h4> - -Each extension should ideally be placed in its own module starting -with the ``mdx_`` prefix (e.g. ``mdx_footnotes.py``). The module must -provide a module-level function called ``makeExtension`` that takes -an optional parameter consisting of a dictionary of configuration over-rides -and returns an instance of the extension. An example from the footnote -extension: - - def makeExtension(configs=None) : - return FootnoteExtension(configs=configs) - -By following the above example, when Markdown is passed the name of your -extension as a string (i.e.: ``'footnotes'``), it will automatically import -the module and call the ``makeExtension`` function initiating your extension. - -You may have noted that the extensions packaged with Python-Markdown do not -use the ``mdx_`` prefix in their module names. This is because they are all -part of the ``markdown.extensions`` package. Markdown will first try to import -from ``markdown.extensions.extname`` and upon failure, ``mdx_extname``. If both -fail, Markdown will continue without the extension. - -However, Markdown will also accept an already existing instance of an extension. -For example: - - import markdown - import myextension - configs = {...} - myext = myextension.MyExtension(configs=configs) - md = markdown.Markdown(extensions=[myext]) - -This is useful if you need to implement a large number of extensions with more -than one residing in a module. - -[Preprocessors]: #preprocessors -[InlinePatterns]: #inlinepatterns -[Treeprocessors]: #treeprocessors -[Postprocessors]: #postprocessors -[BlockParser]: #blockparser -[Working with the ElementTree]: #working_with_et -[Integrating your code into Markdown]: #integrating_into_markdown -[extendMarkdown]: #extendmarkdown -[OrderedDict]: #ordereddict -[registerExtension]: #registerextension -[Config Settings]: #configsettings -[makeExtension]: #makeextension -[ElementTree]: http://effbot.org/zone/element-index.htm |
