Introduction to RST and Sphinx for Textbook Contributors

What is RST?

RST stands for ReStructured Text. It is the standard markup language used for documenting python packages. Sphinx is the Python package that generates an html website from RST files, and it is what we are using to generate this site. To read more about why we chose RST over markdown or Latex, read the following section, Why RST?.

Why RST?

Out of the box, RST supports globally-defined styles, figure numbering and referencing, Latex function rendering, image display customization and more. Furthermore, restructured text was already the language of choice for the AguaClara ecosystem’s documentation.

Setting up RST for Development

There are two ways to quickly view an RST file. The first is using an Atom plugin that renders the view alongside the source code. This is a good initial test to make sure the RST is proper RST and looks mostly correct. However, some functionality, such as any extensions provided by Sphinx won’t run in the preview. In order to see the final html that will display on the website, you’ll need to use the second method, running sphinx locally to fully generate the html code. Once you are satisfied with your work and want to push it to the textbook, you’ll need to incorporate it to the master branch. To do so, refer to Publishing online.

Installing the Atom Plugins

If you are using the Atom IDE to write RST, you can use the rst-preview-pandoc plugin to auto-generate a live RST preview within atom (much like the markdown-preview-plus preview page.) To get rst-preview working, you’ll need to install language-restructuredtext via atom and Pandoc via your command line ($pip install pandoc). If everything worked, you can use ctrl + shift + e to toggle a display window for the live-updated RST preview.

Building RST Locally with Sphinx

We use Sphinx to build RST locally and remotely. Follow these steps to get Sphinx and run it locally:

  1. Get Python 3.5+ and pipenv ($pip install pipenv)

  2. Install all dependencies and setup a virtual environment with $pipenv install

  3. Whenever you want to issue commands, start your virtual environment with $pipenv shell - exit it with $exit

  4. Generate all the html by navigating in the command line to the source directory /Textbook and creating the build in that directory with the command line $make html.

  5. View the html generated in the /Textbook/_build directory by copying the full file path of /Textbook/_build/html/index.html and pasting it into your browser.

Note

Regarding 1. the master branch for the package implementing disqus in sphinx is broken, which is why we use a non-standard pip/online installation. If you already have the incorrect sphinx-disqus version installed, uninstall it with pip uninstall sphinxcontrib-disqus before installing the functioning version.

Building PDF Locally

To check errors with the PDF generation more quickly, it may be worthwhile to setup your local machine to build the PDF.

  • install MikTeX

  • install Perl

  • run the command make latexpdf

  • the PDF will be created in the latex folder inside the _build folder

Publishing Online

We use Travis to ensure this site will always contain functional builds. To publish online, you need to:

  1. Always test your build by first :ref:` building RST locally <heading_building_rst_locally>`, and then following the testing online instructions. Once you like how your build looks, follow the steps below to introduce it into the master branch.

  2. Submit a pull request to master. You’ll need to ask for someone else to review your work at this stage- “request reviewers”. Every pull request must be reviewed by at least one other person.

  3. Travis will build the site using Sphinx, and if there aren’t any errors, Travis will report success to GitHub on the “checks” part of the pull request.

  4. All your requested reviewers must now approve and comment on your commit before the merge is allowed.

  5. Once the PR passes Travis and is approved by another author, feel free to “merge to master.”

  6. To release the master branch, (build the html, pdf, and latex, and upload the pdf to Pages) you’ll need to publish a GitHub release. Include a semver version number as the tag (under “Tag: Choose or create”), and a brief description of the updates under “Release Title”. Finally, for the description, detail the changes as much as you see fit and when ready, hit “Publish release”. Example:

    • Tag name: 0.1.5

    • Release title: Filtration section maintenance

    • Description: Added filter code from aguaclara 0.2.6. Also updated all broken external links.

  7. Travis will rebuild the site and push the html to Pages, and the PDF and LaTeX to GitHub Releases under the tag name.

Important

If your changes to the master branch aren’t pushing to gh-pages, then check the status of the Travis build here.

Testing Online

To test exactly what will be published, we have a test branch. The test branch is built by Travis and contains all the processed html that Travis produces in _build/html. This branch is populated when ANY COMMIT IS PUSHED. Meaning, the last commit to be pushed, if it passes the Travis tests, will be built and the output will be placed in the test branch. Also, if the PDF=True environment variable is triggered for a Travis build, the PDF will also be generated and placed in the test branch. To do this, use the “Trigger Build” option in Travis and put:

script:
    - PDF=True

The website output is viewable here.

Sharing Test Output

if you want to share what your latest branch developments look like without having whoever is viewing it actually have to build it, you can push a commit, and find the rawgit URL with this site by entering the URL of the git file within the test branch that you’d like to share. Furthermore, if you want to point to the commit so that even if someone else pushes, the URL will still point to the code you intend it to, make sure to include the commit SHA within the rawgit URL like so: https://rawgit.com/AguaClara/Textbook/e5693e0485702b95e11d4d6bdf76d07c42fdbf99/html/index.html. That link will never change where it is pointing. To share the PDF output, follow the testing online instructions to build the PDF, and point to the commit with the PDF. Happy testing!

Brief Best Practices

When writing RST, there are often many ways to write the same thing. Almost always, the way with the fewest number of characters is the best way. Ideally, never copy and paste.

How do I write RST?

RST is friendly to learn and powerful. There are many useful cheatsheets, like this one. When you start writing RST, look at the cheat sheets all the time. Use ctrl-f all the time to find whatever you need.

Things not covered in most cheat sheets which are of critical importance:

  • A document is referred to by its title, as defined between the ***** signs at the top of the page, NOT the filename. So it is critical to have a title.

  • Anything else you’d like to add for the future…

Example to Start From

This file is written in RST. You can start there! Just click on “View page source” at the top of the page.

Converting Markdown to RST

Ideally, use pandoc to do the conversion in the command line: pandoc --from=markdown --to=rst --output=my_file.rst my_file.md. Raw html will not be converted (because it is not actually markdown), and tables are converted poorly. You’ll need to carefully review any page converted with pandoc.