A pyproject.toml Developer’s Cheat Sheet | by Ricardo Mendes | Jul, 2022

Ideas and notes on the way to arrange clear pyproject.toml recordsdata

Background picture by Klára Vernarcová on Unsplash

Round three years in the past, I wrote a information on the way to design a clean Python package structure that depends on the Setuptools construct system with setup.py and setup.cfg recordsdata. The Python ecosystem constantly evolves, and, these days, builders have extra flexibility to decide on a construct system that higher suits their wants — e.g., Bento, Flit, or Poetry. PEP 517 proposes a normal method to outline various construct programs for Python tasks, relying on pyproject.toml configuration recordsdata launched by PEP 518.

The current piece goals to explain the way to use pyproject.toml and Flit to get comparable outcomes as the previous package deal setup information.

I’m utilizing Flit for 2 completely different causes:

  1. Exhibit the way to use one other construct system than Setuptools
  2. It’s the solely PEP 517 compliant constructing instrument that permits editable installs on the time of this writing in July 2022— which may be very helpful for builders — with out additional configuration. I imagine different instruments will seemingly present such an possibility shortly, although.

The advised package deal construction is offered beneath:

project-root
├──src/
└──package_name/
└──__init__.py
├──assessments/
└──package_name/
├──.gitignore
├──LICENSE
├──pyproject.toml
└──README.md

You’ll discover that pyproject.toml replaces setup.py and setup.cgf. The next code snippet brings an preliminary but working model of the file:

pyproject.toml cheat sheet — preliminary model

Regardless of the TOML format, the content material is similar to setup.py, aside from the [build-system] part (or desk, as such key/worth pairs are referred to as within the TOML specification). It’s used to specify the construct system for the mission — on this case, Flit.

The pyproject.toml file is dealt with by the construct system, and its content material could differ relying on the system you select. For instance, Poetry requires the authors array components as title <e-mail> strings (source), whereas Flit requires a listing of tables with title and e-mail keys (source). Please all the time discuss with the construct system documentation when writing your pyproject.toml recordsdata to keep away from syntax points. You may simply discover such docs by googling <construct system> pyproject.toml (e.g., flit pyproject.toml or poetry pyproject.toml).

Within the coming sections, you will note the way to add nice-to-have options akin to growth dependency administration, automated assessments, and linting to the mission. A completely working instance is offered on the companion repository:

Earlier than operating flit instructions, it’s essential to set up Flit from PyPI. Run pip set up flit~=3.7.1 (exchange 3.7.1 together with your most popular model) to take action.

Growth dependencies

Flit’s pyproject.toml documentation describes the way to add non-obligatory dependencies to a given mission. Elective, on this context, means dependencies not required by the package deal to do its core job however for testing or documentation functions, as an illustration.

The [project.optional-dependencies] desk comprises lists of packages wanted for each non-obligatory function. (source)

The code beneath is used within the companion mission to enumerate the event dependencies:

[project.optional-dependencies]
dev = [
"pylint ~=2.14.0",
"pytest-cov ~=3.0.0",
"toml ~=0.10.2",
"yapf ~=0.32.0",
]

To put in them, simply run flit set up --deps=develop -s (yow will discover extra concerning the -s flag within the Editable installs part beneath).

Code formatting

YAPF is an instance of a code formatting instrument that already supports pyproject.toml, offering a substitute for setup.cfg. The next code snippet exhibits the way to use it:

[tool.yapf]
blank_line_before_nested_class_or_def = true
column_limit = 88

Please discuss with your most popular formatter’s documentation to verify in addition they help pyproject.toml.

Automated assessments

Assist for pyproject.toml was introduced in Pytest 6.0. Since then, one can use the [tool.pytest.ini_options] desk to arrange Pytest for a given mission. Within the companion repository, I used the beneath strains to set the default choices:

[tool.pytest.ini_options]
addopts = "--cov --cov-report html --cov-report term-missing --cov-fail-under 95"

And

[tool.coverage.run]
supply = ["src"]

to find out which listing pytest-cov ought to think about when calculating check protection.

Then, simply run pytest from the mission root folder.

Linting

Flake8 is often my first selection for linting, given its simplicity. Sadly, it nonetheless didn’t help pyproject.toml for configuration once I wrote this weblog put up, so I made a decision for Pylint:

[tool.pylint]
max-line-length = 88
disable = [
"C0103", # (invalid-name)
"C0114", # (missing-module-docstring)
"C0115", # (missing-class-docstring)
"C0116", # (missing-function-docstring)
"R0903", # (too-few-public-methods)
"R0913", # (too-many-arguments)
"W0105", # (pointless-string-statement)
]

Once more, discuss with your most popular linter’s documentation to make sure they help pyproject.toml.

Editable installs

Builders utilizing legacy builds depend on pip set up -e . (aka editable set up or growth mode) to imitate putting in packages from the mission’s supply code and check modifications on the fly. Flit gives the identical function via flit set up -s. You will discover extra about flit set up within the official docs.

Sidenote: I’ve observed that pytest-cov requires the mission’s supply code to be put in with the -s flag in an effort to report protection accordingly, which implies the flag must be used even when operating assessments on CI pipelines.

Constructing with Flit is simple, and yow will discover the obtainable choices within the official docs. Let’s do an area construct and set up to see what it appears to be like like.

  1. Take away packages from earlier installs: pip unistall -y pyproject-toml-cheat-sheet stringcase.
  2. Construct a wheel and a sdist (tarball) from the supply: flit construct. When the construct finishes, it’s best to see a message just like Buit wheel: dist/pyproject_toml_cheat_sheet-1.0.0-py3-none-any.whl within the console.
  3. Set up utilizing pip to verify the ensuing package deal is pip-compatible: pip set up dist/pyproject_toml_cheat_sheet-1.0.0-py3-none-any.whl.

Now, let’s name the pyproject_cheat_sheet.StringFormatter.format_to_snakecase methodology utilizing the Python Interactive Shell:

python
>>> from pyproject_cheat_sheet import StringFormatter
>>> print(StringFormatter.format_to_snakecase('FooBar'))
foo_bar
>>> exit()

As you may see, foo_bar is the output for StringFormatter.format_to_snakecase('FooBar'), which implies the package deal works as anticipated.

Publishing can be quite simple. Please discuss with the Flit documentation for additional particulars.

We’re good with Flit, however what if you happen to nonetheless need to use Setuptools for any purpose, akin to avoiding breaking modifications to present construct pipelines? No worries… it’s only a matter of configuring the construct system via pyproject.toml, beginning with one thing like:

[build-system]
build-backend = "setuptools.build_meta"
requires = [
"setuptools ~=63.2.0",
"wheel ~=0.37.1",
]

You then most likely need to test Setuptools-specific configurations, and there’s loads of data in their docs.

Including options and dependencies to the code base makes it extra advanced, however we are able to positively do it moderately. Some key advantages of preserving code as clear as doable are onboarding new staff members and selling long-term upkeep with much less burden for concerned people. I actually imagine it begins with how we manage our packages, which is why I share such ideas.

I hope you loved it!

More Posts