New Features
- New pytest.mark.skip mark, which unconditionally skips marked tests.
- --doctest-glob may now be passed multiple times in the command-line.
- New -rp and -rP reporting options give the summary and full output of passing
tests, respectively.
- pytest.mark.xfail now has a strict option which makes XPASS tests to fail the
test suite, defaulting to False. There’s also a xfail_strict ini option that
can be used to configure it project-wise.
- Parser.addini now supports options of type bool.
- New ALLOW_BYTES doctest option strips b prefixes from byte strings in doctest
output (similar to ALLOW_UNICODE).
- give a hint on KeyboardInterrupt to use the –fulltrace option to show the
errors
- catch IndexError exceptions when getting exception source location. This
fixes pytest internal error for dynamically generated code (fixtures and
tests) where source lines are fake by intention
Changes
- Important: py.code has been merged into the pytest repository as
pytest._code. This decision was made because py.code had very few uses
outside pytest and the fact that it was in a different repository made it
difficult to fix bugs on its code in a timely manner. The team hopes with
this to be able to better refactor out and improve that code. This change
shouldn’t affect users, but it is useful to let users aware if they encounter
any strange behavior.
Keep in mind that the code for pytest._code is private and experimental, so
you definitely should not import it explicitly!
Please note that the original py.code is still available in pylib.
- pytest_enter_pdb now optionally receives the pytest config object.
- Removed code and documentation for Python 2.5 or lower versions, including
removal of the obsolete _pytest.assertion.oldinterpret module.
- Comparisons now always show up in full when CI or BUILD_NUMBER is found in
the environment, even when -vv isn’t used.
- --lf and --ff now support long names: --last-failed and --failed-first
respectively.
- Added expected exceptions to pytest.raises fail message
- Collection only displays progress (“collecting X items”) when in a terminal.
This avoids cluttering the output when using --color=yes to obtain colors in
CI integrations systems
Bug Fixes
- The -s and -c options should now work under xdist; Config.fromdictargs now
represents its input much more faithfully.
- support Python 3.5’s @ operator in assertion rewriting.
- Fix formatting utf-8 explanation messages.
- Fix traceback style docs to describe all of the available options
(auto/long/short/line/native/no), with auto being the default since v2.6.
- junit record_xml_property doesn’t allow multiple records with same name.
- Warn if using xmlconfig.file, self.loadZCML is the preferred option.
- Avoid false reports by suffixing an opening parenthesis on all methods.
- Add decorators from zope.interface and zope.component.
Due to the bugfix we can also remove the D001 suppression.
- The ``no_xvfb``-marker is now registered automatically so pytest doesn't fail
when run with ``--strict``.
- The ``xvfb`` fixture is now session-scoped.
Before this change, adding a new logging message involving logging e.g. the
default duckduckgo setting value failed.
Now we basically use a black- instead of a whitelist and only fail if we get a
load status message for duckduckgo.
- PropertyMock is now accessible from mocker.
- Fix regression using one of the assert_* methods in patched functions which
receive a parameter named method.
- Remove duplication from the environment section
- Dropped support for Python 3.2
- Indicated setup and teardown in report
- Fixed colour of errors in report
- Fix version parsing of py.test to work with py.test release candidates
- More general handling of the health check problem where things could fail
because of a cache miss - now one "free" example is generated before the start
of the health check run.
- Fix a bug where Hypothesis would error when running on Python 2.7.3 or earlier
because it was trying to pass a bytearray object to struct.unpack (which is
only supported since 2.7.4).
Trying to get the device location while running the tests can trigger all kind
of funny effects.
Since we can't easily mock the GPS responses, we only run those on the CI where
we at least have some predictable setup.
Fixes#1297.
We used to use flake8-pep257 because docstrings claims no codes are ignorable
(which is wrong). However, that doesn't work with pydocstyle, so we had a
separate pydocstyle environment (and flake8-pep257 to check tests with relaxed
rules).
Now we only use flake8-docstrings + pydocstyle (which it switches to if it's
available, even though it still depends on pep257), and drop the pydocstyle tox
env.
As soon as flake8-docstrings is updated to drop the (unneeded) pep257
dependency, we can drop it as well.
This means we can now run things like running pylint --version without having
to set PYTHONPATH correctly now.
When using skip_install=true, the plugins wouldn't work as they need to import
qutebrowser.
We have to keep setting PYTHONPATH in run_pylint_on_tests.py, otherwise we get
this error I don't quite understand:
F: 1, 0: error while code parsing: Unable to load file
'/home/florian/proj/qutebrowser/git/__init__.py' ([Errno 2] No such
file or directory: '...') (parse-error)
(3.0.2: performance fix)
------------------
3.0.1 - 2016-02-18
------------------
* Fix a case where it was possible to trigger an "Unreachable" assertion when
running certain flaky stateful tests.
* Improve shrinking of large stateful tests by eliminating a case where it was
hard to delete early steps.
* Improve efficiency of drawing binary(min_size=n, max_size=n) significantly by
provide a custom implementation for fixed size blocks that can bypass a lot
of machinery.
* Set default home directory based on the current working directory at the
point Hypothesis is imported, not whenever the function first happens to be
called.
------------------
3.0.0 - 2016-02-17
------------------
Codename: This really should have been 2.1.
Externally this looks like a very small release. It has one small breaking change
that probably doesn't affect anyone at all (some behaviour that never really worked
correctly is now outright forbidden) but necessitated a major version bump and one
visible new feature.
Internally this is a complete rewrite. Almost nothing other than the public API is
the same.
New features:
* Addition of data() strategy which allows you to draw arbitrary data interactively
within the test.
* New "exploded" database format which allows you to more easily check the example
database into a source repository while supporting merging.
* Better management of how examples are saved in the database.
* Health checks will now raise as errors when they fail. It was too easy to have
the warnings be swallowed entirely.
New limitations:
* choices and streaming strategies may no longer be used with find(). Neither may
data() (this is the change that necessitated a major version bump).
Feature removal:
* The ForkingTestCase executor has gone away. It may return in some more working
form at a later date.
Performance improvements:
* A new model which allows flatmap, composite strategies and stateful testing to
perform *much* better. They should also be more reliable.
* Filtering may in some circumstances have improved significantly. This will
help especially in cases where you have lots of values with individual filters
on them, such as lists(x.filter(...)).
* Modest performance improvements to the general test runner by avoiding expensive
operations
In general your tests should have got faster. If they've instead got significantly
slower, I'm interested in hearing about it.
Data distribution:
The data distribution should have changed significantly. This may uncover bugs the
previous version missed. It may also miss bugs the previous version could have
uncovered. Hypothesis is now producing less strongly correlated data than it used
to, but the correlations are extended over more of the structure.
Shrinking:
Shrinking quality should have improved. In particular Hypothesis can now perform
simultaneous shrinking of separate examples within a single test (previously it
was only able to do this for elements of a single collection). In some cases
performance will have improved, in some cases it will have got worse but generally
shouldn't have by much.
This is based on HTML files with a global YAML comment, currently with "target"
as the only allowed key.
The tests then do this:
- Open a HTML file in data/hints/html
- Start hinting
- Make sure only one hint is visible
- Follow it, and make sure the page mentioned in "target:" is reached
Some ideas for the future:
- A "scroll" key, to scroll before hinting
- A "zoom" key, to zoom
- Multiple hints via a list
- Checking position of hints?
- A mode to manually check the pages (to check hint positions)
Issue #1214
Now uses a sensible filename for data: links instead of the whole base64
content. For PDF.js, it even uses the correct pdf filename.
TODO: Produces "QPainter:🔚 Painter ended with 2 saved states" while
running the tests here (Arch Linux):
CPython: 3.5.1
Qt: 5.5.1, runtime: 5.5.1
PyQt: 5.5.1
- Fixed Jython compatibility
- Fixed HTML formatter output with leading empty lines
- Added a mapping table for LaTeX encodings and added utf8
- Fixed image formatter font searching on Macs
- Fixed deepcopy-ing of Token instances
- Fixed Julia string interpolation
- Fixed statefulness of HttpLexer between get_tokens calls
- Many smaller fixes to various lexers
