ClosedLess/clog
ClosedLess/clog
2
1
Fork 0
Code Issues 1 Pull Requests 1 Projects Releases 2 Wiki Activity

Added documentation TESTING.md

Browse Source 
Added new documentation regarding the process of conducting new unit
tests for the project. Documentation covers what libraries are used in
test-driven development, along with how they are used.

Documentation covers how to create new unit tests according to a defined
standard, as well as using pytest fixtures.
This commit is contained in:
Ethan Smith-Coss 2022-05-16 17:17:07 +01:00
parent 8ca27b5f81
commit 2339aabeb1
Signed by: TheOnePath
GPG Key ID: 4E7D436CE1A0BAF1
1 changed files with 268 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

268
testing/TESTING.md Normal file
View File

@ -0,0 +1,268 @@
# Testing CLog
Clog makes use of a few development libraries for automated testing and unit
testing. Those libraries (currently) are primaily pytest, my[py], flake8,
and tox.
Here is a brief understanding of each library for how it's used:
* Pytest – automated unit testing of each function/method/class structure in
the code base. Pytest is also used with the `coverage` extension to indicate
how much of the code base is touched by the unit tests.
* Mypy – using static type-checking to ensure that type annotations are as
accurate as possible. Also identify bad practices that may appear when writing
statements/code blocks before refactorisation, and raise errors when mypy is
ran.
* Flake8 – to keep written code as close to the defined PEP8 standard, this
library is used to enforce style guidance. However, not all styling is
conventient and can be annoying to deal with when own standards are used. For
this reason, some flake8 rules are ignored in configuration files.
* tox – ensure that the code base is backwards compatible with earlier versions
(currently 3.6 or later) of Python. If any version is listed as failing, then
modules should be reconsidered to support older, compatible, syntax up to the
latest version of Python.
With these, Clog can ensure that code behaves the way it is expected, best
practices are adopted and forced were appropriate, and it's backwards
compatible with earlier versions and not just the latest release.
## Test-Driven Development
If you wish to test the library against the defined systems, then a few
prerequisites are required in order to setup the environment correctly. The
basic steps are as followed: clone project – setup tools with pip – run tools.
### Setting Up the Environment
It's usually always simpler and easier to clone the repository if you intend to
use a library, either for building from scratch or to develop on the library.
For this, you will require Git, and to run the following commands:
```
git clone https://git.closedless.xyz/ClosedLess/clog.git
cd clog
```
Afterwards, you can run pip to install and setup the required tools and
environment settings in editable mode.
#### For Linux
```
pip3 install -e .
pip3 install -r requirements-dev.txt
```
#### For Windows
```
pip install -e .
pip install -r requirements-dev.txt
```
This will install all the required packages necessarily for carrying out
test-driven development. Those are the following packages (with versions):
* `flake8==4.0.1`
* `tox==3.25.0`
* `pytest==7.1.2`
* `pytest-cov==3.0.0`
* `mypy==0.950`
### Running the Tools and Expectations
Assuming that `pip` did not report any errors when installing the required
packages, all the tools required should be available from the console window.
From either a terminal, Command-Prompt, or PowerShell, the following commands
can be ran with the defined inputs:
```
mypy clog # 'mypy .' will cause mypy to run on the entire workspace
flake8
pytest
tox
```
Each of the above lines are commands to run each tool individually. Mypy is the
only command which requires a file, or directory, to point to when it runs.
Every tool is expected to return no errors or warnings once it has finished
executing. If mypy is ran using `mypy .`, then some errors and/or warning may
be shown due to static type-checking of modules found in the `testing/` and
root directory spaces. These files are not important for ensuring type-checking
and may be ignored entirely (this is why `clog` is given as the directory).
Flake8 is a bit too strict regarding some guidance rules. For this reason,
certain rules are defined to be ignored by flake8 when analysing, and
additional rules are ignored for specific files. For example, the rules W503
and W504 define where a break should occur when dealing with multiline
operators. According to PEP8, a line-break should be placed before the operator
and not afterwards to help readability. With this, the W504 rules is ignored:
```py
# best practice
if (my_var == 1
and other_var is not None): # W503 & PEP8 standard
# do something...
# bad practice
if (my_var == 1 and
other_var is not None): # W504 & non-PEP8 standard
# do something...
```
Other rules may be ignored from the configuration, and testing modules are
completely ignored using the descriptor tag at the start of the module. To
find out which rules are ignored, consult the `setup.cfg` configuration file
in the root directory of the project.
If tox is producing errors, check that all Python interpreters are installed
to the local machine, and whose path to executable is located in the Path
environment variable. In the case that not all interpreters are available, it
would be reasonable to ignore the errors produced by tox and ensure that tox
passes for at least the installed interpreters. Note that if development to
the project is to be conducted, then **all** interpreters required by tox are
to be installed on the local machine to guarantee compatibility of code with
earlier versions of Python. (CAUTION: SUBMITTED PRs WHICH **DO NOT** PASS ALL
TOX TESTS **WILL BE REJECTED** UNTIL CORRECTED.)
Pytest is expected to pass 80 tests and skip 1 (v0.1.0). The tests are designed
to ensure robustness of each module and its contents within the code base. Some
test parameters will check that certain conditions fail purposefully to
indicate that erroneous arguments are handled accordingly. If any tests are
identified as failing on a local machine, an Issue may be raised, subject to
the code base not being modified. In the event that code has been modified,
this must be explicitly stated, and any new functionality added **should** have
their own unit tests defined appropriately. (CAUTION: SUBMITTED PRs WHICH
**DO NOT** PASS ALL PYTEST TESTS **WILL BE REJECTED** UNTIL CORRECTED.)
## Performing Pytest Unit Testing
If you wish to add new features to the project and submit a PR with a new
enhancement, then it's important to create unit tests for the additional
features that have been added. This section will define how to go about
defining new unit tests.
### Naming Testing Modules
If a new module in the code base has been added with new features, then a new
module should also be defined in the `testing/` directory. The name of this
module should start with the prefix `test_` followed by the suffix
`[new module name].py`. Therefore, if a new module created was `helloworld.py`,
then the corresponding test module should be named `test_helloworld.py`, and
all unit tests related to `helloworld.py` should be placed in this module.
If a new feature was added to an existing module in the code base, then the new
unit tests for this may be added to the respectful, existing test module.
### Defining Pytest Unit Tests
Unit tests are defined using a verbose function header, and potentially a class
body to simply group different tests together. For a function to become a
pytest unit test, the prefix `test` must be present on a function header. For
this reason, the adopted prefix is `test_` followed by a verbose summary of
the unit test, using the `snake_case` naming convention.
Here is a basic example of how to implement a valid unit test for Clog, which
is taken in part from existing modules in the code base.
**`helloworld.py`**
```py
class Foo:
"""A dummy Singleton class"""
__instance__ = None
def __new__(cls):
"""Construct a new instance of the class,
or return an existing instance."""
# check if an instance does not already exist
if cls.__instance__ is None:
# make a new Singleton instance
cls.__instance__ = super(Foo, cls).__new__(cls)
return cls.__instance__ # always return the Singleton instance
```
**`test_helloworld.py`**
```py
import pytest
from helloworld import Foo
bar = Foo() # instantiate a new global Singleton instance
class TestFoo:
"""class represents grouping of unit tests related to Foo."""
# using pytest `mark.parametrize`, multiple values can be passed
# to the function, and if one assertion fails, other values can
# still be tested by pytest.
@pytest.mark.parametrize(
"value, expected", [
(Foo(), Foo()),
(bar, bar),
(Foo(), bar),
(bar, Foo())
]
)
def test_new_foo_instance(self, value, expected):
"""Verfiy the identity of all `value`s against what is `expected`.
If any assertion fails, then the class is not a Singleton."""
assert value is expected # assert identity is the same
```
Note, docstrings and comments are not mandatory in testing modules. Given the
verboseness of the function header, there's a clear indication of what the
test is checking. Comments may be adopted if function bodies contain setup code
for a test – most tests will simply contain an `assert` statement.
### Creating Pytest Fixtures
You may want to construct your own fixture for testing to help with assisting.
Clog already defines a fixture called `capture_stdpipe`, which uses
`monkeypatch` to interrupt a call to write to a standard PIPE, and store the
write value to a temporary buffer, which can be accessed when required. All
fixtures should be defined in the `conftest.py` module, found in the root
directory of the project.
#### Using the `capture_stdpipe` fixture
If you are testing functionality regarding standard PIPE streams, this fixture
will be very handy for ensuring integrity of the message's body through a
write process to a PIPE, with respect to any formatting. Both `STDOUT` and
`STDERR` can be accessed via this fixture.
**`conftest.py`**
```py
import sys
import pytest
@pytest.fixture
def capture_stdpipe(monkeypatch):
stream_buf = {"stdout": "", "stderr": "", "writes": 0}
def mimic_stdout(chars):
stream_buf['stdout'] += chars
stream_buf['writes'] += 1
def mimic_stderr(chars):
stream_buf['stderr'] += chars
stream_buf['writes'] += 1
monkeypatch.setattr(sys.stdout, 'write', mimic_stdout)
monkeypatch.setattr(sys.stderr, 'write', mimic_stderr)
return stream_buf
```
The stream buffer can hold one single write to both standard PIPE streams
before data is then overwritten and lost. To access the data from the buffer in
a unit test function, the fixture can be passed as a function parameter.
**`helloworld.py`**
```py
def check_that_the_fixture_captures_output(capture_stdpipe):
# print will go to STDOUT by default
print("Hello World!")
# access the buffer and compare
assert capture_stdpipe['stdout'] == "Hello World!"
```
New fixtures defined for unit testing can be used in a similar fashion to how
`capture_stdpipe` is used. For more information regarding how to use fixtures
and further pytest features, consult the official documentations.
Author: Ethan Smith-Coss.
Copyright (c) 2022-23