Development process
===================

This page described the process that should be followed for contributing to Twisted.


If you're going to be doing development on Twisted itself,
or if you want to take advantage of bleeding-edge features (or bug fixes) that are not yet available in a numbered release,
you'll probably want to check out a tree from the Twisted Git repository.

The `trunk` is the main branch and is where all current development takes place.


Checkout and initial dev virtual environment
--------------------------------------------

Twisted uses Git to track code changes.
Git tutorials can be found elsewhere,
see in particular `Git and GitHub learning resources <https://help.github.com/articles/good-resources-for-learning-git-and-github/>`_ :

.. code-block:: console

    $ git clone https://github.com/twisted/twisted twisted
    $ python3 -m venv ./venv
    $ . venv/bin/activate
    $ pip install -e .[dev]
    $ pip install pre-commit

Please note that all subsequent commands in this document assume that your Twisted virtual environment is active.  (However, we will not assume you've put it in `./venv`; feel free to use whatever Python environment manager you are comfortable with.)

The output of ``git blame`` `will be better <https://github.com/psf/black#migrating-your-code-style-without-ruining-git-blame>`_ if you configure it to use our ignore file:

.. code-block:: console

    $ cd twisted
    $ git config blame.ignoreRevsFile .git-blame-ignore-revs


Running tests
-------------

The tests are handled by the `trial` testing framework.
It knows about deferred and any other good stuff provided by Twisted.
To run the full unit-test suite, do:

.. code-block:: console

    trial twisted

To run a single test file (like ``twisted/test/test_defer.py`` ), do one of:

.. code-block:: console

    trial twisted.test.test_defer
    # or
    trial twisted/test/test_defer.py

To run any tests that are related to a code file, like ``twisted/protocols/imap4.py`` , do:

.. code-block:: console

    trial --testmodule twisted/mail/imap4.py

This depends upon the ``.py`` file having an appropriate "test-case-name" tag that indicates which test cases provide coverage.
See the :doc:`Test Standards </development/test-standard>` document for
details about using "test-case-name".
In this example, the ``twisted.mail.test.test_imap`` test will be run.

Some tests create temporary files in `./_trial_temp`, which you can inspect for debugging purposes, but this directory will be deleted on each subsequent test run, so if you're debugging some filesystem code you may want to move it aside first.

Building docs
-------------

Twisted's narrative documentation — i.e. the documentation sourced from `.rst` files in `./docs/` — is generated by `Sphinx <https://sphinx-doc.org/>`_.
Its API reference documentation — i.e. the documentation sourced from docstrings in `.py` files — is generated by `pydoctor <https://pydoctor.readthedocs.io/>`_.

To build the HTML form of the docs into the ``doc/`` directory, do the following.
This will trigger a full build including the API docs:

.. code-block:: console

    tox -e narrativedocs
    firefox docs/_build/index.html


Committing and pre-commit hooks
--------------------------------

When submitting a PR, create a GitHub issue first, and prefix the name of the for the PR branch with the associated GitHub issue number, so that we can easily cross-reference them.
For example, use `1234-some-brach-name` as the name of the branch working to fix issue `1234`.

To speed up acceptance of your PR, you may want to make sure it passes our pre-commit code style checks locally.
If you skip this step, you may find some reformatting commits generated by the service we use, `pre-commit <https://pre-commit.com/>`_ . We don't mind, so feel free to let the computer do the job instead, but it may extend the amount of time you're waiting for automated tests to run.

To set up `git` to run these checks automatically every time you commit code,
run this::

    pre-commit install

Or to manually trigger the checks before each commit, run::

    pre-commit


Release notes management
------------------------

It is up to the authors of individual changes to write high-level descriptions for their changes.
They are not a commit or a merge message.
These descriptions will be aggregated into the release notes distributed with Twisted releases.
They should be written in such a way to help other users decide whether or not they need to upgrade,
or whether a certain defect is fixed in a release.

During development, to avoid generating commit conflicts, we use the `towncrier <https://pypi.python.org/pypi/towncrier>`_ tool to manage separate news fragments for each change.
At the release time, all these "release fragments" are aggregated into our single `NEWS <https://github.com/twisted/twisted/blob/trunk/NEWS.rst>`_ release file.

Changes must be accompanied by a file whose content describes that change in at least one `newsfragments` directory.
There are `newsfragments` directories for each subproject and one root directory for core Twisted changes.
If a change affects multiple areas of Twisted, then each affected area can have a newsfragments entry to detail the relevant changes.
An entry must be a file named `<issue number>.<change type>` (eg. `1234.bugfix`).
You should replace `<issue number>` with the issue number which is being resolved by the change (if multiple issues are resolved, multiple files with the same contents should be added).
The `<change type>` extension is replaced by one of the following literal strings:

* **feature** - Issues which are adding a new feature
* **bugfix** - Issues which are fixing a bug
* **doc** - Issues primarily about fixing or improving documentation (any variety)
* **removal** - Issues which are deprecating something or removing something which was already deprecated
* **misc** - Issues which are very minor and not worth summarizing outside of the git changelog.  These should be empty (their contents will be ignored)

To get a sense of how the text in these files is presented to users, take a look at `NEWS.rst <[https://github.com/twisted/twisted/blob/trunk/NEWS.rst>`_ the real overall news file].
The goal when writing the content for one of these files is to produce text that will fit well into the overall news files.

Here are a few which should help you write good news fragments:

* The entry SHOULD contain a high-level description of the change suitable for end users.
* When the changes touch Python code, the grammatical subject of the sentence SHOULD be a Python class/method/function/interface/variable/etc, and the verb SHOULD be something that the object does. The verb MAY be prefixed with "now".
* For bugfix, it MAY contain a reference to the version in which the bug was introduced.

Here are some examples.

Features::

    twisted.protocols.amp now raises InvalidSignature when bad arguments are passed to Command.makeArguments

    The new module twisted.internet.endpoints provides an interface for specifying address families separately from socket types.

Bugfix::

    twisted.internet.ssl.Certificate(...).getPublicKey().keyHash() now produces a stable value regardless of OpenSSL version. Unfortunately this means that it is different than the value produced by older Twisted versions.

    twisted.names.secondary.SecondaryAuthority can now answer queries again (broken since 13.2.0).

    The SSL server string endpoint parser (twisted.internet.endpoints.serverFromString) now constructs endpoints which, by default, disable the insecure SSLv3 protocol.

Deprecations::

    twisted.trial.util.findObject is now deprecated.

    twisted.conch.insults.colors is now deprecated in favor of twisted.conch.insults.helper.

    twisted.runner.procmon.ProcessMonitor's active, consistency, and consistencyDelay attributes are now deprecated.

Removals::

    twisted.internet.interfaces.IReactorTime.cancelCallLater, deprecated since Twisted 2.5, has been removed.

    Support for versions of pyOpenSSL older than 0.10 has been removed.

Documentation::

    The documentation for twisted.internet.defer.DeferredSemaphore now describes the actual usage for `limit` and `tokens` instance attributes.

    The docstring for twisted.conch.ssh.userauth.SSHUserAuthClient is now clearer on how the preferredOrder instance variable is handled.

    twisted.mail.alias now has full API documentation.

    The howto document page of Deferred now has documentation about the cancellation.


You don't need to worry about newlines in the file; the contents will be rewrapped when added to the NEWS files.


Review process
--------------

Any change to be accepted into the main branch must pass the review process.

See :doc:`the dedicated review process page </development/review-process>` for more details.


Reverting a change
------------------

If a change set somehow introduces a test suite regression or is otherwise found to be undesirable,
it is to be reverted.

Any developer may revert a commit that introduces a test suite regression on a supported platform.
The revert message should be as explicit as possible.
If it's a failure,
put the message of the error in the commit message, possible with more details about the test environment.
If there are too many failures,
it can be put in the issue tracker,
with a reference in the message.

Use the "Reopens" tag in the commit message to reference the relevant issue::

    Revert revision-sha: Brief description

    A description of the problem, or a traceback if pertinent

    Reopens: #issue-number

Reverted branches are to be reviewed again before being merged.