python 应该如何记录单元测试？

How should unit tests be documented?

提问by ddbeck

I'm trying to improve the number and quality of tests in my Python projects. One of the the difficulties I've encountered as the number of tests increase is knowing what each test does and how it's supposed to help spot problems. I know that part of keeping track of tests is better unit test names (which has been addressed elsewhere), but I'm also interested in understanding how documentation and unit testing go together.

我正在尝试提高 Python 项目中测试的数量和质量。随着测试数量的增加，我遇到的困难之一是了解每个测试的作用以及它应该如何帮助发现问题。我知道跟踪测试的部分是更好的单元测试名称（已在别处解决），但我也有兴趣了解文档和单元测试如何结合在一起。

How can unit tests be documented to improve their utility when those tests fail in the future? Specifically, what makes a good unit test docstring?

当这些测试在未来失败时，如何记录单元测试以提高其效用？具体来说，什么是好的单元测试文档字符串？

I'd appreciate both descriptive answers and examples of unit tests with excellent documentation. Though I'm working exclusively with Python, I'm open to practices from other languages.

我很欣赏具有出色文档的单元测试的描述性答案和示例。尽管我只使用 Python，但我对其他语言的实践持开放态度。

采纳答案by Kaleb Brasee

I document most on my unit tests with the method name exclusively:

我只用方法名称记录了我的单元测试中的大部分内容：

testInitializeSetsUpChessBoardCorrectly()
testSuccessfulPromotionAddsCorrectPiece()

For almost 100% of my test cases, this clearly explains what the unit test is validating and that's all I use. However, in a few of the more complicated test cases, I'll add a few comments throughout the method to explain what several lines are doing.

对于我几乎 100% 的测试用例，这清楚地解释了单元测试正在验证的内容，这就是我使用的全部内容。但是，在一些更复杂的测试用例中，我将在整个方法中添加一些注释来解释几行的作用。

I've seen a tool before (I believe it was for Ruby) that would generate documentation files by parsing the names of all the test cases in a project, but I don't recall the name. If you had test cases for a chess Queen class:

我以前见过一个工具（我相信它是用于 Ruby 的），它可以通过解析项目中所有测试用例的名称来生成文档文件，但我不记得名称了。如果您有国际象棋皇后课程的测试用例：

testCanMoveStraightUpWhenNotBlocked()
testCanMoveStraightLeftWhenNotBlocked()

the tool would generate an HTML doc with contents something like this:

该工具将生成一个 HTML 文档，内容如下：

Queen requirements:
 - can move straight up when not blocked.
 - can move straight left when not blocked.

回答by Mike Mazur

Perhaps the issue isn't in how best to write test docstrings, but how to write the tests themselves? Refactoring tests in such a way that they're self documenting can go a long way, and your docstring won't go stale when the code changes.

也许问题不在于如何最好地编写测试文档字符串，而在于如何自己编写测试？以自我记录的方式重构测试可以有很长的路要走，并且当代码更改时，您的文档字符串不会过时。

There's a few things you can do to make the tests clearer:

您可以做一些事情来使测试更清晰：

• clear & descriptive test method names (already mentioned)
• test body should be clear and concise (self documenting)
• abstract away complicated setup/teardown etc. in methods
• more?

• 清晰和描述性的测试方法名称（已经提到）
• 测试主体应该清晰简洁（自我记录）
• 在方法中抽象出复杂的设置/拆卸等
• 更多的？

For example, if you have a test like this:

例如，如果您有这样的测试：

def test_widget_run_returns_0():
    widget = Widget(param1, param2, "another param")
    widget.set_option(true)
    widget.set_temp_dir("/tmp/widget_tmp")
    widget.destination_ip = "10.10.10.99"

    return_value = widget.run()

    assert return_value == 0
    assert widget.response == "My expected response"
    assert widget.errors == None

You might replace the setup statements with a method call:

您可以使用方法调用替换设置语句：

def test_widget_run_returns_0():
    widget = create_basic_widget()
    return_value = widget.run()
    assert return_value == 0
    assert_basic_widget(widget)

def create_basic_widget():
    widget = Widget(param1, param2, "another param")
    widget.set_option(true)
    widget.set_temp_dir("/tmp/widget_tmp")
    widget.destination_ip = "10.10.10.99"
    return widget

def assert_basic_widget():
    assert widget.response == "My expected response"
    assert widget.errors == None

Note that your test method is now composed of a series of method calls with intent-revealing names, a sort of DSL specific to your tests. Does a test like that still need documentation?

请注意，您的测试方法现在由一系列具有意图揭示名称的方法调用组成，这是一种特定于您的测试的 DSL。这样的测试还需要文档吗？

Another thing to note is that your test method is mainly at one level of abstraction. Someone reading the test method will see the algorithm is:

另一件要注意的事情是您的测试方法主要处于一个抽象级别。阅读测试方法的人会看到算法是：

• creating a widget
• calling run on the widget
• asserting the code did what we expect

• 创建小部件
• 在小部件上调用运行
• 断言代码做了我们所期望的

Their understanding of the test method is not muddied by the details of setting up the widget, which is one level of abstraction lower than the test method.

他们对测试方法的理解不会被设置widget的细节所迷惑，这比测试方法低一级抽象。

The first version of the test method follows the Inline Setuppattern. The second version follows Creation Methodand Delegated Setuppatterns.

测试方法的第一个版本遵循内联设置模式。第二个版本遵循创建方法和委托设置模式。

Generally I'm against comments, except where they explain the "why" of the code. Reading Uncle Bob Martin's Clean Codeconvinced me of this. There is a chapter on comments, and there is a chapter on testing. I recommend it.

一般来说，我反对评论，除非他们解释了代码的“为什么”。阅读 Uncle Bob Martin 的Clean Code使我确信这一点。有一章是评论，一章是测试。我推荐它。

For more on automated testing best practices, do check out xUnit Patterns.

有关自动化测试最佳实践的更多信息，请查看xUnit Patterns。

回答by Bill the Lizard

The name of the test method should describe exactly what you are testing. The documentation should say what makes the test fail.

测试方法的名称应准确描述您正在测试的内容。文档应该说明是什么导致测试失败。

回答by Chris Lacasse

You should use a combination of descriptive method names and comments in the doc string. A good way to do it is including a basic procedure and verification steps in the doc string. Then if you run these tests from some kind of testing framework that automates running the tests and collecting results, you can have the framework log the contents of the doc string for each test method along with its stdout+stderr.

您应该在文档字符串中使用描述性方法名称和注释的组合。一个很好的方法是在文档字符串中包含一个基本程序和验证步骤。然后，如果您从某种自动运行测试并收集结果的测试框架运行这些测试，您可以让框架记录每个测试方法的文档字符串的内容及其 stdout+stderr。

Here's a basic example:

这是一个基本示例：

class SimpelTestCase(unittest.TestCase):
    def testSomething(self):
        """ Procedure:
            1. Print something
            2. Print something else
            ---------
            Verification:
            3. Verify no errors occurred
        """
        print "something"
        print "something else"

Having the procedure with the test makes it much easier to figure out what the test is doing. And if you include the docstring with the test output it makes figuring out what went wrong when going through the results later much easier. The previous place I worked at did something like this and it worked out very well when failures occurred. We ran the unit tests on every checkin automatically, using CruiseControl.

拥有测试程序可以更容易地弄清楚测试在做什么。如果您在测试输出中包含文档字符串，则可以更轻松地在稍后查看结果时找出问题所在。我以前工作的地方就是这样的，当发生故障时效果很好。我们使用 CruiseControl 在每次签到时自动运行单元测试。

回答by Tim Ottinger

When the test fails (which should be before it ever passes) you should see the error message and be able to tell what's up. That only happens if you plan it that way.

当测试失败时（应该在它通过之前），您应该会看到错误消息并能够知道发生了什么。只有当你这样计划时才会发生这种情况。

It's entirely a matter of the naming of the test class, the test method, and the assert message. When a test fails, and you can't tell what is up from these three clues, then rename some things or break up some tests classes.

这完全是测试类的命名、测试方法和断言消息的问题。当测试失败时，您无法从这三个线索中看出发生了什么，然后重命名一些东西或分解一些测试类。

It doesn't happen if the name of the fixture is ClassXTests and the name of the test is TestMethodX and the error message is "expected true, returned false". That's a sign of sloppy test writing.

如果夹具的名称是 ClassXTests 并且测试的名称是 TestMethodX 并且错误消息是“预期为真，返回假”，则不会发生这种情况。这是测试写作草率的迹象。

Most of the time you shouldn't have to read the test or any comments to know what has happened.

大多数情况下，您不必阅读测试或任何评论来了解发生了什么。