Python 单元测试测试顺序

Unittest tests order

提问by nmb.ten

How do I be sure of the unittest methods order? Is the alphabetical or numeric prefixes the proper way?

我如何确定单元测试方法的顺序？字母或数字前缀是否正确？

class TestFoo(TestCase):
    def test_1(self):
        ...
    def test_2(self):
        ...

or

或者

class TestFoo(TestCase):
    def test_a(self):
        ...
    def test_b(self):
        ...

采纳答案by max

You can disable it by setting sortTestMethodsUsing to None: http://docs.python.org/2/library/unittest.html#unittest.TestLoader.sortTestMethodsUsing

您可以通过将 sortTestMethodsUsing 设置为 None 来禁用它：http://docs.python.org/2/library/unittest.html#unittest.TestLoader.sortTestMethodsUsing

import unittest
unittest.TestLoader.sortTestMethodsUsing = None

For pure unittests, you folks are right; but for component tests and integration tests... I do not agree that you shall assume nothing about the state. What if you are testing the state. For example, your test validates that a service is auto-started upon installation. If in your setup, you start the service, then do the assertion, then you are no longer testing the state but you are testing the "service start" functionality.

对于纯单元测试，你们是对的；但是对于组件测试和集成测试......我不同意你对状态不做任何假设。如果您正在测试状态怎么办。例如，您的测试验证服务在安装时自动启动。如果在您的设置中，您启动服务，然后进行断言，那么您将不再测试状态，而是测试“服务启动”功能。

Another example is when your setup takes a long time or requires a lot of space and it just becomes impractical to run the setup frequently.

另一个例子是当您的设置需要很长时间或需要大量空间并且频繁运行设置变得不切实际时。

Many developers tend to use "unittest" frameworks for component testing...so stop and ask yourself, am I doing unittesting or component testing.

许多开发人员倾向于使用“单元测试”框架进行组件测试……所以停下来问问自己，我是在做单元测试还是组件测试。

回答by zoul

Why do you need specific test order? The tests should be isolated and therefore it should be possible to run them in any order, or even in parallel.

为什么需要特定的测试订单？测试应该是隔离的，因此应该可以以任何顺序甚至并行运行它们。

If you need to test something like user unsubscribing, the test could create a fresh database with a test subscription and then try to unsubscribe. This scenario has its own problems, but in the end it's better than having tests depend on each other. (Note that you can factor out common test code, so that you don't have to repeat the DB setup code or create testing data ad nauseam.)

如果您需要测试诸如用户取消订阅之类的内容，该测试可以使用测试订阅创建一个新数据库，然后尝试取消订阅。这种情况有其自身的问题，但最终总比让测试相互依赖要好。（请注意，您可以分解出常见的测试代码，这样您就不必重复数据库设置代码或创建令人作呕的测试数据。）

回答by zoul

Don't rely on the order. If they use some common state like the filesystem or database, then you should create setUpand tearDownmethods that get your environment into a testable state, then clean up after the tests have run. Each test should assume that the environment is as defined in setUp, and should make no further assumptions.

不要依赖顺序。如果他们使用一些常见的状态，如文件系统或数据库，那么您应该创建setUp和tearDown那个让你的环境变成一个可测试状态的方法，然后清理试验运行后。每个测试都应该假设环境如 中定义的那样setUp，并且不应该做进一步的假设。

回答by eradman

There are a number of reasons for prioritizing tests, not the least of which is productivity, which is what JUnit Max is geared for. It's sometimes helpful to keep very slow tests in their own module so that you can get quick feedback from the those tests that that don't suffer from the same heavy dependencies. Ordering is also helpful in tracking down failures from tests that are not completely self-contained.

优先考虑测试的原因有很多，其中最重要的是生产力，而这正是 JUnit Max 所面向的。有时在它们自己的模块中保留非常慢的测试是有帮助的，这样您就可以从那些没有受到同样严重依赖的测试中获得快速反馈。排序也有助于从不完全独立的测试中追踪失败。

回答by Elmar Zander

If you use 'nose' and you write your test cases as functions (and not as methods of some TestCase derived class) 'nose' doesn't fiddle with the order, but uses the order of the functions as defined in the file. In order to have the assert_* methods handy without needing to subclass TestCase I usually use the testing module from numpy. Example:

如果您使用 'nose' 并且您将测试用例编写为函数（而不是某些 TestCase 派生类的方法），则 'nose' 不会摆弄顺序，而是使用文件中定义的函数顺序。为了在不需要子类化 TestCase 的情况下使用 assert_* 方法，我通常使用来自 numpy 的测试模块。例子：

from numpy.testing import *

def test_aaa():
    assert_equal(1, 1)

def test_zzz():
    assert_equal(1, 1)

def test_bbb():
    assert_equal(1, 1)

Running that with ''nosetest -vv'' gives:

使用 ''nosetest -vv'' 运行它会得到：

test_it.test_aaa ... ok
test_it.test_zzz ... ok
test_it.test_bbb ... ok
----------------------------------------------------------------------
Ran 3 tests in 0.050s
OK

Note to all those who contend that unit tests shouldn't be ordered: while it is true that unit tests should be isolated and can run independently, your functions and classes are usually not independent. They rather build up on another from simpler/low-level functions to more complex/high-level functions. When you start optimising your low-level functions and mess up (for my part, I do that frequently; if you don't, you probably don't need unit test anyway;-) then it's a lot better for diagnosing the cause, when the tests for simple functions come first, and tests for functions that depend on those functions later. If the tests are sorted alphabetically the real cause usually gets drowned among one hundred failed assertions, which are not there because the function under test has a bug, but because the low-level function it relies on has.

请注意所有主张不应订购单元测试的人：虽然单元测试确实应该被隔离并且可以独立运行，但您的函数和类通常不是独立的。他们宁愿建立在另一个从更简单/低级功能到更复杂/高级功能的基础上。当你开始优化你的低级函数并搞砸时（就我而言，我经常这样做；如果你不这样做，你可能无论如何都不需要单元测试；-）那么诊断原因要好得多，当首先测试简单函数，然后测试依赖于这些函数的函数时。如果测试按字母顺序排序，真正的原因通常会淹没在一百个失败的断言中，因为被测函数有一个错误，所以不存在这些断言，

That's why I want to have my unit tests sorted the way I specified them: not to use state that was built up in early tests in later tests, but as a very helpful tool in diagnosing problems.

这就是为什么我想让我的单元测试按照我指定的方式排序：不要在以后的测试中使用早期测试中建立的状态，而是作为诊断问题的非常有用的工具。

回答by jiakai

See the example of WidgetTestCaseon http://docs.python.org/library/unittest.html#organizing-test-code, it says that

请参阅http://docs.python.org/library/unittest.html#organizing-test-codeWidgetTestCase上的示例，它说

Class instances will now each run one of the test_*() methods, with self.widget created and destroyed separately for each instance.

类实例现在将分别运行 test_*() 方法之一，并为每个实例分别创建和销毁 self.widget。

So it might be of no use to specify the order of test cases, if you do not access global variables.

因此，如果您不访问全局变量，那么指定测试用例的顺序可能没有用。

回答by Jonas Geiregat

There are scenarios where the order can be important and where setUp and Teardown come in as to limited. There's only one setUp and tearDown method, which is logical but you can only put so much information in them until it get's unclear what setUp or tearDown might actually be doing.

在某些情况下，顺序可能很重要，而 setUp 和 Teardown 的作用有限。只有一种 setUp 和 tearDown 方法，这是合乎逻辑的，但您只能在其中放入这么多信息，直到不清楚 setUp 或 tearDown 实际在做什么。

Take this integration test as an example:

以本次集成测试为例：

You are writing tests to see if the registration form and the login form are working correctly. In such a case the order is important, as you can't login without an existing account. More importantly the order of your tests represents some kind of user interaction. Where each test might represent a step in the whole process or flow you're testing.

您正在编写测试以查看注册表和登录表单是否正常工作。在这种情况下，顺序很重要，因为没有现有帐户就无法登录。更重要的是，您的测试顺序代表了某种用户交互。每个测试可能代表您正在测试的整个过程或流程中的一个步骤。

Dividing your code in those logical pieces has several advantages.

将您的代码分成这些逻辑部分有几个优点。

Might not be the best solution but, I often use one method that kicks off the actual tests.

可能不是最好的解决方案，但我经常使用一种方法来启动实际测试。

def test_registration_login_flow(self):
    _test_registration_flow()
    _test_login_flow()

回答by Jason Wirth

I half agree with the idea that tests souldn't be ordered. In some cases it helps (it's easier damn it!) to have them in order... after all that's the reason for the 'unit' in UnitTest.

我一半同意不应该订购测试的想法。在某些情况下，按顺序排列它们会有所帮助（更容易该死！）……毕竟这就是 UnitTest 中“单元”的原因。

That said one alternative is to use mock objects to mockout and patch the items that should run before that specific code under test. You can also put a dummy function in there to monkey patch your code. For more info check out Mock, which is part of the standard library now. Mock

也就是说，一种替代方法是使用模拟对象来模拟和修补应该在被测特定代码之前运行的项目。您还可以在其中放置一个虚拟函数来修补您的代码。有关更多信息，请查看 Mock，它现在是标准库的一部分。 嘲笑

Here are some YouTube videos if you haven't used Mock before.

如果您以前没有使用过 Mock，这里有一些 YouTube 视频。

Video 1

视频 1

Video 2

视频2

Video 3

视频3

More to the point, try using class methods to structure your code, then place all the class methods in one main test method.

更重要的是，尝试使用类方法来构建代码，然后将所有类方法放在一个主要测试方法中。

import unittest
import sqlite3

class MyOrderedTest(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        cls.create_db()
        cls.setup_draft()
        cls.draft_one()
        cls.draft_two()
        cls.draft_three()

    @classmethod
    def create_db(cls):
        cls.conn = sqlite3.connect(":memory:")

    @classmethod
    def setup_draft(cls):
        cls.conn.execute("CREATE TABLE players ('draftid' INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, 'first', 'last')")

    @classmethod
    def draft_one(cls):
        player = ("Hakeem", "Olajuwon")
        cls.conn.execute("INSERT INTO players (first, last) VALUES (?, ?)", player)

    @classmethod
    def draft_two(cls):
        player = ("Sam", "Bowie")
        cls.conn.execute("INSERT INTO players (first, last) VALUES (?, ?)", player)

    @classmethod
    def draft_three(cls):
        player = ("Michael", "Jordan")
        cls.conn.execute("INSERT INTO players (first, last) VALUES (?, ?)", player)

    def test_unordered_one(self):
        cur = self.conn.execute("SELECT * from players")
        draft = [(1, u'Hakeem', u'Olajuwon'), (2, u'Sam', u'Bowie'), (3, u'Michael', u'Jordan')]
        query = cur.fetchall()
        print query
        self.assertListEqual(query, draft)

    def test_unordered_two(self):
        cur = self.conn.execute("SELECT first, last FROM players WHERE draftid=3")
        result = cur.fetchone()
        third = " ".join(result)
        print third
        self.assertEqual(third, "Michael Jordan")

回答by Seren Seraph

There is no reason given that you can't build on what was done in a previous test or should rebuild it all from scratch for the next test. At least no reason is usually offered but instead people just confidently say "you shouldn't". That isn't helpful.

没有理由认为您不能建立在先前测试中所做的工作，或者应该为下一次测试从头开始重建。至少通常不会提供任何理由，而是人们只是自信地说“你不应该”。那没有帮助。

In general I am tired of reading too many answers here that say basically "you shouldn't do that" instead of giving any information on how to best do it if in the questioners judgment there is good reason to do so. If I wanted someone's opinion on whether I should do something then I would have asked for opinions on whether doing it is a good idea.

总的来说，我厌倦了在这里阅读太多基本上说“你不应该这样做”的答案，而不是提供任何关于如何最好地做到这一点的信息，如果在提问者的判断中有充分的理由这样做。如果我想要某人对我是否应该做某事的意见，那么我会就这样做是否是一个好主意征求意见。

That out of the way, if you read say loadTestsFromTestCase and what it calls it ultimately scans for methods with some name pattern in whatever order they are encountered in the classes method dictionary, so basically in key order. It take this information and makes a testsuite of mapping it to the TestCase class. Giving it instead a list ordered as you would like is one way to do this. I am not so sure of the most efficient/cleanest way to do it but this does work.

顺便说一下，如果你读到 loadTestsFromTestCase 和它所调用的内容，它最终会按照在类方法字典中遇到的任何顺序扫描具有某种名称模式的方法，所以基本上是按关键顺序。它获取此信息并制作一个将其映射到 TestCase 类的测试套件。给它一个按照您想要的顺序排列的列表是实现此目的的一种方法。我不太确定最有效/最干净的方法来做到这一点，但这确实有效。

回答by gregorySalvan

Contrary to what was said here: - tests have to run in isolation (order must not matters for that) AND - ordering them is important because they describe what the system do and how the developper implements it.

与这里所说的相反： - 测试必须单独运行（顺序必须无关紧要）并且 - 对它们进行排序很重要，因为它们描述了系统做什么以及开发人员如何实现它。

IOW, each test brings you informations of the system and the developper logic.

IOW，每次测试都会为您带来系统和开发人员逻辑的信息。

So if these informations are not ordered that can make your code difficult to understand.

因此，如果这些信息未按顺序排列，可能会使您的代码难以理解。

回答by gahcep

Ok, may be a bit later, but anyway...

好吧，可能会晚一点，但无论如何......

You should try proboscislibrary. It will allow you to make tests order as well as set up any test dependencies. I use it and this library is awesome truly.

你应该试试长鼻图书馆。它将允许您制定测试顺序以及设置任何测试依赖项。我使用它，这个库真的很棒。

For example, if test case #1from module Ashould depend on test case #3from module Byou CANset this behaviour using the library.

例如，如果test case #1从module A应取决于test case #3从module B您的CAN使用图书馆的这种行为。