java 使用 JUnit 作为验收测试框架

Using JUnit as an acceptance test framework

提问by Chris Knight

OK, so I work for a company who has openly adopted agile practices for development in recent years. Our unit tests and code quality are improving. One area we still are working on is to find what works best for us in the automated acceptance test arena. We want to take our well formed user stories and use these to drive the code in a test driven manner. This will also give us acceptance level tests for each user story which we can then automate.

好的，所以我在一家近年来公开采用敏捷开发实践的公司工作。我们的单元测试和代码质量正在提高。我们仍在努力的一个领域是在自动化验收测试领域找到最适合我们的方法。我们希望采用格式良好的用户故事，并使用它们以测试驱动的方式驱动代码。这也将为我们提供每个用户故事的接受度测试，然后我们可以将其自动化。

To date, we've tried Fit, Fitnesse and Selenium. Each have their advantages, but we've also had real issues with them as well. With Fit and Fitnesse, we can't help but feel they overcomplicate things and we've had many technical issues using them. The business haven't fully bought in these tools and aren't particularly keen on maintaining the scripts all the time (and aren't big fans of the table style). Selenium is really good, but slow and relies on real time data and resources.

迄今为止，我们已经尝试过 Fit、Fitnesse 和 Selenium。每个人都有自己的优势，但我们也遇到了真正的问题。使用 Fit 和 Fitnesse，我们不禁感到它们使事情变得过于复杂，而且我们在使用它们时遇到了许多技术问题。该企业还没有完全购买这些工具，也不是特别热衷于一直维护脚本（并且不是表格样式的忠实拥护者）。Selenium 非常好，但速度慢并且依赖于实时数据和资源。

One approach we are now considering is the use of the JUnit framework to provide similiar functionality. Rather than testing just a small unit of work using JUnit, why not use it to write a test (using the JUnit framework) to cover an acceptance level swath of the application? I.e. take a new story ("As a user I would like to see basic details of my policy...") and write a test in JUnit which starts executing application code at the point of entry for the policy details link but covers all code and logic down to the stubbed data access layer and back to the point of forwarding to the next page in the application, asserting on what data the user should see on that page.

我们现在正在考虑的一种方法是使用 JUnit 框架来提供类似的功能。与其仅使用 JUnit 测试一个小的工作单元，为什么不使用它来编写测试（使用 JUnit 框架）以覆盖应用程序的接受级别呢？即拿一个新故事（“作为用户，我想查看我的政策的基本细节......”）并在 JUnit 中编写一个测试，该测试在政策详细信息链接的入口点开始执行应用程序代码，但涵盖所有代码和逻辑向下到存根数据访问层，然后返回到转发到应用程序中的下一页的点，断言用户应该在该页面上看到哪些数据。

This seems to me to have the following advantages:

在我看来，这具有以下优点：

• Simplicity (no additional frameworks required)
• Zero effort to integrate with our Continuous Integration build server (since it already handles our JUnit tests)
• Full skillset already present in the team (its just a JUnit test after all)

• 简单（不需要额外的框架）
• 零努力与我们的持续集成构建服务器集成（因为它已经处理了我们的 JUnit 测试）
• 团队中已经存在完整的技能集（毕竟它只是一个 JUnit 测试）

And the downsides being:

缺点是：

• Less customer involvement (though they are heavily involved in writing the user stories in the first place from which the acceptance tests will be written)
• Perhaps more difficult to understand (or make understood) the user story and acceptance criteria in a JUnit class verses a freetext specification ala Fit or Fitnesse

• 较少的客户参与（尽管他们在编写验收测试时首先大量参与编写用户故事）
• 也许更难理解（或理解）JUnit 类中的用户故事和接受标准与自由文本规范 ala Fit 或 Fitnesse

So, my question is really, have you ever tried this method? Ever considered it? What are your thoughts? What do you like and dislike about this approach? Finally, please only mention alternative frameworks if you can say why you like or dislike them more than this approach.

所以，我的问题是，你有没有尝试过这种方法？有没有考虑过？你觉得呢？你有没有什么想法？您喜欢和不喜欢这种方法的哪些方面？最后，如果您能说出比这种方法更喜欢或不喜欢它们的原因，请仅提及替代框架。

回答by user1704737

My experience using JUnit for an Acceptance Testing framework.

我将 JUnit 用于验收测试框架的经验。

Our company did a very similar thing where we started with FitNesse and then went to JUnit. We did this mainly because FitNesse was sitting on top of Java already, so switching to JUnit was easy. FitNesse seemed to not fit (no pun intended) our needs.

我们公司做了一个非常相似的事情，从 FitNesse 开始，然后转到 JUnit。我们这样做主要是因为 FitNesse 已经位于 Java 之上，所以切换到 JUnit 很容易。FitNesse 似乎不适合（不是双关语）我们的需求。

Here are my pros and cons of using JUnit:

以下是我使用 JUnit 的利弊：

Pros:

优点：

• JUnit has a lot of flexibility (because it's Java), so it's relatively easy to create an Internal Domain Specific Language[Fowler]. The DSL can be modified so that it's easy for domain experts (not software engineers) to read/write.
• Since it's JUnit you can use Eclipse as an IDE, which gives you auto-complete, syntax checking, debugging, etc.
• Specific for our application, we have a CORBA interface to the UI. Java can easily use this CORBA interface as well.

• JUnit 具有很大的灵活性（因为它是 Java），因此创建内部域特定语言[Fowler]相对容易。可以修改 DSL，以便领域专家（而非软件工程师）可以轻松读/写。
• 由于它是 JUnit，您可以将 Eclipse 用作 IDE，它为您提供自动完成、语法检查、调试等功能。
• 对于我们的应用程序，我们有一个到 UI 的 CORBA 接口。Java 也可以轻松地使用这个 CORBA 接口。

Cons:

缺点：

• People hear JUnit and think unit test. This gets confusing when people start referring to the acceptance tests as "JUnit tests".
• To provide a simplified testing environment you have to spend some time extending the framework and developing your DSL.

• 人们听到 JUnit 并想到单元测试。当人们开始将验收测试称为“JUnit 测试”时，这会让人感到困惑。
• 为了提供一个简化的测试环境，您必须花费一些时间来扩展框架和开发您的 DSL。

So in summary, JUnit works just fine for Acceptance Tests if you are willing to spend the time to extend it into you own domain specific testing environment. Otherwise I think I would look elsewhere for something more out of the box.

总而言之，如果您愿意花时间将 JUnit 扩展到您自己的特定领域测试环境中，JUnit 就可以很好地用于验收测试。否则，我想我会在别处寻找更多开箱即用的东西。

回答by Jan Schaefer

It is perfectly valid to use JUnit to write acceptance tests. JUnit is just a test execution framework, it is completely irrelevant whether you are actually writing unit tests with it, component tests, integration tests or acceptance tests. As user1704737 already said it is important that you have a readable DSL for writing your tests.

使用 JUnit 编写验收测试是完全有效的。JUnit 只是一个测试执行框架，它与您是否实际使用它编写单元测试、组件测试、集成测试或验收测试完全无关。正如 user1704737 已经说过的，重要的是你有一个可读的 DSL 来编写你的测试。

To write you tests in JUnit I suggest to give JGivena try. It is a lightweight framework to make it easy to write acceptance tests in the given-when-then notation in plain Java. It uses JUnit (or TestNG) as a test executor, so that it can easily be integrated into existing test infrastructures. You also get HTML reports for your acceptance tests which you can show business to validate the tests against the requirements.

要在 JUnit 中编写测试，我建议尝试JGiven。它是一个轻量级框架，可以很容易地在纯 Java 中以 given-when-then 表示法编写验收测试。它使用 JUnit（或 TestNG）作为测试执行器，因此可以轻松集成到现有的测试基础架构中。您还可以获得验收测试的 HTML 报告，您可以向业务展示这些报告以根据要求验证测试。

Disclaimer: I am the author of JGiven

免责声明：我是 JGiven 的作者

回答by yoosiba

Business and UAT
Business most of the time will loose interest in any testing tool sooner or later. They are Business not testers after all, so they won't commit much to writing/maintaining test scripts. They are Business, they want to do business. From their point of view testers/developers/other IT guys are responsible for delivering them software of certain quality.
Even if Agile is your way, and you get some level of commitment from business don't expect them to behave like tester with every sprint/iteration or whatever you have. It is really not their job.
One digression: User Acceptance Tests are be manual tests executed by the user(s), so he(they) can say is product what he(they) asked for.
So whenever feature (or set of features) is ready, do presentation for business, let them play with it, let them say is it what they need. Don't force them to check this feature with every iteration.

业务和 UAT
业务在大多数情况下迟早会对任何测试工具失去兴趣。他们毕竟是业务人员而不是测试人员，因此他们不会在编写/维护测试脚本方面投入太多精力。他们是生意人，他们想做生意。从他们的角度来看，测试人员/开发人员/其他 IT 人员负责为他们提供特定质量的软件。
即使敏捷是您的方式，并且您从业务中获得了一定程度的承诺，也不要期望他们在每次冲刺/迭代或您拥有的任何东西时都表现得像测试人员。这真的不是他们的工作。
题外话：用户验收测试是由用户执行的手动测试，因此他（他们）可以说是他（他们）要求的产品。
因此，每当功能（或一组功能）准备就绪时，就为业务进行演示，让他们玩弄它，让他们说这是他们需要的。不要强迫他们在每次迭代时检查这个特性。

Now back to youracceptance tests. Whenever you have story from client to do, implement test(s) for it in your test infrastructure. You write it, you run it, you maintain it. It may be wise to have test for feature and drive development with ADD/BDD/TDD or whatever you want to call it. It should be obvious that this test(s) than contribute to your regression tests suite in next iterations.

现在回到你的验收测试。每当您有来自客户的故事要做时，请在您的测试基础架构中为其实施测试。你编写它，运行它，维护它。使用 ADD/BDD/TDD 或任何您想称呼的方式对功能和驱动开发进行测试可能是明智之举。很明显，此测试对您的下一次迭代中的回归测试套件有所贡献。

So my point of view is, Business won't be keen on writing/maintaining tests, especially when set of features grows. Don't fight with it, adapt to it. Let them do just (manual) User Acceptance Tests at the end of iteration for new features. For your own sake create tests for features they want have. Create feature acceptance tests for each feature. Let those tests become your Regression Testing Suit. Accept that t is your responsibility to maintain it.

所以我的观点是，业务不会热衷于编写/维护测试，尤其是当功能集增长时。不要与之抗争，适应它。让他们在迭代结束时对新功能进行（手动）用户验收测试。为了您自己，为他们想要的功能创建测试。为每个功能创建功能验收测试。让这些测试成为您的回归测试套装。接受维护它是您的责任。

Acceptance testing Framework
JUnit, huh? Then try with WatiJfor web, and Abbotfor desktop. Keep in mind that those test won't be simple Unit Tests. You want something that will tests features of real application, you want Test Script to execute specific Scenario/Business Process. This means you need to approach writing those tests like writing application they are testing: DRY,KISS, YAGNI, Design Patterns - everything applies. In the end, it will be writing software, that just happens to test other software you write.

验收测试框架
JUnit，嗯？然后尝试使用Web 版WatiJ和桌面版Abbot。请记住，这些测试不会是简单的单元测试。您想要一些可以测试真实应用程序功能的东西，您想要测试脚本来执行特定的场景/业务流程。这意味着您需要像编写他们正在测试的应用程序一样编写这些测试：DRY、KISS、YAGNI、设计模式——一切都适用。最后，它将是编写软件，这恰好是为了测试您编写的其他软件。

Will those test become large and complex with this approach? Well, larger and more complicated than unit tests. But they are not testing single unit but whole application. Besides they will be far more simpler and smaller than application you are writing in the first place.

使用这种方法，这些测试会变得庞大而复杂吗？嗯，比单元测试更大更复杂。但他们不是在测试单个单元，而是整个应用程序。此外，它们将比您最初编写的应用程序更简单、更小。

Will you bee writing testing framework? If you want to succeed with this approach, than yes, you will be writing testing framework - collection of test classes and helper classes that have some knowledge about your application implemented. But you won't be extending JUnit. JUnit here is just some API you use in your framework.

你会编写测试框架吗？如果您想使用这种方法取得成功，那么您将编写测试框架 - 测试类和帮助类的集合，它们对您的应用程序实现了一些了解。但是您不会扩展 JUnit。这里的 JUnit 只是您在框架中使用的一些 API。

回答by Diego

A few recommendations if you follow that path:

如果您遵循该路径，有一些建议：

• Put your acceptance tests in another project, since probably they will run slower and have more dependencies than your usual tests
• Create builders (maybe using a fluent interface) to setup your model in test fixtures, because you will find that setting up the required objects for your test will be the most boring and difficult part to maintain.
• Take a look into Groovy, and the Spock Framework: it's compatible with JUnit, but provides you with a nice DSL to make your tests readable.

• 将您的验收测试放在另一个项目中，因为它们可能会比通常的测试运行得更慢并且具有更多的依赖项
• 创建构建器（可能使用流畅的界面）在测试装置中设置模型，因为您会发现设置测试所需的对象将是最无聊和最难维护的部分。
• 看看 Groovy 和Spock 框架：它与 JUnit 兼容，但为您提供了一个很好的 DSL，使您的测试可读。

回答by Xu Yi

Try robotframework (www.robotframework.org), I've been using it for years, Pekka Klarck from Helsinki developed it (Nokia Siemens Networks is supporting it since the beginning, now it's also open sourced).

试试robotframework（www.robotframework.org），我已经用了很多年了，来自赫尔辛基的Pekka Klarck开发了它（诺基亚西门子网络从一开始就支持它，现在它也开源了）。

robotframework uses tabular format test cases, supporting HTML, CSV or plain text (written in certain grammer). It has a library mechanism, which supports to import Selenium functions. You can write library for robotframework in Python or Java.

robotsframework 使用表格格式的测试用例，支持 HTML、CSV 或纯文本（以特定语法编写）。它有一个库机制，支持导入Selenium 函数。您可以使用 Python 或 Java 为机器人框架编写库。

For the JUnit as Acceptance Test Framework, of course you can, but it doesn't worth doing. Higher level testing should rely much less even none on a software's internal implementation, while write Acceptance Test using JUnit (which is a unit test framework), you introduced a lot dependencies on its e.g. internal function calls. Which will be a big headache while there is any changes with the software design, which a high level test should not be affected and you're.

对于 JUnit 作为验收测试框架，当然可以，但不值得这样做。更高级别的测试应该更少依赖于软件的内部实现，甚至不依赖于软件的内部实现，而使用 JUnit（这是一个单元测试框架）编写验收测试时，您引入了很多对其内部函数调用的依赖关系。当软件设计发生任何变化时，这将是一个令人头疼的问题，高级测试不应该受到影响，而您会受到影响。

回答by Matthew Farwell

You mentioned some good points about the pros and cons. Can I add:

你提到了一些关于利弊的好点。我可以添加：

Upsides: I like the idea of being able to test everything without having to have a database, a web server. This helps enormously in understanding the code, and in refactoring.

优点：我喜欢无需数据库、Web 服务器即可测试所有内容的想法。这对理解代码和重构非常有帮助。

Downsides:

缺点：

The JUnit code will be complex and hard to maintain. You will have to reproduce all of the interactions between code and user within the Junit test, which very quickly becomes complex. If all you are doing is testing one single entry point, then this could well work, but if you are testing multiple screens (a wizard or something like that), then it can quickly become fragile and unmanageable. The problem in my experience is almost always the plumbing code between pages, or the setup code required for a page.

JUnit 代码将很复杂且难以维护。您必须在 Junit 测试中重现代码和用户之间的所有交互，这很快就会变得复杂。如果您所做的只是测试一个单一的入口点，那么这可能很有效，但如果您正在测试多个屏幕（向导或类似的东西），那么它很快就会变得脆弱且无法管理。根据我的经验，问题几乎总是页面之间的管道代码，或者页面所需的设置代码。

One other thing to consider is the flexibility of what you're trying to do. With Fitnesse, it is fairly easy to add another test if you discover a bug. In fact, this is the idea behind Fitnesse: the 'user' can add another test without having to change the code. Don't underestimate the value of this.

要考虑的另一件事是您尝试做的事情的灵活性。使用 Fitnesse，如果您发现错误，添加另一个测试是相当容易的。事实上，这就是 Fitnesse 背后的想法：“用户”可以添加另一个测试而无需更改代码。不要低估它的价值。

So I would suggest two things:

所以我建议两件事：

1) Try it. Take a user story (not too complex, not too simple), and do it in JUnit. Compare what you've done with the Selenium/Fitnesse etc, and see if it is worthwhile. See if you 1. actually find bugs, 2. produce brittle, hard to manage code.

1）试一试。拿一个用户故事（不要太复杂，不要太简单），并在 JUnit 中进行。比较你对 Selenium/Fitnesse 等所做的事情，看看它是否值得。看看你是否 1. 真正发现了错误， 2. 产生脆弱的、难以管理的代码。

2) Could you use the data created by the Fitnesse as input to your JUnit tests (thus removing the need for a web server etc). You could possibly read the data and call the methods in the Java classes directly. Again, you may have problems with setup & database code.

2) 您能否使用 Fitnesse 创建的数据作为 JUnit 测试的输入（从而消除对 Web 服务器等的需求）。您可以直接读取数据并调用 Java 类中的方法。同样，您可能会遇到设置和数据库代码的问题。

回答by ring bearer

IMHO Not a good idea. Apart from the 2 cons that you mentioned, how about the vast amount of effort required to build an acceptance test framework that will deal all kinds of web/UI/glue scenarios that you might have? Your point "Simplicity (no additional frameworks required)" is incorrect as your organization would have to invest on building a more elaborate framework on top of JUnit. It may be more than JUnit.

恕我直言，这不是一个好主意。除了您提到的 2 个缺点，构建验收测试框架所需的大量工作如何处理您可能拥有的各种 Web/UI/胶水场景？您的观点“简单（不需要额外的框架）”是不正确的，因为您的组织将不得不投资在 JUnit 之上构建一个更复杂的框架。它可能不仅仅是 JUnit。