iOS 测试/规范 TDD/BDD 和集成与验收测试

iOS Tests/Specs TDD/BDD and Integration & Acceptance Testing

提问by ma11hew28

What are the best technologies to use for behavior-driven development on the iPhone? And what are some open source example projects that demonstrate sound use of these technologies? Here are some options I've found:

在 iPhone 上进行行为驱动开发的最佳技术是什么？哪些开源示例项目展示了这些技术的合理使用？以下是我找到的一些选项：

---

---

Unit Testing

单元测试

Test::UnitStyle

测试::单元样式

1. OCUnit/SenTestingKitas explained in iOS Development Guide: Unit Testing Applications& other OCUnit references.
   • Examples: iPhoneUnitTests, Three20
2. CATCH
3. GHUnit
4. Google Toolbox for Mac: iPhone Unit Testing

1. OCUnit/SenTestingKit，如iOS 开发指南：单元测试应用程序和其他OCUnit 参考中所述。
   • 示例：iPhoneUnitTests，Three20
2. 抓住
3. 温室气体单位
4. 适用于 Mac 的 Google 工具箱：iPhone 单元测试

RSpecStyle

RSpec样式

1. Kiwi(which also comes with mocking & expectations)
2. Cedar
3. Jasminewith UI Automationas shown in dexterous' iOS-Acceptance-Testing specs

1. 猕猴桃（也带有嘲讽和期望）
2. 雪松
3. Jasmine具有UI 自动化功能，如灵巧的 iOS 验收测试规范所示

---

---

Acceptance Testing

验收测试

SeleniumStyle

硒风格

1. UI Automation (works on device)

   • UI Automation Instruments Guide
   • UI Automation reference documentation
   • Tuneup js- cool library for using with UIAutomation.

   • Capturing User Interface Actions into Automation Scripts

     It's possible to use Cucumber (written in JavaScript)to drive UI Automation. This would be a great open-source project. Then, we could write Gherkinto run UI Automation testing. For now, I'll just write Gherkin as comments.

   UPDATE: Zucchini Frameworkseems to blend Cucumber & UI Automation! :)

   Old Blog Posts:

   • Alex Vollmer's UI Automation tutorial
   • O'Reilly Answers UI Automation tutorial
   • Adi Saxena's UI Automation tutorial

2. UISpecwith UISpecRunner

   • UISpec is open source on Google Code.
   • UISpec has comprehensive documentation.

3. FoneMonkey

1. UI 自动化（适用于设备）

   • UI自动化仪器指南
   • UI 自动化参考文档
   • Tuneup js- 与 UIAutomation 一起使用的酷库。

   • 将用户界面操作捕获到自动化脚本中

     可以使用Cucumber（用 JavaScript 编写）来驱动 UI 自动化。这将是一个伟大的开源项目。然后，我们可以编写Gherkin来运行 UI 自动化测试。现在，我只会写 Gherkin 作为评论。

   更新：Zucchini 框架似乎融合了 Cucumber 和 UI 自动化！:)

   旧博客文章：

   • Alex Vollmer 的 UI 自动化教程
   • O'Reilly 回答 UI 自动化教程
   • Adi Saxena 的 UI 自动化教程

2. UISpec与UISpecRunner

   • UISpec 在 Google Code 上是开源的。
   • UISpec 有全面的文档。

3. FoneMonkey

CucumberStyle

黄瓜风格

1. Frankand iCuke(based on the Cucumber meets iPhone talk)

   • The Frank Google Grouphas much more activity than the iCuke Google Group.
   • Frank runs on both device and simulator, while iCuke only runs in simulator.
   • Frank seems to have a more comprehensive set of step definitions than iCuke's step definitions. And, Frank also has a step definition compendium on their wiki.
   • I proposed that we merge iCuke & Frank(similar to how Merb & Rails merged) since they have the same common goal: Cucumber for iOS.

2. KIF (Keep It Functional)by Square

3. Zucchini Frameworkuses Cucumber syntax for writing tests and uses CoffeeScript for step definitions.

1. Frank和iCuke（基于Cucumber 遇到 iPhone 谈话）

   • 在弗兰克谷歌集团具有比更活动iCuke谷歌集团。
   • Frank 可以在设备和模拟器上运行，而 iCuke 只在模拟器中运行。
   • Frank 似乎有一套比iCuke 的步骤定义更全面的步骤定义。而且，Frank 在他们的 wiki 上也有一个步骤定义纲要。
   • 我提议我们合并 iCuke 和 Frank（类似于 Merb 和 Rails 的合并方式），因为他们有相同的共同目标：iOS 版 Cucumber。

2. KIF（保持功能）由广场

3. Zucchini Framework使用 Cucumber 语法编写测试并使用 CoffeeScript 进行步骤定义。

Additions

添加

• OCMockfor mocking
• OCHamcrestand/or Expectafor expectations

• OCMock的嘲讽
• OCHamcrest和/或Expecta的期望

Conclusion

结论

Well, obviously, there's no right answer to this question, but here's what I'm choosing to go with currently:

嗯，显然，这个问题没有正确的答案，但这是我目前选择的：

For unit testing, I used to use OCUnit/SenTestingKitin XCode 4. It's simple & solid. But, I prefer the language of BDD over TDD (Why is RSpec better than Test::Unit?) because our words create our world. So now, I use Kiwi with ARC& Kiwi code completion/autocompletion. I prefer Kiwi over Cedar because it's built on top of OCUnit and comes with RSpec-style matchers & mocks/stubs. UPDATE: I'm now looking into OCMock because, currently, Kiwi doesn't support stubbing toll-free bridged objects.

对于单元测试，我曾经在 XCode 4 中使用OCUnit/SenTestingKit。它简单而可靠。但是，我更喜欢 BDD 的语言而不是 TDD（为什么 RSpec 比 Test::Unit 更好？）因为我们的语言创造了我们的世界。所以现在，我将Kiwi 与 ARC和Kiwi 代码完成/自动完成一起使用。我更喜欢 Kiwi 而不是 Cedar，因为它建立在 OCUnit 之上，并带有 RSpec 风格的匹配器和模拟/存根。更新：我现在正在研究 OCMock，因为目前Kiwi 不支持存根免费桥接对象。

For acceptance testing, I use UI Automation because it's awesome. It lets you record each test case, making writing tests automatic. Also, Apple develops it, and so it has a promising future. It also works on the device and from Instruments, which allows for other cool features, like showing memory leaks. Unfortunately, with UI Automation, I don't know how to run Objective-C code, but with Frank & iCuke you can. So, I'll just test the lower-level Objective-C stuff with unit tests, or create UIButtons only for the TESTbuild configuration, which when clicked, will run Objective-C code.

对于验收测试，我使用 UI 自动化，因为它很棒。它可以让您记录每个测试用例，从而自动编写测试。此外，Apple 开发了它，因此它有一个充满希望的未来。它也适用于设备和 Instruments，它允许其他很酷的功能，比如显示内存泄漏。不幸的是，使用 UI 自动化，我不知道如何运行 Objective-C 代码，但是使用 Frank & iCuke 可以。因此，我将使用单元测试来测试较低级别的 Objective-C 内容，或者UIButton仅为TEST构建配置创建s ，单击该配置将运行 Objective-C 代码。

Which solutions do you use?

您使用哪些解决方案？

Related Questions

相关问题

• Is there a BDD solution that presently works well with iOS4 and Xcode4?
• SenTestingKit (integrated with XCode) versus GHUnit on XCode 4 for Unit Testing?
• Testing asynchronous code on iOS with OCunit
• SenTestingKit in Xcode 4: Asynchronous testing?
• How does unit testing on the iPhone work?

• 目前是否有适用于 iOS4 和 Xcode4 的 BDD 解决方案？
• SenTestingKit（与 XCode 集成）与 XCode 4 上的 GHUnit 进行单元测试？
• 使用 OCunit 在 iOS 上测试异步代码
• Xcode 4 中的 SenTestingKit：异步测试？
• iPhone 上的单元测试是如何工作的？

回答by Adam Milligan

tl;dr

tl;博士

At Pivotal we wrote Cedar because we use and love Rspec on our Ruby projects. Cedar isn't meant to replace or compete with OCUnit; it's meant to bring the possibility of BDD-style testing to Objective C, just as Rspec pioneered BDD-style testing in Ruby, but hasn't eliminated Test::Unit. Choosing one or the other is largely a matter of style preferences.

在 Pivotal，我们编写了 Cedar，因为我们在 Ruby 项目中使用并喜欢 Rspec。Cedar 并不是要取代 OCUnit 或与 OCUnit 竞争。它旨在为 Objective C 带来 BDD 风格测试的可能性，就像 Rspec 在 Ruby 中开创 BDD 风格测试一样，但并没有消除 Test::Unit。选择其中一个很大程度上取决于风格偏好。

In some cases we designed Cedar to overcome some shortcomings in the way OCUnit works for us. Specifically, we wanted to be able to use the debugger in tests, to run tests from the command line and in CI builds, and get useful text output of test results. These things may be more or less useful to you.

在某些情况下，我们设计 Cedar 是为了克服 OCUnit 为我们工作的方式中的一些缺点。具体来说，我们希望能够在测试中使用调试器，从命令行和 CI 构建中运行测试，并获得有用的测试结果文本输出。这些东西可能或多或少对你有用。

Long answer

长答案

Deciding between two testing frameworks like Cedar and OCUnit (for example) comes down to two things: preferred style, and ease of use. I'll start with the style, because that's simply a matter of opinion and preference; ease of use tends to be a set of tradeoffs.

在两个测试框架（例如 Cedar 和 OCUnit）之间做出决定归结为两件事：首选风格和易用性。我将从风格开始，因为这只是意见和偏好的问题；易用性往往是一组权衡。

Style considerations transcend what technology or language you use. xUnit-style unit testing has been around for far longer than BDD-style testing, but the latter has rapidly gained in popularity, largely due to Rspec.

风格考虑超越了您使用的技术或语言。xUnit 风格的单元测试比 BDD 风格的测试存在的时间要长得多，但后者迅速流行起来，主要是由于 Rspec。

The primary advantage of xUnit-style testing is its simplicity, and wide adoption (amongst developers who write unit tests); nearly any language you could consider writing code in has an xUnit-style framework available.

xUnit 样式测试的主要优点是它的简单性和广泛采用（在编写单元测试的开发人员中）；几乎任何您可以考虑用其编写代码的语言都有一个可用的 xUnit 风格的框架。

BDD-style frameworks tend to have two main differences when compared to xUnit-style: how you structure the test (or specs), and the syntax for writing your assertions. For me, the structural difference is the main differentiator. xUnit tests are one-dimensional, with one setUp method for all tests in a given test class. The classes that we test, however, aren't one-dimensional; we often need to test actions in several different, potentially conflicting, contexts. For example, consider a simple ShoppingCart class, with an addItem: method (for the purposes of this answer I'll use Objective C syntax). The behavior of this method may differ when the cart is empty compared to when the cart contains other items; it may differ if the user has entered a discount code; it may differ if the specified item can't be shipped by the selected shipping method; etc. As these possible conditions intersect with one another you end up with a geometrically increasing number of possible contexts; in xUnit-style testing this often leads to a lot of methods with names like testAddItemWhenCartIsEmptyAndNoDiscountCodeAndShippingMethodApplies. The structure of BDD-style frameworks allows you to organize these conditions individually, which I find makes it easier to make sure I cover all cases, as well as easier to find, change, or add individual conditions. As an example, using Cedar syntax, the method above would look like this:

与 xUnit 风格相比，BDD 风格的框架往往有两个主要区别：如何构建测试（或规范），以及编写断言的语法。对我来说，结构差异是主要的区别。xUnit 测试是一维的，对给定测试类中的所有测试都有一个 setUp 方法。然而，我们测试的类不是一维的；我们经常需要在几个不同的、可能相互冲突的环境中测试操作。例如，考虑一个简单的 ShoppingCart 类，它有一个 addItem: 方法（为了这个答案，我将使用 Objective C 语法）。当购物车为空时，与购物车包含其他物品时相比，此方法的行为可能有所不同；如果用户输入了折扣代码，则可能会有所不同；如果指定的项目可以，它可能会有所不同' t 通过选择的运输方式运输；等等。当这些可能的条件相互交叉时，你最终会得到数量呈几何级数增加的可能上下文；在 xUnit 样式的测试中，这通常会导致很多方法的名称类似于 testAddItemWhenCartIsEmptyAndNoDiscountCodeAndShippingMethodApplies。BDD 风格框架的结构允许您单独组织这些条件，我发现这样可以更轻松地确保涵盖所有情况，也更易于查找、更改或添加单个条件。例如，使用 Cedar 语法，上面的方法如下所示：在 xUnit 样式的测试中，这通常会导致很多方法的名称类似于 testAddItemWhenCartIsEmptyAndNoDiscountCodeAndShippingMethodApplies。BDD 风格框架的结构允许您单独组织这些条件，我发现这样可以更轻松地确保涵盖所有情况，也更易于查找、更改或添加单个条件。例如，使用 Cedar 语法，上面的方法如下所示：在 xUnit 样式的测试中，这通常会导致很多方法的名称类似于 testAddItemWhenCartIsEmptyAndNoDiscountCodeAndShippingMethodApplies。BDD 风格框架的结构允许您单独组织这些条件，我发现这样可以更轻松地确保涵盖所有情况，也更易于查找、更改或添加单个条件。例如，使用 Cedar 语法，上面的方法如下所示：

describe(@"ShoppingCart", ^{
    describe(@"addItem:", ^{
        describe(@"when the cart is empty", ^{
            describe(@"with no discount code", ^{
                describe(@"when the shipping method applies to the item", ^{
                    it(@"should add the item to the cart", ^{
                        ...
                    });

                    it(@"should add the full price of the item to the overall price", ^{
                        ...
                    });
                });

                describe(@"when the shipping method does not apply to the item", ^{
                    ...
                });
            });

            describe(@"with a discount code", ^{
                ...
            });
        });

        describe(@"when the cart contains other items, ^{
            ...
        });
    });
});

In some cases you'll find contexts in that contain the same sets of assertions, which you can DRY up using shared example contexts.

在某些情况下，您会在其中找到包含相同断言集的上下文，您可以使用共享示例上下文来完成这些断言。

The second main difference between BDD-style frameworks and xUnit-style frameworks, assertion (or "matcher") syntax, simply makes the style of the specs somewhat nicer; some people really like it, others don't.

BDD 风格的框架和 xUnit 风格的框架之间的第二个主要区别，断言（或“匹配器”）语法，只是使规范的风格更好一些；有些人真的很喜欢它，有些人则不喜欢。

That leads to the question of ease of use. In this case, each framework has its pros and cons:

这导致了易用性的问题。在这种情况下，每个框架都有其优点和缺点：

• OCUnit has been around much longer than Cedar, and is integrated directly into Xcode. This means it's simple to make a new test target, and, most of the time, getting tests up and running "just works." On the other hand, we found that in some cases, such as running on an iOS device, getting OCUnit tests to work was nigh impossible. Setting up Cedar specs takes some more work than OCUnit tests, since you have get the library and link against it yourself (never a trivial task in Xcode). We're working on making setup easier, and any suggestions are more than welcome.

• OCUnit runs tests as part of the build. This means you don't need to run an executable to make your tests run; if any tests fail, your build fails. This makes the process of running tests one step simpler, and test output goes directly into your build output window which makes it easy to see. We chose to have Cedar specs build into an executable which you run separately for a few reasons:

  • We wanted to be able to use the debugger. You run Cedar specs just like you would run any other executable, so you can use the debugger in the same way.
  • We wanted easy console logging in tests. You can use NSLog() in OCUnit tests, but the output goes into the build window where you have to unfold the build step in order to read it.
  • We wanted easy to read test reporting, both on the command line and in Xcode. OCUnit results appear nicely in the build window in Xcode, but building from the command line (or as part of a CI process) results in test output intermingled with lots and lots of other build output. With separate build and run phases Cedar separates the output so the test output is easy to find. The default Cedar test runner copies the standard style of printing "." for each passing spec, "F" for failing specs, etc. Cedar also has the ability to use custom reporter objects, so you can have it output results any way you like, with a little effort.

• OCUnit is the official unit testing framework for Objective C, and is supported by Apple. Apple has basically limitless resources, so if they want something done it will get done. And, after all, this is Apple's sandbox we're playing in. The flip side of that coin, however, is that Apple receives on the order of a bajillion support requests and bug reports each day. They're remarkably good about handling them all, but they may not be able to handle issues you report immediately, or at all. Cedar is much newer and less baked than OCUnit, but if you have questions or problems or suggestions send a message to the Cedar mailing list ([email protected]) and we'll do what we can to help you out. Also, feel free to fork the code from Github (github.com/pivotal/cedar) and add whatever you think is missing. We make our testing frameworks open source for a reason.

• Running OCUnit tests on iOS devices can be difficult. Honestly, I haven't tried this for quite some time, so it may have gotten easier, but the last time I tried I simply couldn't get OCUnit tests for any UIKit functionality to work. When we wrote Cedar we made sure that we could test UIKit-dependent code both on the simulator and on devices.

• OCUnit 的存在时间比 Cedar 长得多，并且直接集成到 Xcode 中。这意味着创建新的测试目标很简单，而且在大多数情况下，启动和运行测试“正常工作”。另一方面，我们发现在某些情况下，例如在 iOS 设备上运行，让 OCUnit 测试工作几乎是不可能的。设置 Cedar 规范比 OCUnit 测试需要更多的工作，因为您已经获得了库并自己链接到它（在 Xcode 中绝不是一项微不足道的任务）。我们正在努力使设置更容易，任何建议都非常受欢迎。

• OCUnit 运行测试作为构建的一部分。这意味着您不需要运行可执行文件来运行您的测试；如果任何测试失败，你的构建失败。这使得运行测试的过程更简单，并且测试输出直接进入您的构建输出窗口，使其易于查看。我们选择将 Cedar 规范构建到您单独运行的可执行文件中，原因如下：

  • 我们希望能够使用调试器。您可以像运行任何其他可执行文件一样运行 Cedar 规范，因此您可以以相同的方式使用调试器。
  • 我们想要简单的控制台登录测试。您可以在 OCUnit 测试中使用 NSLog()，但输出会进入构建窗口，您必须在其中展开构建步骤才能读取它。
  • 我们想要易于阅读的测试报告，无论是在命令行还是在 Xcode 中。OCUnit 结果很好地显示在 Xcode 的构建窗口中，但是从命令行构建（或作为 CI 过程的一部分）会导致测试输出与大量其他构建输出混合在一起。通过单独的构建和运行阶段，Cedar 将输出分开，因此测试输出很容易找到。默认的 Cedar 测试运行器复制标准的打印样式“.”。对于每个通过的规范，“F”表示失败的规范等。Cedar 还能够使用自定义报告器对象，因此您可以轻松地以任何您喜欢的方式输出结果。

• OCUnit 是 Objective C 的官方单元测试框架，并得到 Apple 的支持。苹果基本上拥有无限的资源，所以如果他们想完成某件事，它就会完成。毕竟，这是我们正在玩的苹果沙盒。然而，硬币的另一面是苹果每天收到大量支持请求和错误报告。他们非常擅长处理所有问题，但他们可能无法立即处理您报告的问题，或者根本无法处理。与 OCUnit 相比，Cedar 更新且不那么成熟，但如果您有任何疑问、问题或建议，请向 Cedar 邮件列表 ([email protected]) 发送消息，我们将竭尽所能为您提供帮助。另外，请随意从 Github (github.com/pivotal/cedar) fork 代码并添加您认为缺少的任何内容。

• 在 iOS 设备上运行 OCUnit 测试可能很困难。老实说，我已经有一段时间没有尝试过这个了，所以它可能变得更容易了，但是上次我尝试时，我根本无法让任何 UIKit 功能的 OCUnit 测试工作。当我们编写 Cedar 时，我们确保我们可以在模拟器和设备上测试与 UIKit 相关的代码。

Finally, we wrote Cedar for unit testing, which means it's not really comparable with projects like UISpec. It's been quite a while since I tried using UISpec, but I understood it to be focused primarily on programmatically driving the UI on an iOS device. We specifically decided not to try to have Cedar support these types of specs, since Apple was (at the time) about to announce UIAutomation.

最后，我们为单元测试编写了 Cedar，这意味着它无法与 UISpec 等项目相提并论。自从我尝试使用 UISpec 已经有一段时间了，但我理解它主要专注于以编程方式驱动 iOS 设备上的 UI。我们特别决定不尝试让 Cedar 支持这些类型的规范，因为 Apple（当时）即将宣布 UIAutomation。

回答by raidfive

I'm going to have to toss Frankinto the acceptance testing mix. This is a fairly new addition but has worked excellent for me so far. Also, it is actually being actively worked on, unlike icuke and the others.

我将不得不把弗兰克扔到验收测试组合中。这是一个相当新的补充，但到目前为止对我来说效果很好。此外，与 iuke 和其他人不同，它实际上正在积极开展工作。

回答by Richard J. Ross III

For test driven development, I like to use GHUnit, its a breeze to set up, and works great for debugging too.

对于测试驱动开发，我喜欢使用GHUnit，它的设置轻而易举，而且对调试也很有效。

回答by mzaks

Great List!

很棒的名单！

I found another interesting solution for UI testing iOS applications.

我发现了另一个有趣的 UI 测试 iOS 应用程序的解决方案。

Zucchini Framework

西葫芦框架

It is based on UIAutomation. The framework let you write screen centric scenarios in Cucumber like style. The scenarios can be executed in Simulator and on device from a console (it is CI friendly).

它基于UIAutomation. 该框架允许您以类似 Cucumber 的风格编写以屏幕为中心的场景。这些场景可以在模拟器和设备上从控制台执行（它是 CI 友好的）。

The assertions are screenshot based. Sounds inflexible, but it gets you nice HTML report, with highlighted screen comparison and you can provide masks which define the regions you want to have pixel exact assertion.

断言是基于屏幕截图的。听起来不灵活，但它为您提供了不错的 HTML 报告，突出显示的屏幕比较，您可以提供定义要进行像素精确断言的区域的掩码。

Each screen has to be described in CoffeScriptand the tool it self is written in ruby. It is kind of polyglott nightmare, but the tool provides a nice abstraction for UIAutomationand when the screens are described it is manageable even for QA person.

每个屏幕都必须在其中进行描述，CoffeScript并且它本身的工具是用 ruby​​ 编写的。这是一种多语言的噩梦，但该工具提供了一个很好的抽象，UIAutomation并且当描述屏幕时，即使对于 QA 人员也是可以管理的。

回答by SamCee

I would choose iCuke for acceptance tests and Cedar for unit tests. UIAutomation is a step in the right direction for Apple, but the tools need better support for continuous integration; automatically running UIAutomation tests with Instruments is currently not possible, for example.

我会选择 iCuke 进行验收测试，选择 Cedar 进行单元测试。UIAutomation 对 Apple 来说是朝着正确方向迈出的一步，但这些工具需要更好地支持持续集成；例如，当前无法使用 Instruments 自动运行 UIAutomation 测试。

回答by Rob

GHUnit is good for unit tests; for integration tests, I've used UISpec with some success (github fork here: https://github.com/drync/UISpec), but am looking forward to trying iCuke, since it promises to be a lightweight setup, and you can use the rails testing goodness, like RSpec and Cucumber.

GHUnit 适用于单元测试；对于集成测试，我已经使用 UISpec 取得了一些成功（这里的 github 分支：https: //github.com/drync/UISpec），但我期待尝试 iCuke，因为它承诺是一个轻量级的设置，你可以使用Rails进行测试善良，喜欢的RSpec和黄瓜。

回答by Keith Smiley

I currently use spectafor rspec like setups and it's partner (as mentioned above) expectawhich has ton's of awesome matching options.

我目前将specta用于rspec之类的设置，它的合作伙伴（如上所述）expecta有很多很棒的匹配选项。

回答by Eric Smith

I happen to really like OCDSpec2 but I'm biased, I wrote OCDSpec and contribute to the second.

我碰巧真的很喜欢 OCDSpec2 但我有偏见，我写了 OCDSpec 并为第二个做出了贡献。

It's very fast even on iOS, in part because it's built from the ground up rather than being put on top of OCUnit. It has an RSpec/Jasmine syntax as well.

即使在 iOS 上它也非常快，部分原因是它是从头开始构建的，而不是放在 OCUnit 之上。它也有一个 RSpec/Jasmine 语法。

https://github.com/ericmeyer/ocdspec2

https://github.com/ericmeyer/ocdspec2