Java 如何自动化Kafka测试

How to automate Kafka Testing

提问by Alpana Chauhan

We have developed a small system,using kafka to queue the data and later consume that data to place orders for users. We have tested certain things manually, but now our aim is automate the process. Is there any client available to test it.While googling things out i found out ways to Unit test it using kafka client itself, but my aim is to test the system as whole. Please Guide

我们开发了一个小系统，使用 kafka 对数据进行排队，然后使用该数据为用户下订单。我们已经手动测试了某些东西，但现在我们的目标是自动化这个过程。是否有任何客户端可用于测试它。在谷歌搜索时，我找到了使用 kafka 客户端本身对其进行单元测试的方法，但我的目标是测试整个系统。请指导

EDIT: our purpose is just API testing i.e., just the back-end not the UI

编辑：我们的目的只是 API 测试，即只是后端而不是 UI

采纳答案by Pawe? Szymczyk

You can start Kafka programmatically in your integration test, Kafka uses Zookeeper so firsly look at Zookeeper TestingServer- instance of this class creates and starts the Zk server using the given port.

您可以在集成测试中以编程方式启动 Kafka，Kafka 使用 Zookeeper，因此首先查看 Zookeeper TestingServer-此类的实例使用给定端口创建并启动 Zk 服务器。

Next look at KafkaServerStartable.scala, you have to provide configuration that points to your in memory Zk server and invoke startup()method, here is some code:

接下来看KafkaServerStartable.scala，你必须提供指向内存中 Zk 服务器的配置和调用startup()方法，这里是一些代码：

import kafka.server.KafkaConfig; 
import kafka.server.KafkaServerStartable;
import java.util.Properties;

public KafkaTest() {
    Properties properties = createProperties();
    KafkaConfig kafkaConfig = new KafkaConfig(properties);
    KafkaServerStartable kafka = new KafkaServerStartable(kafkaConfig);
    kafka.startup();
}

Hope these help:)

希望这些有帮助:)

回答by Arunmozhi

Below answer is for someone who still looking for a working sample.

以下答案适用于仍在寻找工作样本的人。

Created sample in-memory kafka for integration testing purpose using KafkaServerStartable. Anyone could extend this functionality for their use-case.

使用 KafkaServerStartable 创建了用于集成测试目的的示例内存 kafka。任何人都可以为他们的用例扩展此功能。

• No modification needed. Easy download and run.
• Added sample test case to cover produce and consume message to a topic.

• 无需修改。轻松下载并运行。
• 添加了示例测试用例以涵盖主题的生产和消费消息。

Checkout the link here --> Kafka Integration Test

在此处查看链接 --> Kafka 集成测试

回答by Sidd Gautama

You can go for integration-testing or end-to-end testing by bringing up Kafka in a dockercontainer. If you use Apachekafka-clients:2.1.0, then you don't need to deal with ZooKeeperat the API level while producing or consuming the records.

您可以通过在docker容器中启动 Kafka 来进行集成测试或端到端测试。如果您使用Apachekafka-clients:2.1.0，则在生成或使用记录时无需在 API 级别处理ZooKeeper。

Dockerizing Kafka, and testing helps to cover the scenarios in a single node as well as multi-node Kafka cluster. This way you don't have to test against Mock/In-Memory Kafka once, then real Kafka later.

Dockerizing Kafka，测试有助于覆盖单节点和多节点Kafka集群中的场景。这样你就不必对 Mock/In-Memory Kafka 进行一次测试，然后再对真正的 Kafka 进行测试。

If you have too many test scenarios to cover, you can go for Kafka Declarative Testinglike docker-composestyle, by which you can eliminate the Kafka client API coding.

如果你有太多的测试场景需要覆盖，你可以选择类似Kafka Declarative Testing 的docker-compose风格，通过它你可以消除 Kafka 客户端 API 编码。

Checkout some handy examples here for validating produceand consume.

在此处查看一些方便的示例以验证生产和消费。