java 如何使用IDE在storm生产集群中提交拓扑
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/15781176/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How to submit a topology in storm production cluster using IDE
提问by abhi
I am facing an issue Must submit topologies using the 'storm' client script so that StormSubmitter knows which jar to uploadwhile submitting a topology to a production cluster using IDE, while the same thing if i perform in command line using storm jarcommand, its running like heaven. I have seen examples of the same from githublink.
我在Must submit topologies using the 'storm' client script so that StormSubmitter knows which jar to upload使用 IDE 将拓扑提交到生产集群时遇到了一个问题，而如果我使用storm jar命令在命令行中执行同样的事情，它的运行就像天堂一样。我从githublink看到了相同的例子。
For submitting topology i am using these set of lines
为了提交拓扑，我正在使用这些行
conf.put(Config.NIMBUS_HOST, NIMBUS_NODE);
conf.put(Config.NIMBUS_THRIFT_PORT,6627);
conf.put(Config.STORM_ZOOKEEPER_PORT,2181);
conf.put(Config.STORM_ZOOKEEPER_SERVERS,ZOOKEEPER_ID);
conf.setNumWorkers(20);
conf.setMaxSpoutPending(5000);
StormSubmitter submitter = new StormSubmitter();
submitter.submitTopology("test", conf, builder.createTopology());
Please suggest me if this is the correct approach to run?
请建议我这是否是正确的运行方法？
回答by abhi
Well found the solution. When we ran "storm jar" it trigger a property flag for storm.jar in the submitted jar. So if we want to programmatically submit a jar then simply set the flag this way
很好地找到了解决方案。当我们运行“storm jar”时，它会在提交的jar 中触发storm.jar 的属性标志。因此，如果我们想以编程方式提交一个 jar，那么只需以这种方式设置标志
System.setProperty("storm.jar", <path-to-jar>);
System.setProperty("storm.jar", <path-to-jar>);
For example:
例如：
System.setProperty("storm.jar", "/Users/programming/apache-storm-1.0.1/lib/storm-core-1.0.1.jar");
StormSubmitter.submitTopology("myTopology", config, builder.createTopology());
回答by Nishu Tayal
For submitting a topology to remote Storm cluster, you need to upload that jar to nimbus machine and then submit that jar to Cluster using NimbusClient.
You can do it like this:
要将拓扑提交到远程 Storm 集群，您需要将该 jar 上传到 nimbus 机器，然后使用 NimbusClient 将该 jar 提交到集群。
你可以这样做：
Map storm_conf = Utils.readStormConfig();
storm_conf.put("nimbus.host", "<Nimbus Machine IP>");
Client client = NimbusClient.getConfiguredClient(storm_conf)
.getClient();
String inputJar = "C:\workspace\TestStormRunner\target\TestStormRunner-0.0.1-SNAPSHOT-jar-with-dependencies.jar";
NimbusClient nimbus = new NimbusClient(storm_conf, "<Nimbus Machine IP>",
<Nimbus Machine Port>);
// upload topology jar to Cluster using StormSubmitter
String uploadedJarLocation = StormSubmitter.submitJar(storm_conf,
inputJar);
String jsonConf = JSONValue.toJSONString(storm_conf);
nimbus.getClient().submitTopology("testtopology",
<uploadedJarLocation>, jsonConf, builder.createTopology());
Here is the working example : Submitting a topology to Remote Storm Cluster
这是工作示例：向远程风暴集群提交拓扑
回答by Alex
I didn't run java code for submitting myself, but I checked storm command - and it's a python file, which runs java and http://nathanmarz.github.com/storm/doc/backtype/storm/StormSubmitter.htmlclass
我没有运行用于提交自己的 java 代码，但我检查了 Storm 命令 - 它是一个 python 文件，它运行 java 和http://nathanmarz.github.com/storm/doc/backtype/storm/StormSubmitter.html类
The only thing I think you should worry about - is to include all needed libraries, when executing it.
我认为您应该担心的唯一一件事是在执行时包含所有需要的库。
回答by leon
I have resolved this this problem based on @abhi and @Nishu Tayal's answers, I'd like to post my code here:
我已经根据@abhi 和@Nishu Tayal 的回答解决了这个问题，我想在这里发布我的代码：
public static void submitLocalTopologyWay1(String topologyName, Config topologyConf,
StormTopology topology, String localJar) {
try {
//get default storm config
Map defaultStormConf = Utils.readStormConfig();
defaultStormConf.putAll(topologyConf);
//set JAR
System.setProperty("storm.jar",localJar);
//submit topology
StormSubmitter.submitTopology(topologyName, defaultStormConf, topology);
} catch (Exception e) {
String errorMsg = "can't deploy topology " + topologyName + ", " + e.getMessage();
System.out.println(errorMsg);
e.printStackTrace();
}
}
public static void submitLocalTopologyWay2(String topologyName, Config topologyConf,
StormTopology topology, String localJar) {
try {
//get nimbus client
Map defaultStormConf = Utils.readStormConfig();
defaultStormConf.putAll(topologyConf);
Client client = NimbusClient.getConfiguredClient(defaultStormConf).getClient();
//upload JAR
String remoteJar = StormSubmitter.submitJar(defaultStormConf, localJar);
//submit topology
client.submitTopology(topologyName, remoteJar, JSONValue.toJSONString(topologyConf), topology);
} catch (Exception e) {
String errorMsg = "can't deploy topology " + topologyName + ", " + e.getMessage();
System.out.println(errorMsg);
e.printStackTrace();
}
}
then here is a test, and you must build your code to a JAR file first.
那么这里是一个测试，您必须先将代码构建到 JAR 文件中。
public void testSubmitTopologySubmitLocalTopologyWay1() {
Config config = new Config();
config.put(Config.NIMBUS_HOST,"9.119.84.179");
config.put(Config.NIMBUS_THRIFT_PORT, 6627);
config.put(Config.STORM_ZOOKEEPER_SERVERS, Arrays.asList("9.119.84.177","9.119.84.178","9.119.84.176"));
config.put(Config.STORM_ZOOKEEPER_PORT,2181);
config.put(Config.TOPOLOGY_WORKERS, 3);
RemoteSubmitter.submitLocalTopologyWay1("word-count-test-1", config,
WordCountTopology.buildTopology(), // your topology
"C:\MyWorkspace\project\storm-sample-0.0.1-SNAPSHOT-jar-with-dependencies.jar");//the JAR file
}
