java 使用 HTTP POST 时 Solr 查询字符串是否有大小或期限限制?

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/16201195/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-10-31 22:11:21  来源:igfitidea点击:

Is there a size or term limit for a Solr query string when using HTTP POST?

javasolrhttp-post

提问by mattgmg1990

I'm using Java to query a Solr server for results that have IDs within a set of known IDs that I am interested in.

我正在使用 Java 查询 Solr 服务器以获取在我感兴趣的一组已知 ID 中具有 ID 的结果。

The best way I could think to get just these results that I am interested in was to create a long query string that looks something like this:

我认为获得这些我感兴趣的结果的最好方法是创建一个长查询字符串,如下所示:

q=(item_id:XXX33-3333 OR item_id:YYY42-3445 OR item_id:JFDE-3838)

q=(item_id:XXX33-3333 OR item_id:YYY42-3445 OR item_id:JFDE-3838)

I generate this String, queryString, before making my request, and there are over 1500 such ids included in the request I would eventually like to make. I am using an HTTP POST to make the query as such:

queryString在提出请求之前生成了这个字符串,并且在我最终想要提出的请求中包含了超过 1500 个这样的 id。我正在使用 HTTP POST 进行查询:

        HttpPost post = new HttpPost(url);
        post.setHeader("Content-Type", "application/x-www-form-urlencoded; charset=utf-8");

        StringEntity entity = new StringEntity(queryString, "UTF-8");
        entity.setContentType("application/x-www-form-urlencoded; charset=utf-8");
        post.setEntity(entity);

        HttpClient client = new DefaultHttpClient();
        HttpResponse response = client.execute(post);

If I limit the query to just the first 1000 ids, it succeeds and I get the results back as I would expect. However, if I increase the query to include all 1500 that I am really interested in, I get an HTTP 400 response code with the following error:

如果我将查询限制为前 1000 个 ID,它就会成功,并且我会如我所愿地得到结果。但是,如果我增加查询以包含我真正感兴趣的所有 1500,我会收到一个 HTTP 400 响应代码,并显示以下错误:

HTTP/1.1 400 org.apache.lucene.queryParser.ParseException: Cannot parse '[my query here...]

HTTP/1.1 400 org.apache.lucene.queryParser.ParseException: Cannot parse '[my query here...]

Is there a limit to the number of ids that I can OR together in a Solr query? Is there another reason this might be failing when I go past 1000? I have experimented and it fails at around 1024 (my ids are all almost the same length) so it seems to suggest there is a character or term limit.

我可以在 Solr 查询中 OR 在一起的 ID 数量是否有限制?当我超过 1000 时,这是否还有其他原因可能会失败?我已经尝试过,它在 1024 左右失败(我的 ID 长度几乎相同),因此它似乎表明存在字符或期限限制。

Or, if someone has a good suggestion of how I can retrieve the items I'm looking for in another, smarter, way, I would love to hear it. My backup solution is just to query Solr for allitems, parse the results, and use the ones that belong to the set I am interested in. I would prefer not to do this, since the data source could have tens of thousands of items, and it would be inefficient.

或者,如果有人对我如何以另一种更智能的方式检索我正在寻找的项目有很好的建议,我很想听听。我的备份解决方案只是查询 Solr 的所有项目,解析结果,并使用属于我感兴趣的集合的那些。我不想这样做,因为数据源可能有数万个项目,这将是低效的。

回答by nikhil500

There is no limit on the Solr side - we regularly use Solr in a similar way with tens of thousands of IDs in the query.

Solr 方面没有限制——我们经常以类似的方式使用 Solr,查询中有数万个 ID。

You need to look at the settings for your servlet container (Tomcat, Jetty etc.) and increase the maximum POST size. Look up maxPostSizeif you are using Tomcat and maxFormContentSizeif you are using Jetty.

您需要查看 servlet 容器(Tomcat、Jetty 等)的设置并增加最大 POST 大小。查询maxPostSize您是否使用 Tomcat 以及maxFormContentSize是否使用 Jetty。

回答by kellyfj

As of Solr 6.0 there is a maxBooleanClausesconfiguration within Solr - defaults to 1024.

从 Solr 6.0 开始,Solr 中有一个maxBooleanClauses配置 - 默认为 1024。

I wrote a unit test to confirm and confirmed the limitation (with Solr 5.3).

我写了一个单元测试来确认和确认限制(使用 Solr 5.3)。

See more here https://wiki.apache.org/solr/SolrConfigXml#The_Query_Section

在此处查看更多信息https://wiki.apache.org/solr/SolrConfigXml#The_Query_Section

FWIW there is an open Solr JIRA to remove it so it may be removed in the future https://issues.apache.org/jira/browse/SOLR-4586

FWIW 有一个开放的 Solr JIRA 可以删除它,因此将来可能会删除它 https://issues.apache.org/jira/browse/SOLR-4586