Java Jackson->Jackson + HttpPost = "无效的 UTF-8 中间字节"，设置 Mime 和编码

Hymanson->Hymanson + HttpPost = "Invalid UTF-8 middle byte", Setting Mime and Encoding

提问by Mark Bennett

I'm using the Apache HTTP Client libs and Hymanson in my client. When I post JSON to the server I get the error:

我在我的客户端中使用 Apache HTTP 客户端库和 Hymanson。当我将 JSON 发布到服务器时，出现错误：

org.codehaus.Hymanson.JsonParseException: Invalid UTF-8 middle byte 0x65
 at [Source: HttpInputOverHTTP@22a4ac95; line: 1, column: 81]

If I don't set any headers than I get invalid media type, which makes sense.

如果我没有设置任何标题而不是我得到的invalid media type，这是有道理的。

If I use curl and the same headers, the server accepts it, so I think the server is OK (and just a coincidence that it's also using Hymanson)

如果我使用 curl 和相同的标头，服务器会接受它，所以我认为服务器没问题（只是巧合，它也在使用 Hymanson）

These is the document; I've hard coded it as a Java literal using only 8 bit characters to avoid any other place for mangling to happen

这些是文件；我仅使用 8 位字符将其硬编码为 Java 文字，以避免在任何其他地方发生重整

// "Stra\u00DFe" = "Stra?e"
static String TINY_UTF8_DOC = "[{ \"id\" : \"2\", \"fields\" : { \"subject\" : [{ \"name\" : \"subject\", \"value\" : \"Stra\u00DFe\" }] } }]";

Here's the code I've been using, and comments with the various attempts:

这是我一直在使用的代码，以及对各种尝试的评论：

HttpClient httpClient = new DefaultHttpClient();
HttpPost post = new HttpPost( url );

// Attempt A
// post.setEntity(  new StringEntity( content )  );

// Attempt B
// post.setEntity(  new StringEntity( content )  );
// post.setHeader("Content-Type", "application/json; charset=utf-8");

// Attempt C
// post.setEntity(  new StringEntity( content, ContentType.create("application/json") )  );

// Attempt D
// post.setEntity(  new StringEntity( content, ContentType.create("application/json; charset=UTF-8") )  );

// Attempt F
// post.setEntity(  new StringEntity( content, ContentType.create("application/json; charset=utf-8") )  );

// Attempt G
// StringEntity params = new StringEntity( content );
// params.setContentType("application/json; charset=UTF-8");
// post.setEntity(params);

// And then send to server
HttpResponse response = httpClient.execute( post );
int code = response.getStatusLine().getStatusCode();
// ...etc...

Other weird things I've noticed:

我注意到的其他奇怪的事情：

• For a while this behaved differently on Eclipse on the Mac vs. running a .jar on Linux; clearly that's a symptom of platform-specific encoding or decoding, but I don't know where. Ironically that brokewhen I set Eclipse to treat code as UTF-8 (vs. ASCII) I suspect this is an important clue, but not sure where it fits.
• I've seen times when instead of 2 bytes there's 4 bytes in the stream, though this might have been a different encoding problem when writing to disk, though I was specifically setting UTF-8 on file IO
• When I look at the string entity in the debugger, I see the bytes, but the 8-bit character is a negative number. When you run through the Two's Compliment math, it is still the correct Unicode code point, so nominally OK, assuming httpclient isn't buggy.

• 有一段时间，这在 Mac 上的 Eclipse 上与在 Linux 上运行 .jar 的表现不同；显然，这是特定于平台的编码或解码的症状，但我不知道在哪里。具有讽刺意味的是打破了，当我设置Eclipse来对待代码为UTF-8（ASCII对比），我怀疑这是一个重要的线索，但不知道在哪里它适合。
• 我见过流中有 4 个字节而不是 2 个字节的情况，尽管这在写入磁盘时可能是一个不同的编码问题，尽管我专门在文件 IO 上设置了 UTF-8
• 当我在调试器中查看字符串实体时，我看到的是字节，但 8 位字符是负数。当您运行 Two's Compliment 数学时，它仍然是正确的 Unicode 代码点，因此名义上可以，假设 httpclient 没有问题。

Really out of ideas, and as I said, it works with curl, so I think the server is OK.

真的没有想法，正如我所说，它适用于curl，所以我认为服务器还可以。

Edit:

编辑：

curl works when posting to the server, but I can't share the server code. It was pointed out that since curl isn't written in Java, and so perhaps it behaves differently, and therefore the server code could still be suspect.

curl 在发布到服务器时有效，但我无法共享服务器代码。有人指出，由于 curl 不是用 Java 编写的，因此它的行为可能有所不同，因此服务器代码仍可能受到怀疑。

So as a further test, the code below does NOTuse the Apache httpclient library, and DOES work when posting to the server. This proves that the server is fine and there's still something wrong with how I'm using the Apache library on the client side (or maybe it's buggy).

因此，作为进一步的测试，下面的代码不使用 Apache httpclient 库，并且在发布到服务器时确实有效。这证明服务器很好，而且我在客户端使用 Apache 库的方式仍然有问题（或者它可能有问题）。

Non-apache-httpclient code, which does work:

非 apache-httpclient 代码，它确实有效：

import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.net.HttpURLConnection;
import java.net.URL;

class PostUtf8 {
    static String POST_URL = "http://...";

    // \u00DF = LATIN SMALL LETTER SHARP S, looks like letter B
    static String TINY_UTF8_DOC = "[{ \"id\" : \"2\", \"fields\" : { \"subject\" : [{ \"name\" : \"subject\", \"value\" : \"Stra\u00DFe\" }] } }]";

    public static void main( String [] args ) throws Exception {
        System.out.println( "Posting to " + POST_URL );
        URL url = new URL( POST_URL );
        HttpURLConnection conn = (HttpURLConnection) url.openConnection();
        conn.setRequestProperty( "Content-Type", "application/json; charset=UTF-8" );
        conn.setRequestMethod("POST");
        conn.setDoOutput(true);
        OutputStream sout = conn.getOutputStream();
        OutputStreamWriter wout = new OutputStreamWriter(sout, "UTF-8" );
        wout.write( TINY_UTF8_DOC );
        wout.flush();
        int result = conn.getResponseCode();
        System.out.println( "Result = " + result );
    }
}

采纳答案by Alexey Gavrilov

It looks like the problem is how the ContentTypeparameter for the HttpClient's StringEntityconstructor is being created.

看起来问题是如何创建ContentTypeHttpClientStringEntity构造函数的参数。

Using the ContentType.APPLICATION_JSONconstant as a parameter (which corresponds to the "application/json; charset=utf-8" mime type) makes everything work.

使用ContentType.APPLICATION_JSON常量作为参数（对应于“application/json; charset=utf-8”mime 类型）使一切正常。

Here is an example posting the JSON string to a public http service that echoes the request back to the client:

下面是一个将 JSON 字符串发布到公共 http 服务的示例，该服务将请求回显给客户端：

public class HttpClientEncoding {

    static String TINY_UTF8_DOC = "[{ \"id\" : \"2\", \"fields\" : { \"subject\" : " +
            "[{ \"name\" : \"subject\", \"value\" : \"Stra\u00DFe\" }] } }]";

    public static void main(String[] args) throws IOException {
        HttpClient httpClient = new DefaultHttpClient();
        HttpPost post = new HttpPost("http://httpbin.org/post");
        StringEntity entity = new StringEntity(TINY_UTF8_DOC, ContentType.APPLICATION_JSON);
        //StringEntity entity = new StringEntity(TINY_UTF8_DOC, ContentType.create("application/json; charset=utf-8"));
        post.setEntity(entity);
        HttpResponse response = httpClient.execute(post);
        String result = EntityUtils.toString(response.getEntity());
        System.out.println(result);
        ObjectMapper mapper = new ObjectMapper();
        JsonNode node = mapper.readValue(result, JsonNode.class);
        System.out.println(node.get("json").get(0).get("fields").get("subject").get(0).get("value").asText());
    }
}

Output:

输出：

{
  "origin": "46.9.77.167",
  "url": "http://httpbin.org/post",
  "args": {},
  "data": "[{ \"id\" : \"2\", \"fields\" : { \"subject\" : [{ \"name\" : \"subject\", \"value\" : \"Stra\u00dfe\" }] } }]",
  "files": {},
  "form": {},
  "headers": {
    "Content-Length": "90",
    "User-Agent": "Apache-HttpClient/4.3.3 (java 1.5)",
    "Host": "httpbin.org",
    "Connection": "close",
    "X-Request-Id": "c02864cc-a1d6-434c-9cff-1f6187ceb080",
    "Content-Type": "application/json; charset=UTF-8"
  },
  "json": [
    {
      "id": "2",
      "fields": {
        "subject": [
          {
            "value": "Stra\u00dfe",
            "name": "subject"
          }
        ]
      }
    }
  ]
}
Stra?e

回答by Hpalle

This worked for me:(Specifying "UTF-8" to StringEntity)

这对我有用:(将“UTF-8”指定为 StringEntity）

        Object obj = parser.parse(new FileReader(
                "/home/user/paylod.txt"));

        org.json.simple.JSONObject jsonObject = (org.json.simple.JSONObject) obj;

        StringEntity input = new StringEntity( jsonObject.toString(),"UTF-8");
        input.setContentType("application/json");