Linux 有没有办法从 UTF8 转换为 iso-8859-1？
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/11156473/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Is there a way to convert from UTF8 to iso-8859-1?
提问by fazineroso
My software is getting some strings in UTF8 than I need to convert to ISO 8859 1. I know that UTF8 domain is bigger than iso 8859. But the data in UTF8 has been previously upconverted from ISO, so I should not miss anything.
我的软件在 UTF8 中获取了一些字符串，而不是我需要转换为 ISO 8859 1。我知道 UTF8 域比 iso 8859 大。但是 UTF8 中的数据以前已经从 ISO 上转换过，所以我不应该错过任何东西。
I would like to know if there is an easy / direct way to convert from UTF8 to iso-8859-1.
我想知道是否有一种简单/直接的方法可以将 UTF8 转换为 iso-8859-1。
Thanks
谢谢
采纳答案by Nominal Animal
Here is a function you might find useful: utf8_to_latin9(). It converts to ISO-8859-15(including EURO, which ISO-8859-1does not have), but also works correctly for the UTF-8->ISO-8859-1conversion part of a ISO-8859-1->UTF-8->ISO-8859-1round-trip.
这是一个您可能会觉得有用的函数：utf8_to_latin9(). 它转换为ISO-8859-15（包括没有的 EURO ISO-8859-1），但也适用于-> ->往返的UTF-8->ISO-8859-1转换部分。ISO-8859-1UTF-8ISO-8859-1
The function ignores invalid code points similar to //IGNOREflag for iconv, but does not recompose decomposed UTF-8 sequences; that is, it won't turn U+006E U+0303into U+00F1. I don't bother recomposing because iconv does not either.
该函数会忽略类似于//IGNOREiconv 标志的无效代码点，但不会重组分解的 UTF-8 序列；也就是说，它不会U+006E U+0303变成U+00F1. 我不打扰重组，因为 iconv 也没有。
The function is very careful about the string access. It will never scan beyond the buffer. The output buffer must be one byte longer than length, because it always appends the end-of-string NUL byte. The function returns the number of characters (bytes) in output, not including the end-of-string NUL byte.
该函数对字符串访问非常小心。它永远不会扫描超出缓冲区。输出缓冲区必须比长度长一个字节，因为它总是附加字符串结尾的 NUL 字节。该函数返回输出中的字符（字节）数，不包括字符串结尾的 NUL 字节。
/* UTF-8 to ISO-8859-1/ISO-8859-15 mapper.
* Return 0..255 for valid ISO-8859-15 code points, 256 otherwise.
*/
static inline unsigned int to_latin9(const unsigned int code)
{
/* Code points 0 to U+00FF are the same in both. */
if (code < 256U)
return code;
switch (code) {
case 0x0152U: return 188U; /* U+0152 = 0xBC: OE ligature */
case 0x0153U: return 189U; /* U+0153 = 0xBD: oe ligature */
case 0x0160U: return 166U; /* U+0160 = 0xA6: S with caron */
case 0x0161U: return 168U; /* U+0161 = 0xA8: s with caron */
case 0x0178U: return 190U; /* U+0178 = 0xBE: Y with diaresis */
case 0x017DU: return 180U; /* U+017D = 0xB4: Z with caron */
case 0x017EU: return 184U; /* U+017E = 0xB8: z with caron */
case 0x20ACU: return 164U; /* U+20AC = 0xA4: Euro */
default: return 256U;
}
}
/* Convert an UTF-8 string to ISO-8859-15.
* All invalid sequences are ignored.
* Note: output == input is allowed,
* but input < output < input + length
* is not.
* Output has to have room for (length+1) chars, including the trailing NUL byte.
*/
size_t utf8_to_latin9(char *const output, const char *const input, const size_t length)
{
unsigned char *out = (unsigned char *)output;
const unsigned char *in = (const unsigned char *)input;
const unsigned char *const end = (const unsigned char *)input + length;
unsigned int c;
while (in < end)
if (*in < 128)
*(out++) = *(in++); /* Valid codepoint */
else
if (*in < 192)
in++; /* 10000000 .. 10111111 are invalid */
else
if (*in < 224) { /* 110xxxxx 10xxxxxx */
if (in + 1 >= end)
break;
if ((in[1] & 192U) == 128U) {
c = to_latin9( (((unsigned int)(in[0] & 0x1FU)) << 6U)
| ((unsigned int)(in[1] & 0x3FU)) );
if (c < 256)
*(out++) = c;
}
in += 2;
} else
if (*in < 240) { /* 1110xxxx 10xxxxxx 10xxxxxx */
if (in + 2 >= end)
break;
if ((in[1] & 192U) == 128U &&
(in[2] & 192U) == 128U) {
c = to_latin9( (((unsigned int)(in[0] & 0x0FU)) << 12U)
| (((unsigned int)(in[1] & 0x3FU)) << 6U)
| ((unsigned int)(in[2] & 0x3FU)) );
if (c < 256)
*(out++) = c;
}
in += 3;
} else
if (*in < 248) { /* 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx */
if (in + 3 >= end)
break;
if ((in[1] & 192U) == 128U &&
(in[2] & 192U) == 128U &&
(in[3] & 192U) == 128U) {
c = to_latin9( (((unsigned int)(in[0] & 0x07U)) << 18U)
| (((unsigned int)(in[1] & 0x3FU)) << 12U)
| (((unsigned int)(in[2] & 0x3FU)) << 6U)
| ((unsigned int)(in[3] & 0x3FU)) );
if (c < 256)
*(out++) = c;
}
in += 4;
} else
if (*in < 252) { /* 111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx */
if (in + 4 >= end)
break;
if ((in[1] & 192U) == 128U &&
(in[2] & 192U) == 128U &&
(in[3] & 192U) == 128U &&
(in[4] & 192U) == 128U) {
c = to_latin9( (((unsigned int)(in[0] & 0x03U)) << 24U)
| (((unsigned int)(in[1] & 0x3FU)) << 18U)
| (((unsigned int)(in[2] & 0x3FU)) << 12U)
| (((unsigned int)(in[3] & 0x3FU)) << 6U)
| ((unsigned int)(in[4] & 0x3FU)) );
if (c < 256)
*(out++) = c;
}
in += 5;
} else
if (*in < 254) { /* 1111110x 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx */
if (in + 5 >= end)
break;
if ((in[1] & 192U) == 128U &&
(in[2] & 192U) == 128U &&
(in[3] & 192U) == 128U &&
(in[4] & 192U) == 128U &&
(in[5] & 192U) == 128U) {
c = to_latin9( (((unsigned int)(in[0] & 0x01U)) << 30U)
| (((unsigned int)(in[1] & 0x3FU)) << 24U)
| (((unsigned int)(in[2] & 0x3FU)) << 18U)
| (((unsigned int)(in[3] & 0x3FU)) << 12U)
| (((unsigned int)(in[4] & 0x3FU)) << 6U)
| ((unsigned int)(in[5] & 0x3FU)) );
if (c < 256)
*(out++) = c;
}
in += 6;
} else
in++; /* 11111110 and 11111111 are invalid */
/* Terminate the output string. */
*out = '#include <iconv.h>
#include <stdio.h>
int main (void) {
iconv_t cd = iconv_open("ISO_8859-1", "UTF-8");
if (cd == (iconv_t) -1) {
perror("iconv_open failed!");
return 1;
}
char input[] = "Test ??ü";
char *in_buf = &input[0];
size_t in_left = sizeof(input) - 1;
char output[32];
char *out_buf = &output[0];
size_t out_left = sizeof(output) - 1;
do {
if (iconv(cd, &in_buf, &in_left, &out_buf, &out_left) == (size_t) -1) {
perror("iconv failed!");
return 1;
}
} while (in_left > 0 && out_left > 0);
*out_buf = 0;
iconv_close(cd);
printf("%s -> %s\n", input, output);
return 0;
}
';
return (size_t)(out - (unsigned char *)output);
}
Note that you can add custom transliteration for specific code points in the to_latin9()function, but you are limited to one-character replacements.
请注意，您可以为to_latin9()函数中的特定代码点添加自定义音译，但仅限于一个字符替换。
As it is currently written, the function can do in-place conversion safely: input and output pointers can be the same. The output string will never be longer than the input string. If your input string has room for an extra byte (for example, it has the NUL terminating the string), you can safely use the above function to convert it from UTF-8 to ISO-8859-1/15. I deliberately wrote it this way, because it should save you some effort in an embedded environment, although this approach is a bit limited wrt. customization and extension.
正如当前编写的那样，该函数可以安全地进行就地转换：输入和输出指针可以相同。输出字符串永远不会比输入字符串长。如果您的输入字符串有空间容纳额外的字节（例如，它有 NUL 终止字符串），您可以安全地使用上述函数将其从 UTF-8 转换为 ISO-8859-1/15。我故意这样写的，因为它应该在嵌入式环境中为您节省一些精力，尽管这种方法有点有限。定制和扩展。
Edit:
编辑：
I included a pair of conversion functions in an edit to this answerfor both Latin-1/9 to/from UTF-8 conversion (ISO-8859-1 or -15 to/from UTF-8); the main difference is that those functions return a dynamically allocated copy, and keep the original string intact.
我在此答案的编辑中包含了一对转换函数，用于 Latin-1/9 到/从 UTF-8 转换（ISO-8859-1 或 -15 到/从 UTF-8）；主要区别在于这些函数返回一个动态分配的副本，并保持原始字符串不变。
回答by kay - SE is evil
iconv - perform character set conversion
size_t iconv(iconv_t cd, char **inbuf, size_t *inbytesleft, char **outbuf, size_t *outbytesleft);
iconv_t iconv_open(const char *tocode, const char *fromcode);
iconv - 执行字符集转换
size_t iconv(iconv_t cd, char **inbuf, size_t *inbytesleft, char **outbuf, size_t *outbytesleft);
iconv_t iconv_open(const char *tocode, const char *fromcode);
tocodeis "ISO_8859-1"and fromcodeis "UTF-8".
tocode是"ISO_8859-1"并且fromcode是"UTF-8"。
Working example:
工作示例：
##代码##