69,369
社区成员
发帖
与我相关
我的任务
分享iconv函数转换GB2312与unicode编码遇到问题。
利用iconv的函数：
//UNICODE码转为GB2312码
int u2g(char *inbuf,int inlen,char *outbuf,int outlen)
{
return code_convert("utf-8","gb2312",inbuf,inlen,outbuf,outlen);
}
//GB2312码转为UNICODE码
int g2u(char *inbuf,size_t inlen,char *outbuf,size_t outlen)
{
return code_convert("gb2312","utf-8",inbuf,inlen,outbuf,outlen);
}
GB2312转换unicode时（g2u时），遇到0x00后面就没了，有没有什么解决办法？
例如英文字符w ，gbk编码是单字节，unicode编码是两个字节，低位为0x00,转换完后w后面的其他字符都没了....
//UNICODE码转为GB2312码
int u2g(char *inbuf,int inlen,char *outbuf,int outlen)
{
return code_convert("utf-8","gb2312",inbuf,inlen,outbuf,outlen);
}
//GB2312码转为UNICODE码
int g2u(char *inbuf,size_t inlen,char *outbuf,size_t outlen)
{
return code_convert("gb2312","utf-8",inbuf,inlen,outbuf,outlen);
}
GB2312转换unicode时（g2u时），遇到0x00后面就没了，有没有什么解决办法？
例如英文字符w ，gbk编码是单字节，unicode编码是两个字节，低位为0x00,转换完后w后面的其他字符都没了....
...全文
请发表友善的回复…
发表回复
fefe82 2016-09-21
- 打赏
- 举报
是utf16.那该怎么取长度，我打印了看的，如果是“我的w我的”，gbk长度是9即inlen为9，转换成unicode后长度变成了7……[/quote]
iconv 不是有一个 outlen 的输出嘛 ..[/quote]
我的意思是我转换完的文本再用strlen测长度只有7。。。。用printf按%c打印，00的后面也没有内容。[/quote]
utf16 的不能用 strlen 。strlen 只适用于“已零为结束的字符串”。utf16 编码串不是。%s 同理。
这里只能自己写一个长度检测的函数。
打印的话，还得转回 gbk 或 utf8 。[/quote]
C11 现在有 char16_t 用于存储 utf16 的字符（串），但是貌似还没有相应的 strlen ，printf 也不接受。
fefe82 2016-09-21
- 打赏
- 举报
是utf16.那该怎么取长度，我打印了看的，如果是“我的w我的”，gbk长度是9即inlen为9，转换成unicode后长度变成了7……[/quote]
iconv 不是有一个 outlen 的输出嘛 ..[/quote]
我的意思是我转换完的文本再用strlen测长度只有7。。。。用printf按%c打印，00的后面也没有内容。[/quote]
utf16 的不能用 strlen 。strlen 只适用于“已零为结束的字符串”。utf16 编码串不是。%s 同理。
这里只能自己写一个长度检测的函数。
打印的话，还得转回 gbk 或 utf8 。
sinat_36023721 2016-09-21
- 打赏
- 举报
是utf16.那该怎么取长度，我打印了看的，如果是“我的w我的”，gbk长度是9即inlen为9，转换成unicode后长度变成了7……[/quote]
iconv 不是有一个 outlen 的输出嘛 ..[/quote]
我的意思是我转换完的文本再用strlen测长度只有7。。。。用printf按%c打印，00的后面也没有内容。
sinat_36023721 2016-09-21
- 打赏
- 举报
是utf16.那该怎么取长度，我打印了看的，如果是“我的w我的”，gbk长度是9即inlen为9，转换成unicode后长度变成了7……[/quote]
iconv 不是有一个 outlen 的输出嘛 ..[/quote]outlen我是直接定长145
fefe82 2016-09-21
- 打赏
- 举报
是utf16.那该怎么取长度，我打印了看的，如果是“我的w我的”，gbk长度是9即inlen为9，转换成unicode后长度变成了7……[/quote]
iconv 不是有一个 outlen 的输出嘛 ..
sinat_36023721 2016-09-21
- 打赏
- 举报
是utf16.那该怎么取长度，我打印了看的，如果是“我的w我的”，gbk长度是9即inlen为9，转换成unicode后长度变成了7……
fefe82 2016-09-21
- 打赏
- 举报
utf8 没有 0x00 。'w' 也是一个字节。
你会不会是 utf-16
utf-16 的话，不要用 strlen 取长度
slmax1 2016-09-21
- 打赏
- 举报
outbuf 的空间要先申请好.
slmax1 2016-09-21
- 打赏
- 举报
不过我是转成UTF-8
slmax1 2016-09-21
- 打赏
- 举报
outlen的长度一般要是inlen长度的3倍.
size_t outLen = inLen * 3 + 1;
我是这样处理的.
赵4老师 2016-09-21
- 打赏
- 举报
//iconv_linux下字符集编码转换轻松实现
(1) iconv_t iconv_open(const char *tocode, const char *fromcode);
//此函数说明将要进行哪两种编码的转换,tocode是目标编码,fromcode是原编码,该函数返回一个转换句柄,供以下两个函数使用。
(2) size_t iconv(iconv_t cd,char **inbuf,size_t *inbytesleft,char **outbuf,size_t *outbytesleft);
//此函数从inbuf中读取字符,转换后输出到outbuf中,inbytesleft用以记录还未转换的字符数,outbytesleft用以记录输出缓冲的剩余空间。
(3) int iconv_close(iconv_t cd);
//此函数用于关闭转换句柄,释放资源。
//例子1: 用C语言实现的转换示例程序
/* f.c : 代码转换示例C程序 */
#include <iconv.h>
#define OUTLEN 255
main()
{
char *in_utf8 = "姝ｅ?ㄥ??瑁?";
char *in_gb2312 = "正在安装";
char out[OUTLEN];
/*unicode码转为gb2312码*/
rc = u2g(in_utf8,strlen(in_utf8),out,OUTLEN);
printf("unicode-->gb2312 out=%sn",out);
//gb2312码转为unicode码
rc = g2u(in_gb2312,strlen(in_gb2312),out,OUTLEN);
printf("gb2312-->unicode out=%sn",out);
}
/*代码转换:从一种编码转为另一种编码*/
int code_convert(char *from_charset,char *to_charset,char *inbuf,int inlen,char *outbuf,int outlen)
{
iconv_t cd;
int rc;
char **pin = &inbuf;
char **pout = &outbuf;
cd = iconv_open(to_charset,from_charset);
if (cd==0) return -1;
memset(outbuf,0,outlen);
if (iconv(cd,pin,&inlen,pout,&outlen)==-1) return -1;
iconv_close(cd);
return 0;
}
/*UNICODE码转为GB2312码*/
int u2g(char *inbuf,int inlen,char *outbuf,int outlen)
{
return code_convert("utf-8","gb2312",inbuf,inlen,outbuf,outlen);
}
/*GB2312码转为UNICODE码*/
int g2u(char *inbuf,size_t inlen,char *outbuf,size_t outlen)
{
return code_convert("gb2312","utf-8",inbuf,inlen,outbuf,outlen);
}
//例子2: 用C++语言实现的转换示例程序
/* f.cpp : 代码转换示例C++程序 */
#include <iconv.h>
#include <iostream>
#define OUTLEN 255
using namespace std;
// 代码转换操作类
class CodeConverter {
private:
iconv_t cd;
public:
// 构造
CodeConverter(const char *from_charset,const char *to_charset) {
cd = iconv_open(to_charset,from_charset);
}
// 析构
~CodeConverter() {
iconv_close(cd);
}
// 转换输出
int convert(char *inbuf,int inlen,char *outbuf,int outlen) {
char **pin = &inbuf;
char **pout = &outbuf;
memset(outbuf,0,outlen);
return iconv(cd,pin,(size_t *)&inlen,pout,(size_t *)&outlen);
}
};
int main(int argc, char **argv)
{
char *in_utf8 = "姝ｅ?ㄥ??瑁?";
char *in_gb2312 = "正在安装";
char out[OUTLEN];
// utf-8-->gb2312
CodeConverter cc = CodeConverter("utf-8","gb2312");
cc.convert(in_utf8,strlen(in_utf8),out,OUTLEN);
cout << "utf-8-->gb2312 in=" << in_utf8 << ",out=" << out << endl;
// gb2312-->utf-8
CodeConverter cc2 = CodeConverter("gb2312","utf-8");
cc2.convert(in_gb2312,strlen(in_gb2312),out,OUTLEN);
cout << "gb2312-->utf-8 in=" << in_gb2312 << ",out=" << out << endl;
}
二、利用iconv命令进行编码转换
iconv命令用于转换指定文件的编码,默认输出到标准输出设备,亦可指定输出文件。
用法： iconv [选项...] [文件...]
有如下选项可用:
输入/输出格式规范：
-f, --from-code=名称 原始文本编码
-t, --to-code=名称 输出编码
信息：
-l, --list 列举所有已知的字符集
输出控制：
-c 从输出中忽略无效的字符
-o, --output=FILE 输出文件
-s, --silent 关闭警告
--verbose 打印进度信息
-?, --help 给出该系统求助列表
--usage 给出简要的用法信息
-V, --version 打印程序版本号
例子:
iconv -f utf-8 -t gb2312 aaa.txt >bbb.txt
这个命令读取aaa.txt文件，从utf-8编码转换为gb2312编码,其输出定向到bbb.txt文件。
小结: Linux为我们提供了强大的编码转换工具,给我们带来了方便。
赵4老师 2016-09-21
- 打赏
- 举报
strlen, wcslen, _mbslen, _mbstrlen
Get the length of a string.
size_t strlen( const char *string );
size_t wcslen( const wchar_t *string );
size_t _mbslen( const unsigned char *string );
size_t _mbstrlen( const char *string );
Routine Required Header Compatibility
strlen <string.h> ANSI, Win 95, Win NT
wcslen <string.h> or <wchar.h> ANSI, Win 95, Win NT
_mbslen <mbstring.h> Win 95, Win NT
_mbstrlen <stdlib.h> Win 95, Win NT
For additional compatibility information, see Compatibility in the Introduction.
Libraries
LIBC.LIB Single thread static library, retail version
LIBCMT.LIB Multithread static library, retail version
MSVCRT.LIB Import library for MSVCRT.DLL, retail version
Return Value
Each of these functions returns the number of characters in string, excluding the terminal NULL. No return value is reserved to indicate an error.
Parameter
string
Null-terminated string
Remarks
Each of these functions returns the number of characters in string, not including the terminating null character. wcslen is a wide-character version of strlen; the argument of wcslen is a wide-character string. wcslen and strlen behave identically otherwise.
Generic-Text Routine Mappings
TCHAR.H Routine _UNICODE & _MBCS Not Defined _MBCS Defined _UNICODE Defined
_tcslen strlen strlen wcslen
_tcsclen strlen _mbslen wcslen
_mbslen and _mbstrlen return the number of multibyte characters in a multibyte-character string. _mbslen recognizes multibyte-character sequences according to the multibyte code page currently in use; it does not test for multibyte-character validity. _mbstrlen tests for multibyte-character validity and recognizes multibyte-character sequences according to the LC_CTYPE category setting of the current locale. For more information about the LC_CTYPE category, see setlocale.
Example
/* STRLEN.C */
#include <string.h>
#include <stdio.h>
#include <conio.h>
#include <dos.h>
void main( void )
{
char buffer[61] = "How long am I?";
int len;
len = strlen( buffer );
printf( "'%s' is %d characters long\n", buffer, len );
}
Output
'How long am I?' is 14 characters long
String Manipulation Routines | Locale Routines
See Also setlocale, strcat, strcmp, strcoll Functions, strcpy, strrchr, _strset, strspn
printf, wprintf
Print formatted output to the standard output stream.
int printf( const char *format [, argument]... );
int wprintf( const wchar_t *format [, argument]... );
Routine Required Header Compatibility
printf <stdio.h> ANSI, Win 95, Win NT
wprintf <stdio.h> or <wchar.h> ANSI, Win 95, Win NT
For additional compatibility information, see Compatibility in the Introduction.
Libraries
LIBC.LIB Single thread static library, retail version
LIBCMT.LIB Multithread static library, retail version
MSVCRT.LIB Import library for MSVCRT.DLL, retail version
Return Value
Each of these functions returns the number of characters printed, or a negative value if an error occurs.
Parameters
format
Format control
argument
Optional arguments
Remarks
The printf function formats and prints a series of characters and values to the standard output stream, stdout. If arguments follow the format string, the format string must contain specifications that determine the output format for the arguments. printf and fprintf behave identically except that printf writes output to stdout rather than to a destination of type FILE.
wprintf is a wide-character version of printf; format is a wide-character string. wprintf and printf behave identically otherwise.
Generic-Text Routine Mappings
TCHAR.H Routine _UNICODE & _MBCS Not Defined _MBCS Defined _UNICODE Defined
_tprintf printf printf wprintf
The format argument consists of ordinary characters, escape sequences, and (if arguments follow format) format specifications. The ordinary characters and escape sequences are copied to stdout in order of their appearance. For example, the line
printf("Line one\n\t\tLine two\n");
produces the output
Line one
Line two
Format specifications always begin with a percent sign (%) and are read left to right. When printf encounters the first format specification (if any), it converts the value of the first argument after format and outputs it accordingly. The second format specification causes the second argument to be converted and output, and so on. If there are more arguments than there are format specifications, the extra arguments are ignored. The results are undefined if there are not enough arguments for all the format specifications.
Example
/* PRINTF.C: This program uses the printf and wprintf functions
* to produce formatted output.
*/
#include <stdio.h>
void main( void )
{
char ch = 'h', *string = "computer";
int count = -9234;
double fp = 251.7366;
wchar_t wch = L'w', *wstring = L"Unicode";
/* Display integers. */
printf( "Integer formats:\n"
"\tDecimal: %d Justified: %.6d Unsigned: %u\n",
count, count, count, count );
printf( "Decimal %d as:\n\tHex: %Xh C hex: 0x%x Octal: %o\n",
count, count, count, count );
/* Display in different radixes. */
printf( "Digits 10 equal:\n\tHex: %i Octal: %i Decimal: %i\n",
0x10, 010, 10 );
/* Display characters. */
printf("Characters in field (1):\n%10c%5hc%5C%5lc\n", ch, ch, wch, wch);
wprintf(L"Characters in field (2):\n%10C%5hc%5c%5lc\n", ch, ch, wch, wch);
/* Display strings. */
printf("Strings in field (1):\n%25s\n%25.4hs\n\t%S%25.3ls\n",
string, string, wstring, wstring);
wprintf(L"Strings in field (2):\n%25S\n%25.4hs\n\t%s%25.3ls\n",
string, string, wstring, wstring);
/* Display real numbers. */
printf( "Real numbers:\n\t%f %.2f %e %E\n", fp, fp, fp, fp );
/* Display pointer. */
printf( "\nAddress as:\t%p\n", &count);
/* Count characters printed. */
printf( "\nDisplay to here:\n" );
printf( "1234567890123456%n78901234567890\n", &count );
printf( "\tNumber displayed: %d\n\n", count );
}
Output
Integer formats:
Decimal: -9234 Justified: -009234 Unsigned: 4294958062
Decimal -9234 as:
Hex: FFFFDBEEh C hex: 0xffffdbee Octal: 37777755756
Digits 10 equal:
Hex: 16 Octal: 8 Decimal: 10
Characters in field (1):
h h w w
Characters in field (2):
h h w w
Strings in field (1):
computer
comp
Unicode Uni
Strings in field (2):
computer
comp
Unicode Uni
Real numbers:
251.736600 251.74 2.517366e+002 2.517366E+002
Address as: 0012FFAC
Display to here:
123456789012345678901234567890
Number displayed: 16
Floating-Point Support Routines | Stream I/O Routines | Locale Routines
See Also fopen, fprintf, scanf, sprintf, vprintf Functions
