急!!!!求助各路大侠各路高手,小弟跪求解决办法。

baiying15 2012-06-07 11:39:25
我在linux下面有一个接口日志,里面有几十万的记录,每条记录的格式有点乱而且长度不等,具体如下:
2012-06-07 00:00:10,956 INFO CenterMusicServiceImpl:282 - accessTime=2012-06-07 00:00:10.911 invokeTime=44 com.chinamobile.soapserver.client.musicmanage.GetUserSer
vicesEvt@7c47b61f{ DID=1128017 DIDPwd=B9F1110B4933CA50FEB04EE9D4A56FFC SEQ=11280172012060700001000066142 accessPlatformID=0140901 role=001
roleCode=彩铃用户号码 MSISDN=15012466146 serviceType=2 __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.Ge
tUserServicesResp@8510d7d2{ nextService=null orderedService=com.chinamobile.soapserver.client.musicmanage.ServiceInfo@534ceb76{ allowtimes=50 hold1=20
11-01-26 07:40:50 hold2=002 serviceid=600906002000005001 status=00 __equalsCalc=null __hashCodeCalc=false } returnCode=000000
description=[UMS]操作成功 __equalsCalc=null __hashCodeCalc=false }
2012-06-07 00:00:13,105 INFO CenterMusicServiceImpl:282 - accessTime=2012-06-07 00:00:13.45 invokeTime=59 com.chinamobile.soapserver.client.musicmanage.GetUserSer
vicesEvt@f24f86bc{ DID=1128017 DIDPwd=89C176928600529375D306AA07FB7E36 SEQ=11280172012060700001300066143 accessPlatformID=0140901 role=001
roleCode=彩铃用户号码 MSISDN=13829820066 serviceType=2 __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.Ge
tUserServicesResp@e66307ac{ nextService=null orderedService=com.chinamobile.soapserver.client.musicmanage.ServiceInfo@b49f1b50{ allowtimes=50 hold1=20
10-11-03 17:07:12 hold2=001 serviceid=600906002000005001 status=00 __equalsCalc=null __hashCodeCalc=false } returnCode=000000
description=[UMS]操作成功 __equalsCalc=null __hashCodeCalc=false }
2012-06-07 00:00:16,198 INFO CenterMusicServiceImpl:282 - accessTime=2012-06-07 00:00:16.161 invokeTime=36 com.chinamobile.soapserver.client.musicmanage.GetUserSer
vicesEvt@ddee3404{ DID=1128017 DIDPwd=1C2460BB72544E3C2393C9861823CC78 SEQ=11280172012060700001600066144 accessPlatformID=0140901 role=001
roleCode=彩铃用户号码 MSISDN=13927077133 serviceType=2 __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.Ge
tUserServicesResp@31c3ec5c{ nextService=null orderedService=null returnCode=000000 description=[UMS]操作成功 __equalsCalc=null __hashCo
deCalc=false }
2012-06-07 00:00:16,313 INFO CenterMusicServiceImpl:282 - accessTime=2012-06-07 00:00:16.272 invokeTime=39 com.chinamobile.soapserver.client.musicmanage.GetUserSer
vicesEvt@e85dcd9d{ DID=1128017 DIDPwd=187F1B8FF34F67E743D61BD2EBE5D183 SEQ=11280172012060700001600066146 accessPlatformID=0140901 role=001
roleCode=彩铃用户号码 MSISDN=15012466220 serviceType=2 __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.Ge
tUserServicesResp@31c3ec5c{ nextService=null orderedService=null returnCode=000000 description=[UMS]操作成功 __equalsCalc=null __hashCo
deCalc=false }
2012-06-07 00:00:16,383 INFO CenterMusicServiceImpl:282 - accessTime=2012-06-07 00:00:16.271 invokeTime=111 com.chinamobile.soapserver.client.musicmanage.GetUserSer
vicesEvt@ae9aab0f{ DID=1128017 DIDPwd=8B14D92EC6591D729720DB43754B963B SEQ=11280172012060700001600066145 accessPlatformID=0140901 role=001
roleCode=彩铃用户号码 MSISDN=13927077133 serviceType=2 __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.Ge
tUserServicesResp@31c3ec5c{ nextService=null orderedService=null returnCode=000000 description=[UMS]操作成功 __equalsCalc=null __hashCo
deCalc=false }
2012-06-07 00:00:16,517 INFO CenterMusicServiceImpl:303 - accessTime=2012-06-07 00:00:16.394 invokeTime=121 com.chinamobile.soapserver.client.musicmanage.GetBizInfo
Evt@6f4dd8b9{ DID=1128017 DIDPwd=110D14642614C16625736963B8BE72B5 SEQ=11280172012060700001600066147 accessPlatformID=0140901 role=001 roleCode
=彩铃用户号码 MSISDN=13927077133 contentID=600902000009206193 copyRightID=600582223262 sessionID=null __equalsCalc=null __hashCodeCalc=false
} com.chinamobile.soapserver.client.musicmanage.GetBizInfoRsp@b4d29b0a{ bizInfos=[com.chinamobile.soapserver.client.musicmanage.BizInfo@4c2bfab5{ bizCode=
600902002004738617 bizType=11 desc=按次 offReason=1 originalPrice=200 salePrice=200 __equalsCalc=null __hashCodeCalc=false }][com.c
hinamobile.soapserver.client.musicmanage.BizInfo@4c31405b{ bizCode=600902002004738617 bizType=30 desc=赠送 offReason=1 originalPrice=200
salePrice=200 __equalsCalc=null __hashCodeCalc=false }][com.chinamobile.soapserver.client.musicmanage.BizInfo@27b94e6c{ bizCode=600906002000005001
bizType=21 desc=包月1 offReason=1 originalPrice=500 salePrice=500 __equalsCalc=null __hashCodeCalc=false }][com.chinamobile.soapserver.cl
ient.musicmanage.BizInfo@27e5b543{ bizCode=600906002000005002 bizType=22 desc=包月2 offReason=1 originalPrice=1000 salePrice=1000 __equals
Calc=null __hashCodeCalc=false }] monLevel=0 nextOrderInfo=null orderedService=null returnCode=000000 returnDesc=[IBUS]成功 __equals
Calc=null __hashCodeCalc=false }

我想从这些日志里面取出一下的字段:2012-06-07 00:00:16|^|accessTime=2012-06-07 00:00:16|^|invokeTime=39|^|GetBizInfoEvt|^|accessPlatformID=0140901|^|MSISDN=13927077133|^|returnCode=000000|^|description=[UMS]操作成功
请问各位高手怎么用脚本实现啊。。非常感谢。。。小弟跪拜了!
...全文
138 6 打赏 收藏 转发到动态 举报
写回复
用AI写文章
6 条回复
切换为时间正序
请发表友善的回复…
发表回复
baiying15 2012-06-07
  • 打赏
  • 举报
回复
非常感谢楼上美女的回复,不过好像还是没有截出我需要的字段,另外还想有个SHELL的解决办法。。谢谢
bugs2k 2012-06-07
  • 打赏
  • 举报
回复
#!/usr/bin/env perl

my $date = '\d+-\d+-\d+ \d+:\d+:\d+';
my $pat = qr/^($date).*?(accessTime=$date).*?(invokeTime=\d+).*?(accessPlatformID=\d+).*?(MSISDN=\d+).*?description=(.*?) /;
for (<DATA>) {
if (/$pat/gi) {
my $text = join '|^|', $1, $2, $3, $4, $5, $6;
print "$text\n";
}
}

__DATA__
2012-06-07 00:00:10,956 INFO CenterMusicServiceImpl:282 - accessTime=2012-06-07 00:00:10.911 invokeTime=44 com.chinamobile.soapserver.client.musicmanage.GetUserServicesEvt@7c47b61f{ DID=1128017 DIDPwd=B9F1110B4933CA50FEB04EE9D4A56FFC SEQ=11280172012060700001000066142 accessPlatformID=0140901 role=001 roleCode=彩铃用户号码 MSISDN=15012466146 serviceType=2 __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.GetUserServicesResp@8510d7d2{ nextService=null orderedService=com.chinamobile.soapserver.client.musicmanage.ServiceInfo@534ceb76{ allowtimes=50 hold1=20 11-01-26 07:40:50 hold2=002 serviceid=600906002000005001 status=00 __equalsCalc=null __hashCodeCalc=false } returnCode=000000 description=[UMS]操作成功 __equalsCalc=null __hashCodeCalc=false }
2012-06-07 00:00:13,105 INFO CenterMusicServiceImpl:282 - accessTime=2012-06-07 00:00:13.45 invokeTime=59 com.chinamobile.soapserver.client.musicmanage.GetUserSer vicesEvt@f24f86bc{ DID=1128017 DIDPwd=89C176928600529375D306AA07FB7E36 SEQ=11280172012060700001300066143 accessPlatformID=0140901 role=001 roleCode=彩铃用户号码 MSISDN=13829820066 serviceType=2 __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.GetUserServicesResp@e66307ac{ nextService=null orderedService=com.chinamobile.soapserver.client.musicmanage.ServiceInfo@b49f1b50{ allowtimes=50 hold1=20 10-11-03 17:07:12 hold2=001 serviceid=600906002000005001 status=00 __equalsCalc=null __hashCodeCalc=false } returnCode=000000 description=[UMS]操作成功 __equalsCalc=null __hashCodeCalc=false }
2012-06-07 00:00:16,198 INFO CenterMusicServiceImpl:282 - accessTime=2012-06-07 00:00:16.161 invokeTime=36 com.chinamobile.soapserver.client.musicmanage.GetUserServicesEvt@ddee3404{ DID=1128017 DIDPwd=1C2460BB72544E3C2393C9861823CC78 SEQ=11280172012060700001600066144 accessPlatformID=0140901 role=001 roleCode=彩铃用户号码 MSISDN=13927077133 serviceType=2 __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.GetUserServicesResp@31c3ec5c{ nextService=null orderedService=null returnCode=000000 description=[UMS]操作成功 __equalsCalc=null __hashCodeCalc=false }
2012-06-07 00:00:16,313 INFO CenterMusicServiceImpl:282 - accessTime=2012-06-07 00:00:16.272 invokeTime=39 com.chinamobile.soapserver.client.musicmanage.GetUserServicesEvt@e85dcd9d{ DID=1128017 DIDPwd=187F1B8FF34F67E743D61BD2EBE5D183 SEQ=11280172012060700001600066146 accessPlatformID=0140901 role=001 roleCode=彩铃用户号码 MSISDN=15012466220 serviceType=2 __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.GetUserServicesResp@31c3ec5c{ nextService=null orderedService=null returnCode=000000 description=[UMS]操作成功 __equalsCalc=null __hashCodeCalc=false }
2012-06-07 00:00:16,383 INFO CenterMusicServiceImpl:282 - accessTime=2012-06-07 00:00:16.271 invokeTime=111 com.chinamobile.soapserver.client.musicmanage.GetUserSer vicesEvt@ae9aab0f{ DID=1128017 DIDPwd=8B14D92EC6591D729720DB43754B963B SEQ=11280172012060700001600066145 accessPlatformID=0140901 role=001 roleCode=彩铃用户号码 MSISDN=13927077133 serviceType=2 __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.GetUserServicesResp@31c3ec5c{ nextService=null orderedService=null returnCode=000000 description=[UMS]操作成功 __equalsCalc=null __hashCodeCalc=false }
2012-06-07 00:00:16,517 INFO CenterMusicServiceImpl:303 - accessTime=2012-06-07 00:00:16.394 invokeTime=121 com.chinamobile.soapserver.client.musicmanage.GetBizInfoEvt@6f4dd8b9{ DID=1128017 DIDPwd=110D14642614C16625736963B8BE72B5 SEQ=11280172012060700001600066147 accessPlatformID=0140901 role=001 roleCode =彩铃用户号码 MSISDN=13927077133 contentID=600902000009206193 copyRightID=600582223262 sessionID=null __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.GetBizInfoRsp@b4d29b0a{ bizInfos=[com.chinamobile.soapserver.client.musicmanage.BizInfo@4c2bfab5{ bizCode= 600902002004738617 bizType=11 desc=按次 offReason=1 originalPrice=200 salePrice=200 __equalsCalc=null __hashCodeCalc=false }][com.c hinamobile.soapserver.client.musicmanage.BizInfo@4c31405b{ bizCode=600902002004738617 bizType=30 desc=赠送 offReason=1 originalPrice=200 salePrice=200 __equalsCalc=null __hashCodeCalc=false }][com.chinamobile.soapserver.client.musicmanage.BizInfo@27b94e6c{ bizCode=600906002000005001 bizType=21 desc=包月1 offReason=1 originalPrice=500 salePrice=500 __equalsCalc=null __hashCodeCalc=false }][com.chinamobile.soapserver.cl ient.musicmanage.BizInfo@27e5b543{ bizCode=600906002000005002 bizType=22 desc=包月2 offReason=1 originalPrice=1000 salePrice=1000 __equals Calc=null __hashCodeCalc=false }] monLevel=0 nextOrderInfo=null orderedService=null returnCode=000000 returnDesc=[IBUS]成功 __equals Calc=null __hashCodeCalc=false }
baiying15 2012-06-07
  • 打赏
  • 举报
回复
咋都没人来回复了啊?求各位过路的神仙来帮小弟出谋划策吧;谢谢啦。。。管理也来帮帮忙吧。谢谢谢谢。!!
baiying15 2012-06-07
  • 打赏
  • 举报
回复
我用shell是这么写的:cat /opt/logs/soap-node*/center.log.log|grep INFO |grep GetUserServicesEvt |awk -F"[ '=']+" '{printf $1" "$2"|"&7" "$8"|";for(i=9;i<=NF;i++){if($i=="invokeTime" || $i=="returnCode")printf "%s",$i"="$(i+1)"|"};printf "\n"}'
但是执行的时候报这个错误:
awk: {printf $1" "$2"|"&7" "$8"|";for(i=9;i<=NF;i++){if($i=="invokeTime" || $i=="returnCode")printf "%s",$i"="$(i+1)"|"};printf "\n"}
awk: ^ syntax error
由于水平太低,始终看不出来问题出在哪儿;跪求高手指点啊。。
baiying15 2012-06-07
  • 打赏
  • 举报
回复
其实,我就是想从每条记录里面,取这几个字段出来;其他的东西我都不要。取出这种标准的格式以后,方便我导入到数据库中。
bugs2k 2012-06-07
  • 打赏
  • 举报
回复
没看得明白,大致认为是正则匹配吧。

#!/usr/bin/env perl

my $date = '\d+-\d+-\d+ \d+:\d+:\d+';
my $pat = qr/^($date).*?(accessTime=$date).*?(invokeTime=\d+).*\.([^@]+)@.*?(accessPlatformID=\d+).*?(MSISDN=\d+).*?description=(.*?) /;
while (<DATA>) {
if (/$pat/gi) {
my $text = join '|', $1, $2, $3, $4, $5, $6, $7;
print "$text\n";
}
}

__DATA__
2012-06-07 00:00:10,956 INFO CenterMusicServiceImpl:282 - accessTime=2012-06-07 00:00:10.911 invokeTime=44 com.chinamobile.soapserver.client.musicmanage.GetUserServicesEvt@7c47b61f{ DID=1128017 DIDPwd=B9F1110B4933CA50FEB04EE9D4A56FFC SEQ=11280172012060700001000066142 accessPlatformID=0140901 role=001 roleCode=彩铃用户号码 MSISDN=15012466146 serviceType=2 __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.GetUserServicesResp@8510d7d2{ nextService=null orderedService=com.chinamobile.soapserver.client.musicmanage.ServiceInfo@534ceb76{ allowtimes=50 hold1=20 11-01-26 07:40:50 hold2=002 serviceid=600906002000005001 status=00 __equalsCalc=null __hashCodeCalc=false } returnCode=000000 description=[UMS]操作成功 __equalsCalc=null __hashCodeCalc=false }
2012-06-07 00:00:13,105 INFO CenterMusicServiceImpl:282 - accessTime=2012-06-07 00:00:13.45 invokeTime=59 com.chinamobile.soapserver.client.musicmanage.GetUserServicesEvt@f24f86bc{ DID=1128017 DIDPwd=89C176928600529375D306AA07FB7E36 SEQ=11280172012060700001300066143 accessPlatformID=0140901 role=001 roleCode=彩铃用户号码 MSISDN=13829820066 serviceType=2 __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.GetUserServicesResp@e66307ac{ nextService=null orderedService=com.chinamobile.soapserver.client.musicmanage.ServiceInfo@b49f1b50{ allowtimes=50 hold1=20 10-11-03 17:07:12 hold2=001 serviceid=600906002000005001 status=00 __equalsCalc=null __hashCodeCalc=false } returnCode=000000 description=[UMS]操作成功 __equalsCalc=null __hashCodeCalc=false }
2012-06-07 00:00:16,198 INFO CenterMusicServiceImpl:282 - accessTime=2012-06-07 00:00:16.161 invokeTime=36 com.chinamobile.soapserver.client.musicmanage.GetUserServicesEvt@ddee3404{ DID=1128017 DIDPwd=1C2460BB72544E3C2393C9861823CC78 SEQ=11280172012060700001600066144 accessPlatformID=0140901 role=001 roleCode=彩铃用户号码 MSISDN=13927077133 serviceType=2 __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.GetUserServicesResp@31c3ec5c{ nextService=null orderedService=null returnCode=000000 description=[UMS]操作成功 __equalsCalc=null __hashCodeCalc=false }
2012-06-07 00:00:16,313 INFO CenterMusicServiceImpl:282 - accessTime=2012-06-07 00:00:16.272 invokeTime=39 com.chinamobile.soapserver.client.musicmanage.GetUserServicesEvt@e85dcd9d{ DID=1128017 DIDPwd=187F1B8FF34F67E743D61BD2EBE5D183 SEQ=11280172012060700001600066146 accessPlatformID=0140901 role=001 roleCode=彩铃用户号码 MSISDN=15012466220 serviceType=2 __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.GetUserServicesResp@31c3ec5c{ nextService=null orderedService=null returnCode=000000 description=[UMS]操作成功 __equalsCalc=null __hashCodeCalc=false }
2012-06-07 00:00:16,383 INFO CenterMusicServiceImpl:282 - accessTime=2012-06-07 00:00:16.271 invokeTime=111 com.chinamobile.soapserver.client.musicmanage.GetUserServicesEvt@ae9aab0f{ DID=1128017 DIDPwd=8B14D92EC6591D729720DB43754B963B SEQ=11280172012060700001600066145 accessPlatformID=0140901 role=001 roleCode=彩铃用户号码 MSISDN=13927077133 serviceType=2 __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.GetUserServicesResp@31c3ec5c{ nextService=null orderedService=null returnCode=000000 description=[UMS]操作成功 __equalsCalc=null __hashCodeCalc=false }
2012-06-07 00:00:16,517 INFO CenterMusicServiceImpl:303 - accessTime=2012-06-07 00:00:16.394 invokeTime=121 com.chinamobile.soapserver.client.musicmanage.GetBizInfoEvt@6f4dd8b9{ DID=1128017 DIDPwd=110D14642614C16625736963B8BE72B5 SEQ=11280172012060700001600066147 accessPlatformID=0140901 role=001 roleCode =彩铃用户号码 MSISDN=13927077133 contentID=600902000009206193 copyRightID=600582223262 sessionID=null __equalsCalc=null __hashCodeCalc=false } com.chinamobile.soapserver.client.musicmanage.GetBizInfoRsp@b4d29b0a{ bizInfos=[com.chinamobile.soapserver.client.musicmanage.BizInfo@4c2bfab5{ bizCode= 600902002004738617 bizType=11 desc=按次 offReason=1 originalPrice=200 salePrice=200 __equalsCalc=null __hashCodeCalc=false }][com.c hinamobile.soapserver.client.musicmanage.BizInfo@4c31405b{ bizCode=600902002004738617 bizType=30 desc=赠送 offReason=1 originalPrice=200 salePrice=200 __equalsCalc=null __hashCodeCalc=false }][com.chinamobile.soapserver.client.musicmanage.BizInfo@27b94e6c{ bizCode=600906002000005001 bizType=21 desc=包月1 offReason=1 originalPrice=500 salePrice=500 __equalsCalc=null __hashCodeCalc=false }][com.chinamobile.soapserver.cl ient.musicmanage.BizInfo@27e5b543{ bizCode=600906002000005002 bizType=22 desc=包月2 offReason=1 originalPrice=1000 salePrice=1000 __equals Calc=null __hashCodeCalc=false }] monLevel=0 nextOrderInfo=null orderedService=null returnCode=000000 returnDesc=[IBUS]成功 __equals Calc=null __hashCodeCalc=false }


zhu@ubuntu:~/perl$ perl log.pl
2012-06-07 00:00:10|accessTime=2012-06-07 00:00:10|invokeTime=44|GetUserServicesEvt|accessPlatformID=0140901|MSISDN=15012466146|[UMS]操作成功
2012-06-07 00:00:13|accessTime=2012-06-07 00:00:13|invokeTime=59|GetUserServicesEvt|accessPlatformID=0140901|MSISDN=13829820066|[UMS]操作成功
2012-06-07 00:00:16|accessTime=2012-06-07 00:00:16|invokeTime=36|GetUserServicesEvt|accessPlatformID=0140901|MSISDN=13927077133|[UMS]操作成功
2012-06-07 00:00:16|accessTime=2012-06-07 00:00:16|invokeTime=39|GetUserServicesEvt|accessPlatformID=0140901|MSISDN=15012466220|[UMS]操作成功
2012-06-07 00:00:16|accessTime=2012-06-07 00:00:16|invokeTime=111|GetUserServicesEvt|accessPlatformID=0140901|MSISDN=13927077133|[UMS]操作成功
zhu@ubuntu:~/perl$

19,612

社区成员

发帖
与我相关
我的任务
社区描述
系统使用、管理、维护问题。可以是Ubuntu, Fedora, Unix等等
社区管理员
  • 系统维护与使用区社区
加入社区
  • 近7日
  • 近30日
  • 至今
社区公告
暂无公告

试试用AI创作助手写篇文章吧