java服务挂掉，贴出日志求解

[思无欲]

===== BEGIN DUMP =============================================================
JRockit dump produced after 0 days, 06:07:19 on Sun Apr 22 19:29:22 2012

**************************************************************************
* If you see this dump, please go to *
* http://download.oracle.com/docs/cd/E15289_01/go2troubleshooting.html *
* for troubleshooting information. *
**************************************************************************

Additional information is available in:
/usr/local/resin/jrockit.12088.dump
No snapshot file (core dump) will be created because core dumps have been
disabled. To enable core dumping, try "ulimit -c unlimited"
before starting JRockit again.

Error Message: Illegal memory access. [54]
Signal info : si_signo=11, si_code=1 si_addr=0x480000000
Version : Oracle JRockit(R) R28.0.1-21-133393-1.6.0_20-20100512-2126-linux-x86_64
CPU : Intel Core i7 (HT) SSE SSE2 SSE3 SSSE3 SSE4.1 SSE4.2 Core Intel64
Number CPUs : 8
Tot Phys Mem : 16823226368 (16043 MB)
OS version : CentOS release 5.7 (Final)
Linux version 2.6.18-274.el5 ([e-mail]) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-50)) #1 SMP Fri Jul 22 04:43:29 EDT 2011 (x86_64)
Thread System: Linux NPTL
LibC release : 2.5-stable
Java locking : Normal
State : JVM is running
Command Line : -Djava.util.logging.manager=com.caucho.log.LogManagerImpl -Djava.system.class.loader=com.caucho.loader.SystemClassLoader -Djavax.management.builder.initial=com.caucho.jmx.MBeanServerBuilderImpl -Djava.awt.headless=true -Dresin.home=/usr/local/resin/ -server -Xmx14336m -Xms14336m -Xss1m -Xdebug -Dcom.sun.management.jmxremote -Xmanagement -Djrockit.managementserver.autodiscovery=true -Xdebug -Dcom.sun.management.jmxremote -Djava.util.logging.manager=com.caucho.log.LogManagerImpl -Djavax.management.builder.initial=com.caucho.jmx.MBeanServerBuilderImpl -Djava.awt.headless=true -Dresin.home=/usr/local/resin/ -Dresin.root=/usr/local/resin/ -Dsun.java.launcher=SUN_STANDARD -Dsun.java.launcher=SUN_STANDARD -Dsun.java.launcher=SUN_STANDARD -Djava.util.logging.manager=com.caucho.log.LogManagerImpl -Djavax.management.builder.initial=com.caucho.jmx.MBeanServerBuilderImpl -Djava.awt.headless=true -Dresin.home=/usr/local/resin/ -Dresin.root=/usr/local/resin/ -Dsun.java.launcher=SUN_STANDARD -Dsun.java.launcher=SUN_STANDARD -Dsun.java.launcher=SUN_STANDARD -Dsun.java.launcher=SUN_STANDARD com.caucho.server.resin.Resin --root-directory /usr/local/resin/ -conf /usr/local/resin/conf/resin.conf -socketwait 61157 restart
Repository : /tmp/2012_04_22_13_22_05_12088
java.home : /usr/local/jrmc-4.0.1-1.6.0/jre
j.class.path : ./../lib/resin.jar:/usr/local/jrmc-4.0.1-1.6.0/lib/tools.jar:/usr/local/resin/lib/pro.jar:/usr/local/resin/lib/resin.jar:/usr/local/resin/lib/jaxrpc-15.jar:/usr/local/resin/lib/hessian.jar:/usr/local/resin/lib/jstl-11.jar:/usr/local/resin/lib/commons-lang-2.4.jar:/usr/local/resin/lib/cbhessian-3.0.20.jar:/usr/local/resin/lib/saaj.jar:/usr/local/resin/lib/jta-101.jar:/usr/local/resin/lib/cbcrypto-0.13.00.jar:/usr/local/resin/lib/webutil.jar:/usr/local/resin/lib/javamail-141.jar:/usr/local/resin/lib/jsdk-15.jar:/usr/local/resin/lib/jca-15.jar:/usr/local/resin/lib/script-10.jar:/usr/local/resin/lib/jaxws-api.jar:/usr/local/resin/lib/jms-11.jar:/usr/local/resin/lib/activation.jar:/usr/local/resin/lib/portlet-10.jar:/usr/local/resin/lib/resin-util.jar:/usr/local/resin/lib/jsf-12.jar:/usr/local/resin/lib/ejb-15.jar:/usr/local/resin/lib/jsr88.jar:/usr/local/resin/lib/jpa-15.jar:/usr/local/resin/lib/quercus.jar:/usr/local/resin/lib/resin-deploy.jar:/usr/local/resin/lib/jws-15.jar:/usr/local/resin/lib/license.jar:/usr/local/resin/lib/j2ee-management-10.jar:/usr/local/resin/lib/webbeans-16.jar
j.lib.path : /usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64/jrockit:/usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64:/usr/local/jrmc-4.0.1-1.6.0/jre/../lib/amd64:/usr/local/resin/libexec64:/usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64/jrockit:/usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64:/usr/local/jrmc-4.0.1-1.6.0/jre/../lib/amd64:/usr/local/resin/libexec64
JAVA_HOME : /usr/local/jrmc-4.0.1-1.6.0
_JAVA_OPTIONS: <not set>
LD_LIBRARY_PATH: /usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64/jrockit:/usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64:/usr/local/jrmc-4.0.1-1.6.0/jre/../lib/amd64:/usr/local/resin/libexec64:/usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64/jrockit:/usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64:/usr/local/jrmc-4.0.1-1.6.0/jre/../lib/amd64:/usr/local/resin/libexec64
LD_ASSUME_KERNEL: <not set>
LD_PRELOAD : <not set>
StackOverFlow: 0 StackOverFlowErrors have occured
OutOfMemory : 0 OutOfMemoryErrors have occured
C Heap : Good; no memory allocations have failed
GC Strategy : Mode: throughput, with strategy: genparpar (basic strategy: genparpar)
GC Status : OC is not running. Last finished OC was OC#4.
: YC is not running. Last finished YC was YC#2.
YC Promotion : Last YC successfully promoted all objects
YC History : Ran 2 YCs before OC#1.
: Ran 0 YCs before OC#2.
: Ran 0 YCs before OC#3.
: Ran 0 YCs before OC#4.
: Ran 0 YCs since last OC.
Heap : 0x100000000 - 0x480000000 (Size: 14336 MB)
Compaction : (no compaction area)
NurseryList : 0x1026847a8 - 0x13a6a6968
KeepArea : 0x12c6a6920 - 0x13a6a6968
KA Markers : [ 0x11e6a6938, 0x12c6a6920 , 0x13a6a6968 ]
Forbidden A : (none)
Previous KA : 0x1dfffffd8 - 0x24fffffc8
Previous FA : (none)
CompRefs : References are compressed, with heap base 0x0 and shift 3.

Registers (from ThreadContext: 0x47dd94e0:
rax = 000000000000000c rcx = 00000002026edae0
rdx = 00000000ca6a6968 rbx = 0000000000000010
rsp = 0000000047dd9920 rbp = 000000013a6a6968
rsi = 000000027d912518 rdi = 00000002ed912518
r8 = 000000008ffffffc r9 = 0000000000000002
r10 = 0000000000000000 r11 = 000000375cc79f18
r12 = 0000000047dd9a40 r13 = 000000007fffffff
r14 = 00000000115c31d0 r15 = 0000000000000000
cs = 0000000000000033 fs = 0000000400000000
gs = 0004000000000000
rip = 00002aaac8fedb44 flags = 0000000000000202

Loaded modules:
(* denotes the module where the exception occured)
0000000000400000-0000000000412703 /usr/local/jrmc-4.0.1-1.6.0/bin/java
00007fffd8bfd000-00007fffd8bfddaf /usr/local/jrmc-4.0.1-1.6.0/bin/java
000000375d000000-000000375d001f93 /lib64/libdl.so.2
0000003b30600000-0000003b30615b4f /lib64/libpthread.so.0
000000375cc00000-000000375cd4ca47 /lib64/libc.so.6
000000375c800000-000000375c81bbe7 /lib64/ld-linux-x86-64.so.2
00002b1b6b669000-00002b1b6b9620ab /usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64/jrockit/libjvm.so
00002b1b6bc3a000-00002b1b6bc581db /usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64/libosal.so
00002b1b6be5b000-00002b1b6be67313 /usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64/libutil.so
000000375e400000-000000375e48122f /lib64/libm.so.6
0000003b30e00000-0000003b30e06f5b /lib64/librt.so.1
00002aaaaabb8000-00002aaaaabc164b /lib64/libnss_files.so.2
00002aaaaadc3000-00002aaaaadce72b /usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64/libjfr.so
00002aab090a5000-00002aab090b1337 /usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64/libverify.so
00002aab091b4000-00002aab091dc10b /usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64/libjava.so
0000003760c00000-0000003760c14107 /lib64/libnsl.so.1
00002aab09323000-00002aab093295bf /usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64/native_threads/libhpi.so
00002aab0cacb000-00002aab0cad88cb /usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64/libzip.so
00002aab0cedf000-00002aab0cef198f /usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64/libnet.so
00002aab0d337000-00002aab0d33c49b /usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64/libmanagement.so
00002aab0d985000-00002aab0d98f027 /usr/local/resin/libexec64/libresin.so
00002aab0e422000-00002aab0e423437 /usr/local/resin/libexec64/libresin_os.so
00002aab0ed64000-00002aab0ed6a8ab /usr/local/jrmc-4.0.1-1.6.0/jre/lib/amd64/libnio.so
00002aab0f9ec000-00002aab0f9efd93 /lib64/libnss_dns.so.2
000000375ec00000-000000375ec108a7 /lib64/libresolv.so.2

Stack:
(* marks the word pointed to by the stack pointer)
0000000047dd9920: 00000002ed912518* 0000000000000010 000000013a6a6968 0000000100000010
0000000047dd9940: 000000008ffffffc 00002aaa00000002 00000000115c31d0 0000000000000000
0000000047dd9960: 000000013a6a6968 00000002026edae0 0000000100000010 7fffffff00000001
0000000047dd9980: 0000000200000000 0000000000000000 0000000000000000 0000000000000000
0000000047dd99a0: 00002aaac8f63a80 0000000000000002 0000000000000000 0000000047dd9960
0000000047dd99c0: 00000002ed912518 00002aaac8fed89e 00000002ed912518 0000000000000010

Code:
(* marks the word pointed to by the instruction pointer)
00002aaac8fedae0: 48020c8950247489 0004054858245489 7560244489480000 63480000009ae9d4
00002aaac8fedb00: d0634808245489f2 44c0634918244489 37348d4820244489 11148d48243c8948
00002aaac8fedb20: c1894810244c8948 0148c829f0e18348 48d9f748ca0148ce 4805eb1a7400f983
00002aaac8fedb40: 0e04100f48244c8b* 10c183480a04110f 48e87548244c8948 d8f748c20148c601
00002aaac8fedb60: 3074000000003d48 483024748b480feb 24448b483824548b 748948060c8b6640
00002aaac8fedb80: 8948020c89663024 0000020548382454 d275402444894800 5b000000a0c48148

[思无欲]

未知，正常运作中突发。无高并发等情况。怀疑是程序异常引起。
所以贴出日志求大牛指点

[cseu]

再等一段时间，可能只是偶发情况。就像莫名其妙的蓝屏一样

[思无欲]

Last optimized methods:
No methods optimized.

Thread:
"pool-msg-consume-5-thread-2" id=119 idx=0x1f4 tid=12216 lastJavaFrame=(nil)
Stack 0: start=0x47cd8000, end=0x47ddc000, guards=0x47cdd000 (ok), forbidden=0x47cdb000

Thread Stack Trace:
-- Java stack --
at jrockit/vm/ArrayCopy.memcpyBlock(ArrayCopy.java:593)
at jrockit/vm/ArrayCopy.fcopy(ArrayCopy.java:545)
at jrockit/vm/ArrayCopy.copy_checks_done(ArrayCopy.java:462)
at jrockit/vm/ArrayCopy.copy(ArrayCopy.java:447)
at java/util/Arrays.copyOf(Arrays.java:2883)
at java/lang/AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
at java/lang/AbstractStringBuilder.append(AbstractStringBuilder.java:572)
at java/lang/StringBuffer.append(StringBuffer.java:320)
at com/taobao/api/internal/util/json/JSONReader.add(JSONReader.java:196)
at com/taobao/api/internal/util/json/JSONReader.add(JSONReader.java:201)
at com/taobao/api/internal/util/json/JSONReader.string(JSONReader.java:187)
at com/taobao/api/internal/util/json/JSONReader.read(JSONReader.java:79)
at com/taobao/api/internal/util/json/JSONReader.object(JSONReader.java:110)
at com/taobao/api/internal/util/json/JSONReader.read(JSONReader.java:83)
at com/taobao/api/internal/util/json/JSONReader.read(JSONReader.java:63)
at com/taobao/api/internal/util/json/JSONReader.read(JSONReader.java:71)
at com/server/listener/RealizeTopCometMessageListener.onReceiveMsg(RealizeTopCometMessageListener.java:90)
at com/taobao/api/internal/stream/AbstractStreamImpl$StreamEvent.run(AbstractStreamImpl.java:66)
at java/util/concurrent/Executors$RunnableAdapter.call(Executors.java:441)
at java/util/concurrent/FutureTask$Sync.innerRun(FutureTask.java:303)
at java/util/concurrent/FutureTask.run(FutureTask.java:138)
at java/util/concurrent/ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java/lang/Thread.run(Thread.java:619)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace

Memory usage report:
Total mapped 17013920KB (reserved=1348212KB)
- Java heap 14680064KB (reserved=0KB)
- GC tables 491020KB
- Thread stacks 250948KB (#threads=245)
- Compiled code 1048576KB (used=5527KB)
- Internal 1864KB
- OS 205604KB
- Other 294116KB
- Classblocks 2304KB (malloced=2166KB #6003)
Not tracing sites.

- Java class data 38400KB (malloced=37756KB #27965 in 6003 classes)
Not tracing sites.

- Native memory tracking 1024KB (malloced=228KB #10)
Not tracing sites.


Set the env variable TRACE_ALLOC_SITES=1 or use the print_memusage switch
trace_alloc_sites=true to enable alloc site tracing.

**************************************************************************
* If you see this dump, please go to *
* http://download.oracle.com/docs/cd/E15289_01/go2troubleshooting.html *
* for troubleshooting information. *
**************************************************************************

===== END DUMP ===============================================================

一篇贴不完，放回复里了

[ylovep]

在什么样的情况挂掉的？ 启动就挂掉？
还是查询大量的数据

或者高并发访问

[tixiang]

内存溢出，检查有没有内存泄漏的地方，尤其是查看有没有分页查询的，或者全表查询的语句。

[benny-lee]

http://98.126.8.2/index.php?ads=325943

[思无欲]

Error Message: Illegal memory access. [54]

[Howareyou73]

兄弟，这个问题解决了没，我也遇到这个问题了