ORA-00445: background process W000 did not start after 120 seconds导致数据库死机

huangdh12 2014-08-04 09:21:30
问题描述：
在数据库运行过程中发现两次 smon 进程僵死导致的数据库down机。 在这个时候， 连接数 一次为400，一次为141. 出现这样的情况的时候， sqlplus /nolog 然后 conn / as sysdba 无响应。
数据库服务器配置为 linux redhat 64位 内存 32g ， oracle 为11203. sga分配12g。大家帮忙看看错误原因。
alert中的描述如下
======================7-10================================
Thu Jul 10 14:02:36 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_smco_34722.trc (incident=537285):
ORA-00445: background process "W009" did not start after 120 seconds
Incident details in: /oracle/diag/rdbms/app/app/incident/incdir_537285/app_smco_34722_i537285.trc
Dumping diagnostic data in directory=[cdmp_20140710140240], requested by (instance=1, osid=34722 (SMCO)), summary=[incident=537285].
Thu Jul 10 14:02:42 2014
Thu Jul 10 14:03:36 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:04:36 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:06:07 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:07:07 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:07:13 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_smco_34722.trc (incident=537286):
ORA-00445: background process "W004" did not start after 120 seconds
Incident details in: /oracle/diag/rdbms/app/app/incident/incdir_537286/app_smco_34722_i537286.trc
Thu Jul 10 14:08:38 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:09:18 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_smco_34722.trc (incident=537287):
ORA-00445: background process "W005" did not start after 120 seconds
Incident details in: /oracle/diag/rdbms/app/app/incident/incdir_537287/app_smco_34722_i537287.trc
Thu Jul 10 14:09:39 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:11:09 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:11:38 2014
Thu Jul 10 14:12:09 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:13:40 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:14:40 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:16:11 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:17:11 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:18:42 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:19:28 2014
Thu Jul 10 14:19:43 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:21:14 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:22:14 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:23:45 2014
PMON failed to acquire latch, see PMON dump
Thu Jul 10 14:28:35 2014
============================================================
========================8-4====================================
Sun Aug 03 23:03:42 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_smco_5482.trc (incident=681013):
ORA-00445: background process "W000" did not start after 120 seconds
Incident details in: /oracle/diag/rdbms/app/app/incident/incdir_681013/app_smco_5482_i681013.trc
Sun Aug 03 23:03:44 2014
Dumping diagnostic data in directory=[cdmp_20140803230344], requested by (instance=1, osid=5482 (SMCO)), summary=[incident=681013].
Sun Aug 03 23:07:32 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_smco_5482.trc (incident=681014):
ORA-00445: background process "W002" did not start after 120 seconds
Incident details in: /oracle/diag/rdbms/app/app/incident/incdir_681014/app_smco_5482_i681014.trc
Sun Aug 03 23:21:07 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=681197):
ORA-00445: background process "q002" did not start after 120 seconds
Incident details in: /oracle/diag/rdbms/app/app/incident/incdir_681197/app_qmnc_5253_i681197.trc
Sun Aug 03 23:23:20 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=681198):
ORA-00445: background process "q002" did not start after 120 seconds
Incident details in: /oracle/diag/rdbms/app/app/incident/incdir_681198/app_qmnc_5253_i681198.trc
Sun Aug 03 23:25:32 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=681199):
ORA-00445: background process "q002" did not start after 120 seconds
Incident details in: /oracle/diag/rdbms/app/app/incident/incdir_681199/app_qmnc_5253_i681199.trc
Sun Aug 03 23:27:44 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=681200):
ORA-00445: background process "q002" did not start after 120 seconds
Sun Aug 03 23:29:55 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=681201):
ORA-00445: background process "q002" did not start after 120 seconds
Sun Aug 03 23:32:06 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=681202):
ORA-00445: background process "q002" did not start after 120 seconds
Sun Aug 03 23:34:17 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=681203):
ORA-00445: background process "q002" did not start after 120 seconds
Sun Aug 03 23:36:28 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=681204):
ORA-00445: background process "q002" did not start after 120 seconds
Sun Aug 03 23:38:39 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704837):
ORA-00445: background process "q002" did not start after 120 seconds
Sun Aug 03 23:40:50 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704838):
ORA-00445: background process "q002" did not start after 120 seconds
Sun Aug 03 23:43:01 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704839):
ORA-00445: background process "q002" did not start after 120 seconds
Sun Aug 03 23:45:12 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704840):
ORA-00445: background process "q002" did not start after 120 seconds
Sun Aug 03 23:47:23 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704841):
ORA-00445: background process "q002" did not start after 120 seconds
Sun Aug 03 23:49:35 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704842):
ORA-00445: background process "q002" did not start after 120 seconds
Sun Aug 03 23:51:46 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704843):
ORA-00445: background process "q002" did not start after 120 seconds
Sun Aug 03 23:53:57 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704844):
ORA-00445: background process "q002" did not start after 120 seconds
Sun Aug 03 23:56:08 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704845):
ORA-00445: background process "q002" did not start after 120 seconds
Sun Aug 03 23:58:19 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704846):
ORA-00445: background process "q002" did not start after 120 seconds
Mon Aug 04 00:00:30 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704847):
ORA-00445: background process "q002" did not start after 120 seconds
Mon Aug 04 00:02:41 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704848):
ORA-00445: background process "q002" did not start after 120 seconds
Mon Aug 04 00:04:52 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704849):
ORA-00445: background process "q002" did not start after 120 seconds
Incident details in: /oracle/diag/rdbms/app/app/incident/incdir_704849/app_qmnc_5253_i704849.trc
Mon Aug 04 00:07:04 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704850):
ORA-00445: background process "q002" did not start after 120 seconds
Incident details in: /oracle/diag/rdbms/app/app/incident/incdir_704850/app_qmnc_5253_i704850.trc
Mon Aug 04 00:09:16 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704851):
ORA-00445: background process "q002" did not start after 120 seconds
Incident details in: /oracle/diag/rdbms/app/app/incident/incdir_704851/app_qmnc_5253_i704851.trc
Mon Aug 04 00:11:28 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704852):
ORA-00445: background process "q002" did not start after 120 seconds
Mon Aug 04 00:13:40 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704853):
ORA-00445: background process "q002" did not start after 120 seconds
Mon Aug 04 00:15:51 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704854):
ORA-00445: background process "q002" did not start after 120 seconds
Mon Aug 04 00:18:02 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704855):
ORA-00445: background process "q002" did not start after 120 seconds
Mon Aug 04 00:20:13 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704856):
ORA-00445: background process "q002" did not start after 120 seconds
Mon Aug 04 00:22:24 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704857):
ORA-00445: background process "q002" did not start after 120 seconds
Mon Aug 04 00:23:23 2014
PMON failed to acquire latch, see PMON dump
Mon Aug 04 00:24:23 2014
PMON failed to acquire latch, see PMON dump
Mon Aug 04 00:24:35 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704858):
ORA-00445: background process "q002" did not start after 120 seconds
Mon Aug 04 00:25:53 2014
PMON failed to acquire latch, see PMON dump
Mon Aug 04 00:26:46 2014
Errors in file /oracle/diag/rdbms/app/app/trace/app_qmnc_5253.trc (incident=704859):
ORA-00445: background process "q003" did not start after 120 seconds
Mon Aug 04 00:26:53 2014
PMON failed to acquire latch, see PMON dump
Mon Aug 04 00:28:23 2014
PMON failed to acquire latch, see PMON dump
Mon Aug 04 00:29:23 2014
PMON failed to acquire latch, see PMON dump
Mon Aug 04 00:30:53 2014
PMON failed to acquire latch, see PMON dump
...全文
2801 19 打赏 收藏 转发到动态 举报
写回复
用AI写文章
19 条回复
切换为时间正序
请发表友善的回复…
发表回复
huangdh12 2014-08-10
  • 打赏
  • 举报
 回复
引用 18 楼 sjcss 的回复:
楼主问题处理完成要分享哦~ 搞不好,大版给你个精华,呵~
因为是生产服务器， 所以不能够随便创建。 linux红帽6.2 安装vnc后 好像曾经导致过 df -h 无法执行完成的情况。 现在禁用掉vnc ，看看错误是否还会重现。 主要是 坑爹的机房，最近天气太热了，停电了几次。
美到心痛 2014-08-08
  • 打赏
  • 举报
 回复
楼主问题处理完成要分享哦~ 搞不好,大版给你个精华,呵~
美到心痛 2014-08-08
  • 打赏
  • 举报
 回复
不知道怎么处理你的问题~ 如果我遇到这种情况,又不知道怎么处理. 先现这台服务器上再建一个数据库,看会不会出这样的错,如果会,就是系统和软件的问题. 如果不会,把参数文件最小化,再nomount,再mount,再open~ 看在那里出错.一步步确认.
huangdh12 2014-08-05
  • 打赏
  • 举报
 回复
listener.ora --------------------------------------------------------------------------------- # listener.ora Network Configuration File: /oracle/product/11.2.0/db_1/network/admin/listener.ora # Generated by Oracle configuration tools. SID_LIST_LISTENER = (SID_LIST = (SID_DESC = (SID_NAME = PLSExtProc) (ORACLE_HOME = /oracle/product/11.2.0/db_1) (PROGRAM = extproc) ) (SID_DESC = (GLOBAL_DBNAME = APP) (ORACLE_HOME = /oracle/product/11.2.0/db_1) (SID_NAME = APP) ) ) LISTENER = (DESCRIPTION_LIST = (DESCRIPTION = (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = APPDBSERVER)(PORT = 1521)) ) ) ADR_BASE_LISTENER = /oracle DIAG_ADR_ENABLED_LISTENER = OFF INBOUND_CONNECT_TIME_LISTENER = 0 --------------------------------------------------------------------------------- tnsnames.ora --------------------------------------------------------------------------------- # tnsnames.ora Network Configuration File: /oracle/product/11.2.0/db_1/network/admin/tnsnames.ora # Generated by Oracle configuration tools. APP = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = APPDBSERVER)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = app) ) ) --------------------------------------------------------------------------------- sqlnet.ora --------------------------------------------------------------------------------- # sqlnet.ora Network Configuration File: /oracle/product/11.2.0/db_1/network/admin/sqlnet.ora # Generated by Oracle configuration tools. NAMES.DIRECTORY_PATH= (TNSNAMES, EZCONNECT) ADR_BASE = /oracle tcp.validnode_checking=no DIAG_ADR_ENABLED = OFF SQLNET.INBOUND_CONNECT_TIMEOUT=0 ---------------------------------------------------------------------------------
huangdh12 2014-08-05
  • 打赏
  • 举报
 回复
8.3 的时候 发生过机房停电。 两次。
huangdh12 2014-08-05
  • 打赏
  • 举报
 回复
引用 13 楼 wildwave 的回复:
Aug 3 03:40:00 APPDBSERVER fcoemon: error 98 Address already in use Aug 3 03:40:00 APPDBSERVER fcoemon: clif bind failed 系统日志里一直在报这个错误。不知道是否和宕机时一样。8月4日出故障，你给了8月3日的日志，无语... 这个错误貌似和存储有关，检查一下
只有这个。 8月4号没有。
小灰狼W 2014-08-05
  • 打赏
  • 举报
 回复
Aug 3 03:40:00 APPDBSERVER fcoemon: error 98 Address already in use Aug 3 03:40:00 APPDBSERVER fcoemon: clif bind failed 系统日志里一直在报这个错误。不知道是否和宕机时一样。8月4日出故障，你给了8月3日的日志，无语... 这个错误貌似和存储有关，检查一下
huangdh12 2014-08-05
  • 打赏
  • 举报
 回复
引用 3 楼 bw555 的回复:
你看看这个和你现象一致吗？参考下吧 http://www.blogjava.net/decode360/archive/2009/06/23/287745.html
这个 我以前已经设置过了。 好像 不行。
huangdh12 2014-08-05
  • 打赏
  • 举报
 回复
引用 6 楼 wildwave 的回复:
里面没有我说的两类文件... /var/log/message & *pmon*.trc
csdn有办法传附件的吗？传到itpub了 http://www.itpub.net/forum.php?mod=viewthread&tid=1881616&page=1#pid22526226 在6#
huangdh12 2014-08-05
  • 打赏
  • 举报
 回复
引用 8 楼 sjcss 的回复:
把你的spfile 贴出来看看呗~
app.__db_cache_size=5100273664 app.__java_pool_size=33554432 app.__large_pool_size=33554432 app.__oracle_base='/oracle'#ORACLE_BASE set from environment app.__pga_aggregate_target=10737418240 app.__sga_target=12884901888 app.__shared_io_pool_size=0 app.__shared_pool_size=7616856064 app.__streams_pool_size=33554432 *.audit_file_dest='/oracle/admin/app/adump' *.audit_trail='db' *.compatible='11.2.0.0.0' *.control_files='/oracle/oradata/app/control01.ctl','/oracle/fast_recovery_area/app/control02.ctl','/home/oracle/oradata/app/control03.ctl' *.db_block_size=8192 *.db_name='app' *.db_recovery_file_dest='/oracle/fast_recovery_area' *.db_recovery_file_dest_size=4322230272 *.diagnostic_dest='/oracle' *.dispatchers='(PROTOCOL=TCP) (SERVICE=appXDB)' *.open_cursors=300 *.pga_aggregate_target=10737418240 *.processes=3000 *.remote_login_passwordfile='EXCLUSIVE' *.sec_case_sensitive_logon=FALSE *.sessions=3105 *.sga_target=12884901888 *.undo_tablespace='UNDOTBS1'
bw555 2014-08-04
  • 打赏
  • 举报
 回复
你看看这个和你现象一致吗？参考下吧 http://www.blogjava.net/decode360/archive/2009/06/23/287745.html
CT_LXL 2014-08-04
  • 打赏
  • 举报
 回复
引用 1 楼 huangdh12 的回复:
Stack:
找大版主看看嘛，DBA方面的知识我也比较欠缺啊
huangdh12 2014-08-04
  • 打赏
  • 举报
 回复
附件发不上来。 Trace file /oracle/diag/rdbms/app/app/trace/app_smco_5482.trc Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options ORACLE_HOME = /oracle/product/11.2.0/db_1 System name: Linux Node name: appDBSERVER Release: 2.6.32-131.0.15.el6.x86_64 Version: #1 SMP Tue May 10 15:42:40 EDT 2011 Machine: x86_64 Instance name: app Redo thread mounted by this instance: 1 Oracle process number: 22 Unix process pid: 5482, image: oracle@appDBSERVER (SMCO) *** 2014-08-03 23:02:41.726 *** SESSION ID:(4161.3) 2014-08-03 23:02:41.726 *** CLIENT ID:() 2014-08-03 23:02:41.726 *** SERVICE NAME:(SYS$BACKGROUND) 2014-08-03 23:02:41.726 *** MODULE NAME:(KTSJ) 2014-08-03 23:02:41.726 *** ACTION NAME:(KTSJ Coordinator) 2014-08-03 23:02:41.726 Waited for process W000 to initialize for 60 seconds *** 2014-08-03 23:02:41.749 Process diagnostic dump for W000, OS id=72980 ------------------------------------------------------------------------------- os thread scheduling delay history: (sampling every 1.000000 secs) 0.000000 secs at [ 23:02:41 ] NOTE: scheduling delay has not been sampled for 0.203651 secs 0.000000 secs from [ 23:02:37 - 23:02:42 ], 5 sec avg 0.000000 secs from [ 23:01:42 - 23:02:42 ], 1 min avg 0.000000 secs from [ 22:57:42 - 23:02:42 ], 5 min avg loadavg : 2.07 1.18 0.62 Memory (Avail / Total) = 471.81M / 32095.73M Swap (Avail / Total) = 63987.36M / 63999.99M F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD 0 S oracle 72980 1 0 80 0 - 3204958 semtim 23:01 ? 00:00:00 ora_w000_app *** 2014-08-03 23:02:46.648 Stack: [Thread debugging using libthread_db enabled] 0x0000003faf8e8f17 in semop () from /lib64/libc.so.6 #0 0x0000003faf8e8f17 in semop () from /lib64/libc.so.6 #1 0x0000000009355767 in sskgpwwait () #2 0x000000000935455e in skgpwwait () #3 0x0000000008f96c19 in kslges () #4 0x0000000008f965c8 in kslgetl () #5 0x0000000008f93536 in ksfglt () #6 0x000000000932859a in kghalo () #7 0x0000000001e9c3de in kksInitSessionState () #8 0x0000000000b8bb95 in ksucri () #9 0x0000000000ba15f6 in ksucre () #10 0x0000000000bde883 in ksvrdp () #11 0x0000000002547a4a in opirip () #12 0x00000000017e8d4d in opidrv () #13 0x0000000001de40bb in sou2o () #14 0x0000000000a0b136 in opimai_real () #15 0x0000000001dea02c in ssthrdmain () #16 0x0000000000a0b01d in main () A debugging session is active. Inferior 1 [process 72980] will be detached. Quit anyway? (y or n) [answered Y; input not from terminal] ------------------------------------------------------------------------------- Process diagnostic dump actual duration=4.870000 sec (max dump time=30.000000 sec) *** 2014-08-03 23:02:46.649 *** 2014-08-03 23:02:57.674 Waited for process W000 to initialize for 76 seconds *** 2014-08-03 23:02:57.674 Process diagnostic dump for W000, OS id=72980 ------------------------------------------------------------------------------- os thread scheduling delay history: (sampling every 1.000000 secs) 0.000000 secs at [ 23:02:57 ] NOTE: scheduling delay has not been sampled for 0.016198 secs 0.000000 secs from [ 23:02:53 - 23:02:58 ], 5 sec avg 0.000000 secs from [ 23:01:58 - 23:02:58 ], 1 min avg 0.000000 secs from [ 22:57:58 - 23:02:58 ], 5 min avg loadavg : 2.34 1.28 0.66 Memory (Avail / Total) = 385.38M / 32095.73M Swap (Avail / Total) = 63987.36M / 63999.99M F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD 0 S oracle 72980 1 0 80 0 - 3204958 semtim 23:01 ? 00:00:00 ora_w000_app *** 2014-08-03 23:03:01.327 Stack: [Thread debugging using libthread_db enabled] 0x0000003faf8e8f17 in semop () from /lib64/libc.so.6 #0 0x0000003faf8e8f17 in semop () from /lib64/libc.so.6 #1 0x0000000009355767 in sskgpwwait () #2 0x000000000935455e in skgpwwait () #3 0x0000000008f96c19 in kslges () #4 0x0000000008f965c8 in kslgetl () #5 0x0000000008f93536 in ksfglt () #6 0x000000000932859a in kghalo () #7 0x0000000001e9c3de in kksInitSessionState () #8 0x0000000000b8bb95 in ksucri () #9 0x0000000000ba15f6 in ksucre () #10 0x0000000000bde883 in ksvrdp () #11 0x0000000002547a4a in opirip () #12 0x00000000017e8d4d in opidrv () #13 0x0000000001de40bb in sou2o () #14 0x0000000000a0b136 in opimai_real () #15 0x0000000001dea02c in ssthrdmain () #16 0x0000000000a0b01d in main () A debugging session is active. Inferior 1 [process 72980] will be detached. Quit anyway? (y or n) [answered Y; input not from terminal] ------------------------------------------------------------------------------- Process diagnostic dump actual duration=3.650000 sec (max dump time=30.000000 sec) *** 2014-08-03 23:03:01.327 *** 2014-08-03 23:03:13.339 Waited for process W000 to initialize for 91 seconds *** 2014-08-03 23:03:13.339 Process diagnostic dump for W000, OS id=72980 ------------------------------------------------------------------------------- os thread scheduling delay history: (sampling every 1.000000 secs) 0.000000 secs at [ 23:03:12 ] NOTE: scheduling delay has not been sampled for 0.606208 secs 0.000000 secs from [ 23:03:08 - 23:03:13 ], 5 sec avg 0.000000 secs from [ 23:02:13 - 23:03:13 ], 1 min avg 0.000000 secs from [ 22:58:14 - 23:03:13 ], 5 min avg loadavg : 2.49 1.37 0.70 Memory (Avail / Total) = 559.61M / 32095.73M Swap (Avail / Total) = 63987.36M / 63999.99M F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD 0 S oracle 72980 1 0 80 0 - 3204958 semtim 23:01 ? 00:00:00 ora_w000_app *** 2014-08-03 23:03:16.960 Stack: [Thread debugging using libthread_db enabled] 0x0000003faf8e8f17 in semop () from /lib64/libc.so.6 #0 0x0000003faf8e8f17 in semop () from /lib64/libc.so.6 #1 0x0000000009355767 in sskgpwwait () #2 0x000000000935455e in skgpwwait () #3 0x0000000008f96c19 in kslges () #4 0x0000000008f965c8 in kslgetl () #5 0x0000000008f93536 in ksfglt () #6 0x000000000932859a in kghalo () #7 0x0000000001e9c3de in kksInitSessionState () #8 0x0000000000b8bb95 in ksucri () #9 0x0000000000ba15f6 in ksucre () #10 0x0000000000bde883 in ksvrdp () #11 0x0000000002547a4a in opirip () #12 0x00000000017e8d4d in opidrv () #13 0x0000000001de40bb in sou2o () #14 0x0000000000a0b136 in opimai_real () #15 0x0000000001dea02c in ssthrdmain () #16 0x0000000000a0b01d in main () A debugging session is active. Inferior 1 [process 72980] will be detached. Quit anyway? (y or n) [answered Y; input not from terminal] ------------------------------------------------------------------------------- Process diagnostic dump actual duration=3.620000 sec (max dump time=30.000000 sec) *** 2014-08-03 23:03:16.961 *** 2014-08-03 23:03:28.973 Waited for process W000 to initialize for 107 seconds *** 2014-08-03 23:03:28.973 Process diagnostic dump for W000, OS id=72980 ------------------------------------------------------------------------------- os thread scheduling delay history: (sampling every 1.000000 secs) 0.000000 secs at [ 23:03:28 ] NOTE: scheduling delay has not been sampled for 0.160286 secs 0.000000 secs from [ 23:03:24 - 23:03:29 ], 5 sec avg 0.000000 secs from [ 23:02:29 - 23:03:29 ], 1 min avg 0.000000 secs from [ 22:58:29 - 23:03:29 ], 5 min avg loadavg : 2.60 1.45 0.73 Memory (Avail / Total) = 544.25M / 32095.73M Swap (Avail / Total) = 63987.36M / 63999.99M F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD 0 S oracle 72980 1 0 80 0 - 3204958 semtim 23:01 ? 00:00:00 ora_w000_app *** 2014-08-03 23:03:32.423 Stack: [Thread debugging using libthread_db enabled] 0x0000003faf8e8f17 in semop () from /lib64/libc.so.6 #0 0x0000003faf8e8f17 in semop () from /lib64/libc.so.6 #1 0x0000000009355767 in sskgpwwait () #2 0x000000000935455e in skgpwwait () #3 0x0000000008f96c19 in kslges () #4 0x0000000008f965c8 in kslgetl () #5 0x0000000008f93536 in ksfglt () #6 0x000000000932859a in kghalo () #7 0x0000000001e9c3de in kksInitSessionState () #8 0x0000000000b8bb95 in ksucri () #9 0x0000000000ba15f6 in ksucre () #10 0x0000000000bde883 in ksvrdp () #11 0x0000000002547a4a in opirip () #12 0x00000000017e8d4d in opidrv () #13 0x0000000001de40bb in sou2o () #14 0x0000000000a0b136 in opimai_real () #15 0x0000000001dea02c in ssthrdmain () #16 0x0000000000a0b01d in main () A debugging session is active. Inferior 1 [process 72980] will be detached. Quit anyway? (y or n) [answered Y; input not from terminal] ------------------------------------------------------------------------------- Process diagnostic dump actual duration=3.450000 sec (max dump time=30.000000 sec) *** 2014-08-03 23:03:32.423 *** 2014-08-03 23:03:41.653 Killing process (ospid 72980): (reason=KSOREQ_WAIT_CANCELLED error=0) ... and the process is still alive after kill! Incident 681013 created, dump file: /oracle/diag/rdbms/app/app/incident/incdir_681013/app_smco_5482_i681013.trc ORA-00445: background process "W000" did not start after 120 seconds *** 2014-08-03 23:06:32.456 Waited for process W002 to initialize for 60 seconds *** 2014-08-03 23:06:32.456 Process diagnostic dump for W002, OS id=73074 ------------------------------------------------------------------------------- os thread scheduling delay history: (sampling every 1.000000 secs) 0.000000 secs at [ 23:06:31 ] NOTE: scheduling delay has not been sampled for 0.732194 secs 0.000000 secs from [ 23:06:27 - 23:06:32 ], 5 sec avg 0.000000 secs from [ 23:05:33 - 23:06:32 ], 1 min avg 0.000000 secs from [ 23:01:33 - 23:06:32 ], 5 min avg loadavg : 1.83 1.67 0.95 Memory (Avail / Total) = 548.94M / 32095.73M Swap (Avail / Total) = 63987.37M / 63999.99M F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD 0 S oracle 73074 1 0 80 0 - 3204958 semtim 23:05 ? 00:00:00 ora_w002_app *** 2014-08-03 23:06:35.493 Stack:
sych888 2014-08-04
  • 打赏
  • 举报
 回复
学习学习..............
美到心痛 2014-08-04
  • 打赏
  • 举报
 回复
把你的spfile 贴出来看看呗~
IT_lau 2014-08-04
  • 打赏
  • 举报
 回复
这方面知识匮乏啊
小灰狼W 2014-08-04
  • 打赏
  • 举报
 回复
里面没有我说的两类文件... /var/log/message & *pmon*.trc
huangdh12 2014-08-04
  • 打赏
  • 举报
 回复
引用 4 楼 wildwave 的回复:
提供下故障时段的操作系统日志以及pmon的trace文件 可能和系统问题有关
这里好像没看见上传附件的。 http://www.itpub.net/forum.php?mod=viewthread&tid=1881616&page=1#pid22524952 这里面我把trace文件上传了。 看看。
小灰狼W 2014-08-04
  • 打赏
  • 举报
 回复
提供下故障时段的操作系统日志以及pmon的trace文件 可能和系统问题有关
ORA-00445: background process “J000“ did not start after 120 seconds
客户反馈数据库宕机： 查看alert日志： 1 Mon Dec 30 08:56:01 2019 2 WARNING: inbound connection timed out (ORA-3136) 3 Mon Dec 30 08:56:04 2019 4 Errors in file /u01/app/oracle/diag/rdbms/ecology/ecology/trace/ecology_cjq0_25270.trc (incident=300282): 5 ORA-00445: ba.
ORA-00445: background process "J000" did not start after 120 seconds
原文地址：https://blog.csdn.net/lwei_998/article/details/9296001Anything that can go wrong will go wrong. ：“凡事只要有可能出错，那就一定会出错。”   墨菲定律再次被验证，早上数据库宕机，日志如下：Dump continued from file: /opt/ora11g/diag/rdbms/pro...
ORA-00445报错问题的处理
原文地址：http://blog.itpub.net/12679300/viewspace-1169288/一、问题背景：操作系统信息：Linux x86-64数据库信息：ORACLE 11.2.0.3-64问题出现前系统的变更：由于业务的需要，每天定时把本机的备份传输到本地的另外一台LINUX测试服务器，并在那台服务器上面实现自动恢复，实现测试库的数据保证和正式库的数据同步； 二、问题症状：在检...
ORA-00445 实例挂起
ORA-00445 实例挂起
一次数据库宕机问题的分析(r6笔记第5天)
今天来到办公室，发现有一台服务器中的数据库实例停掉了。这种情况真是意料之外，尤其是我还不是很熟悉这台机器的服务。赶紧查看数据库日志，可以看到数据库在昨晚停掉了，从日志来看...
基础和管理

17,377

社区成员

95,128

社区内容

发帖
与我相关
我的任务
社区描述
Oracle 基础和管理
社区管理员
  • 基础和管理社区
加入社区
  • 近7日
  • 近30日
  • 至今

加载中

查看更多榜单
社区公告
暂无公告

试试用AI创作助手写篇文章吧

+ 用AI写文章