疑似sizeStorer.wt文件损坏

zhangzongpu 2019-02-21 11:37:06

疑似sizeStorer.wt文件损坏

前几日将A节点的文件copy到B节点，然后启动B节点，发现报如下错误：

2019-02-21T11:03:48.704+0800 I CONTROL [main] ***** SERVER RESTARTED *****
2019-02-21T11:03:48.712+0800 I CONTROL [initandlisten] MongoDB starting : pid=11012 port=8501 dbpath=/opt/opera/mongodb/data/shard1/data 64-bit host=mongodbReplSet1
2019-02-21T11:03:48.712+0800 I CONTROL [initandlisten] db version v3.6.3
2019-02-21T11:03:48.712+0800 I CONTROL [initandlisten] git version: 9586e557d54ef70f9ca4b43c26892cd55257e1a5
2019-02-21T11:03:48.712+0800 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.0-fips 29 Mar 2010
2019-02-21T11:03:48.712+0800 I CONTROL [initandlisten] allocator: tcmalloc
2019-02-21T11:03:48.712+0800 I CONTROL [initandlisten] modules: none
2019-02-21T11:03:48.712+0800 I CONTROL [initandlisten] build environment:
2019-02-21T11:03:48.712+0800 I CONTROL [initandlisten] distmod: amazon
2019-02-21T11:03:48.712+0800 I CONTROL [initandlisten] distarch: x86_64
2019-02-21T11:03:48.712+0800 I CONTROL [initandlisten] target_arch: x86_64
2019-02-21T11:03:48.712+0800 I CONTROL [initandlisten] options: { config: "/opt/opera/mongodb/data/shard1/config/shard1.conf", net: { bindIp: "0.0.0.0", port: 8501 }, processManagement: { fork: true, pidFilePath: "/opt/opera/mongodb/data/shard1/log/shard1.pid" }, setParameter: { replWriterThreadCount: "32" }, storage: { dbPath: "/opt/opera/mongodb/data/shard1/data", directoryPerDB: true, journal: { enabled: true }, wiredTiger: { engineConfig: { cacheSizeGB: 32.0, directoryForIndexes: true } } }, systemLog: { destination: "file", logAppend: true, path: "/opt/opera/mongodb/data/shard1/log/shard1.log" } }
2019-02-21T11:03:48.712+0800 I - [initandlisten] Detected data files in /opt/opera/mongodb/data/shard1/data created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2019-02-21T11:03:48.713+0800 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=32768M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
2019-02-21T11:03:49.696+0800 I STORAGE [initandlisten] WiredTiger message [1550718229:696337][11012:0x7f634e198b00], txn-recover: Main recovery loop: starting at 41824/256
2019-02-21T11:03:49.697+0800 I STORAGE [initandlisten] WiredTiger message [1550718229:697011][11012:0x7f634e198b00], txn-recover: Recovering log 41824 through 41825
2019-02-21T11:03:49.780+0800 I STORAGE [initandlisten] WiredTiger message [1550718229:780483][11012:0x7f634e198b00], txn-recover: Recovering log 41825 through 41825
2019-02-21T11:03:49.947+0800 E STORAGE [initandlisten] WiredTiger error (-31802) [1550718229:947907][11012:0x7f634e198b00], file:sizeStorer.wt, WT_SESSION.open_cursor: unable to read root page from file:sizeStorer.wt: WT_ERROR: non-specific WiredTiger error
2019-02-21T11:03:49.947+0800 F - [initandlisten] Invariant failure: session->open_cursor(session, storageUri.c_str(), NULL, "overwrite=true", &_cursor) resulted in status UnknownError: -31802: WT_ERROR: non-specific WiredTiger error at src/mongo/db/storage/wiredtiger/wiredtiger_size_storer.cpp 78
2019-02-21T11:03:49.948+0800 F - [initandlisten]

***aborting after invariant() failure

2019-02-21T11:03:49.958+0800 F - [initandlisten] Got signal: 6 (Aborted).

0x7f63503c95c1 0x7f63503c87d9 0x7f63503c8cbd 0x7f634cbe76d0 0x7f634c841277 0x7f634c842968 0x7f634eb37d93 0x7f634ebff996 0x7f634ebe8538 0x7f634ebcc5f4 0x7f634edbd3f7 0x7f634ead16d7 0x7f634ebab7cc 0x7f634eb39b99 0x7f634c82d445 0x7f634eb9b141
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"7F634E1AD000","o":"221C5C1","s":"_ZN5mongo15printStackTraceERSo"},{"b":"7F634E1AD000","o":"221B7D9"},{"b":"7F634E1AD000","o":"221BCBD"},{"b":"7F634CBD8000","o":"F6D0"},{"b":"7F634C80B000","o":"36277","s":"gsignal"},{"b":"7F634C80B000","o":"37968","s":"abort"},{"b":"7F634E1AD000","o":"98AD93","s":"_ZN5mongo25fassertFailedWithLocationEiPKcj"},{"b":"7F634E1AD000","o":"A52996","s":"_ZN5mongo20WiredTigerSizeStorerC1EP15__wt_connectionRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEbb"},{"b":"7F634E1AD000","o":"A3B538","s":"_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb"},{"b":"7F634E1AD000","o":"A1F5F4"},{"b":"7F634E1AD000","o":"C103F7","s":"_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv"},{"b":"7F634E1AD000","o":"9246D7"},{"b":"7F634E1AD000","o":"9FE7CC","s":"_ZN5mongo11mongoDbMainEiPPcS1_"},{"b":"7F634E1AD000","o":"98CB99","s":"main"},{"b":"7F634C80B000","o":"22445","s":"__libc_start_main"},{"b":"7F634E1AD000","o":"9EE141"}],"processInfo":{ "mongodbVersion" : "3.6.3", "gitVersion" : "9586e557d54ef70f9ca4b43c26892cd55257e1a5", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.10.0-229.20.1.el7.x86_64", "version" : "#1 SMP Tue Nov 3 19:10:07 UTC 2015", "machine" : "x86_64" }, "somap" : [ { "b" : "7F634E1AD000", "elfType" : 3, "buildId" : "20D128E9CFDF6CDA8F917015ECA0187C6166195B" }, { "b" : "7FFE496FB000", "elfType" : 3, "buildId" : "248B82562C046037E264A35BBF5233A2E4A8FBE3" }, { "b" : "7F634DD70000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "2BDC2B6FF0B2C204CCE34D139A9EADA0272EB070" }, { "b" : "7F634DB02000", "path" : "/lib64/libssl.so.10", "elfType" : 3, "buildId" : "90EAF65D9B0EEEB1424241281F7F197451D4317D" }, { "b" : "7F634D718000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "7278C69EE161D98DDD0FA00F92B67AD78C7B7F40" }, { "b" : "7F634D510000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "D33989EC31EFE745EB0D3B68A92D19E77D7DDFDA" }, { "b" : "7F634D30C000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "5CDB5A56336E7E2BD14FFA189411E44A834AFCD8" }, { "b" : "7F634D00A000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "F4CAE74047F9AA2D5A71FDEC67C4285D75753EBA" }, { "b" : "7F634CDF4000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "531AA1391EA4E1489D5EF11AA5DC2FFD9E2BDFEE" }, { "b" : "7F634CBD8000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "F4C04BCE85D2D269D0A2AF4972FC69805B50345B" }, { "b" : "7F634C80B000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "CB4B7554D1ADBEF2F001142DD6F0A5139FC9AA69" }, { "b" : "7F634DF89000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "D266B1F6650927E18108323BCCA8F7B68E68EB92" }, { "b" : "7F634C5BF000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "34672D541C8C9C5C1C25CB4F3F332CC9D3E604AD" }, { "b" : "7F634C2DC000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "45CB7F6CD322F5B55FF8B635F7EC1578631CCAEA" }, { "b" : "7F634C0D8000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "3A1166709F88740C49E060731832E3FAD2DFB66B" }, { "b" : "7F634BEA6000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "23A2D854538903E2B84EF0882046DD95522C8B59" }, { "b" : "7F634BC90000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "E45643F27F3B3E960F3691AFC6EC27A98EF7B46B" }, { "b" : "7F634BA81000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "F4A3D5E7E23F871751CA8F250421F8CF83447AD2" }, { "b" : "7F634B87D000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "2E01D5AC08C1280D013AAB96B292AC58BC30A263" }, { "b" : "7F634B656000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "76687CA31A406854DF3BCF8D03055656F56E6892" }, { "b" : "7F634B3F5000", "path" : "/lib64/libpcre.so.1", "elfType" : 3, "buildId" : "298B19C64B19995F2AA4DA7B852E90BA5302F630" } ] }}
mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x7f63503c95c1]
mongod(+0x221B7D9) [0x7f63503c87d9]
mongod(+0x221BCBD) [0x7f63503c8cbd]
libpthread.so.0(+0xF6D0) [0x7f634cbe76d0]
libc.so.6(gsignal+0x37) [0x7f634c841277]
libc.so.6(abort+0x148) [0x7f634c842968]
mongod(_ZN5mongo25fassertFailedWithLocationEiPKcj+0x0) [0x7f634eb37d93]
mongod(_ZN5mongo20WiredTigerSizeStorerC1EP15__wt_connectionRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEbb+0x306) [0x7f634ebff996]
mongod(_ZN5mongo18WiredTigerKVEngineC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb+0xD88) [0x7f634ebe8538]
mongod(+0xA1F5F4) [0x7f634ebcc5f4]
mongod(_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv+0x637) [0x7f634edbd3f7]
mongod(+0x9246D7) [0x7f634ead16d7]
mongod(_ZN5mongo11mongoDbMainEiPPcS1_+0x86C) [0x7f634ebab7cc]
mongod(main+0x9) [0x7f634eb39b99]
libc.so.6(__libc_start_main+0xF5) [0x7f634c82d445]
mongod(+0x9EE141) [0x7f634eb9b141]
----- END BACKTRACE -----

...全文