hbase运维：hbase宕机 org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler

[qqqwww159]

大量数据导入hbase后，hbase警告空间不足。晚上hbase宕机，查看日志报错
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler
Failed open of region=**,201032001279-001-004,1529571351771.ee5454ce5412f84872d9a86868f79316., starting to roll back the global memstore size.
org.apache.hadoop.security.AccessControlException: Permission denied: user=hbase, access=WRITE, inode="/hbase/data/**/ee5454ce5412f84872d9a86868f79316/recovered.edits":hdfs:hbase:drwxr-xr-x

这个报错是权限问题，但是其他表操作步骤一样，其他的表没有权限问题。。

最近被hbase宕机搞得头皮发麻，有没有大神请教一下怎么很好运维HBASE。

其他错误日志：
INFO org.apache.hadoop.hbase.zookeeper.MetaTableLocator: Failed verification of hbase:meta,,1 at address=slave3,60020,1527752753309, exception=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is not online on slave3,60020,1527775126262
at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2964)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1055)

这个好像是是找不到region。。

[qqqwww159]

还有这个：
org.apache.hadoop.hbase.zookeeper.MetaTableLocator: Failed verification of hbase:meta,,1 at address=slave2,60020,1528184649556, exception=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is not online on slave2,60020,1528249996819
at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2964)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1055)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegionInfo(RSRpcServices.java:1339)
at org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:22233)