20,848
社区成员




Scan s = new Scan();
System.out.print(s.getMaxResultSize()+" ");
s.setMaxResultSize(100);
System.out.println(s.getMaxResultSize());
ResultScanner ss = table.getScanner(s);
int i =0;
for(Result r:ss){
//....
i++;
}
ss.close();
System.out.println("result :" + i);
String StartRow = "XXXXXXXXXXXXX";
String StopRow = "YYYYYYYYYYYYY";
Scan MyScan = new Scan(StartRow.getBytes(), StopRow.getBytes());
......................................................................................
TableMapReduceUtil.initTableMapperJob(table, MyScan, mapper, outputKeyClass, outputValueClass, job)
[/quote][/quote]
你的rowkey是随机数的话那就没法确定起始点了吧,那么你是想从表中随意取出固定量的数据吗? 不然的话你肯定是根据一个规则来取数据的吧,那么就利用那个规则使用endpoint在region层过滤数据,也可使用filter在客户端层过滤,但是在客户端层过滤会加大网络传输消耗,在regionserver层过滤会加大regionserver的负担,权衡下吧[/quote]
filter是客户端过滤?呵呵,笑抽了
String StartRow = "XXXXXXXXXXXXX";
String StopRow = "YYYYYYYYYYYYY";
Scan MyScan = new Scan(StartRow.getBytes(), StopRow.getBytes());
......................................................................................
TableMapReduceUtil.initTableMapperJob(table, MyScan, mapper, outputKeyClass, outputValueClass, job)
[/quote][/quote]
你的rowkey是随机数的话那就没法确定起始点了吧,那么你是想从表中随意取出固定量的数据吗? 不然的话你肯定是根据一个规则来取数据的吧,那么就利用那个规则使用endpoint在region层过滤数据,也可使用filter在客户端层过滤,但是在客户端层过滤会加大网络传输消耗,在regionserver层过滤会加大regionserver的负担,权衡下吧
String StartRow = "XXXXXXXXXXXXX";
String StopRow = "YYYYYYYYYYYYY";
Scan MyScan = new Scan(StartRow.getBytes(), StopRow.getBytes());
......................................................................................
TableMapReduceUtil.initTableMapperJob(table, MyScan, mapper, outputKeyClass, outputValueClass, job)
final int MaxRowCount = 100;
int RowCount = 0;
Scan s = new Scan();
s.setCaching(MaxRowCount);
ResultScanner ss = table.getScanner(s);
for (int i = 0; i < MaxRowCount; i++) {
Result r = ss.next();
System.out.println("result :" + new String(r.getRow()));
RowCount++;
}
ss.close();
System.out.println("result :" + RowCount);
或者用endpoint解决限制行数scan的问题,以及scan性能问题
可以参考我的博客http://blog.csdn.net/tntzbzc/article/details/8918463
setMaxVersion控制的是单行最大TS版本数,不是行数
String StartRow = "XXXXXXXXXXXXX";
String StopRow = "YYYYYYYYYYYYY";
Scan MyScan = new Scan(StartRow.getBytes(), StopRow.getBytes());
......................................................................................
TableMapReduceUtil.initTableMapperJob(table, MyScan, mapper, outputKeyClass, outputValueClass, job)
[/quote]