Hbase not support MemStore to BlockCache

原文地址： http://web.archiveorange.com/archive/v/kaweJLpl8BCoXAfi3S8p

 

提问题的这个哥们还是蛮有想法的，今天还想HBase的写操作会不会经过LruBlockCache，看完代码以后，答案是否定地~

we are trying to read efficiently a hot column family (in_memory=true,

blockcaching=true) that get writes at say 500 qps and reads at 10,000 qps.



- as long as writes are in memstore we get them from memstore and its fast

- if we have read it once it will be at least in block cache (gets priority

due to in_memory=true) and subsequent reads are faster

- however memstore flush puts records on disk which demands for disk IO to

get them back in block cache



is there a way for memstore flush to go to blockcache?





-- Abhijit



-----------------------



Not currently.



Blockcache is made of blocks pulled from HDFS.  It'd be a little

awkward inserting hot records into the block cache w/o going via HDFS.



But, yes, you have a good point that flush is disruptive of hot records.



In the past we talked of a keyvalue cache on top of the block cache

but it fell out of favor because block cache seemed to be good enough

but sounds like we need to revive it or do some fancy dancing if

column family is marked in-memory, we keep around the snapshot of

memstore until we know the block cache has been populated?



Any other suggestions?



File an issue.  We need to come up w/ a fix for this case.



Thanks for writing the list,



St.Ack



------------------------



thanks bunch stack for the response.



agree, we should work towards it. very attractive feature for onlinestore

use case of hbase



i will open a Jira and as we ramp-up on HBase here, might be able to discuss

solutions and potentially work on adding this feature



-- Abhijit