hbase的参数说明
hbase.balancer.period
Period at which the region balancer runs in the master.
Type: int
Default:
300000 (5 mins)
Unit:
milliseconds
hbase.client.keyvalue.maxsize
Specifies the combined maximum allowed size of a
KeyValue
instance. This is to
set an upper boundary for a single entry saved in a storage file. Since they cannot
be split, it helps avoiding that a region cannot be split any further because the data
is too large. It seems wise to set this to a fraction of the maximum region size.
Setting it to zero or less disables the check.
Type:
int
Default:
10485760
Unit:
bytes
hbase.client.pause
General client pause value. Used mostly as value to wait before running a retry of
a failed get, region lookup, etc.
475
Type:
long
Default:
1000 (1 sec)
Unit:
milliseconds
hbase.client.retries.number
Maximum retries. Used as maximum for all retryable operations such as fetching
of the root region from root region server, getting a cell’s value, starting a row
update, etc.
Type:
int
Default:
10
Unit:
number
hbase.client.scanner.caching
Number of rows that will be fetched when calling
next
on a scanner if it is not
served from (local, client) memory. Higher caching values will enable faster scanners
but will eat up more memory and some calls of
next
may take longer and
longer time when the cache is empty. Do not set this value such that the time
between invocations is greater than the scanner timeout; i.e.
hbase.region
server.lease.period
.
Type:
int
Default:
1
Unit:
number
hbase.client.write.buffer
Default size of the
HTable
client write buffer in bytes. A bigger buffer takes more
memory—on both the client and server side since server instantiates the passed
write buffer to process it—but a larger buffer size reduces the number of RPCs
made. For an estimate of server-side memory-used, evaluate
hbase.cli
ent.write.buffer * hbase.regionserver.handler.count
.
Type:
long
Default:
2097152
Unit:
bytes
hbase.cluster.distributed
The mode the cluster will be in. Possible values are
false
for standalone mode and
true
for distributed mode. If
false
, startup will run all HBase and ZooKeeper daemons
together in the one JVM.
Type:
boolean
Default:
false
hbase.coprocessor.master.classes
A comma-separated list of
org.apache.hadoop.hbase.coprocessor.MasterObserver
coprocessors that are loaded by default on the active
HMaster
process. For any
implemented coprocessor methods, the listed classes will be called in order. After
476 | Appendix A: HBase Configuration Properties
implementing your own
MasterObserver
, just put it in HBase’s classpath and add
the fully qualified class name here.
Type: class names
Default:
<empty>
hbase.coprocessor.region.classes
A comma-separated list of Coprocessors that are loaded by default on all tables.
For any override coprocessor method, these classes will be called in order. After
implementing your own Coprocessor, just put it in HBase’s classpath and add the
fully qualified class name here. A coprocessor can also be loaded on demand by
setting
HTableDescriptor
.
Type: class names
Default:
<empty>
hbase.defaults.for.version.skip
Set to
true
to skip the
hbase.defaults.for.version
check. Setting this to
true
can
be useful in contexts other than the other side of a maven generation; i.e., running
in an IDE. You’ll want to set this boolean to
true
to avoid seeing the RuntimeException
complaint
"hbase-default.xml file seems to be for an old ver
sion of HBase (@@@VERSION@@@), this version is X.X.X-SNAPSHOT"
.
Type:
boolean
Default:
false
hbase.hash.type
The hashing algorithm for use in
HashFunction
. Two values are supported now:
murmur
(MurmurHash) and
jenkins
(JenkinsHash). Used by Bloom filters.
Type:
string
Default:
murmur
hbase.hregion.majorcompaction
The time (in milliseconds) between major compactions of all
HStoreFile
s in a region.
Default: 1 day. Set to 0 to disable automated major compactions.
Type:
long
Default:
86400000 (1 day)
Unit:
milliseconds
hbase.hregion.max.filesize
Maximum
HStoreFile
size. If any one of a column families’
HStoreFiles
has grown
to exceed this value, the hosting
HRegion
is split in two.
Type:
long
Default:
268435456 (256 * 1024 * 1024)
Unit:
bytes
HBase Configuration Properties | 477
hbase.hregion.memstore.block.multiplier
Block updates if memstore has
hbase.hregion.block.memstore
time
hbase.hregion.flush.size
bytes. Useful for preventing runaway memstore during
spikes in update traffic. Without an upper bound, the memstore fills such that
when it flushes, the resultant flush files take a long time to compact or split, or
worse, we OOME.
Type:
int
Default:
2
Unit:
number
hbase.hregion.memstore.flush.size
Memstore will be flushed to disk if size of the memstore exceeds this number of
bytes. Value is checked by a thread that runs every
hbase.server.thread.wakefre
quency
.
Type:
long
Default:
67108864 (1024*1024*64L)
Unit:
bytes
hbase.hregion.memstore.mslab.enabled
Enables the MemStore-Local Allocation Buffer, a feature which works to prevent
heap fragmentation under heavy write loads. This can reduce the frequency of stopthe-
world GC pauses on large heaps.
Type:
boolean
Default:
true
hbase.hregion.preclose.flush.size
If the memstores in a region are this size or larger when we go to close, run a “preflush”
to clear out memstores before we put up the region closed flag and take the
region offline. On close, a flush is run under the close flag to empty memory. During
this time the region is offline and we are not taking on any writes. If the memstore
content is large, this flush could take a long time to complete. The preflush is meant
to clean out the bulk of the memstore before putting up the close flag and taking
the region offline so the flush that runs under the close flag has little to do.
Type:
long
Default:
5242880 (1024 * 1024 * 5)
Unit:
bytes
hbase.hstore.blockingStoreFiles
If more than this number of
StoreFile
s in any one
Store
(one
StoreFile
is written
per flush of
MemStore
) then updates are blocked for this
HRegion
until a compaction
is completed, or until
hbase.hstore.blockingWaitTime
has been exceeded.
Type:
int
Default:
7, hardcoded: -1
478 | Appendix A: HBase Configuration Properties
Unit:
number
hbase.hstore.blockingWaitTime
The time an
HRegion
will block updates for after hitting the
StoreFile
limit defined
by
hbase.hstore.blockingStoreFiles
. After this time has elapsed, the
HRegion
will
stop blocking updates even if a compaction has not been completed.
Type:
int
Default:
90000
Unit:
milliseconds
hbase.hstore.compaction.max
Max number of
HStoreFiles
to compact per minor compaction.
Type:
int
Default:
10
Unit:
number
hbase.hstore.compactionThreshold
If more than this number of
HStoreFiles
in any one
HStore
(one
HStoreFile
is written
per flush of memstore) then a compaction is run to rewrite all
HStoreFiles
files
as one. Larger numbers put off compaction, but when it runs, it takes longer to
complete.
Type:
int
Default:
3, hardcoded: 2
Unit:
number
hbase.mapreduce.hfileoutputformat.blocksize
The
mapreduce HFileOutputFormat
writes store files/HFiles. This is the minimum
HFile
blocksize
to emit. Usually in HBase, when writing HFiles, the
blocksize
is
gotten from the table schema (
HColumnDescriptor
) but in the MapReduce
output
format
context, we don’t have access to the schema, so we get the
blocksize
from
the configuation. The smaller you make the
blocksize
, the bigger your index will
be and the less you will fetch on a random access. Set the
blocksize
down if you
have small cells and want faster random access of individual cells.
Type:
int
Default:
65536
Unit:
bytes
hbase.master.dns.interface
The name of the network interface from which a master should report its IP address.
Type:
string
Default: “default”
HBase Configuration Properties | 479
hbase.master.dns.nameserver
The hostname or IP address of the name server (DNS) which a master should use
to determine the hostname used for communication and display purposes.
Type:
string
Default: “default”
hbase.master.info.bindAddress
The bind address for the HBase Master web UI.
Type:
String
Default:
0.0.0.0
hbase.master.info.port
The port for the HBase Master web UI. Set to -1 if you do not want a UI instance run.
Type:
int
Default:
60010
Unit:
number
hbase.master.kerberos.principal
Example: “hbase/[email protected]”. The Kerberos principal name that
should be used to run the HMaster process. The principal name should be in the
form: user/hostname@DOMAIN. If “_HOST” is used as the hostname portion, it
will be replaced with the actual hostname of the running instance.
Type:
string
Default:
hbase.master.keytab.file
Full path to the Kerberos keytab file to use for logging in the configured HMaster
server principal.
Type:
string
Default:
hbase.master.logcleaner.plugins
A comma-separated list of
LogCleanerDelegate
s invoked by the LogsCleaner service.
These WAL/HLog cleaners are called in order, so put the HLog cleaner that
prunes the most HLog files in front. To implement your own
LogCleanerDele
gate
, just put it in HBase’s classpath and add the fully qualified class name here.
Always add the above default log cleaners in the list.
Type:
string
Default:
org.apache.hadoop.hbase.master.TimeToLiveLogCleaner
hbase.master.logcleaner.ttl
Maximum time an HLog can stay in the .oldlogdir directory, after which it will be
cleaned by a master thread.
Type:
long
480 | Appendix A: HBase Configuration Properties
Default:
600000
Unit:
milliseconds
hbase.master.port
The port the HBase Master should bind to.
Type:
int
Default:
60000
Unit:
number
hbase.regions.slop
Rebalance if any region server has average + (average * slop) regions. Default is
20% slop.
Type:
Default:
0.2
Unit:
float (percent)
hbase.regionserver.class
The RegionServer interface to use. Used by the client opening proxy to remote
region server.
Type: class name
Default:
org.apache.hadoop.hbase.ipc.HRegionInterface
hbase.regionserver.dns.interface
The name of the network interface from which a region server should report its IP
address.
Type:
string
Default: “default”
hbase.regionserver.dns.nameserver
The hostname or IP address of the name server (DNS) which a region server should
use to determine the hostname used by the master for communication and display
purposes.
Type:
string
Default: “default”
hbase.regionserver.global.memstore.lowerLimit
When memstores are being forced to flush to make room in memory, keep flushing
until we hit this mark. Defaults to 35% of heap. This value equal to
hbase.region
server.global.memstore.upperLimit
causes the minimum possible flushing to occur
when updates are blocked due to memstore limiting.
Type:
float
Default:
0.35, hardcoded: 0.25
Unit:
float (percent)
HBase Configuration Properties | 481
hbase.regionserver.global.memstore.upperLimit
Maximum size of all memstores in a region server before new updates are blocked
and flushes are forced. Defaults to 40% of heap.
Type:
float
Default:
0.4
Unit:
float (percent)
hbase.regionserver.handler.count
Count of RPC Listener instances spun up on RegionServers. The same property is
used by the master for count of master handlers.
Type:
int
Default:
10
Unit:
number
hbase.regionserver.hlog.reader.impl
The HLog file reader implementation.
Type: class name
Default:
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader
hbase.regionserver.hlog.writer.impl
The HLog file writer implementation.
Type: class name
Default:
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter
hbase.regionserver.info.bindAddress
The address for the HBase RegionServer web UI.
Type:
string
Default:
0.0.0.0
hbase.regionserver.info.port
The port for the HBase RegionServer web UI. Set to -1 if you do not want the
RegionServer UI to run.
Type:
int
Default:
60030
Unit:
number
hbase.regionserver.info.port.auto
Whether or not the Master or RegionServer UI should search for a port to bind to.
Enables automatic port search if
hbase.regionserver.info.port
is already in use.
Useful for testing; turned off by default.
Type:
boolean
Default:
false
482 | Appendix A: HBase Configuration Properties
hbase.regionserver.kerberos.principal
Example: “hbase/[email protected]”. The Kerberos principal name that
should be used to run the HRegionServer process. The principal name should be
in the form user/hostname@DOMAIN. If “_HOST” is used as the hostname portion,
it will be replaced with the actual hostname of the running instance. An entry
for this principal must exist in the file specified in
hbase.regionserver.keytab.file
.
Type:
string
Default:
<empty>
hbase.regionserver.keytab.file
Full path to the Kerberos keytab file to use for logging in the configured HRegion-
Server server principal.
Type:
string
Default:
<empty>
hbase.regionserver.lease.period
HRegion server lease period in milliseconds. Default is 60 seconds. Clients must
report in within this period else they are considered dead.
Type:
long
Default:
60000 (1 min)
Unit:
milliseconds
hbase.regionserver.logroll.period
Period at which we will roll the commit log regardless of how many edits it has.
Type:
long
Default:
3600000
Unit:
milliseconds
hbase.regionserver.msginterval
Interval between messages from the RegionServer to the HBase Master in milliseconds.
Type:
int
Default:
3000 (3 secs)
Unit:
milliseconds
hbase.regionserver.nbreservationblocks
The number of reservoir blocks of memory released on OOME so we can clean up
properly before server shutdown.
Type:
int
Default:
4
Unit:
number
HBase Configuration Properties | 483
hbase.regionserver.optionallogflushinterval
Sync the HLog to the HDFS after this interval if it has not accumulated enough
entries to trigger a sync.
Type:
long
Default:
1000 (1 sec)
Unit:
milliseconds
hbase.regionserver.port
The port the HBase RegionServer binds to.
Type:
int
Default:
60020
Unit:
number
hbase.regionserver.regionSplitLimit
Limit for the number of regions after which no more region splitting should take
place. This is not a hard limit for the number of regions, but acts as a guideline for
the RegionServer to stop splitting after a certain limit. Default is set to
MAX_INT
; that
is, do not block splitting.
Type:
int
Default:
2147483647
Unit:
number
hbase.rest.port
The port for the HBase REST server.
Type:
int
Default:
8080, hardcoded: 9090
Unit:
number
hbase.rest.readonly
Defines the mode the REST server will be started in. Possible values are
false
,
which means all HTTP methods are permitted (
GET
,
PUT
,
POST
, and
DELETE
); and
true
, which means only the
GET
method is permitted.
Type:
boolean
Default:
false
hbase.rootdir
The directory shared by region servers and into which HBase persists. The URL
should be fully qualified to include the filesystem scheme. For example, to specify
the HDFS directory /hbase where the HDFS instance’s namenode is running at
namenode.example.org
on port 9000, set this value to
hdfs://namenode.exam
ple.org:9000/hbase
. By default, HBase writes into /tmp. Change this configuration
else all data will be lost on machine restart.
Type:
string
484 | Appendix A: HBase Configuration Properties
Default:
file:///tmp/hbase-${user.name}/hbase
hbase.rpc.engine
Implementation of
org.apache.hadoop.hbase.ipc.RpcEngine
to be used for client/
server RPC call marshaling.
Type: class name
Default:
org.apache.hadoop.hbase.ipc.WritableRpcEngine
hbase.server.thread.wakefrequency
Time to sleep in between searches for work (in milliseconds). Used as sleep interval
by service threads such as log roller.
Type:
int
Default:
10000 (10 secs)
Unit:
milliseconds
hbase.tmp.dir
Temporary directory on the local filesystem. Change this setting to point to a location
more permanent than /tmp (the /tmp directory is often cleared on machine
restart).
Type:
string
Default:
/tmp/hbase-${user.name}
hbase.zookeeper.dns.interface
The name of the network interface from which a ZooKeeper server should report
its IP address.
Type:
string
Default: “default”
hbase.zookeeper.dns.nameserver
The hostname or IP address of the name server (DNS) which a ZooKeeper server
should use to determine the hostname used by the master for communication and
display purposes.
Type:
string
Default: “default”
hbase.zookeeper.leaderport
Port used by ZooKeeper for leader election. See http://hadoop.apache.org/zookeeper/
docs/r3.1.1/zookeeperStarted.html#sc_RunningReplicatedZooKeeper for
more information.
Type:
int
Default:
3888
Unit:
number
HBase Configuration Properties | 485
hbase.zookeeper.peerport
Port used by ZooKeeper peers to talk to each other. See http://hadoop.apache.org/
zookeeper/docs/r3.1.1/zookeeperStarted.html#sc_RunningReplicatedZooKeeper
for more information.
Type:
int
Default:
2888
Unit:
number
hbase.zookeeper.property.clientPort
Property from ZooKeeper’s zoo.cfg configuration file. The port at which the clients
will connect.
Type:
int
Default:
2181
Unit:
number
hbase.zookeeper.property.dataDir
Property from ZooKeeper’s zoo.cfg configuration file. The directory where the
snapshot is stored.
Type:
string
Default:
${hbase.tmp.dir}/zookeeper
hbase.zookeeper.property.initLimit
Property from ZooKeeper’s zoo.cfg configuration file. The number of ticks that the
initial synchronization phase can take.
Type:
int
Default:
10
Unit:
number
hbase.zookeeper.property.maxClientCnxns
Property from ZooKeeper’s zoo.cfg configuration file. Limit on number of concurrent
connections (at the socket level) that a single client, identified by IP address,
may make to a single member of the ZooKeeper ensemble. Set high to avoid Zoo-
Keeper connection issues running standalone and pseudodistributed.
Type:
int
Default:
30
Unit:
number
hbase.zookeeper.property.syncLimit
Property from ZooKeeper’s zoo.cfg configuration file. The number of ticks that can
pass between sending a request and getting an acknowledgment.
Type:
int
Default:
5
Unit:
number
486 | Appendix A: HBase Configuration Properties
hbase.zookeeper.quorum
Comma-separated list of servers in the ZooKeeper Quorum. For example, by default,
“host1.mydomain.com,host2.mydomain.com,host3.mydomain.com” is set
to localhost for local and pseudodistributed modes of operation. For a fully distributed
setup, this should be set to a full list of ZooKeeper quorum servers. If
HBASE_MANAGES_ZK
is set in hbase-env.sh, this is the list of servers on which we will
start/stop ZooKeeper.
Type:
string
Default:
localhost
hfile.block.cache.size
Percentage of maximum heap (
-Xmx
setting) to allocate to block cache used by
HFile
/
StoreFile
. Default of 0.2 means allocate 20%. Set to 0 to disable.
Type:
float
Default:
0.2
Unit:
float (percent)
zookeeper.session.timeout
ZooKeeper session timeout. HBase passes this to the ZooKeeper quorum as the
suggested maximum time for a session (this setting becomes ZooKeeper’s
maxSes
sionTimeout
). See
http://hadoop.apache.org/zookeeper/docs/current/zookeeperProg
rammers.html#ch_zkSessions
. “The client sends a requested timeout, the server
responds with the timeout that it can give the client.”
Type:
int
Default:
180000
Unit:
milliseconds
zookeeper.znode.parent
Root znode for HBase in ZooKeeper. All of HBase’s ZooKeeper files that are configured
with a relative path will go under this node. By default, all of HBase’s
ZooKeeper file paths are configured with a relative path, so they will all go under
this directory unless changed.
Type:
string
Default:
/hbase
zookeeper.znode.rootserver
Path to znode holding root region location. This is written by the master and read
by clients and region servers. If a relative path is given, the parent folder will be $
{zookeeper.znode.parent}. By default, this means the root location is stored at /
hbase/root-region-server.
Type:
string
Default:
root-region-server