Phoenix（十一）二级索引之— —性能优化篇

1. 说明

在使用phoenix二级索引的时候可以进行一些参数的修改来优化性能，这个没有经过实际使用，但是在这里也记录一下以供有需要的人参考，内容来自官方文档。

http://phoenix.apache.org/secondary_indexing.html

2. 优化

All the following parameters must be set in hbase-site.xml - they are true for the entire cluster and all index tables, as well as across all regions on the same server (so, for instance, a single server would not write to too many different index tables at once).

1. index.builder.threads.max
   
   • Number of threads to used to build the index update from the primary table update
   • Increasing this value overcomes the bottleneck of reading the current row state from the underlying HRegion. Tuning this value too high will just bottleneck at the HRegion as it will not be able to handle too many concurrent scan requests as well as general thread-swapping concerns.
   • Default: 10
2. index.builder.threads.keepalivetime
   
   • Amount of time in seconds after we expire threads in the builder thread pool.
   • Unused threads are immediately released after this amount of time and not core threads are retained (though this last is a small concern as tables are expected to sustain a fairly constant write load), but simultaneously allows us to drop threads if we are not seeing the expected load.
   • Default: 60
3. index.writer.threads.max
   
   • Number of threads to use when writing to the target index tables.
   • The first level of parallelization, on a per-table basis - it should roughly correspond to the number of index tables
   • Default: 10
4. index.writer.threads.keepalivetime
   
   • Amount of time in seconds after we expire threads in the writer thread pool.
   • Unused threads are immediately released after this amount of time and not core threads are retained (though this last is a small concern as tables are expected to sustain a fairly constant write load), but simultaneously allows us to drop threads if we are not seeing the expected load.
   • Default: 60
5. hbase.htable.threads.max
   
   • Number of threads each index HTable can use for writes.
   • Increasing this allows more concurrent index updates (for instance across batches), leading to high overall throughput.
   • Default: 2,147,483,647
6. hbase.htable.threads.keepalivetime
   
   • Amount of time in seconds after we expire threads in the HTable’s thread pool.
   • Using the “direct handoff” approach, new threads will only be created if it is necessary and will grow unbounded. This could be bad but HTables only create as many Runnables as there are region servers; therefore, it also scales when new region servers are added.
   • Default: 60
7. index.tablefactory.cache.size
   
   • Number of index HTables we should keep in cache.
   • Increasing this number ensures that we do not need to recreate an HTable for each attempt to write to an index table. Conversely, you could see memory pressure if this value is set too high.
   • Default: 10
8. org.apache.phoenix.regionserver.index.priority.min
   
   • Value to specify to bottom (inclusive) of the range in which index priority may lie.
   • Default: 1000
9. org.apache.phoenix.regionserver.index.priority.max
   
   • Value to specify to top (exclusive) of the range in which index priority may lie.
   • Higher priorites within the index min/max range do not means updates are processed sooner.
   • Default: 1050
10. org.apache.phoenix.regionserver.index.handler.count
    
    • Number of threads to use when serving index write requests for global index maintenance.
    • Though the actual number of threads is dictated by the Max(number of call queues, handler count), where the number of call queues is determined by standard HBase configuration. To further tune the queues, you can adjust the standard rpc queue length parameters (currently, there are no special knobs for the index queues), specifically ipc.server.max.callqueue.length and ipc.server.callqueue.handler.factor. See the HBase Reference Guide for more details.
    • Default: 30