HBase Data Model——HBase 数据模型（翻译）
在HBase中，数据是存储在有行有列的表格中。这是与关系型数据库重复的术语，并不是有用的类比。相反，HBase可以被认为是一个多维度的映射。
HBase数据模型术语
Table（表格）
一个HBase表格由多行组成。
Row（行）
HBase中的行里面包含一个key和一个或者多个包含值的列。行按照行的key字母顺序存储在表格中。因为这个原因，行的key的设计就显得非常重要。数据的存储目标是相近的数据存储到一起。一个常用的行的key的格式是网站域名。如果你的行的key是域名，你应该将域名进行反转(org.apache.www, org.apache.mail, org.apache.jira)再存储。这样的话，所有Apache域名将会存储在一起，好过基于子域名的首字母分散在各处。
Column（列）
HBase中的列包含用：分隔开的列族和列的限定符。
Column Family（列族）
因为性能的原因，列族物理上包含一组列和它们的值。每一个列族拥有一系列的存储属性，例如值是否缓存在内存中，数据是否要压缩或者他的行key是否要加密等等。表格中的每一行拥有相同的列族，尽管一个给定的行可能没有存储任何数据在一个给定的列族中。
Column Qualifier（列的限定符）
列的限定符是列族中数据的索引。例如给定了一个列族content，那么限定符可能是content:html，也可以是content:pdf。列族在创建表格时是确定的了，但是列的限定符是动态地并且行与行之间的差别也可能是非常大的。
Cell（单元）
单元是由行、列族、列限定符、值和代表值版本的时间戳组成的。
Timestamp（时间戳）
时间戳是写在值旁边的一个用于区分值的版本的数据。默认情况下，时间戳表示的是当数据写入时RegionSever的时间点，但你也可以在写入数据时指定一个不同的时间戳。
19.概念视图
你可以读一下 Jim R写的Understanding HBase and BigTable 博客来简单了解一下HBase的数据模型，另一个好的解释是Amandeep Khurana.的 Introduction to Basic Schema Design 。
学习不同的方面的资料可能会帮助你更透彻地了解HBase的设计。所链接的文章覆盖本部分所讲的信息。
接下来的例子是 取自BigTable 中第二页中的例子，在此基础上做了些许的改变。一个名为webable的表格，表格中有两行（com.cnn.www 和 com.example.www）和三个列族（contents, anchor, 和 people）。在这个例子当中，第一行(com.cnn.www)中anchor包含两列（anchor:cssnsi.com, anchor:my.look.ca）和content包含一列（contents:html）。这个例子中com.cnn.www拥有5个版本而com.example.www有一个版本。contents:html列中包含给定网页的整个HTML。anchor限定符包含能够表示行的站点以及链接中文本。People列族表示跟站点有关的人。
| Table 4. Table webtable |
|||||
| Row Key |
Time Stamp |
ColumnFamily contents |
ColumnFamily anchor |
ColumnFamily people |
|
| 列名 按照所定义好的，一个列名的格式为列族名前缀加限定符。例如，列contents:html由列族contents和html限定符。冒号（:）用于将列族和列限定符分开。 |
|||||
| "com.cnn.www" |
t9 |
anchor:cnnsi.com = "CNN" |
|||
| "com.cnn.www" |
t8 |
anchor:my.look.ca = "CNN.com" |
|||
| "com.cnn.www" |
t6 |
contents:html = "…" |
|||
| "com.cnn.www" |
t5 |
contents:html = "…" |
|||
| "com.cnn.www" |
t3 |
contents:html = "…" |
|||
| com.example.www |
t5 |
contents:html: "..." |
|
people:author: "John Doe" |
|
在HBase中，表格中的单元如果是空将不占用空间或者事实上不存在。这就使得HBase看起来“稀疏”。表格视图不是唯一方式来查看HBase中数据，甚至不是最精确的。下面的方式以多维度映射的方式来表达相同的信息。下面只是一个用于说明目的的模型可能不是百分百的精确。
{
"com.cnn.www": {
contents: {
t6: contents:html: "..."
t5: contents:html: "..."
t3: contents:html: "..."
}
anchor: {
t9: anchor:cnnsi.com = "CNN"
t8: anchor:my.look.ca = "CNN.com"
}
people: {}
}
"com.example.www": {
contents: {
t5: contents:html: "..."
}
anchor: {}
people: {
t5: people:author: "John Doe"
}
}
}
20. 物理视图
尽管一个概念层次的表格可能看起来是由一些列稀疏的行组成，但他们是通过列族来存储的。一个新建的限定符(column_family:column_qualifier)可以随时地添加到已存在的列族中。
| Table 5. ColumnFamily anchor |
||
| Row Key |
Time Stamp |
Column Family anchor |
| "com.cnn.www" |
t9 |
anchor:cnnsi.com = "CNN" |
| "com.cnn.www" |
t8 |
anchor:my.look.ca = "CNN.com" |
| Table 6. ColumnFamily contents |
||
| Row Key |
Time Stamp |
ColumnFamily contents: |
| "com.cnn.www" |
t6 |
contents:html = "…" |
| "com.cnn.www" |
t5 |
contents:html = "…" |
| "com.cnn.www" |
t3 |
contents:html = "…" |
概念视图中的空单元实际上是没有进行存储的。因此对于返回时间戳为t8的contents:html的值的请求，结果为空。同样的，一个返回时间戳为t9的anchor:my.look.ca的值的请求，结果也为空。然而，如果没有指定时间戳的话，那么会返回特定列的最新值。对有多个版本的列，优先返回最新的值，因为时间戳是按照递减顺序存储的。因此对于一个返回com.cnn.www里面所有的列的值并且没有指定时间戳的请求，返回的结果会是时间戳为t6的contents:html 的值、时间戳 t9的anchor:cnnsi.com f的值和时间戳t8的 anchor:my.look.ca 。
关于Apache Hbase如何存储数据的内部细节，请查看 regions.arch.
21. 命名空间
命名空间是一个类似于关系型数据库系统中的数据库的逻辑上的表分组的概念。这个抽象的概念为即将到来的多租户相关特性奠定了基础：
-
- Quota Management (HBASE-8410) - Restrict the amount of resources (i.e. regions, tables) a namespace can consume.
- Namespace Security Administration (HBASE-9206) - Provide another level of security administration for tenants.
- Region server groups (HBASE-6721) - A namespace/table can be pinned onto a subset of RegionServers thus guaranteeing a course level of isolation.
21.1. 命名空间管理
命名空间可以被创建、移除和修改。命名空间关系的指定是在创建表格通过指定一个完全限定表名的形式完成的：
| This behavior represents a fix for an unexpected change that was introduced in HBase 0.94, and was fixed in HBASE-10118. The change has been backported to HBase 0.94 and newer branches. |
27.3. 当前的局限性
27.3.1. Deletes mask Puts删除覆盖插入/更新
删除操作覆盖插入/更新操作，即使put在delete之后执行的。可以查看 HBASE-2256. 还记得一个删除写入一个墓碑，只有当下一次精简操作发生时才会执行真正地删除操作。假设你执行了一个删除全部小于等于T的操作。在此之外又做了一个时间戳为T的put操作。这个put操作即使是发生在delete之后，也会被delete墓碑所覆盖。执行put的时候不会报错，不过当你执行一个get的时候会发现执行无效。你会在精简操作之后重新开始工作。如果你在put的使用的递增的版本，那么这些问题将不会出现。但如果你不在意时间，在执行delelte后立刻执行put的话，那么它们将有可能发生在同一时间点，这将会导致上述问题的出现。
27.3.2. 精简操作影响查询结果
创建三个版本为t1,t2,t3的单元，并且设置最大版本数为2.所以当我们查询所有版本时，只会返回t2和t3。但是当你删除版本t2和t3的时候，版本t1会重新出现。显然，一旦重要精简工作运行之后，这样的行为就不会再出现。（查看 Bending time in HBase.）
28. 排序次序
HBase中所有的数据模型操作返回的数据都是经过排序的。首先是行排序，其次是列族，接着是列限定符，最后是时间戳（递减排序，左右最新的记录最先返回）
29. 列元数据
所有列的元数据都存储在一个列族的内部KeyValue实例中。因此，HBsase不仅支持一行中有多列，而且支持行之间的列的差异多样化。跟踪列名是你的责任。
唯一获取一个列族的所有列的方法是处理所有的行。查看 keyvalue获得更多关于HBase内部如何存储数据的信息。
30. Joins
HBase是否支持join是一个常见的问题，答案是没有，至少没办法像RDBMS那样支持（例如等价式join或者外部join）。正如本章节所阐述的，HBase中读取数据的操作是Get和Scan。
然而，这不意味着等价式join功能没办法在你的应用中实现，但是你必须自己实现。两种主要策略是将数据非结构化地写到HBase中，或者查找表格然后在应用中或者MapReduce代码中实现join操作（正如RDBMS所演示的，将根据表格的大小会有几种不同的策略，例如嵌套使循环和hash-join）。哪个是最好的方法？这将依赖于你想做什么，没有一种方案能够应对各种情况。
31. ACID
查看 ACID Semantics. Lars Hofhansl也写了一份报告 ACID in HBase.
下面是原文
Data Model
In HBase, data is stored in tables, which have rows and columns. This is a terminology overlap with relational databases (RDBMSs), but this is not a helpful analogy. Instead, it can be helpful to think of an HBase table as a multi-dimensional map.
HBase Data Model Terminology
Table
An HBase table consists of multiple rows.
Row
A row in HBase consists of a row key and one or more columns with values associated with them. Rows are sorted alphabetically by the row key as they are stored. For this reason, the design of the row key is very important. The goal is to store data in such a way that related rows are near each other. A common row key pattern is a website domain. If your row keys are domains, you should probably store them in reverse (org.apache.www, org.apache.mail, org.apache.jira). This way, all of the Apache domains are near each other in the table, rather than being spread out based on the first letter of the subdomain.
Column
A column in HBase consists of a column family and a column qualifier, which are delimited by a : (colon) character.
Column Family
Column families physically colocate a set of columns and their values, often for performance reasons. Each column family has a set of storage properties, such as whether its values should be cached in memory, how its data is compressed or its row keys are encoded, and others. Each row in a table has the same column families, though a given row might not store anything in a given column family.
Column Qualifier
A column qualifier is added to a column family to provide the index for a given piece of data. Given a column family content, a column qualifier might be content:html, and another might be content:pdf. Though column families are fixed at table creation, column qualifiers are mutable and may differ greatly between rows.
Cell
A cell is a combination of row, column family, and column qualifier, and contains a value and a timestamp, which represents the value’s version.
Timestamp
A timestamp is written alongside each value, and is the identifier for a given version of a value. By default, the timestamp represents the time on the RegionServer when the data was written, but you can specify a different timestamp value when you put data into the cell.
19. Conceptual View
You can read a very understandable explanation of the HBase data model in the blog post Understanding HBase and BigTable by Jim R. Wilson. Another good explanation is available in the PDF Introduction to Basic Schema Design by Amandeep Khurana.
It may help to read different perspectives to get a solid understanding of HBase schema design. The linked articles cover the same ground as the information in this section.
The following example is a slightly modified form of the one on page 2 of the BigTable paper. There is a table called webtable that contains two rows (com.cnn.www and com.example.www) and three column families named contents, anchor, and people. In this example, for the first row (com.cnn.www), anchor contains two columns (anchor:cssnsi.com, anchor:my.look.ca) and contents contains one column (contents:html). This example contains 5 versions of the row with the row key com.cnn.www, and one version of the row with the row key com.example.www. The contents:html column qualifier contains the entire HTML of a given website. Qualifiers of the anchor column family each contain the external site which links to the site represented by the row, along with the text it used in the anchor of its link. The people column family represents people associated with the site.
| Table 4. Table webtable |
|||||
| Row Key |
Time Stamp |
ColumnFamily contents |
ColumnFamily anchor |
ColumnFamily people |
|
| Column Names By convention, a column name is made of its column family prefix and a qualifier. For example, the column contents:html is made up of the column family contents and the html qualifier. The colon character (:) delimits the column family from the column family qualifier. |
|||||
| "com.cnn.www" |
t9 |
anchor:cnnsi.com = "CNN" |
|||
| "com.cnn.www" |
t8 |
anchor:my.look.ca = "CNN.com" |
|||
| "com.cnn.www" |
t6 |
contents:html = "…" |
|||
| "com.cnn.www" |
t5 |
contents:html = "…" |
|||
| "com.cnn.www" |
t3 |
contents:html = "…" |
|||
Cells in this table that appear to be empty do not take space, or in fact exist, in HBase. This is what makes HBase "sparse." A tabular view is not the only possible way to look at data in HBase, or even the most accurate. The following represents the same information as a multi-dimensional map. This is only a mock-up for illustrative purposes and may not be strictly accurate.
{
"com.cnn.www": {
contents: {
t6: contents:html: "..."
t5: contents:html: "..."
t3: contents:html: "..."
}
anchor: {
t9: anchor:cnnsi.com = "CNN"
t8: anchor:my.look.ca = "CNN.com"
}
people: {}
}
"com.example.www": {
contents: {
t5: contents:html: "..."
}
anchor: {}
people: {
t5: people:author: "John Doe"
}
}
}
20. Physical View
Although at a conceptual level tables may be viewed as a sparse set of rows, they are physically stored by column family. A new column qualifier (column_family:column_qualifier) can be added to an existing column family at any time.
| Table 5. ColumnFamily anchor |
||
| Row Key |
Time Stamp |
Column Family anchor |
| "com.cnn.www" |
t9 |
anchor:cnnsi.com = "CNN" |
| "com.cnn.www" |
t8 |
anchor:my.look.ca = "CNN.com" |
| Table 6. ColumnFamily contents |
||
| Row Key |
Time Stamp |
ColumnFamily contents: |
| "com.cnn.www" |
t6 |
contents:html = "…" |
| "com.cnn.www" |
t5 |
contents:html = "…" |
| "com.cnn.www" |
t3 |
contents:html = "…" |
The empty cells shown in the conceptual view are not stored at all. Thus a request for the value of the contents:html column at time stamp t8 would return no value. Similarly, a request for an anchor:my.look.cavalue at time stamp t9 would return no value. However, if no timestamp is supplied, the most recent value for a particular column would be returned. Given multiple versions, the most recent is also the first one found, since timestamps are stored in descending order. Thus a request for the values of all columns in the row com.cnn.www if no timestamp is specified would be: the value of contents:html from timestamp t6, the value of anchor:cnnsi.com from timestamp t9, the value of anchor:my.look.ca from timestamp t8.
For more information about the internals of how Apache HBase stores data, see regions.arch.
21. Namespace
A namespace is a logical grouping of tables analogous to a database in relation database systems. This abstraction lays the groundwork for upcoming multi-tenancy related features:
-
- Quota Management (HBASE-8410) - Restrict the amount of resources (i.e. regions, tables) a namespace can consume.
- Namespace Security Administration (HBASE-9206) - Provide another level of security administration for tenants.
- Region server groups (HBASE-6721) - A namespace/table can be pinned onto a subset of RegionServers thus guaranteeing a course level of isolation.
21.1. Namespace management
A namespace can be created, removed or altered. Namespace membership is determined during table creation by specifying a fully-qualified table name of the form:
| This behavior represents a fix for an unexpected change that was introduced in HBase 0.94, and was fixed in HBASE-10118. The change has been backported to HBase 0.94 and newer branches. |
27.3. Current Limitations
27.3.1. Deletes mask Puts
Deletes mask puts, even puts that happened after the delete was entered. See HBASE-2256. Remember that a delete writes a tombstone, which only disappears after then next major compaction has run. Suppose you do a delete of everything ⇐ T. After this you do a new put with a timestamp ⇐ T. This put, even if it happened after the delete, will be masked by the delete tombstone. Performing the put will not fail, but when you do a get you will notice the put did have no effect. It will start working again after the major compaction has run. These issues should not be a problem if you use always-increasing versions for new puts to a row. But they can occur even if you do not care about time: just do delete and put immediately after each other, and there is some chance they happen within the same millisecond.
27.3.2. Major compactions change query results
…create three cell versions at t1, t2 and t3, with a maximum-versions setting of 2. So when getting all versions, only the values at t2 and t3 will be returned. But if you delete the version at t2 or t3, the one at t1 will appear again. Obviously, once a major compaction has run, such behavior will not be the case anymore… (See Garbage Collection in Bending time in HBase.)
28. Sort Order
All data model operations HBase return data in sorted order. First by row, then by ColumnFamily, followed by column qualifier, and finally timestamp (sorted in reverse, so newest records are returned first).
29. Column Metadata
There is no store of column metadata outside of the internal KeyValue instances for a ColumnFamily. Thus, while HBase can support not only a wide number of columns per row, but a heterogeneous set of columns between rows as well, it is your responsibility to keep track of the column names.
The only way to get a complete set of columns that exist for a ColumnFamily is to process all the rows. For more information about how HBase stores data internally, see keyvalue.
30. Joins
Whether HBase supports joins is a common question on the dist-list, and there is a simple answer: it doesn’t, at not least in the way that RDBMS' support them (e.g., with equi-joins or outer-joins in SQL). As has been illustrated in this chapter, the read data model operations in HBase are Get and Scan.
However, that doesn’t mean that equivalent join functionality can’t be supported in your application, but you have to do it yourself. The two primary strategies are either denormalizing the data upon writing to HBase, or to have lookup tables and do the join between HBase tables in your application or MapReduce code (and as RDBMS' demonstrate, there are several strategies for this depending on the size of the tables, e.g., nested loops vs. hash-joins). So which is the best approach? It depends on what you are trying to do, and as such there isn’t a single answer that works for every use case.
31. ACID
See ACID Semantics. Lars Hofhansl has also written a note on ACID in HBase.
转载于:https://www.cnblogs.com/zhaohongtian/p/6810270.html