Impala的SYNC_DDL

那天去面试，面试官问我知不知道Impala同步数据的方式，我回答invalidate metadata和refresh table [partition]，后来他问我还有没有其他的方式，当时我没想到别的，因为工作中用到impala都是和hive结合的，spark把数据落到hive中，然后刷新impala，impala提供查询。

后来他提了下问我是否知道Impala的SYNC_DDL。说实话这个我也是第一次听过，后来去官网查了下用法，发现这个impala自己运行DDL时候同步用的，挺好的，又长知识了。

这个命令是为了解决这个问题的。在某个节点的impala-shell运行DDL语句，可能会出现其他节点查看不到的情况，因为impala的所有元数据都是用catalogd来管理的。一个impalad进行DDL操作会发送到catalogd，由catalogd在广播给其他的impalad服务，这时候可能会出现延迟，导致有的impalad查询不到发生的DDL。SYNC_DDL就是解决这个问题的。

官网的描述如下

When enabled, causes any DDL operation such as CREATE TABLE or ALTER TABLE to return only when the changes have been propagated to all other Impala nodes in the cluster by the Impala catalog service. That way, if you issue a subsequent CONNECT statement in impala-shell to connect to a different node in the cluster, you can be sure that other node will already recognize any added or changed tables. (The catalog service automatically broadcasts the DDL changes to all nodes automatically, but without this option there could be a period of inconsistency if you quickly switched to another node, such as by issuing a subsequent query through a load-balancing proxy.)
Although INSERT is classified as a DML statement, when the SYNC_DDL option is enabled, INSERT statements also delay their completion until all the underlying data and metadata changes are propagated to all Impala nodes. Internally, Impala inserts have similarities with DDL statements in traditional database systems, because they create metadata needed to track HDFS block locations for new files and they potentially add new partitions to partitioned tables.
Note: Because this option can introduce a delay after each write operation, if you are running a sequence of CREATE DATABASE, CREATE TABLE, ALTER TABLE, INSERT, and similar statements within a setup script, to minimize the overall delay you can enable the SYNC_DDL query option only near the end, before the final DDL statement.
Type: Boolean; recognized values are 1 and 0, or true and false; any other value interpreted as false
Default: false (shown as 0 in output of SET statement)