SQL : hive sql 一些小语句记录[删除连续分区 drop partition]

---

想要除个别字段外的剩余所有字段

这是HIVE中查询语句的一个小技巧，一个表有很多字段，我们想要除个别字段外的剩余所有字段，全部列出来不方便且不美观，实际上hive语句可以解决这个问题。

选择tableName表中除了name、id、pwd之外的所有字段：

set hive.support.quoted.identifiers=None;
select `(name|id|pwd)?+.+` from tableName;

---

 

Requirement

Suppose we are having a hive partition table. This table is partitioned by the year of joining. Our requirement is to drop multiple partitions in hive.

 

Components Involved

• Hive
• HDFS

Sample Data

Let’s say we are having given sample data:

Here, 1 record belongs to 1 partition as we will store data partitioned by the year of joining. In actual, there will be many records for each partition.

Solution

Step 1:  Create Table & Load data

If you already have a partitioned table, then skip this step else read this post for creating a table and loading data into it.

Step 2: Drop Multiple Partitions

If you see sample data, we are having 10 partitions of the year from 2005 to 2014.  Let’s check the partitions in the table:

In case, you want to add multiple partitions in the table, then mention all the partitions in the query like given below:

ALTER TABLE employee ADD partition (YEAR=2005) partition (YEAR=2006) partition (YEAR=2007) partition (YEAR=2008) partition (YEAR=2009) partition (YEAR=2010) partition (YEAR=2011) partition (YEAR=2014);

 Here, all the given partitions will get added to the table in a single query.

CASE I: Drop Specific Partitions

We will use this step’s command if we want to drop some specific partitions from the table. Here, we are going to drop partition 2008, 2009 and 2010 only.

ALTER TABLE db_bdpbase.Employee DROP IF EXISTS PARTITION (YEAR=2008), PARTITION(YEAR=2009), PARTITION(YEAR=2010);

Here, I have mentioned all the specific partitions separated by a comma in the query. It will drop all mentioned partitions in a single query.

CASE II: Drop Range Partition

Here, We want to drop all partition above the value of 2010. That means we have to drop the partition from the value 2011 to 2014.

First, check all available partitions in the table

ALTER TABLE db_bdpbase.Employee DROP IF EXISTS PARTITION(year>2010);

It will drop all partitions from 2011 to 2014.

drop the range partitions :

ALTER TABLE db_bdpbase.Employee DROP IF EXISTS PARTITION(year>2010,year<2014);

Wrapping Up

In this post, we have seen how we can add multiple partitions as well as drop multiple partitions from the hive table. We can drop multiple specific partitions as well as any range kind of partition.

Sharing is caring!