HIVE函数使用案例之----json字符串解析

简单json的解析

一个map情况: {“bssid”:“6C:59:40:21:05:C4”,“ssid”:“MERCURY_05C4”}

select get_json_object('{"bssid":"6C:59:40:21:05:C4","ssid":"MERCURY_05C4"}','$.bssid') as bssid from dual;

map嵌套的情况：{“person”:“tom”,“food”:{“fruit”:“apple”,“meat”:“pig”}}

select get_json_object('{"person":"tom","food":{"fruit":"apple","meat":"pig"}}','$.person') as person,
get_json_object('{"person":"tom","food":{"fruit":"apple","meat":"pig"}}','$.food.fruit') as fruit
from dual ;

通过上面两个例子发现，对于类似这样简单json的解析，不管字段是在第一层map(第一个例子中的bssid)还是在嵌套的map(第二个例子中的fruit)里面，解析起来都挺简单

json 中包含数组

一层数组

select id , get_json_object(users_json,'$.erp') as erp
from fdm.fdm_jdr_sch_db4components_jagile_demand__new_chain  
LATERAL VIEW explode(split(regexp_replace(regexp_replace(task_users , '\\[|\\]',''),'\\}\\,\\{','\\}\\;\\{'),'\\;')) task_users as users_json
where dp='ACTIVE' and id = 578416

第一步 去掉[]

regexp_replace(task_users , '\\[|\\]','')

第二步: },{ 替换成 };{

regexp_replace("XXX",'\\}\\,\\{','\\}\\;\\{')

第三步: 拆分

split("XXX",'\\;')

json 数组嵌套

WITH t1 AS (SELECT
			get_json_object(t.value, '$.base.code') AS code,
			get_json_object(t.value, '$.base.name') AS name,
			get_json_object(t.value, '$.list') AS list
		FROM
			(
				SELECT
					'{"base":{"code":"xm","name":"project"},"list":[{"ACode":"cp1","AName":"Product1","BList":[{"BCode":"gn1","BName":"Feature1"},{"BCode":"gn2","BName":"Feature2"}]},{"ACode":"cp2","AName":"Product2","BList":[{"BCode":"gn1","BName":"Feature1"}]}]}' AS value
			) t )
		
SELECT
	code,
	name,
	ai.ACode,
	ai.AName,
	bi.BCode,
	bi.BName
FROM t1 
    lateral VIEW explode(split(regexp_replace(regexp_extract(list, '^\\[(.+)\\]$', 1), '\\}\\]\\}\\,\\{', '\\}\\]\\}\\|\\|\\{'), '\\|\\|')) list AS a lateral VIEW json_tuple(a, 'ACode', 'AName', 'BList') ai AS ACode,
	AName,
	BList lateral VIEW explode(split(regexp_replace(regexp_extract(BList, '^\\[(.+)\\]$', 1), '\\}\\,\\{', '\\}\\|\\|\\{'), '\\|\\|')) BList AS b lateral VIEW json_tuple(b, 'BCode', 'BName') bi AS BCode,
	BName ;

说明：因为原数据是string（并不是真正的数组类型）类型的，所以无法直接使用explode函数。
1.regexp_extract(‘xxx’,‘^\[(.+)\]$’,1) 这里是把需要解析的json数组去除左右中括号，需要注意的是这里的中括号需要两个转义字符\[。
2.使用split函数拆分成数组，分隔符为’\}\,\{'，其实就是},{。
3.lateral view explode处理2中返回的数组。
4.因为使用},{为分隔符，导致了拆分出来的数据可能头部少了{，或者尾部少了}，需要填充中括号。