Spark将数据写入ElastichSearch

第一步：导包

       org.elasticsearch
       elasticsearch
       6.3.2
 

	   org.elasticsearch
	   elasticsearch-hadoop
	   6.3.2


	   org.elasticsearch.client
	   transport
	   6.3.2

第二步：spark代码中设置ES的相关参数

val conf = new SparkConfig(appName)
      .local()
      .extend(new SparkEsExt(esInfo()))
      .get()

public static Map esInfo() {
        String hostname = "es.hostname";
        String password = "es.password";
        String username = "es.username";
        String port = "es.port";

        return new HashMap(4) {{
            put(EsInfoKey.NODES, hostname);
            put(EsInfoKey.PORT, port);
            put(EsInfoKey.USER, username);
            put(EsInfoKey.PASSWORD, password);
        }};
    }

第三步：调用写入ES的api—saveToEs（）

import org.elasticsearch.spark._
rddSource.saveToEs("info", Map("es.mapping.id" -> "id"))