DataX数据交换,starrockswriter异常解决

使用datax往starrocks里灌数据时一直报错

Caused by: java.lang.NullPointerException
		at com.starrocks.connector.datax.plugin.writer.starrockswriter.manager.StarRocksStreamLoadVisitor.joinRows(StarRocksStreamLoadVisitor.java:111)
		at com.starrocks.connector.datax.plugin.writer.starrockswriter.manager.StarRocksStreamLoadVisitor.doStreamLoad(StarRocksStreamLoadVisitor.java:65)
		at com.starrocks.connector.datax.plugin.writer.starrockswriter.manager.StarRocksWriterManager.asyncFlush(StarRocksWriterManager.java:161)

定位Datax源码com.starrocks.connector.datax.plugin.writer.starrockswriter.manager.StarRocksStreamLoadVisitor
DataX数据交换,starrockswriter异常解决_第1张图片
需要配置loadProps参数。
这里的坑是,我查看官方文档时,他说的是非必填
DataX数据交换,starrockswriter异常解决_第2张图片
现在源码里看到这里需要该参数,我又看文档,下面居然还有这个注意事项

注意事项

导入参数配置

默认传入的数据均会被转为字符串,并以\t作为列分隔符,\n作为行分隔符,组成csv文件进行StreamLoad导入操作。 如需更改列分隔符,则正确配置 loadProps 即可:

"loadProps": {
    "column_separator": "\\x01",
    "row_delimiter": "\\x02"
}

如需更改导入格式为json,则正确配置 loadProps 即可:

"loadProps": {
    "format": "json",
    "strip_outer_array": true
}

因此这里的解决方法就是增加loadProps的参数;完整writer参数为:

    "writer": {
        "name": "starrockswriter", 
        "parameter": {
		"username": "root",
		"password": "****",
		"database": "database",
		"table": "table_name",
		"preSql":[
			],
        "loadProps": {"column_separator": "\\x01","row_delimiter": "\\x02"},
            "column": ["x","x","x"],
			"jdbcUrl": "jdbc:mysql://10.x.x.x:9030",
			"loadUrl": ["ip:8030","ip:8030","ip:8030"]
        }
    }

你可能感兴趣的:(异常解决,big,data)