DataWorks-ODPS数据导出的几种方式

数据量10000行以下

• DataStudio页面下载(对应工作空间需开启下载设置)

  
  

• java-SDK SQLTask接口

import java.util.List;
import com.aliyun.odps.Instance;
import com.aliyun.odps.Odps;
import com.aliyun.odps.OdpsException;
import com.aliyun.odps.account.Account;
import com.aliyun.odps.account.AliyunAccount;
import com.aliyun.odps.data.Record;
import com.aliyun.odps.task.SQLTask;
public class testSql {
    private static final String accessId = "";
    private static final String accessKey = "";       
    private static final String endPoint = "http://service.odps.aliyun.com/api";
    private static final String project = "";
    private static final String sql = "select category from iris;";
    public static void
        main(String[] args) {
        Account account = new AliyunAccount(accessId, accessKey);
        Odps odps = new Odps(account);
        odps.setEndpoint(endPoint);
        odps.setDefaultProject(project);
        Instance i;
        try {
            i = SQLTask.run(odps, sql);
            i.waitForSuccess();
            List records = SQLTask.getResult(i);
            for(Record r:records){
                System.out.println(r.get(0).toString());
            }
        } catch (OdpsException e) {
            e.printStackTrace();
        }
    }
}

数据量10000行以上

• 本地spark读取odps表，写本地文件
• 离线数据集成到OSS, 然后通过OSS下载到本地
• 使用JDBC方式连接MaxCompute
• 使用批量数据通道SDK

    com.aliyun.odps
    odps-sdk-core
    0.31.3-public

注意点：是用0.24.0版本只支持一下数据类型 BIGINT, DOUBLE, BOOLEAN, DATETIME, STRING, DECIMAL, MAP, ARRAY;
报错如下：

ErrorCode=Local Error, ErrorMessage=Invalid json content.
    at com.aliyun.odps.tunnel.TableTunnel$DownloadSession.loadFromJson(TableTunnel.java:1483)
    at com.aliyun.odps.tunnel.TableTunnel$DownloadSession.initiate(TableTunnel.java:1334)
    at com.aliyun.odps.tunnel.TableTunnel$DownloadSession.(TableTunnel.java:1215)
    at com.aliyun.odps.tunnel.TableTunnel.createDownloadSession(TableTunnel.java:371)
    at TunnelCheck.main(TunnelCheck.java:36)
Caused by: java.lang.IllegalArgumentException: No enum constant com.aliyun.odps.OdpsType.INT
    at java.lang.Enum.valueOf(Enum.java:238)
    at com.aliyun.odps.OdpsType.valueOf(OdpsType.java:27)
    at com.aliyun.odps.tunnel.TunnelTableSchema.parseColumn(TunnelTableSchema.java:60)
    at com.aliyun.odps.tunnel.TunnelTableSchema.(TunnelTableSchema.java:36)
    at com.aliyun.odps.tunnel.TableTunnel$DownloadSession.loadFromJson(TableTunnel.java:1480)
    ... 4 more