Android环境上用C++使用libcurl实现4G蜂窝网络双通道的技术探索

查看图片
[TOC]

双通道概述

指设备连接着WiFi的情况下,同时打开蜂窝通道,从而实现双通道同时进行网络请求,提高访问速度。

graph LR
    client---|socket1| WiFi网卡 -->|socket1|Server
    client---|socket2| 4G蜂窝网络网卡 -->|socket2| Server

Android系统下双通结合libCurl的方案概述

不同于iOS的权限简洁,能将网卡直接丢给libCurl使用。Android在native层使用4G网卡,需要费劲很多。

sequenceDiagram
native ->>+ java: 发起请求打开双通

java ->>+ Android OS: 向系统申请4G通道
Note over Android OS : 手机顶部状态栏的4G流量图标常驻
Android OS ->>- java: 持有SocketFactory

java ->>- native:回调开启成功

loop 网络请求

native -->> native:发起网络请求

native ->>+ java:通知创建socket
java ->>+ Android OS: 创建socket
java ->> java: 使用socket解析域名
Android OS ->>- java:已经绑定过目标IP的socket

java ->>- native:获取socket的fd

native ->>+ server: libcurl正常建连
server ->>- native: 完成数据传输

end

native ->>+ java:关闭4G通道
java ->>+ Android OS: 主动释放4G通道
Note over Android OS : 手机顶部状态栏上4G流量图标消失
Android OS ->>- java: 释放成功
java ->>- native:回调关闭成功

关于域名解析服务:

仅使用localHost时:

如果仅使用localHost,那么只需要java层面,用socket绑定域名,不需要额外代码,就能自动的在4G网卡上进行域名解析为ip。
查看socket绑定的ip是socket.getRemoteSocketAddress()

使用自有DNS服务

在指定socket目的IP时,必须主动区分当前socket出口的IP类型,不能讲IPv4与IPv6混用。如果结合自己的DNS方案,那就需要把IPv6作为判断的参数。这里的样例代码,也是仅仅是从已经建联的socket中取出ip地址,没有进一步复杂化。

传递java上层的socket到native

这里利用到的是ParcelFileDescriptor,在java层获取socket的描述符,直接丢给libCurl使用

问题概述

  1. 对比iOS的方案,Android环境下,libCurl不能直接绑定4G网卡,在Android系统层面,限制了直接使用4G网卡。
  2. Android系统绕不开Java层配合。
  3. Android上层保持4G通道的打开,libCurl仍然不能直接使用4G网卡。
  4. Android系统绕不开Java层创建network(可用这个创建socket)
  5. 开启与关闭双通,都需要C++层面通知到Java层。

代码说明

Android双通的基本使用

在普通的使用场景中,我们不关注SocketFactory,也不用关注域名解析,因为系统API返回的connect可以直接进行网络请求,而内部的域名解析细节可以完全忽略:network.openConnection(new URL(url));

if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
        final String TAG = "NetworkCardInfo";
        TlogUtils.d(TAG, "4g通道,尝试开启");
        ConnectivityManager connectivityManager = (ConnectivityManager) getApplication().getSystemService(CONNECTIVITY_SERVICE);

        NetworkRequest request = new NetworkRequest.Builder()
                .addTransportType(NetworkCapabilities.TRANSPORT_CELLULAR)
                .addCapability(NetworkCapabilities.NET_CAPABILITY_INTERNET).build();

        ConnectivityManager.NetworkCallback networkCallback = new ConnectivityManager.NetworkCallback() {
            @Override
            public void onAvailable(Network network) {
                TlogUtils.d(TAG, "4g通道,已经开启");
                // 用network进行网络请求
                try {
                    HttpURLConnection urlConnection = (HttpURLConnection) network.openConnection(new URL(url));
                    int code = urlConnection.getResponseCode();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        };
        connectivityManager.requestNetwork(request, networkCallback);
    }
}

不成功的方案:libCurl直接绑定4G网卡

精准的获取蜂窝网卡名:
Android设备下,网卡非常多,想获取到有效的网卡,必须过滤网卡名,同时必须判断网卡下是否存在有效的IP:

import java.net.Inet6Address;
import java.net.InetAddress;
import java.net.NetworkInterface;
import java.util.Enumeration;

final class NetworkCardInfo {
    private final static String TAG = "NetworkCardInfo";

    /**
     * 获取4G网卡名
     */
    public static String getNetworkCardName(boolean isWifi) {
        String networkCardNameKey = isWifi ? "wlan" : "rmnet";
        final String tag = TAG + "," + networkCardNameKey;
        NameAndIp nameAndIp = getNetworkCardInfoOfCellularImpl(networkCardNameKey, tag);

        TlogUtils.i(tag, "获取到的网卡名:" + nameAndIp);

//        if (isWifi) {
            return nameAndIp.name;
//        } else {
//            return nameAndIp.ip;
//        }
    }

    private static class NameAndIp {
        private String name;
        private String ip;

        public NameAndIp(String name, String ip) {
            this.name = name;
            this.ip = ip;
        }

        @Override
        public String toString() {
            return "NameAndIp{" + "name='" + name + '\'' + ", ip='" + ip + '\'' + '}';
        }
    }

    private static NameAndIp getNetworkCardInfoOfCellularImpl(String networkCardNameKey, String tag) {
        try {
            Enumeration nis = NetworkInterface.getNetworkInterfaces();
            while (nis.hasMoreElements()) {
                NetworkInterface ni = (NetworkInterface) nis.nextElement();
                String name = ni.getName();
                if (name == null) {
                    name = "";
                }
                String nameCompare = name.toLowerCase();
                if (!nameCompare.contains(networkCardNameKey)) {
                    TlogUtils.d(tag, "网卡不符:" + name);
                    continue;
                }
                TlogUtils.d(tag, "网卡可疑:" + name);
                Enumeration ias = ni.getInetAddresses();
                while (ias.hasMoreElements()) {
                    InetAddress ia = ias.nextElement();
//                    // ipv6
//                    if(ia instanceof Inet6Address) {
//                        continue;
//                    }
                    String ip = ia.getHostAddress();
                    // 局域网与广域网IP才认为合理,否则就认为不可用

                    if (ia.isSiteLocalAddress()) {
                        // 判断出当前是否为局域网IP,局域网也是合理的
                        TlogUtils.d(tag, "isSiteLocalAddress:" + ip);
                        return new NameAndIp(name, ip);
                    } else if (ia.isLoopbackAddress()) {
                        TlogUtils.d(tag, "isLoopbackAddress:" + ip);
                    } else if (ia.isAnyLocalAddress()) {
                        TlogUtils.d(tag, "isAnyLocalAddress:" + ip);
                    } else if (ia.isLinkLocalAddress()) {
                        TlogUtils.d(tag, "isLinkLocalAddress:" + ip);
                    } else if (ia.isMulticastAddress()) {
                        TlogUtils.d(tag, "isMulticastAddress:" + ip);
                    } else if (ia.isMCGlobal()) {
                        TlogUtils.d(tag, "isMCGlobal:" + ip);
                    } else if (ia.isMCLinkLocal()) {
                        TlogUtils.d(tag, "isMCLinkLocal:" + ip);
                    } else if (ia.isMCNodeLocal()) {
                        TlogUtils.d(tag, "isMCNodeLocal:" + ip);
                    } else if (ia.isMCOrgLocal()) {
                        TlogUtils.d(tag, "isMCOrgLocal:" + ip);
                    } else if (ia.isMCSiteLocal()) {
                        TlogUtils.d(tag, "isMCSiteLocal:" + ip);
                    } else {
                        // 能走到这里,说明是个普通的广域网IP
                        TlogUtils.d(tag, "普通ip:" + ip);
                        return new NameAndIp(name, ip);
                    }
                }
            }
        } catch (Throwable e) {
            e.printStackTrace();
        }
        return null;
    }
}

让libcurl绑定网卡:

            curl_easy_setopt(dlcurl->curl, CURLOPT_INTERFACE, "name");

得到错误日志:

2020-08-26 20:00:40.660 4103-8729/com.phone I/DownloadNative.v2: YKCLog_v2: XNDczNDUwMDIwOA==-2 ->[ |name= |videoRetry=0 |retryTime=2 |reqUrl=http://vali.cp31.ott.cibntv.net/67756D6080932713CFC02204E/03000500005F057E4E8BB780000000B94C56DC-0A70-4635-B35C-3E5A291ECB30-00002.ts?ccode=01010201&duration=2700&expire=18000&psid=9bc52504cc5c194fcd8a64f56679022c428e5&ups_client_netip=6a0b29dc&ups_ts=1598443186&ups_userid=1340295651&utid=XvD8%2BJtNHPkDAIaQEYw6XqaA&vid=XNDczNDUwMDIwOA%3D%3D&sm=1&operate_type=1&dre=u30&si=76&eo=0&dst=1&iv=1&s=bcecd0971b7f4e449eea&type=flvhdv3&bc=2&hotvt=1&rid=20000000163E8ED36726AEE7B2825AE912CC906B02000000&vkey=B9f5937585ebfe0e1f40685ecb4039c79&f=vali |finalPath=http://vali.cp31.ott.cibntv.net/67756D6080932713CFC02204E/03000500005F057E4E8BB780000000B94C56DC-0A70-4635-B35C-3E5A291ECB30-00002.ts?ccode=01010201&duration=2700&expire=18000&psid=9bc52504cc5c194fcd8a64f56679022c428e5&ups_client_netip=6a0b29dc&ups_ts=1598443186&ups_userid=1340295651&utid=XvD8%2BJtNHPkDAIaQEYw6XqaA&vid=XNDczNDUwMDIwOA%3D%3D&sm=1&operate_type=1&dre=u30&si=76&eo=0&dst=1&iv=1&s=bcecd0971b7f4e449eea&type=flvhdv3&bc=2&hotvt=1&rid=20000000163E8ED36726AEE7B2825AE912CC906B02000000&vkey=B9f5937585ebfe0e1f40685ecb4039c79&f=vali |savePath=/storage/emulated/0/Android/data/com.phone/files/offlinedata/XNDczNDUwMDIwOA==/2 |finalIpUrl= |repCode=7 |httpCode=0 |io_error:code=0desc:Success |retryType=0 |proxyUrl= |ipIndex=1 ip=113.142.161.248 isHttpIp1 |segSize=1119916 |curSpeed=-1 |contentLen=-1 |downloadLen=0 |contentType= |fileSize=0 |ioErrorDesc= |redirectUrls= |curlInfo= => Hostname 'vali.cp31.ott.cibntv.net' was found in DNS cache
     =>   Trying 202.108.249.185...
     => TCP_NODELAY set
     => Local Interface rmnet_data0 is ip 10.229.85.126 using address family 2
     => SO_BINDTODEVICE rmnet_data0 failed with errno 1: Operation not permitted; will do regular bind
     => Local port: 0
     => After 5948ms connect time, move on!
     => connect to 202.108.249.185 port 80 failed: Connection timed out
     =>   Trying 202.108.249.186...
     => TCP_NODELAY set
     => Local Interface rmnet_data0 is ip 10.229.85.126 using address family 2
     => SO_BINDTODEVICE rmnet_data0 failed with errno 1: Operation not permitted; will do regular bind
     => Local port: 0
     => After 2915ms connect time, move on!
     => connect to 202.108.249.186 port 80 failed: Connection timed out
     => Failed to connect to vali.cp31.ott.cibntv.net port 80: Connection timed out
     => Closing connection 21
    ]

这个方案,在iOS系统上,其实是可以正常。但是Android系统存在的权限管理问题,导致C++的libcurl报错。

我们查看curl的源码介绍,并没有找到有效的解决方案

#ifdef SO_BINDTODEVICE
      /* I am not sure any other OSs than Linux that provide this feature,
       * and at the least I cannot test. --Ben
       *
       * This feature allows one to tightly bind the local socket to a
       * particular interface.  This will force even requests to other
       * local interfaces to go out the external interface.
       *
       *
       * Only bind to the interface when specified as interface, not just
       * as a hostname or ip address.
       *
       * interface might be a VRF, eg: vrf-blue, which means it cannot be
       * converted to an IP address and would fail Curl_if2ip. Simply try
       * to use it straight away.
       */
      if(setsockopt(sockfd, SOL_SOCKET, SO_BINDTODEVICE,
                    dev, (curl_socklen_t)strlen(dev) + 1) == 0) {
        /* This is typically "errno 1, error: Operation not permitted" if
         * you're not running as root or another suitable privileged
         * user.
         * If it succeeds it means the parameter was a valid interface and
         * not an IP address. Return immediately.
         */
        return CURLE_OK;
      }
#endif

透传socket方案:

java层申请打开蜂窝通道,创建一个socket,将socket透传给C++


    void openDoubleChanel() {
        TlogUtils.d(TAG, "4g通道,尝试开启");
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
            final ConnectivityManager connectivityManager = (ConnectivityManager) getApplication().getSystemService(CONNECTIVITY_SERVICE);

            NetworkRequest request = new NetworkRequest.Builder()
                    .addTransportType(NetworkCapabilities.TRANSPORT_CELLULAR)
                    .addCapability(NetworkCapabilities.NET_CAPABILITY_INTERNET).build();

            connectivityManager.requestNetwork(request, new NetworkCallbackImpl(connectivityManager));
        }
    }

    @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
    public static class NetworkCallbackImpl extends ConnectivityManager.NetworkCallback {
        final ConnectivityManager connectivityManager;

        public NetworkCallbackImpl(ConnectivityManager connectivityManager) {
            this.connectivityManager = connectivityManager;
        }

        @Override
        public void onAvailable(Network network) {
            TlogUtils.d(TAG, "4g通道,已经开启");
            // 不要测百度,百度不支持IPv6,但是可以测m.baidu.com
            String url = "http://valipl.cp31.ott.cibntv.net/6975030867335716092826D8B/03000300005F56527AB06EB7961451D178FC12-4CD4-44A7-A033-FAE171DF26A0.m3u8?ccode=01010101&duration=5374&expire=18000&psid=851c388bd1f8fc9a0aa11888f95b2854434af&ups_client_netip=2f580582&ups_ts=1599658512&ups_userid=1340295651&utid=XvD8%2BJtNHPkDAIaQEYw6XqaA&vid=XNDgwODE0NDc0MA&vkey=Bfcc9d9a459df2c6a97c2df3e223e65df&sm=1&operate_type=1&dre=u33&si=75&eo=0&dst=1&iv=1&s=aedc187a7603482190d5&type=3gphdv3&bc=2&rid=20000000DB5093085DD73D72365407F5E57F9FBE02000000";//hostEt.getText().toString();
            try {
                URL url1 = new URL(url);
                int port = url1.getPort();
                if (port <= -1) {
                    port = url1.getDefaultPort();
                }

                SocketFactory socketFactory = network.getSocketFactory();

                //java 层创建好socket并绑好
                final Socket socket = socketFactory.createSocket(url1.getHost(), port);
                InetSocketAddress socketAddress = (InetSocketAddress) socket.getRemoteSocketAddress();
                Log.e(TAG, "java层解析到的ip: " + socketAddress);
                final ParcelFileDescriptor parcelFileDescriptor = ParcelFileDescriptor.fromSocket(socket);
                // 开启C++
                IYKCache.testSocket(parcelFileDescriptor.getFd(), url, new SocketOptionImpl(connectivityManager, this, socket, parcelFileDescriptor));
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }

    @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
    private static class SocketOptionImpl implements SocketOption {
        private final ConnectivityManager connectivityManager;
        private final NetworkCallbackImpl networkCallback;
        private final Socket socket;
        private final ParcelFileDescriptor parcelFileDescriptor;

        public SocketOptionImpl(ConnectivityManager connectivityManager, NetworkCallbackImpl networkCallback, Socket socket, ParcelFileDescriptor parcelFileDescriptor) {
            this.connectivityManager = connectivityManager;
            this.networkCallback = networkCallback;
            this.socket = socket;
            this.parcelFileDescriptor = parcelFileDescriptor;
        }

        // 关闭socket
        @Override
        public void closeSocket() {
            try {
                socket.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
            parcelFileDescriptor.detachFd();
            try {
                parcelFileDescriptor.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }

        // 关闭双通
        @Override
        public void closeDoubleChanel() {
            connectivityManager.unregisterNetworkCallback(networkCallback);
        }
    }

对应的C++层发起请求,请求结束后关闭socket,关闭4G通道


#include 

#include 
#include 
#include 
#include 

#ifdef WIN32
#include 
#include 
#include 
#define close closesocket
#else

#include 
#include 
#include 
#include 
#include 

#endif


std::string resStr;

static size_t write_data(void *ptr, size_t size, size_t nmemb, void *stream) {
    resStr += std::string((char *) ptr, size * nmemb);
    return size * nmemb;
}

static curl_socket_t
opensocket(void *clientp, curlsocktype purpose, struct curl_sockaddr *address) {
    curl_socket_t sockfd;
    (void) purpose;
    (void) address;
    sockfd = *(curl_socket_t *) clientp;
    /* the actual externally set socket is passed in via the OPENSOCKETDATA
       option */

    flash_util::log::d("双通道", std::string("opensocket"));
    return sockfd;
}

static int closecb(void *clientp, curl_socket_t item) {
    (void) clientp;
    printf("libcurl wants to close %d now\n", (int) item);
    return 0;
}

std::string errStr;

static int
dl_curl_debug_cb(CURL *handle, curl_infotype type, char *data, size_t size, void *userp) {
    errStr += std::string((char *) data, size);
    return 0;
}


static int sockopt_callback(void *clientp, curl_socket_t curlfd,
                            curlsocktype purpose) {
    (void) clientp;
    (void) curlfd;
    (void) purpose;
    /* This return code was added in libcurl 7.21.5 */
    return CURL_SOCKOPT_ALREADY_CONNECTED;
}
extern "C"
JNIEXPORT void JNICALL
Java_com_flash_downloader_jni_FlashDownloaderJni_testSocket(JNIEnv *env, jclass clazz,
                                                                  jint sockfd, 
                                                                  jstring urlJ,
                                                                  jobject socket_optionJ) {
    CURL *curl = curl_easy_init();

    const std::string &url = parseJStringAndDeleteRef(env, urlJ);
    curl_easy_setopt(curl, CURLOPT_URL, url.c_str());

    /* no progress meter please */
    curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 1L);

    /* send all data to this function  */
    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);

    /* call this function to get a socket */
    curl_easy_setopt(curl, CURLOPT_OPENSOCKETFUNCTION, opensocket);
    curl_easy_setopt(curl, CURLOPT_OPENSOCKETDATA, &sockfd);

    curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
    curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L);

    curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);

    curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
    curl_easy_setopt(curl, CURLOPT_DEBUGFUNCTION, dl_curl_debug_cb);
//    curl_easy_setopt(curl, CURLOPT_DEBUGDATA, dlcurl);

    /* call this function to close sockets */
    curl_easy_setopt(curl, CURLOPT_CLOSESOCKETFUNCTION, closecb);
    curl_easy_setopt(curl, CURLOPT_CLOSESOCKETDATA, &sockfd);

    /* call this function to set options for the socket */
    curl_easy_setopt(curl, CURLOPT_SOCKOPTFUNCTION, sockopt_callback);

//    curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);
    flash_util::log::d("双通道", "begin");
    resStr = "";
    errStr = "";
    CURLcode resCode = curl_easy_perform(curl);
    char *ip;
    curl_easy_getinfo(curl, CURLINFO_PRIMARY_IP, &(ip));


    close(sockfd);
    //回调java层关闭fd引用
    env->CallVoidMethod(socket_optionJ, env->GetMethodID(env->GetObjectClass(socket_optionJ), "closeSocket", "()V"));
    //回调java层关闭双通
    env->CallVoidMethod(socket_optionJ, env->GetMethodID(env->GetObjectClass(socket_optionJ), "closeDoubleChanel", "()V"));

    curl_easy_cleanup(curl);

    flash_util::log::d("双通道",
                       "code:" + std::to_string(resCode) + " ip:" + ip + "\nresStr:" + resStr + "\nerrStr:" + errStr);
}

成功发起请求,且IP为IPv6类型,当前设备仅蜂窝网才有IPv6,所以说明流量走了蜂窝网。

2020-09-10 12:06:51.377 31409-31409/com.phone D/YKDownload.双通道: 4g通道,尝试开启
2020-09-10 12:06:52.085 31409-31557/com.phone D/YKDownload.双通道: 4g通道,已经开启
2020-09-10 12:06:52.471 31409-31557/com.phone E/双通道: java层解析到的ip: valipl.cp31.ott.cibntv.net/2408:871a:1001:51ff::ff53:80
2020-09-10 12:06:53.282 31409-31557/com.phone D/DownloadNative.双通道: begin
2020-09-10 12:06:53.349 31409-31557/com.phone D/DownloadNative.双通道: opensocket
2020-09-10 12:06:53.400 31409-31557/com.phone D/DownloadNative.双通道: code:0 ip:2408:871a:1001:51ff::ff53
    resStr:
    403 Forbidden
    
    

403 Forbidden


openresty
errStr: Trying 101.227.24.225... TCP_NODELAY set Connected to valipl.cp31.ott.cibntv.net (2408:871a:1001:51ff::ff53) port 80 (#0) GET /6975030867335716092826D8B/03000300005F56527AB06EB7961451D178FC12-4CD4-44A7-A033-FAE171DF26A0.m3u8?ccode=01010101&duration=5374&expire=18000&psid=851c388bd1f8fc9a0aa11888f95b2854434af&ups_client_netip=2f580582&ups_ts=1599658512&ups_userid=1340295651&utid=XvD8%2BJtNHPkDAIaQEYw6XqaA&vid=XNDgwODE0NDc0MA&vkey=Bfcc9d9a459df2c6a97c2df3e223e65df&sm=1&operate_type=1&dre=u33&si=75&eo=0&dst=1&iv=1&s=aedc187a7603482190d5&type=3gphdv3&bc=2&rid=20000000DB5093085DD73D72365407F5E57F9FBE02000000 HTTP/1.1 Host: valipl.cp31.ott.cibntv.net Accept: */* HTTP/1.1 403 Forbidden Date: Thu, 10 Sep 2020 04:06:52 GMT Content-Type: text/html Content-Length: 166 Connection: keep-alive anti-detail-code: 403611 Access-Control-Allow-Origin: * Access-Control-Expose-Headers: Content-Length cloud_type: aliyun Server: Tengine 403 Forbidden

403 Forbidden


openresty
Curl_http_done: called premature == 0 Connection #0 to host valipl.cp31.ott.cibntv.net left intact

你可能感兴趣的:(Android环境上用C++使用libcurl实现4G蜂窝网络双通道的技术探索)