代理服务器anyproxy的安装/HTTPS/规则

简介

anyproxy是一款可以高度定制的代理服务器,基于nodejs。

特征

  • 支持https明文代理
  • 支持低网速模拟
  • 支持二次开发,可以用javascript控制代理的全部流程,搭建前端个性化调试环境
  • 提供web版界面,观测请求情况

设计

anyproxy把http通信过程中的各个阶段进行抽离,分解成三个阶段:

  • 收到来自客户端请求之后,允许开发者直接从本地提供返回
  • 在转发请求到服务器前,允许开发者对发送的请求进行修改
  • 在收到服务器响应之后,允许开发者对响应内容进行修改,再返回给客户端

代理服务器anyproxy的安装/HTTPS/规则_第1张图片
对于上述每个阶段,anyproxy都提供了API接口,引入开发者编写自己的规则代码,实时干预通信过程,以此满足各类自定义需求。

具体地,我们提供的接口包括:

  • 收到用户请求之后
    • shouldUseLocalResponse ,是否在本地直接发送响应(不再向服务器发出请求)
    • dealLocalResponse 如果shouldUseLocalResponse返回true,会调用这个函数来获取本地响应内容(异步接口)
  • 向服务端发出请求之前
    • replaceRequestProtocol 替换向服务器发出的请求协议,支持http和https的替换
    • replaceRequestOption 替换向服务器发出的请求参数,即nodeJS中的 request option
    • replaceRequestData 替换请求的body
  • 向用户返回服务端的响应之前
    • replaceResponseStatusCode 替换服务器响应的http状态码
    • replaceResponseHeader 替换服务器响应的http头
    • replaceServerResDataAsync 替换服务器响应的数据(异步接口)
    • pauseBeforeSendingResponse 在请求返回给用户前的延迟时间

快速开始

安装

  • 安装Nodejs
  • npm install -g anyproxy,有可能需要sudo
  • python可选安装

启动

  • 默认启动anyproxy
  • 定制启动端口anyproxy --port 8001
  • 使用某个规则文件anyproxy --rule ./rule_sample/rule_allow_CORS.js
  • 代理https请求anyproxy --intercept(需要安装证书,详情见下文)
  • 其他命令可以通过anyproxy -h查看

加载网页界面

  • 访问http://127.0.0.1:8002,你会在浏览器里看到实时的请求。
  • 要确保是现代浏览器
    代理服务器anyproxy的安装/HTTPS/规则_第2张图片

HTTPS相关教程

  • 生成RootCA
    • 命令行执行 sudo anyproxy –root
  • 找到RootCA文件
    • 方法一: 执行完成之后,会打开证书的安装路径,即可看到 rootCA.crt 文件
    • 方法二: 启动anyproxy,浏览器打开 http://localhost:8002/fetchCrtFile ,也能获取rootCA.crt文件
    • 方法三:启动anyproxy,http://localhost:8002/qr_root 可以获取证书路径的二维码,移动端安装时会比较便捷
  • 打开上述rootCA.crt文件
  • 根据操作系统提示,信任rootCA
    代理服务器anyproxy的安装/HTTPS/规则_第3张图片
    代理服务器anyproxy的安装/HTTPS/规则_第4张图片
  • 其他
    • 如果在访问时出现UNAUTHORIZED_CERTIFICATE一类的安全警告,请重新检查证书的安装情况
    • 证书只需生成一次,使用前每个终端都需要信任它
  • 明文解析HTTPS

    • 需要解析HTTPS时,用intercept参数来启动anyproxy anyproxy --intercept
    • 为终端设置代理,在UI界面就能看到明文的HTTPS请求数据了,带把小锁的就是HTTPS请求
  • 进阶 - 用rule来手动处理https请求(如:代理文件到本地)

    • AnyProxy默认不会解析https请求,你需要引入shouldInterceptHttpsReq这个函数来显式指定解析哪个请求。具体可以参照这份sample : rule_intercept_some_https_requests.js
  • 其他

    • anyproxy --clear可以清除所有已生成的证书。清除后,各终端需要重新安装证书。
    • 日常开发中,不要使用anyproxy --type https来调试。AnyProxy使用https over http的方法来进行代理,而这条命令启动的是一个https代理服务器,两者使用场景完全不同。

AnyProxy规则文件样例

修改请求头:防止CDN返回304

以“防止CDN返回304”这个需求为例,最直接的方案是拦截请求,在发送到CDN前删除header中的if-modified-since字段。在AnyProxy中,配置replaceRequestOption接口,3行代码就能实现这个自定义功能:

//rule file
module.exports = {
  //在向服务器发出请求前,AnyProxy会调用这个接口,可以在此时修改发送请求的参数
  replaceRequestOption : function(req,option){
      var newOption = option;
      delete newOption.headers['if-modified-since'];
      return newOption;
  }
};

修改响应数据

再举个例子,如果你想修改响应数据,在所有html文件最后加个”Hello World”,就需要调用replaceServerResDataAsync接口,并结合content-type字段来进行修改,大约需要8行代码。

//rule file
module.exports = {
    replaceServerResDataAsync: function(req,res,serverResData,callback){
        //append "hello world" to all web pages
        if(/html/i.test(res.headers['content-type'])){
            var newDataStr = serverResData.toString();
            newDataStr += "hello world!";
            callback(newDataStr);
        }else{
            callback(serverResData);
        }
    }
};

把所有的响应延迟1500毫秒

module.exports = {

    pauseBeforeSendingResponse : function(req,res){
        //delay all the response for 1500ms
        return 1500;
    }

};

为ajax请求增加跨域头

//rule scheme :
// Ref: https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS

module.exports = {
    shouldUseLocalResponse : function(req,reqBody){
        //intercept all options request
        if(req.method == "OPTIONS"){
            return true;
        }else{
            return false;
        }
    },

    dealLocalResponse : function(req,reqBody,callback){
        if(req.method == "OPTIONS"){
            callback(200,mergeCORSHeader(req.headers),"");
        }
    },

    replaceResponseHeader: function(req,res,header){
        return mergeCORSHeader(req.headers, header);
    }

};

function mergeCORSHeader(reqHeader,originHeader){
    var targetObj = originHeader || {};

    delete targetObj["Access-Control-Allow-Credentials"];
    delete targetObj["Access-Control-Allow-Origin"];
    delete targetObj["Access-Control-Allow-Methods"];
    delete targetObj["Access-Control-Allow-Headers"];

    targetObj["access-control-allow-credentials"] = "true";
    targetObj["access-control-allow-origin"]      = reqHeader['origin'] || "-___-||";
    targetObj["access-control-allow-methods"]     = "GET, POST, PUT";
    targetObj["access-control-allow-headers"]     = reqHeader['access-control-request-headers'] || "-___-||";

    return targetObj;
}

截获github.com的https请求,再在最后加点文字

module.exports = {


    replaceServerResDataAsync: function(req,res,serverResData,callback){
        //add "hello github" to all github pages
        if(req.headers.host == "github.com"){
            serverResData += "hello github";
        }
        callback(serverResData);
    },

    shouldInterceptHttpsReq :function(req){
        //intercept https://github.com/
        //otherwise, all the https traffic will not go through this proxy

        // return true;
        if(req.headers.host == "github.com"){
            return true;
        }else{
            return false;
        }
    }
};

去除响应头里缓存相关的头

//rule scheme :

module.exports = {
  replaceRequestOption : function(req,option){
      var newOption = option;
      delete newOption.headers['if-none-match'];
      delete newOption.headers['if-modified-since'];

      return newOption;
  },

    replaceResponseHeader: function(req,res,header){
        header = header || {};
        header["Cache-Control"]                    = "no-cache, no-store, must-revalidate";
        header["Pragma"]                           = "no-cache";
        header["Expires"]                          = 0;

        return header;
    }
};

在请求发送到服务端前对参数做一些调整

module.exports = {

    replaceRequestOption : function(req,option){
        //replace request towards http://www.taobao.com
        //                     to http://www.taobao.com/about/

        /*
        option scheme:
        {
            hostname : "www.taobao.com"
            port     : 80
            path     : "/"
            method   : "GET"
            headers  : {cookie:""}
        }
        */
        if(option.hostname == "www.taobao.com" && option.path == "/"){
            option.path = "/about/";
        }
    }
};

改变服务端响应的http状态码

module.exports = {

    replaceResponseStatusCode: function(req,res,statusCode){
        //redirect requests toward http://www.taobao.com/*
        //                      to http://www.etao.com
        //using 302

        if(req.headers.host == "www.taobao.com"){
            statusCode = 302;
        }

        return statusCode;
    },

    replaceResponseHeader: function(req,res,header){
        if(req.headers.host == "www.taobao.com"){
            header.location = "http://www.etao.com";
        }

        return header;
    }
};

把响应映射到本地

//replace all the images with local one
var fs      = require("fs");

var LOCAL_IMAGE = "/Users/path/to/image.png";

module.exports = {

    summary:function(){
        return "replace all the images with local one";
    },

    //mark if use local response
    shouldUseLocalResponse : function(req,reqBody){
        if(/\.(png|gif|jpg|jpeg)$/.test(req.url)){
            req.replaceLocalFile = true;
            return true;
        }else{
            return false;
        }
    },

    dealLocalResponse : function(req,reqBody,callback){
        if(req.replaceLocalFile){
            callback(200, {"content-type":"image/png"}, fs.readFileSync(LOCAL_IMAGE) );
        }
    }
};

整体结构

/*
read the following wiki before using rule file
https://github.com/alibaba/anyproxy/wiki/What-is-rule-file-and-how-to-write-one
*/
module.exports = {
  /*
  These functions will overwrite the default ones, write your own when necessary.
    Comments in Chinese are nothing but a translation of key points. Be relax if you dont understand.
    致中文用户:中文注释都是只摘要,必要时请参阅英文文档。欢迎提出修改建议。
  */
    summary:function(){
        return "this is a blank rule for AnyProxy";
    },




    //=======================
    //when getting a request from user
    //收到用户请求之后
    //=======================

    //是否截获https请求
    //should intercept https request, or it will be forwarded to real server
    shouldInterceptHttpsReq :function(req){
        return false;
    },

    //是否在本地直接发送响应(不再向服务器发出请求)
  //whether to intercept this request by local logic
  //if the return value is true, anyproxy will call dealLocalResponse to get response data and will not send request to remote server anymore
    //req is the user's request sent to the proxy server
  shouldUseLocalResponse : function(req,reqBody){
        return false;
  },

    //如果shouldUseLocalResponse返回true,会调用这个函数来获取本地响应内容
    //you may deal the response locally instead of sending it to server
    //this function be called when shouldUseLocalResponse returns true
    //callback(statusCode,resHeader,responseData)
    //e.g. callback(200,{"content-type":"text/html"},"hello world")
  dealLocalResponse : function(req,reqBody,callback){
        callback(statusCode,resHeader,responseData)
  },



    //=======================
    //when ready to send a request to server
    //向服务端发出请求之前
    //=======================

    //替换向服务器发出的请求协议(http和https的替换)
    //replace the request protocol when sending to the real server
    //protocol : "http" or "https"
    replaceRequestProtocol:function(req,protocol){
      var newProtocol = protocol;
      return newProtocol;
    },

    //替换向服务器发出的请求参数(option)
    //option is the configuration of the http request sent to remote server. You may refers to http://nodejs.org/api/http.html#http_http_request_options_callback
    //you may return a customized option to replace the original one
    //you should not overwrite content-length header in options, since anyproxy will handle it for you
    replaceRequestOption : function(req,option){
        var newOption = option;
        return newOption;
    },

    //替换请求的body
    //replace the request body
    replaceRequestData: function(req,data){
        return data;
    },



    //=======================
    //when ready to send the response to user after receiving response from server
    //向用户返回服务端的响应之前
    //=======================

    //替换服务器响应的http状态码
    //replace the statusCode before it's sent to the user
    replaceResponseStatusCode: function(req,res,statusCode){
      var newStatusCode = statusCode;
      return newStatusCode;
    },

    //替换服务器响应的http头
    //replace the httpHeader before it's sent to the user
    //Here header == res.headers
    replaceResponseHeader: function(req,res,header){
      var newHeader = header;
      return newHeader;
    },

    //替换服务器响应的数据
    //replace the response from the server before it's sent to the user
    //you may return either a Buffer or a string
    //serverResData is a Buffer. for those non-unicode reponse , serverResData.toString() should not be your first choice.
    replaceServerResDataAsync: function(req,res,serverResData,callback){
        callback(serverResData);
    },

    //Deprecated
    // replaceServerResData: function(req,res,serverResData){
    //     return serverResData;
    // },

    //在请求返回给用户前的延迟时间
    //add a pause before sending response to user
    pauseBeforeSendingResponse : function(req,res){
      var timeInMS = 1; //delay all requests for 1ms
      return timeInMS;
    }

};

AnyProxy其他特性

  • 支持Https的中间人(man-in-the-middle)代理,同时提供便捷的根证书安装路径,方便移动端导入证书
  • 低网速网速模拟,协助调试2G/3G下的表现
  • 可以导出所有请求记录,供后期数据分析使用
  • 可以进行模块化调用,做二次封装,合并到现有的前端集成开发环境中,个性化搭建自己的调试环境

文档和支持

HTTPS相关配置的中文文档
What is rule file and how to write one
代理服务器的新轮子:anyproxy

你可能感兴趣的:(nodejs)