varnish高效缓存部署方式及原理详解

缓存:

命中率

文档命中率

字节命中率

缓存类型:

私有缓存

公共缓存

内容路由:

ICP:互联网缓存协议

HTCP:超文本缓存协议

Cache Digest

Cache Pre-filling

CARP:缓存阵列路由协议

缓存处理的具体步骤:

接收请求

解析请求(代理的功能)

查询缓存(检查本地缓存中是否存在对方请求的内容的副本)

副本的新鲜度检测(检查本地缓存的副本是否为最新版本)

构建响应

发送响应

记录日志

如何保证副本的新鲜度?

1,内容过期机制:

HTTP/1.0Expires (过期时间)

绝对时间

HTTP/1.1Cache-Control(max-age=)

相对时长

2,条件式请求:

mtime(时间戳):If-Modified-Since

基于时间的条件式请求

ETag(扩展标记): If-None-Match

基于扩展标签条件式请求

服务器或缓存服务器控制缓存的能力:

由原始服务器定义:

1Cache-Control 

Max-age=

s-max-age=

no-store:不能缓存

no-cache:能缓存,但不能直接使用次缓存对象,缓存对象在使用之前必须做新鲜度验证;

must-revalidate:必须进行新鲜度验证

Private

Public

2Expires

原始服务器不添加任何控制机制,而由缓存服务器自己决定缓存时长。

 

客户端控制是否使用缓存:

Cache-Control

Max-stale:告知缓存机制可以使用国企的文档;

no-cache:告知缓存机制必须进行验证,否则不会接受任何缓存文档;

no-store:告知缓存机制必须尽快删除缓存中的文档

HTTP/1.0

Pragmano-cache  能缓存必须进行新鲜度验证

http协议:

Request

mrthod<url> version

HEADERS

 

BODY

Response:

Version status reason

HEADRES

 

BODY

首部:

通用

Connectionclose|keep-alive

Date:日期时间

Host:请求的主机

Pragma: no-cache

Via:请求或响应在客户端和服务器之间传递时经过的代理

Transfer-Encoding:消息主题的传输编码方式,chunked表示采用块编码的方式

请求

If-Modified-Since

If-None-Match

Referer

User-Agent

 

Host:请求的主机

Accept-Encoding:接受的编码方式

Accept-Language:接受的自然语言

Accept-Charset:接受字符集

Authorization:服务器端发送www-authenticate时,客户端通过此首部提供认证信息;

响应

ETag:内容的标签

Location:重定向后的新位置

Server:服务器软件信息

WWW-Authenticate:要求对客户端进行认证

Age

实体

Content-Encoding

Content-Language

Content-Lenth

Content-Type:内容的MIME格式

Expires

Last-Modified:最近一次的修改时间

Varnish: 淘宝使用的是ATS(apace traffic server)

web缓存丶代理服务器:

  

Varnish 安装:

1,安装jemalloc

http://pkgs.org/download/libjemalloc.so.1()(64bit)  官网

下载地址

rpm -ivh http://dl.fedoraproject.org/pub/epel/6/x86_64/jemalloc-3.6.0-1.el6.x86_64.rpm

2,安装varnish

Varnish 3.0:

rpm --nosignature -i https://repo.varnish-cache.org/redhat/varnish-3.0.el6.rpm

yum install varnish

3,修改配置

使用文件缓存修改方法:

vim /etc/sysconfig/varnish/

VARNISH_STORAGE_FFILE=/var/lib/varnish/varnish_storage.bin

VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"

使用内存缓存修改方法:

VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"

修改配置文件

varnish配置文件:

/etc/sysconfig/varnish

 

varnish控制语言:

/etc/varnish/default.vcl

 

 

一、修改varinsh监听端口:

 

vim /etc/sysconfig/varnish

 

VARNISH_LISTEN_PORT=80

 

二、复制控制语言配置文件

 

probe 后端服务器健康状况检查

 

interval = 1s;   一秒探测一次

window = 5  最多打开五个窗口

threshold = 2;   若打开两个窗口探测都为正常,则此我们说此主机健康。否则抛弃。

 

director webs random(random是一种算法,分配方式随即) webs 是名字

 

purgers 清空缓存

 

 

acl purgers {

        "127.0.0.1";

        "192.168.100.0"/24;

}

允许谁来清空缓存

操作:

cp default.vcl /etc/varnish/defaul.vcl

 

vim /etc/varnish/default.val

修改host

 

修改purgers

acl purgers {

        "127.0.0.1";

        "192.168.100.0"/24;

}

修改域名:

sub vcl_recv {

     if (req.http.host ~ "^www.bbs.com")

修改140

添加html

  if (req.request == "GET" && req.url ~ "\/[0-9]\.(htm|html)$")

 

service varnish restart 

 

外放访问-------

 

使用varnishstat 查看命中率 

 

 

curl -L http://www.bbs.com 查看是否命中

HIT为命中

 

default.vcl配置文件

# This is a basic VCL configuration file for varnish.  See the vcl(7)

# man page for details on VCL syntax and semantics.

# Default backend definition.  Set this to point to your content

# server.

backend web1 {

  .host = "192.168.100.20";

  .port = "80";

  .probe = {

.url = "/";

.interval = 1s;

.window = 5;

.threshold = 2;

}

}


backend web2 {

  .host = "192.168.100.30";

  .port = "80";

  .probe = {

                .url = "/";

                .interval = 1s;

                .window = 5;

                .threshold = 2;

        }

}


director webs random {

.retries = 5;

  {

    .backend = web1;

    .weight  = 2;

  }

  {

    .backend = web2;

    .weight  = 3;

  }

}

acl purgers {

        "127.0.0.1";

        "192.168.0.0"/24;

}


# Below is a commented-out copy of the default VCL logic.  If you

# redefine any of these subroutines, the built-in logic will be

# appended to your code.

 sub vcl_recv {

     if (req.http.host ~ "^www.bbs.com")  

            { set req.backend = webs; }

     else { error 404 "Unknown HostName!"; }

     if (req.request == "PURGE") {

                if (!client.ip ~ purgers) {

                        error 405 "Method not allowed";

                }

                return (lookup);

        }

     if (req.restarts == 0) {

  if (req.http.x-forwarded-for) {

     set req.http.X-Forwarded-For =

  req.http.X-Forwarded-For + ", " + client.ip;

  } else {

     set req.http.X-Forwarded-For = client.ip;

  }

     }

     if (req.request == "GET" && req.url ~ "\.(jpg|png|gif|swf|jpeg|ico)$")

            {

              unset req.http.cookie;

             }

     if (req.request == "GET" && req.url ~ "\.(php)($|\?)")

            { return (pass); }      

     if (req.request != "GET" &&

       req.request != "HEAD" &&

       req.request != "PUT" &&

       req.request != "POST" &&

       req.request != "TRACE" &&

       req.request != "OPTIONS" &&

       req.request != "DELETE") {

         /* Non-RFC2616 or CONNECT which is weird. */

         return (pipe);

     }

     if (req.request != "GET" && req.request != "HEAD") {

         /* We only deal with GET and HEAD by default */

         return (pass);

     }

     if (req.http.Authorization || req.http.Cookie) {

         /* Not cacheable by default */

         return (pass);

     }

     if (req.url ~ "^/fonts/")

           { return (pass); }

     return (lookup);

}

 

 sub vcl_pipe {

#     # Note that only the first request to the backend will have

#     # X-Forwarded-For set.  If you use X-Forwarded-For and want to

#     # have it set for all requests, make sure to have:

#     # set bereq.http.connection = "close";

#     # here.  It is not set by default as it might break some broken web

#     # applications, like IIS with NTLM authentication.

     return (pipe);

 }

 

 sub vcl_pass {

     return (pass);

 }

 

 sub vcl_hash {

     hash_data(req.url);

     if (req.http.host) {

         hash_data(req.http.host);

     } else {

         hash_data(server.ip);

     }

     return (hash);

 }

 

 sub vcl_hit {

     return (deliver); 

 }

 

 sub vcl_miss {

     return (fetch);

 }

 

 sub vcl_fetch {

     if (beresp.ttl <= 0s ||

         beresp.http.Set-Cookie ||

         beresp.http.Vary == "*") {

  /*

  * Mark as "Hit-For-Pass" for the next 2 minutes

  */

  set beresp.ttl = 120 s;

  return (hit_for_pass);

     }

     if (req.request == "GET" && req.url ~ "\.(js|css|mp3|jpg|png|gif|swf|jpeg|ico)$") 

         { set beresp.ttl = 7d; }

     if (req.request == "GET" && req.url ~ "\/[0-9]\.(htm|html)$") 

         { set beresp.ttl = 300s; } 

    return (deliver);

 }

 

 sub vcl_deliver {

       set resp.http.x-hits = obj.hits ; 

       if (obj.hits > 0) 

              { set resp.http.X-Cache = "HIT"; } 

       else { set resp.http.X-Cache = "MISS"; }  

    #return (deliver);

 }

 

 sub vcl_error {

     set obj.http.Content-Type = "text/html; charset=utf-8";

     set obj.http.Retry-After = "5";

     synthetic {"

 <?xml version="1.0" encoding="utf-8"?>

 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"

  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

 <html>

   <head>

     <title>"} + obj.status + " " + obj.response + {"</title>

   </head>

   <body>

     <h1>Error "} + obj.status + " " + obj.response + {"</h1>

     <p>"} + obj.response + {"</p>

     <h3>Guru Meditation:</h3>

     <p>XID: "} + req.xid + {"</p>

     <hr>

     <p>Varnish cache server</p>

   </body>

 </html>

 "};

     return (deliver);

 }

 

 sub vcl_init {

  return (ok);

 }

 

 sub vcl_fini {

  return (ok);

 }


你可能感兴趣的:(互联网,服务器,varnish,mtime,命中率,具体步骤)