Kubernetes(k8s)之分布式文件系统glusterfs

1、为什么使用分布式文件系统glusterfs

答:使用持久化存储nfs,但是使用nfs这个共享存储特别简单,但是后期在扩容和管理方面都特别的不方便,在生产中一般都是使用分布式文件系统,这里使用的是分布式文件系统glusterfs。

2、什么是分布式文件系统glusterfs

答:分布式文件系统glusterfs是一个开源分布式文件系统,具有强大的横向扩展能力,可以支持数PB存储容器和数千客户端,通过网络互联成一个并行的网络文件系统。具有可扩展性、高性能、高可用性等特点。

3、安装分布式文件系统glusterfs,需要所有节点都需要进行安装的,如下所示:

首先在三台节点上都安装glusterfs的yum源,如下所示:

 1 [root@k8s-master ~]# yum install centos-release-gluster -y
 2 Loaded plugins: fastestmirror, langpacks, product-id, search-disabled-repos, subscription-manager
 3 
 4 This system is not registered with an entitlement server. You can use subscription-manager to register.
 5 
 6 Loading mirror speeds from cached hostfile
 7  * base: mirrors.bfsu.edu.cn
 8  * extras: mirrors.bfsu.edu.cn
 9  * updates: mirrors.bfsu.edu.cn
10 base                                                                                                     | 3.6 kB  00:00:00     
11 extras                                                                                                   | 2.9 kB  00:00:00     
12 updates                                                                                                  | 2.9 kB  00:00:00     
13 Resolving Dependencies
14 --> Running transaction check
15 ---> Package centos-release-gluster7.noarch 0:1.0-2.el7.centos will be installed
16 --> Processing Dependency: centos-release-storage-common for package: centos-release-gluster7-1.0-2.el7.centos.noarch
17 --> Running transaction check
18 ---> Package centos-release-storage-common.noarch 0:2-2.el7.centos will be installed
19 --> Finished Dependency Resolution
20 
21 Dependencies Resolved
22 
23 =================================================================================================================================================================================================================
24  Package                                                          Arch                                      Version                                              Repository                                 Size
25 =================================================================================================================================================================================================================
26 Installing:
27  centos-release-gluster7                                          noarch                                    1.0-2.el7.centos                                     extras                                    5.2 k
28 Installing for dependencies:
29  centos-release-storage-common                                    noarch                                    2-2.el7.centos                                       extras                                    5.1 k
30 
31 Transaction Summary
32 =================================================================================================================================================================================================================
33 Install  1 Package (+1 Dependent package)
34 
35 Total download size: 10 k
36 Installed size: 2.4 k
37 Downloading packages:
38 (1/2): centos-release-gluster7-1.0-2.el7.centos.noarch.rpm                                                                                                                                | 5.2 kB  00:00:00     
39 (2/2): centos-release-storage-common-2-2.el7.centos.noarch.rpm                                                                                                                            | 5.1 kB  00:00:00     
40 -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
41 Total                                                                                                                                                                             13 kB/s |  10 kB  00:00:00     
42 Running transaction check
43 Running transaction test
44 Transaction test succeeded
45 Running transaction
46   Installing : centos-release-storage-common-2-2.el7.centos.noarch                                                                                                                                           1/2 
47   Installing : centos-release-gluster7-1.0-2.el7.centos.noarch                                                                                                                                               2/2 
48   Verifying  : centos-release-gluster7-1.0-2.el7.centos.noarch                                                                                                                                               1/2 
49   Verifying  : centos-release-storage-common-2-2.el7.centos.noarch                                                                                                                                           2/2 
50 
51 Installed:
52   centos-release-gluster7.noarch 0:1.0-2.el7.centos                                                                                                                                                              
53 
54 Dependency Installed:
55   centos-release-storage-common.noarch 0:2-2.el7.centos                                                                                                                                                          
56 
57 Complete!
58 [root@k8s-master ~]# 

可以看到yum源里面多了一些文件,默认安装的是CentOS-Gluster-7.repo这个7.x版本的,如下所示:

1 [root@k8s-master ~]# ls /etc/yum.repos.d/
2 CentOS-Base.repo  CentOS-CR.repo  CentOS-Debuginfo.repo  CentOS-fasttrack.repo  CentOS-Gluster-7.repo  CentOS-Media.repo  CentOS-Sources.repo  CentOS-Storage-common.repo  CentOS-Vault.repo
3 [root@k8s-master ~]# 

然后开始在三台节点上安装glusterfs-server包安装上,最主要的就是安装这个包的,如下所示:

  1 [root@k8s-master ~]# yum install install glusterfs-server -y
  2 Loaded plugins: fastestmirror, langpacks, product-id, search-disabled-repos, subscription-manager
  3 
  4 This system is not registered with an entitlement server. You can use subscription-manager to register.
  5 
  6 Loading mirror speeds from cached hostfile
  7  * base: mirrors.bfsu.edu.cn
  8  * centos-gluster7: mirrors.tuna.tsinghua.edu.cn
  9  * extras: mirrors.bfsu.edu.cn
 10  * updates: mirrors.bfsu.edu.cn
 11 centos-gluster7                                                                                                                                                                           | 3.0 kB  00:00:00     
 12 centos-gluster7/7/x86_64/primary_db                                                                                                                                                       |  69 kB  00:00:00     
 13 No package install available.
 14 Resolving Dependencies
 15 --> Running transaction check
 16 ---> Package glusterfs-server.x86_64 0:7.6-1.el7 will be installed
 17 --> Processing Dependency: glusterfs = 7.6-1.el7 for package: glusterfs-server-7.6-1.el7.x86_64
 18 --> Processing Dependency: glusterfs-api = 7.6-1.el7 for package: glusterfs-server-7.6-1.el7.x86_64
 19 --> Processing Dependency: glusterfs-cli = 7.6-1.el7 for package: glusterfs-server-7.6-1.el7.x86_64
 20 --> Processing Dependency: glusterfs-client-xlators = 7.6-1.el7 for package: glusterfs-server-7.6-1.el7.x86_64
 21 --> Processing Dependency: glusterfs-fuse = 7.6-1.el7 for package: glusterfs-server-7.6-1.el7.x86_64
 22 --> Processing Dependency: glusterfs-libs = 7.6-1.el7 for package: glusterfs-server-7.6-1.el7.x86_64
 23 --> Processing Dependency: libgfapi.so.0(GFAPI_6.0)(64bit) for package: glusterfs-server-7.6-1.el7.x86_64
 24 --> Processing Dependency: libgfapi.so.0(GFAPI_PRIVATE_6.0)(64bit) for package: glusterfs-server-7.6-1.el7.x86_64
 25 --> Processing Dependency: libgfapi.so.0(GFAPI_PRIVATE_6.1)(64bit) for package: glusterfs-server-7.6-1.el7.x86_64
 26 --> Processing Dependency: liburcu-bp.so.6()(64bit) for package: glusterfs-server-7.6-1.el7.x86_64
 27 --> Processing Dependency: liburcu-cds.so.6()(64bit) for package: glusterfs-server-7.6-1.el7.x86_64
 28 --> Running transaction check
 29 ---> Package glusterfs.x86_64 0:3.12.2-18.el7 will be updated
 30 ---> Package glusterfs.x86_64 0:7.6-1.el7 will be an update
 31 ---> Package glusterfs-api.x86_64 0:3.12.2-18.el7 will be updated
 32 ---> Package glusterfs-api.x86_64 0:7.6-1.el7 will be an update
 33 ---> Package glusterfs-cli.x86_64 0:3.12.2-18.el7 will be updated
 34 ---> Package glusterfs-cli.x86_64 0:7.6-1.el7 will be an update
 35 ---> Package glusterfs-client-xlators.x86_64 0:3.12.2-18.el7 will be updated
 36 ---> Package glusterfs-client-xlators.x86_64 0:7.6-1.el7 will be an update
 37 ---> Package glusterfs-fuse.x86_64 0:7.6-1.el7 will be installed
 38 ---> Package glusterfs-libs.x86_64 0:3.12.2-18.el7 will be updated
 39 ---> Package glusterfs-libs.x86_64 0:7.6-1.el7 will be an update
 40 ---> Package userspace-rcu.x86_64 0:0.10.0-3.el7 will be installed
 41 --> Finished Dependency Resolution
 42 
 43 Dependencies Resolved
 44 
 45 =================================================================================================================================================================================================================
 46  Package                                                     Arch                                      Version                                          Repository                                          Size
 47 =================================================================================================================================================================================================================
 48 Installing:
 49  glusterfs-server                                            x86_64                                    7.6-1.el7                                        centos-gluster7                                    1.3 M
 50 Installing for dependencies:
 51  glusterfs-fuse                                              x86_64                                    7.6-1.el7                                        centos-gluster7                                    156 k
 52  userspace-rcu                                               x86_64                                    0.10.0-3.el7                                     centos-gluster7                                     93 k
 53 Updating for dependencies:
 54  glusterfs                                                   x86_64                                    7.6-1.el7                                        centos-gluster7                                    640 k
 55  glusterfs-api                                               x86_64                                    7.6-1.el7                                        centos-gluster7                                    114 k
 56  glusterfs-cli                                               x86_64                                    7.6-1.el7                                        centos-gluster7                                    198 k
 57  glusterfs-client-xlators                                    x86_64                                    7.6-1.el7                                        centos-gluster7                                    850 k
 58  glusterfs-libs                                              x86_64                                    7.6-1.el7                                        centos-gluster7                                    425 k
 59 
 60 Transaction Summary
 61 =================================================================================================================================================================================================================
 62 Install  1 Package  (+2 Dependent packages)
 63 Upgrade             ( 5 Dependent packages)
 64 
 65 Total download size: 3.7 M
 66 Downloading packages:
 67 No Presto metadata available for centos-gluster7
 68 warning: /var/cache/yum/x86_64/7/centos-gluster7/packages/glusterfs-cli-7.6-1.el7.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID e451e5b5: NOKEY                             ]  0.0 B/s |    0 B  --:--:-- ETA 
 69 Public key for glusterfs-cli-7.6-1.el7.x86_64.rpm is not installed
 70 (1/8): glusterfs-cli-7.6-1.el7.x86_64.rpm                                                                                                                                                 | 198 kB  00:00:00     
 71 (2/8): glusterfs-api-7.6-1.el7.x86_64.rpm                                                                                                                                                 | 114 kB  00:00:00     
 72 (3/8): glusterfs-libs-7.6-1.el7.x86_64.rpm                                                                                                                                                | 425 kB  00:00:00     
 73 (4/8): glusterfs-fuse-7.6-1.el7.x86_64.rpm                                                                                                                                                | 156 kB  00:00:01     
 74 (5/8): glusterfs-client-xlators-7.6-1.el7.x86_64.rpm                                                                                                                                      | 850 kB  00:00:03     
 75 (6/8): glusterfs-7.6-1.el7.x86_64.rpm                                                                                                                                                     | 640 kB  00:00:03     
 76 (7/8): glusterfs-server-7.6-1.el7.x86_64.rpm                                                                                                                                              | 1.3 MB  00:00:04     
 77 (8/8): userspace-rcu-0.10.0-3.el7.x86_64.rpm                                                                                                                                              |  93 kB  00:00:13     
 78 -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 79 Total                                                                                                                                                                            242 kB/s | 3.7 MB  00:00:15     
 80 Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage
 81 Importing GPG key 0xE451E5B5:
 82  Userid     : "CentOS Storage SIG (http://wiki.centos.org/SpecialInterestGroup/Storage) "
 83  Fingerprint: 7412 9c0b 173b 071a 3775 951a d4a2 e50b e451 e5b5
 84  Package    : centos-release-storage-common-2-2.el7.centos.noarch (@extras)
 85  From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage
 86 Running transaction check
 87 Running transaction test
 88 Transaction test succeeded
 89 Running transaction
 90   Updating   : glusterfs-libs-7.6-1.el7.x86_64                                                                                                                                                              1/13 
 91   Updating   : glusterfs-client-xlators-7.6-1.el7.x86_64                                                                                                                                                    2/13 
 92   Updating   : glusterfs-7.6-1.el7.x86_64                                                                                                                                                                   3/13 
 93   Updating   : glusterfs-api-7.6-1.el7.x86_64                                                                                                                                                               4/13 
 94   Installing : glusterfs-fuse-7.6-1.el7.x86_64                                                                                                                                                              5/13 
 95   Updating   : glusterfs-cli-7.6-1.el7.x86_64                                                                                                                                                               6/13 
 96   Installing : userspace-rcu-0.10.0-3.el7.x86_64                                                                                                                                                            7/13 
 97   Installing : glusterfs-server-7.6-1.el7.x86_64                                                                                                                                                            8/13 
 98   Cleanup    : glusterfs-api-3.12.2-18.el7.x86_64                                                                                                                                                           9/13 
 99   Cleanup    : glusterfs-3.12.2-18.el7.x86_64                                                                                                                                                              10/13 
100   Cleanup    : glusterfs-client-xlators-3.12.2-18.el7.x86_64                                                                                                                                               11/13 
101   Cleanup    : glusterfs-cli-3.12.2-18.el7.x86_64                                                                                                                                                          12/13 
102   Cleanup    : glusterfs-libs-3.12.2-18.el7.x86_64                                                                                                                                                         13/13 
103   Verifying  : glusterfs-libs-7.6-1.el7.x86_64                                                                                                                                                              1/13 
104   Verifying  : glusterfs-api-7.6-1.el7.x86_64                                                                                                                                                               2/13 
105   Verifying  : glusterfs-cli-7.6-1.el7.x86_64                                                                                                                                                               3/13 
106   Verifying  : glusterfs-fuse-7.6-1.el7.x86_64                                                                                                                                                              4/13 
107   Verifying  : glusterfs-client-xlators-7.6-1.el7.x86_64                                                                                                                                                    5/13 
108   Verifying  : glusterfs-server-7.6-1.el7.x86_64                                                                                                                                                            6/13 
109   Verifying  : glusterfs-7.6-1.el7.x86_64                                                                                                                                                                   7/13 
110   Verifying  : userspace-rcu-0.10.0-3.el7.x86_64                                                                                                                                                            8/13 
111   Verifying  : glusterfs-3.12.2-18.el7.x86_64                                                                                                                                                               9/13 
112   Verifying  : glusterfs-cli-3.12.2-18.el7.x86_64                                                                                                                                                          10/13 
113   Verifying  : glusterfs-client-xlators-3.12.2-18.el7.x86_64                                                                                                                                               11/13 
114   Verifying  : glusterfs-libs-3.12.2-18.el7.x86_64                                                                                                                                                         12/13 
115   Verifying  : glusterfs-api-3.12.2-18.el7.x86_64                                                                                                                                                          13/13 
116 
117 Installed:
118   glusterfs-server.x86_64 0:7.6-1.el7                                                                                                                                                                            
119 
120 Dependency Installed:
121   glusterfs-fuse.x86_64 0:7.6-1.el7                                                                      userspace-rcu.x86_64 0:0.10.0-3.el7                                                                     
122 
123 Dependency Updated:
124   glusterfs.x86_64 0:7.6-1.el7        glusterfs-api.x86_64 0:7.6-1.el7        glusterfs-cli.x86_64 0:7.6-1.el7        glusterfs-client-xlators.x86_64 0:7.6-1.el7        glusterfs-libs.x86_64 0:7.6-1.el7       
125 
126 Complete!
127 [root@k8s-master ~]# 

然后开始启动glusterfs,并设置开机自启动,如下所示:

1 [root@k8s-master ~]# systemctl start glusterd.service
2 [root@k8s-master ~]# systemctl enable glusterd.service
3 [root@k8s-master ~]# 

生产环境可以增加一个/dev/sdb或者/dev/sdc用于挂载,将它挂载到某一个目录下面去使用的,默认只有/dev/sda。这里直接增加目录来使用了,在三台机器上都执行下面的创建目录的命令,如下所示:

1 [root@k8s-master ~]# mkdir -p /gfs/test1
2 [root@k8s-master ~]# mkdir -p /gfs/test2
3 [root@k8s-master ~]# 

添加存储资源池,即将节点都组装起来,在master主节点进行操作,可以查看gluster的资源池(如果gluster没有提示,可以先退出xshell,再重新登录即可),如下所示:

 1 [root@k8s-master ~]# gluster pool list
 2 UUID                    Hostname     State
 3 d2ea56bf-6402-49c5-93a2-0d784110b231    localhost    Connected 
 4 [root@k8s-master ~]# gluster pool 
 5 unrecognized command
 6 
 7  Usage: gluster [options]    
 8  Options:
 9  --help  Shows the help information
10  --version  Shows the version
11  --print-logdir  Shows the log directory
12  --print-statedumpdir Shows the state dump directory
13 
14 [root@k8s-master ~]# 

资源池默认只能识别出自己本身,这里需要将另外两个节点加入到资源池中,根据hostname加入到资源池,需要做host解析的,如下所示:

1 [root@k8s-master ~]# gluster peer 
2 detach  probe   status  
3 [root@k8s-master ~]# gluster peer probe k8s-node2
4 peer probe: success. 
5 [root@k8s-master ~]# gluster peer probe k8s-node3
6 peer probe: success. 
7 [root@k8s-master ~]# 

此时,再次查看资源池,发现已经有了三个节点了,如下所示:

1 [root@k8s-master ~]# gluster pool list 
2 UUID                    Hostname     State
3 977fb392-500c-4353-b03c-0ee9e53cee29    k8s-node2    Connected 
4 3b7c328f-2eff-4098-b3ac-ebc730975d22    k8s-node3    Connected 
5 d2ea56bf-6402-49c5-93a2-0d784110b231    localhost    Connected 
6 [root@k8s-master ~]# 

资源池添加完毕之后,开始进行glusterfs卷管理,创建分布式复制卷,如下所示:

可以在资源池里面使用资源池创建一个卷,在生产环境中glusterfs支持七种模式的卷,使用最多的是分布式复制卷,其他有的卷不稳定肯能会丢失书数据,最稳定的一种就是分布式复制卷,分布式复制卷要求至少四个存储单元,就是刚才创建的目录,正常情况下,一个目录就要挂载一个空的硬盘,一个硬盘就是一个存储单元,或者说一个目录就是一个存储单元,glusterfs这个分布式复制卷,最少复制的副本是2,再加上是分布式,至少要有四个目录作为存储单元。

1 [root@k8s-master ~]# gluster volume create biehl replica 2 k8s-master:/gfs/test1 k8s-master:/gfs/test2 k8s-node2:/gfs/test1 k8s-node2:/gfs/test2 force
2 volume create: biehl: success: please start the volume to access data
3 [root@k8s-master ~]# 

卷已经创建成功了,现在需要启动一下才可以访问里面的数据,启动卷并进行查看卷,如下所示:

 1 [root@k8s-master ~]# gluster volume start biehl 
 2 volume start: biehl: success
 3 [root@k8s-master ~]# gluster volume info biehl 
 4  
 5 Volume Name: biehl
 6 Type: Distributed-Replicate
 7 Volume ID: 5729192c-627f-4444-92eb-32a245086f43
 8 Status: Started
 9 Snapshot Count: 0
10 Number of Bricks: 2 x 2 = 4
11 Transport-type: tcp
12 Bricks:
13 Brick1: k8s-master:/gfs/test1
14 Brick2: k8s-master:/gfs/test2
15 Brick3: k8s-node2:/gfs/test1
16 Brick4: k8s-node2:/gfs/test2
17 Options Reconfigured:
18 transport.address-family: inet
19 storage.fips-mode-rchecksum: on
20 nfs.disable: on
21 performance.client-io-threads: off
22 [root@k8s-master ~]# 

下面开始挂载卷,可以在三台机器的任意一个节点上进行挂载,如下所示:

1 [root@k8s-master ~]# mount -t glusterfs 192.168.110.133:/biehl /mnt
2 [root@k8s-master ~]# 

含义就是将192.168.110.133的biehl卷,挂载到mnt目录下面,现在可以进行查看,如下所示:

 1 [root@k8s-master ~]# df -h
 2 Filesystem               Size  Used Avail Use% Mounted on
 3 /dev/mapper/centos-root   18G   14G  3.9G  79% /
 4 devtmpfs                 1.2G     0  1.2G   0% /dev
 5 tmpfs                    1.2G     0  1.2G   0% /dev/shm
 6 tmpfs                    1.2G   40M  1.1G   4% /run
 7 tmpfs                    1.2G     0  1.2G   0% /sys/fs/cgroup
 8 /dev/sda1                197M  157M   41M  80% /boot
 9 tmpfs                    229M   12K  229M   1% /run/user/42
10 overlay                   18G   14G  3.9G  79% /var/lib/docker/overlay2/fbf4133aed53d4e17a706be46fadcbea9fe6eb1e615379bfaa157570f0de7bc6/merged
11 shm                       64M     0   64M   0% /var/lib/docker/containers/5e72b0961647003c757fd41ffe824027c857d1f3938b9a53535645f4f372a40a/shm
12 overlay                   18G   14G  3.9G  79% /var/lib/docker/overlay2/80967f34f76b4afe42411f1b1dc6514b2b19d997644dfa4687a3a0924df06a08/merged
13 overlay                   18G   14G  3.9G  79% /var/lib/docker/overlay2/e085a4fba182d19139db32c664fcc1f08895ca603db3b1f9855183fc071c1adc/merged
14 shm                       64M     0   64M   0% /var/lib/docker/containers/276a98c9d5ccd61f42a0a1ef55c30f76beb2977483ed2d79281dfcec79922029/shm
15 shm                       64M     0   64M   0% /var/lib/docker/containers/0f89914f64c09c0fbd777e637d8f987414420861628d7b78f8a1bafc9054df59/shm
16 overlay                   18G   14G  3.9G  79% /var/lib/docker/overlay2/23afbc15307f6e5b397db4b1fd0e124c943bdb95312234e8d41e4f80a104b0cd/merged
17 shm                       64M     0   64M   0% /var/lib/docker/containers/f686976b5743236b368b8b70e222d230b737612acadc716ca4cbabcd9a187011/shm
18 overlay                   18G   14G  3.9G  79% /var/lib/docker/overlay2/15696b3573a066b5d3fd7aa202086b2bcc4145d9d0bb46e54c7d8d7787d1cc73/merged
19 overlay                   18G   14G  3.9G  79% /var/lib/docker/overlay2/6d9b6978705654a815dd1e72d5501408cead42493c25bdd33e4faa0c9decdcd0/merged
20 overlay                   18G   14G  3.9G  79% /var/lib/docker/overlay2/534837088c8ba6021c80d2ebb4ddb87a075106a784c499cc9ffca541d2e86dce/merged
21 overlay                   18G   14G  3.9G  79% /var/lib/docker/overlay2/8cdf1249e1823f24caf1d14ca9c05caa4e8e569d81292d06c642e618bc90b82b/merged
22 overlay                   18G   14G  3.9G  79% /var/lib/docker/overlay2/3ee0d48c04a32a2f0938af17898697042423d7d147df3ee8c1178bfda6e5fd9a/merged
23 overlay                   18G   14G  3.9G  79% /var/lib/docker/overlay2/31f5db66511cc7fd15cf1e2a23b7178d0eafbf1d7ddbdef1e0f82a254b6f6fc5/merged
24 overlay                   18G   14G  3.9G  79% /var/lib/docker/overlay2/16288e087e06b2c1fc369d19528eeb933cde44b8aa785f35b6756ec4da3a0f60/merged
25 tmpfs                    229M     0  229M   0% /run/user/0
26 192.168.110.133:/biehl    18G   11G  7.1G  61% /mnt
27 [root@k8s-master ~]# df -h /mnt/
28 Filesystem              Size  Used Avail Use% Mounted on
29 192.168.110.133:/biehl   18G   11G  7.1G  61% /mnt
30 [root@k8s-master ~]# 

分布式复制卷扩容,可以先使用df -h命令进行查看扩容前的容量,然后扩容后查看容量,在运行状态也可以进行扩容的哦,如下所示:

1 [root@k8s-master ~]# df -h /mnt/
2 Filesystem              Size  Used Avail Use% Mounted on
3 192.168.110.133:/biehl   18G   11G  7.1G  61% /mnt
4 [root@k8s-master ~]# gluster volume add-brick biehl k8s-node3:/gfs/test1 k8s-node3:/gfs/test2 force
5 volume add-brick: success
6 [root@k8s-master ~]# df -h /mnt/
7 Filesystem              Size  Used Avail Use% Mounted on
8 192.168.110.133:/biehl   23G   15G  8.8G  62% /mnt
9 [root@k8s-master ~]# 

接下来,可以向/mnt目录里面上传一些文件进行观察,如下所示:

1 [root@k8s-master ~]# cd /mnt/
2 [root@k8s-master mnt]# rz -E
3 rz waiting to receive.
4 [root@k8s-master mnt]# ls
5 book-master6.0.war
6 [root@k8s-master mnt]# unzip book-master6.0.war 

解压之后显示一堆文件,这些文件会分别在放到不同的存储单元中,首先查看master主节点的目录单元,如下所示:

 1 [root@k8s-master mnt]# yum install tree
 2 Loaded plugins: fastestmirror, langpacks, product-id, search-disabled-repos, subscription-manager
 3 
 4 This system is not registered with an entitlement server. You can use subscription-manager to register.
 5 
 6 Loading mirror speeds from cached hostfile
 7  * base: mirrors.bfsu.edu.cn
 8  * centos-gluster7: mirrors.tuna.tsinghua.edu.cn
 9  * extras: mirrors.bfsu.edu.cn
10  * updates: mirrors.bfsu.edu.cn
11 Resolving Dependencies
12 --> Running transaction check
13 ---> Package tree.x86_64 0:1.6.0-10.el7 will be installed
14 --> Finished Dependency Resolution
15 
16 Dependencies Resolved
17 
18 =================================================================================================================================================================================================================
19  Package                                         Arch                                              Version                                                 Repository                                       Size
20 =================================================================================================================================================================================================================
21 Installing:
22  tree                                            x86_64                                            1.6.0-10.el7                                            base                                             46 k
23 
24 Transaction Summary
25 =================================================================================================================================================================================================================
26 Install  1 Package
27 
28 Total download size: 46 k
29 Installed size: 87 k
30 Is this ok [y/d/N]: y
31 Downloading packages:
32 tree-1.6.0-10.el7.x86_64.rpm                                                                                                                                                              |  46 kB  00:00:05     
33 Running transaction check
34 Running transaction test
35 Transaction test succeeded
36 Running transaction
37   Installing : tree-1.6.0-10.el7.x86_64                                                                                                                                                                      1/1 
38   Verifying  : tree-1.6.0-10.el7.x86_64                                                                                                                                                                      1/1 
39 
40 Installed:
41   tree.x86_64 0:1.6.0-10.el7                                                                                                                                                                                     
42 
43 Complete!
44 [root@k8s-master mnt]# tree /gfs/

由于我的文件太多,就不进行展示了,使用命令tree /gfs就可以进行查看了,可以在三台机器都进行查看观察的。

 

4、glusterfs作为k8s的后端存储,可以使用命令查看k8s支持的后端存储类型,如下所示:

  1 [root@k8s-master test1]# kubectl explain pv.spec
  2 RESOURCE: spec 
  3 
  4 DESCRIPTION:
  5      Spec defines a specification of a persistent volume owned by the cluster.
  6      Provisioned by an administrator. More info:
  7      http://kubernetes.io/docs/user-guide/persistent-volumes#persistent-volumes
  8 
  9     PersistentVolumeSpec is the specification of a persistent volume.
 10 
 11 FIELDS:
 12    awsElasticBlockStore    
 13      AWSElasticBlockStore represents an AWS Disk resource that is attached to a
 14      kubelet's host machine and then exposed to the pod. More info:
 15      http://kubernetes.io/docs/user-guide/volumes#awselasticblockstore
 16 
 17    cephfs    
 18      CephFS represents a Ceph FS mount on the host that shares a pod's lifetime
 19 
 20    fc    
 21      FC represents a Fibre Channel resource that is attached to a kubelet's host
 22      machine and then exposed to the pod.
 23 
 24    rbd    
 25      RBD represents a Rados Block Device mount on the host that shares a pod's
 26      lifetime. More info:
 27      http://releases.k8s.io/HEAD/examples/volumes/rbd/README.md
 28 
 29    accessModes    <[]Object>
 30      AccessModes contains all ways the volume can be mounted. More info:
 31      http://kubernetes.io/docs/user-guide/persistent-volumes#access-modes
 32 
 33    azureFile    
 34      AzureFile represents an Azure File Service mount on the host and bind mount
 35      to the pod.
 36 
 37    capacity    <object>
 38      A description of the persistent volume's resources and capacity. More info:
 39      http://kubernetes.io/docs/user-guide/persistent-volumes#capacity
 40 
 41    cinder    
 42      Cinder represents a cinder volume attached and mounted on kubelets host
 43      machine More info:
 44      http://releases.k8s.io/HEAD/examples/mysql-cinder-pd/README.md
 45 
 46    flocker    
 47      Flocker represents a Flocker volume attached to a kubelet's host machine
 48      and exposed to the pod for its usage. This depends on the Flocker control
 49      service being running
 50 
 51    nfs    
 52      NFS represents an NFS mount on the host. Provisioned by an admin. More
 53      info: http://kubernetes.io/docs/user-guide/volumes#nfs
 54 
 55    azureDisk    
 56      AzureDisk represents an Azure Data Disk mount on the host and bind mount to
 57      the pod.
 58 
 59    claimRef    
 60      ClaimRef is part of a bi-directional binding between PersistentVolume and
 61      PersistentVolumeClaim. Expected to be non-nil when bound. claim.VolumeName
 62      is the authoritative bind between PV and PVC. More info:
 63      http://kubernetes.io/docs/user-guide/persistent-volumes#binding
 64 
 65    gcePersistentDisk    
 66      GCEPersistentDisk represents a GCE Disk resource that is attached to a
 67      kubelet's host machine and then exposed to the pod. Provisioned by an admin.
 68      More info: http://kubernetes.io/docs/user-guide/volumes#gcepersistentdisk
 69 
 70    persistentVolumeReclaimPolicy    <string>
 71      What happens to a persistent volume when released from its claim. Valid
 72      options are Retain (default) and Recycle. Recycling must be supported by the
 73      volume plugin underlying this persistent volume. More info:
 74      http://kubernetes.io/docs/user-guide/persistent-volumes#recycling-policy
 75 
 76    photonPersistentDisk    
 77      PhotonPersistentDisk represents a PhotonController persistent disk attached
 78      and mounted on kubelets host machine
 79 
 80    vsphereVolume    
 81      VsphereVolume represents a vSphere volume attached and mounted on kubelets
 82      host machine
 83 
 84    flexVolume    
 85      FlexVolume represents a generic volume resource that is
 86      provisioned/attached using an exec based plugin. This is an alpha feature
 87      and may change in future.
 88 
 89    glusterfs    
 90      Glusterfs represents a Glusterfs volume that is attached to a host and
 91      exposed to the pod. Provisioned by an admin. More info:
 92      http://releases.k8s.io/HEAD/examples/volumes/glusterfs/README.md
 93 
 94    hostPath    
 95      HostPath represents a directory on the host. Provisioned by a developer or
 96      tester. This is useful for single-node development and testing only! On-host
 97      storage is not supported in any way and WILL NOT WORK in a multi-node
 98      cluster. More info: http://kubernetes.io/docs/user-guide/volumes#hostpath
 99 
100    iscsi    
101      ISCSI represents an ISCSI Disk resource that is attached to a kubelet's
102      host machine and then exposed to the pod. Provisioned by an admin.
103 
104    quobyte    
105      Quobyte represents a Quobyte mount on the host that shares a pod's lifetime
106 
107 
108 [root@k8s-master test1]# 

可以查看详细的glusterfs配置需要什么,endpoints就是svc里面的一个资源,endpoints就是(clusterIP + 端口号),path就是glusterfs卷的一个路径,readOnly只读模式,如下所示:

 1 [root@k8s-master test1]# kubectl explain pv.spec.glusterfs
 2 RESOURCE: glusterfs 
 3 
 4 DESCRIPTION:
 5      Glusterfs represents a Glusterfs volume that is attached to a host and
 6      exposed to the pod. Provisioned by an admin. More info:
 7      http://releases.k8s.io/HEAD/examples/volumes/glusterfs/README.md
 8 
 9     Represents a Glusterfs mount that lasts the lifetime of a pod. Glusterfs volumes do not support ownership management or SELinux relabeling.
10 
11 FIELDS:
12    endpoints    <string> -required-
13      EndpointsName is the endpoint name that details Glusterfs topology. More
14      info:
15      http://releases.k8s.io/HEAD/examples/volumes/glusterfs/README.md#create-a-pod
16 
17    path    <string> -required-
18      Path is the Glusterfs volume path. More info:
19      http://releases.k8s.io/HEAD/examples/volumes/glusterfs/README.md#create-a-pod
20 
21    readOnly    
22      ReadOnly here will force the Glusterfs volume to be mounted with read-only
23      permissions. Defaults to false. More info:
24      http://releases.k8s.io/HEAD/examples/volumes/glusterfs/README.md#create-a-pod
25 
26 
27 [root@k8s-master test1]# 

开始创建endpoints的glusterfs-ep.yaml配置文件,如下所示:

1 [root@k8s-master k8s]# ls
2 book-master.war  dashboard  dashboard.zip  deploy  health  heapster  hpa  metrics  namespace  pod  rc  skydns  skydns.zip  svc  tomcat_demo  tomcat_demo.zip  volume
3 [root@k8s-master k8s]# mkdir glusterfs
4 [root@k8s-master k8s]# cd glusterfs/
5 [root@k8s-master glusterfs]# ls
6 [root@k8s-master glusterfs]# vim glusterfs-ep.yaml
7 [root@k8s-master glusterfs]# 

具体内容,如下所示:

 1 apiVersion: v1
 2 kind: Endpoints
 3 metadata:
 4   name: glusterfs
 5   namespace: default
 6 subsets:
 7 # k8s连接glusterfs的ip地址和端口号
 8 - addresses:
 9   - ip: 192.168.110.133
10   - ip: 192.168.110.134
11   - ip: 192.168.110.135
12   ports:
13   - port: 49152
14     protocol: TCP
15   

每个节点启动了glusterfs就存在了49152端口号,如下所示:

 1 [root@k8s-node3 test1]# netstat -lntup 49152
 2 Active Internet connections (only servers)
 3 Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
 4 tcp        0      0 0.0.0.0:24007           0.0.0.0:*               LISTEN      47623/glusterd      
 5 tcp        0      0 127.0.0.1:10248         0.0.0.0:*               LISTEN      14515/kubelet       
 6 tcp        0      0 127.0.0.1:10249         0.0.0.0:*               LISTEN      6779/kube-proxy     
 7 tcp        0      0 0.0.0.0:40842           0.0.0.0:*               LISTEN      -                   
 8 tcp        0      0 192.168.110.135:10250   0.0.0.0:*               LISTEN      14515/kubelet       
 9 tcp        0      0 192.168.110.135:10255   0.0.0.0:*               LISTEN      14515/kubelet       
10 tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      1/systemd           
11 tcp        0      0 0.0.0.0:20048           0.0.0.0:*               LISTEN      90097/rpc.mountd    
12 tcp        0      0 0.0.0.0:6000            0.0.0.0:*               LISTEN      7122/X              
13 tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      7304/dnsmasq        
14 tcp        0      0 0.0.0.0:55318           0.0.0.0:*               LISTEN      52362/rpc.statd     
15 tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      6776/sshd           
16 tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      6778/cupsd          
17 tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      7094/master         
18 tcp        0      0 0.0.0.0:49152           0.0.0.0:*               LISTEN      71495/glusterfsd    

开始创建这个endpoint,如下所示:

1 [root@k8s-master glusterfs]# kubectl create -f glusterfs-ep.yaml 
2 endpoints "glusterfs" created
3 [root@k8s-master glusterfs]# 

创建成功可以查看一下,如下所示:

 1 [root@k8s-master glusterfs]# kubectl get ep
 2 NAME         ENDPOINTS                                                           AGE
 3 glusterfs    192.168.110.133:49152,192.168.110.134:49152,192.168.110.135:49152   34s
 4 kubernetes   192.168.110.133:6443                                                25d
 5 mysql        172.16.66.6:3306                                                    5h
 6 myweb        172.16.66.5:8080                                                    6h
 7 [root@k8s-master glusterfs]# kubectl get endpoints 
 8 NAME         ENDPOINTS                                                           AGE
 9 glusterfs    192.168.110.133:49152,192.168.110.134:49152,192.168.110.135:49152   1m
10 kubernetes   192.168.110.133:6443                                                25d
11 mysql        172.16.66.6:3306                                                    5h
12 myweb        172.16.66.5:8080                                                    6h
13 [root@k8s-master glusterfs]# 

接下来,给这个endpoint创建一个Servic,需要注意的是Service和endpoint是靠名称来进行关联的,他们两个的名称必须一致的,如下所示:

1 [root@k8s-master glusterfs]# vim glusterfs-svc.yaml 
2 [root@k8s-master glusterfs]# 

配置内容,如下所示:

 1 apiVersion: v1
 2 kind: Service
 3 metadata:
 4   name: glusterfs
 5   namespace: default
 6 spec:
 7   ports:
 8   - port: 49152
 9     protocol: TCP
10     targetPort: 49152
11   sessionAffinity: None
12   type: ClusterIP
13   

开始进行创建Service,并进行查看,如下所示:

 1 [root@k8s-master glusterfs]# vim glusterfs-svc.yaml 
 2 [root@k8s-master glusterfs]# kubectl create -f glusterfs-svc.yaml 
 3 service "glusterfs" created
 4 [root@k8s-master glusterfs]# kubectl get svc 
 5 NAME         CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
 6 glusterfs    10.254.182.41            49152/TCP        10s
 7 kubernetes   10.254.0.1               443/TCP          25d
 8 mysql        10.254.126.11            3306/TCP         5h
 9 myweb        10.254.188.155          8080:30008/TCP   6h
10 [root@k8s-master glusterfs]# 

可以使用查看详情,看是否已经关联上了,如下所示:

 1 [root@k8s-master glusterfs]# kubectl describe svc glusterfs 
 2 Name:            glusterfs
 3 Namespace:        default
 4 Labels:            
 5 Selector:        
 6 Type:            ClusterIP
 7 IP:            10.254.182.41
 8 Port:                49152/TCP
 9 Endpoints:        192.168.110.133:49152,192.168.110.134:49152,192.168.110.135:49152
10 Session Affinity:    None
11 No events.
12 [root@k8s-master glusterfs]# 

接下来,开始创建glusterfs类型的PV,k8s对接各种后端存储的时候最关键的地方就是创建PV,如下所示:

1 [root@k8s-master glusterfs]# vim glusterfs-pv.yaml
2 [root@k8s-master glusterfs]# 

配置内容,如下所示:

 1 apiVersion: v1
 2 kind: PersistentVolume
 3 metadata:
 4   name: gluster
 5   labels:
 6     type: glusterfs
 7 spec:
 8   capacity:
 9     storage: 50Gi
10   accessModes:
11   - ReadWriteMany
12   glusterfs:
13     endpoints: "glusterfs"
14     path: "biehl"
15     readOnly: false

创建并查看PV,如下所示:

1 [root@k8s-master glusterfs]# gluster volume list 
2 biehl
3 [root@k8s-master glusterfs]# kubectl create -f glusterfs-pv.yaml 
4 persistentvolume "gluster" created
5 [root@k8s-master glusterfs]# kubectl get pv
6 NAME      CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS      CLAIM           REASON    AGE
7 gluster   50Gi       RWX           Retain          Available                             9s
8 mysql     5Gi        RWX           Recycle         Bound       default/mysql             5h
9 [root@k8s-master glusterfs]# 

开始创建PVC,才可以使用PV,开始创建PVC,如下所示:

1 [root@k8s-master glusterfs]# cp ../tomcat_demo/mysql-pvc.yaml .
2 [root@k8s-master glusterfs]# ls
3 glusterfs-ep.yaml  glusterfs-pv.yaml  glusterfs-svc.yaml  mysql-pvc.yaml
4 [root@k8s-master glusterfs]# mv mysql-pvc.yaml glusterfs-pvc.yaml
5 [root@k8s-master glusterfs]# ls
6 glusterfs-ep.yaml  glusterfs-pvc.yaml  glusterfs-pv.yaml  glusterfs-svc.yaml
7 [root@k8s-master glusterfs]# 

配置内容,如下所示:

 1 kind: PersistentVolumeClaim
 2 apiVersion: v1
 3 metadata:
 4   name: gluster
 5 spec:
 6   accessModes:
 7     - ReadWriteMany
 8   resources:
 9     requests:
10       storage: 5Gi

创建并查看PVC、PV的内容,如下所示:

 1 [root@k8s-master glusterfs]# kubectl create -f glusterfs-pvc.yaml 
 2 persistentvolumeclaim "gluster" created
 3 [root@k8s-master glusterfs]# kubectl get pvc 
 4 NAME      STATUS    VOLUME    CAPACITY   ACCESSMODES   AGE
 5 gluster   Bound     gluster   50Gi       RWX           4s
 6 mysql     Bound     mysql     5Gi        RWX           6h
 7 [root@k8s-master glusterfs]# kubectl get pv
 8 NAME      CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS    CLAIM             REASON    AGE
 9 gluster   50Gi       RWX           Retain          Bound     default/gluster             5m
10 mysql     5Gi        RWX           Recycle         Bound     default/mysql               6h
11 [root@k8s-master glusterfs]# 

PVC绑定PV之后就可以使用了,可以对pod进行使用,如下所示:

1 [root@k8s-master glusterfs]# ls
2 glusterfs-ep.yaml  glusterfs-pvc.yaml  glusterfs-pv.yaml  glusterfs-svc.yaml
3 [root@k8s-master glusterfs]# cp ../pod/nginx_pod.yaml .
4 [root@k8s-master glusterfs]# ls
5 glusterfs-ep.yaml  glusterfs-pvc.yaml  glusterfs-pv.yaml  glusterfs-svc.yaml  nginx_pod.yaml
6 [root@k8s-master glusterfs]# vim nginx_pod.yaml 
7 [root@k8s-master glusterfs]# 

将volumeMounts、volumes都配置好,如下所示:

 1 apiVersion: v1
 2 kind: Pod
 3 metadata:
 4   name: nginx
 5   labels:
 6     app: web
 7     env: nginx
 8 spec:
 9   containers:
10     - name: nginx
11       image: 192.168.110.133:5000/nginx:1.13
12       ports:
13         - containerPort: 80
14       volumeMounts:
15         - name: nfs-vol2
16           mountPath: /usr/share/nginx/html
17   volumes:
18     - name: nfs-vol2
19     persistentVolumeClaim:
20       claimName: gluster

创建nginx这个pod,并进行查看,发现已经正常运行起来了,如下所示:

1 [root@k8s-master glusterfs]# kubectl create -f nginx_pod.yaml 
2 pod "nginx" created
3 [root@k8s-master glusterfs]# kubectl get pods -o wide
4 NAME          READY     STATUS    RESTARTS   AGE       IP            NODE
5 mysql-wldks   1/1       Running   0          5h        172.16.66.6   k8s-node3
6 myweb-c8sf6   1/1       Running   1          7h        172.16.66.5   k8s-node3
7 nginx         1/1       Running   0          3s        172.16.66.3   k8s-node3
8 [root@k8s-master glusterfs]# 

然后可以访问一下这个Pod,如下所示:

1 [root@k8s-master glusterfs]# kubectl get pods -o wide
2 NAME          READY     STATUS    RESTARTS   AGE       IP            NODE
3 mysql-wldks   1/1       Running   0          5h        172.16.66.6   k8s-node3
4 myweb-c8sf6   1/1       Running   1          7h        172.16.66.5   k8s-node3
5 nginx         1/1       Running   0          3s        172.16.66.3   k8s-node3
6 [root@k8s-master glusterfs]# curl 172.16.66.3

我们的容器除了SVC里面通过反向代理和端口映射的方式访问之外,还可以让Pod不经过我们的SVC进行访问,可以在Pod里面,新增一个hostPort进行访问。
Kubernetes(k8s)之分布式文件系统glusterfs_第1张图片

将之前的nginx的pod删除掉,再重新创建一个,如下所示:

 1 [root@k8s-master glusterfs]# kubectl get pods -o wide
 2 NAME          READY     STATUS    RESTARTS   AGE       IP            NODE
 3 mysql-wldks   1/1       Running   0          7h        172.16.66.6   k8s-node3
 4 myweb-c8sf6   1/1       Running   1          9h        172.16.66.5   k8s-node3
 5 nginx         1/1       Running   0          2h        172.16.66.3   k8s-node3
 6 [root@k8s-master glusterfs]# kubectl delete pod nginx 
 7 pod "nginx" deleted
 8 [root@k8s-master glusterfs]# kubectl get pods -o wide
 9 NAME          READY     STATUS    RESTARTS   AGE       IP            NODE
10 mysql-wldks   1/1       Running   0          7h        172.16.66.6   k8s-node3
11 myweb-c8sf6   1/1       Running   1          9h        172.16.66.5   k8s-node3
12 [root@k8s-master glusterfs]# kubectl create -f nginx_pod.yaml 
13 pod "nginx" created
14 [root@k8s-master glusterfs]# kubectl get pods -o wide
15 NAME          READY     STATUS    RESTARTS   AGE       IP            NODE
16 mysql-wldks   1/1       Running   0          7h        172.16.66.6   k8s-node3
17 myweb-c8sf6   1/1       Running   1          9h        172.16.66.5   k8s-node3
18 nginx         1/1       Running   0          2s        172.16.66.3   k8s-node3
19 [root@k8s-master glusterfs]# 

可以看到新创建的nginx在k8s-node3这个节点上了,但是此时我发现无论curl访问还是浏览器访问都是403错误,这不都是很郁闷了。

1 [root@k8s-master glusterfs]# curl -I 172.16.66.3
2 HTTP/1.1 403 Forbidden
3 Server: nginx/1.15.12
4 Date: Tue, 30 Jun 2020 14:03:56 GMT
5 Content-Type: text/html
6 Content-Length: 154
7 Connection: keep-alive

后来经过仔细地分析和查找,发现是确实访问不到页面才出现地错误,究其原因,主要就是/mnt目录下面要有可以直接访问地界面才可以地。

1 [root@k8s-master glusterfs]# cd /mnt/
2 [root@k8s-master mnt]# ls
3 css  img  index.html  js

删除之前地nginx地pod,创建新的pod,如下所示:

 1 [root@k8s-master glusterfs]# kubectl get pod -o wide
 2 NAME          READY     STATUS    RESTARTS   AGE       IP            NODE
 3 mysql-wldks   1/1       Running   0          8h        172.16.66.6   k8s-node3
 4 myweb-c8sf6   1/1       Running   1          10h       172.16.66.5   k8s-node3
 5 nginx         1/1       Running   0          6m        172.16.66.3   k8s-node3
 6 nginx2        1/1       Running   0          17m       172.16.74.5   k8s-node2
 7 [root@k8s-master glusterfs]# kubectl delete pod nginx
 8 pod "nginx" deleted
 9 [root@k8s-master glusterfs]# kubectl get pod -o wide
10 NAME          READY     STATUS    RESTARTS   AGE       IP            NODE
11 mysql-wldks   1/1       Running   0          8h        172.16.66.6   k8s-node3
12 myweb-c8sf6   1/1       Running   1          10h       172.16.66.5   k8s-node3
13 nginx2        1/1       Running   0          17m       172.16.74.5   k8s-node2
14 [root@k8s-master glusterfs]# kubectl create -f nginx_pod.yaml 
15 pod "nginx" created
16 [root@k8s-master glusterfs]# kubectl get pod -o wide
17 NAME          READY     STATUS    RESTARTS   AGE       IP            NODE
18 mysql-wldks   1/1       Running   0          8h        172.16.66.6   k8s-node3
19 myweb-c8sf6   1/1       Running   1          10h       172.16.66.5   k8s-node3
20 nginx         1/1       Running   0          2s        172.16.66.3   k8s-node3
21 nginx2        1/1       Running   0          18m       172.16.74.5   k8s-node2
22 [root@k8s-master glusterfs]# curl 172.16.66.3
23 
24 "en">
25 
26 "UTF-8">
27 "viewport" content="width=device-width, initial-scale=1.0">
28 "X-UA-Compatible" content="ie=edge">
29 HTML5ȫı·ɻСԎϷ - Դë֮¼м/title>
<span style="color: #008080;">30</span> 
<span style="color: #008080;">31</span> <style>
<span style="color: #008080;">32</span> <span style="color: #000000;">    html {
</span><span style="color: #008080;">33</span>         width: <span style="color: #800080;">100</span>%<span style="color: #000000;">;
</span><span style="color: #008080;">34</span>         height: <span style="color: #800080;">100</span>%<span style="color: #000000;">;
</span><span style="color: #008080;">35</span>         margin: <span style="color: #800080;">0</span><span style="color: #000000;">;
</span><span style="color: #008080;">36</span>         padding: <span style="color: #800080;">0</span><span style="color: #000000;">;
</span><span style="color: #008080;">37</span> <span style="color: #000000;">        position: relative;
</span><span style="color: #008080;">38</span>         background-image: linear-<span style="color: #000000;">gradient(#2C3E50,#4CA1AF);
</span><span style="color: #008080;">39</span> 
<span style="color: #008080;">40</span> <span style="color: #000000;">    }
</span><span style="color: #008080;">41</span> 
<span style="color: #008080;">42</span> <span style="color: #000000;">    .canvasbig {
</span><span style="color: #008080;">43</span> <span style="color: #000000;">        position: absolute;
</span><span style="color: #008080;">44</span>         left: calc(<span style="color: #800080;">50</span>% -<span style="color: #000000;"> 260px);
</span><span style="color: #008080;">45</span>         top: calc(<span style="color: #800080;">50</span>% -<span style="color: #000000;"> 400px);
</span><span style="color: #008080;">46</span> <span style="color: #000000;">        width: 520px;
</span><span style="color: #008080;">47</span> <span style="color: #000000;">        height: 800px;
</span><span style="color: #008080;">48</span>         
<span style="color: #008080;">49</span> <span style="color: #000000;">    }
</span><span style="color: #008080;">50</span> 
<span style="color: #008080;">51</span> <span style="color: #000000;">    .canvasdiv {
</span><span style="color: #008080;">52</span> <span style="color: #000000;">        position: absolute;
</span><span style="color: #008080;">53</span> <span style="color: #000000;">        cursor: pointer;
</span><span style="color: #008080;">54</span> <span style="color: #000000;">        left: 160px;
</span><span style="color: #008080;">55</span> <span style="color: #000000;">        top: 500px;
</span><span style="color: #008080;">56</span> <span style="color: #000000;">        width: 200px;
</span><span style="color: #008080;">57</span> <span style="color: #000000;">        height: 53px;
</span><span style="color: #008080;">58</span>         background-image: url(img/<span style="color: #000000;">starting.png);
</span><span style="color: #008080;">59</span> <span style="color: #000000;">    }
</span><span style="color: #008080;">60</span> 
<span style="color: #008080;">61</span> <span style="color: #000000;">    .none {
</span><span style="color: #008080;">62</span> <span style="color: #000000;">        display: none;
</span><span style="color: #008080;">63</span> <span style="color: #000000;">    }
</span><span style="color: #008080;">64</span> </style>
<span style="color: #008080;">65</span> 
<span style="color: #008080;">66</span> </head>
<span style="color: #008080;">67</span> <body>
<span style="color: #008080;">68</span> 
<span style="color: #008080;">69</span> <!-- <script src=<span style="color: #800000;">"</span><span style="color: #800000;">js/circle.js</span><span style="color: #800000;">"</span>></script> -->
<span style="color: #008080;">70</span> 
<span style="color: #008080;">71</span> <div <span style="color: #0000ff;">class</span>=<span style="color: #800000;">"</span><span style="color: #800000;">canvasbig</span><span style="color: #800000;">"</span>>
<span style="color: #008080;">72</span>     <div <span style="color: #0000ff;">class</span>=<span style="color: #800000;">"</span><span style="color: #800000;">canvasdiv</span><span style="color: #800000;">"</span>></div>
<span style="color: #008080;">73</span>     <canvas width=<span style="color: #800000;">"</span><span style="color: #800000;">520</span><span style="color: #800000;">"</span> height=<span style="color: #800000;">"</span><span style="color: #800000;">800</span><span style="color: #800000;">"</span> id=<span style="color: #800000;">"</span><span style="color: #800000;">canvas1</span><span style="color: #800000;">"</span>></canvas>
<span style="color: #008080;">74</span> </div>
<span style="color: #008080;">75</span> 
<span style="color: #008080;">76</span> <script src=<span style="color: #800000;">"</span><span style="color: #800000;">js/index.js</span><span style="color: #800000;">"</span>></script>
<span style="color: #008080;">77</span> <div style=<span style="color: #800000;">"</span><span style="color: #800000;">text-align:center;</span><span style="color: #800000;">"</span>>
<span style="color: #008080;">78</span> <p>¸Ϗ·£º<a href=<span style="color: #800000;">"</span><span style="color: #800000;">http://www.mycodes.net/</span><span style="color: #800000;">"</span> target=<span style="color: #800000;">"</span><span style="color: #800000;">_blank</span><span style="color: #800000;">"</span>>Դë֮¼м/a></p>
<span style="color: #008080;">79</span> </div>
<span style="color: #008080;">80</span> </body>
<span style="color: #008080;">81</span> </html>[root@k8s-master glusterfs]# </pre> 
 </div> 
 <p><span style="font-size: 18px;">访问http://192.168.110.135:8889/,在分析地过程中,我将nginx_pod.yaml地hostPort: 8889改成了8889端口号哦。</span></p> 
 <p></p> 
 <p><a href="http://img.e-com-net.com/image/info8/b7f9cf9b845247b6a71b7ff4f715fca6.jpg" target="_blank"><img src="http://img.e-com-net.com/image/info8/b7f9cf9b845247b6a71b7ff4f715fca6.jpg" alt="Kubernetes(k8s)之分布式文件系统glusterfs_第2张图片" width="650" height="305" style="border:1px solid black;"></a></p> 
 <p></p> 
 <p><span style="font-size: 18px;">游戏是可以玩的,功能也算是搞完了。</span></p> 
</div>
                            </div>
                        </div>
                    </div>
                    <!--PC和WAP自适应版-->
                    <div id="SOHUCS" sid="1295497854005092352"></div>
                    <script type="text/javascript" src="/views/front/js/chanyan.js"></script>
                    <!-- 文章页-底部 动态广告位 -->
                    <div class="youdao-fixed-ad" id="detail_ad_bottom"></div>
                </div>
                <div class="col-md-3">
                    <div class="row" id="ad">
                        <!-- 文章页-右侧1 动态广告位 -->
                        <div id="right-1" class="col-lg-12 col-md-12 col-sm-4 col-xs-4 ad">
                            <div class="youdao-fixed-ad" id="detail_ad_1"> </div>
                        </div>
                        <!-- 文章页-右侧2 动态广告位 -->
                        <div id="right-2" class="col-lg-12 col-md-12 col-sm-4 col-xs-4 ad">
                            <div class="youdao-fixed-ad" id="detail_ad_2"></div>
                        </div>
                        <!-- 文章页-右侧3 动态广告位 -->
                        <div id="right-3" class="col-lg-12 col-md-12 col-sm-4 col-xs-4 ad">
                            <div class="youdao-fixed-ad" id="detail_ad_3"></div>
                        </div>
                    </div>
                </div>
            </div>
        </div>
    </div>
    <div class="container">
        <h4 class="pt20 mb15 mt0 border-top">你可能感兴趣的:(Kubernetes(k8s)之分布式文件系统glusterfs)</h4>
        <div id="paradigm-article-related">
            <div class="recommend-post mb30">
                <ul class="widget-links">
                    <li><a href="/article/1773611795592839168.htm"
                           title="《昼颜》里的日本女人:相遇要万种风情,分手要残忍绝情" target="_blank">《昼颜》里的日本女人:相遇要万种风情,分手要残忍绝情</a>
                        <span class="text-muted">迷影咖啡</span>

                        <div>作者:迷之菌子神奇菇迷影咖啡:一本正经做烘焙,胡说八道聊电影漫天萤火虫消散之时良宵就将过去,人们也说含苞待放的花蕾总会开了又谢,因紧紧相拥而面红耳赤的躯体,便是我们经历过这热爱的证明。夫妻关系介绍《昼颜》是2014年电视剧《昼颜:工作日下午三点的恋人们》的续集,故事发在电视剧情节结束的三年后,讲述了已经恢复独身的纱和偶然与曾经的出轨对象北野重逢后再次陷入感情漩涡的故事。《昼颜》制作灵感源自利佳子在</div>
                    </li>
                    <li><a href="/article/1773611668312489984.htm"
                           title="迎接2019" target="_blank">迎接2019</a>
                        <span class="text-muted">唯有杜康1994</span>

                        <div>告别2018这一年是机遇与挑战,痛苦与喜悦,失去与收获的一年一月:收获了第一份爱情,开始真正想去了解一个人三月:对工作有了更深入的认识,靠自己的力量完成晋升五月:搬家,住进了自己理想的公寓,一间属于自己的屋子。满地的书六月:外调广州,升经理,有了自己的第一个团队。七月:怀着自我否定,第一次完成了部门任务八月:第一个员工流失,痛哭不已明白无不散之筵席九月:员工陆续离开,经济是一切的根本。十月:陪员工</div>
                    </li>
                    <li><a href="/article/1773611163691581440.htm"
                           title="亲子日记之祝姑姥姥生日快乐(282)" target="_blank">亲子日记之祝姑姥姥生日快乐(282)</a>
                        <span class="text-muted">冰心雨露_d504</span>

                        <div>2021年7月18日,周日,晴周日上班的不上班,上学的不上学,全家都属于休息状态,洗衣做饭是上午的主要任务,中午休息一会儿,下午比较晚了出去给梦怡买了二年级上册的口算题卡,然后去参加姑姑的生日聚餐,本来姑姑应该是周一生日,因为周一都要上班,就提前到周日过了,说是过生日其实就是想借此机会一家人聚聚,毕竟平常都忙,没有时间聚在一起,梦怡还给姑姥姥做了生日贺卡,虽然长相一般,重在心意。生日快乐</div>
                    </li>
                    <li><a href="/article/1773604712738783232.htm"
                           title="android 自定义曲线图,Android自定义View——贝赛尔曲线" target="_blank">android 自定义曲线图,Android自定义View——贝赛尔曲线</a>
                        <span class="text-muted">weixin_39767513</span>
<a class="tag" taget="_blank" href="/search/android/1.htm">android</a><a class="tag" taget="_blank" href="/search/%E8%87%AA%E5%AE%9A%E4%B9%89%E6%9B%B2%E7%BA%BF%E5%9B%BE/1.htm">自定义曲线图</a>
                        <div>个人博客:haichenyi.com。感谢关注本文针对有一定自定义View的童鞋,最好对贝赛尔曲线有辣么一丢丢了解,不了解也没关系。花5分钟看一下GcsSloop的安卓自定义View进阶-Path之贝塞尔曲线。本文的最终效果图:最终效果图.gif思路首先他是一个只有上半部分的正弦形状的水波纹,很规则。其次,他这个正弦图左右在移动。然后,就是它这个自定义View,上下也在移动,是慢慢增加的最后,优化</div>
                    </li>
                    <li><a href="/article/1773581925500387328.htm"
                           title="信任" target="_blank">信任</a>
                        <span class="text-muted">饮冰伊乔</span>

                        <div>随着社会的发展,微信和支付宝交易给人们带来了极大的方便,越来越多的人出门都只选择拿一部手机即可,方便安全,可昨天我就遇到了一件比较尴尬的事。昨天傍晚,我从公司出来,感觉有点饿,决定索性吃了饭再回去,来到去过几次的一个店里,如往常一样叫了餐,当时店里吃饭的不多,老板麻利的先去做了,正要扫微信付账的时候发现手机没电了,迷之尴尬,我只好跟老板说不用做了,手机没电了,我身上又没现金,付不了帐了。老板娘很热</div>
                    </li>
                    <li><a href="/article/1773554866065571840.htm"
                           title="Django之Debug篇" target="_blank">Django之Debug篇</a>
                        <span class="text-muted">菜鸟之编程</span>
<a class="tag" taget="_blank" href="/search/Django/1.htm">Django</a><a class="tag" taget="_blank" href="/search/django/1.htm">django</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E5%90%8E%E7%AB%AF/1.htm">后端</a>
                        <div>一、DebugToolBar基本使用1.1、概述Django框架的调试工具栏使用django-debug-toolbar库,是一组可配置的面板,显示有关当前请求/响应的各种调试信息,点击时,显示有关面板内容的更多详细信息。官方文档:DjangoDebugToolbar—DjangoDebugToolbar4.3.0documentation1.2、安装pipinstalldjango-debug-</div>
                    </li>
                    <li><a href="/article/1773520124586819584.htm"
                           title="helm 部署 Kube-Prometheus + Grafana + 钉钉告警部署 Kube-Prometheus" target="_blank">helm 部署 Kube-Prometheus + Grafana + 钉钉告警部署 Kube-Prometheus</a>
                        <span class="text-muted">zxj19880502</span>
<a class="tag" taget="_blank" href="/search/grafana/1.htm">grafana</a><a class="tag" taget="_blank" href="/search/prometheus/1.htm">prometheus</a>
                        <div>背景角色IPK8S版本容器运行时k8s-master-1172.16.16.108v1.24.1containerd://1.6.8k8s-node-1172.16.16.109v1.24.1containerd://1.6.8k8s-node-2172.16.16.110v1.24.1containerd://1.6.8安装kube-prometheusmkdir-p/data/yaml/kub</div>
                    </li>
                    <li><a href="/article/1773515880156692480.htm"
                           title="以高质量调查研究助力组织工作提质增效" target="_blank">以高质量调查研究助力组织工作提质增效</a>
                        <span class="text-muted">木同木同</span>

                        <div>近日,中共中央办公厅印发了《关于在全党大兴调查研究的工作方案》(后简称《方案》),要求各地区各部门结合实际认真贯彻落实。调查研究是我们党的传家宝,是谋事之基、成事之道。组织部门作为党管干部的核心部门,肩负着管党员、管干部、管人才的重要职责,调查研究更是把握组织工作规律、破解组织工作难题的重要办法。要用好调查研究这一法宝,把准政治方向,贯穿问题导向,坚持实事求是,以高质量调查研究助力组织工作提质增效</div>
                    </li>
                    <li><a href="/article/1773514618988199936.htm"
                           title="地低为海人低为王" target="_blank">地低为海人低为王</a>
                        <span class="text-muted">缘梦草</span>

                        <div>地低为海,人低为王。低,高之基,高,低之顶。谦虚才能收益,咄咄逼人会招损。高粱熟了,头会低下。受益反而会深藏不露。捧人之术,把人捧高,然后控制。受控之人,大都是虚荣心作祟。活成别人想象中的样子,像别人想象中那么有面子的活着是悲哀的。穷不可怕,怕的是虚荣心。当你好面子,你就会被控制。一旦受虚荣心控制,你不得不谨小慎微的维护面子,付出多余的能量,希望得到别人的一句赞。面子就是金子打造的牢笼,让你在赞美</div>
                    </li>
                    <li><a href="/article/1773499744581124096.htm"
                           title="计划比盲目做重要" target="_blank">计划比盲目做重要</a>
                        <span class="text-muted">一祉微笑</span>

                        <div>坚持分享第99天。一次次会议提醒着老师们假期余额不足,马上面临停机状态。50多天掐指而过,想想放假时的计划,对照如今的完成情况,感觉差太远。想着好好看书,如今50多天过去了第6本还处在未完待续状态;想着假期好好陪陪孩子,在玩中学一些知识,如今想想,孩子学的真不多;想着暑假坚持跑步,有时还是容易给自己找借口,休息三两天。给这个假期一个综合评价,只能说只完成了计划的百分之五六十。想想为什么临近开学没达</div>
                    </li>
                    <li><a href="/article/1773477172762181632.htm"
                           title="081天 约柜失而复得" target="_blank">081天 约柜失而复得</a>
                        <span class="text-muted">宁宁2018</span>

                        <div>撒迦利亚书4:6他对我说:“这是耶和华指示所罗巴伯的。万军之耶和华说:‘不是倚靠势力,不是倚靠才能,乃是倚靠我的灵方能成事。’</div>
                    </li>
                    <li><a href="/article/1773468344071028736.htm"
                           title="《金文成〈正蒙〉日记124。2020-11-9》" target="_blank">《金文成〈正蒙〉日记124。2020-11-9》</a>
                        <span class="text-muted">金吾生</span>

                        <div>《金文成〈正蒙〉日记124。2020-11-9》今天是庚子丁亥丙子,九月廿四,2020年11月9日星期一。作者篇7【舜之孝,汤、武之武,虽顺逆不同,其为不幸均矣。】船山讲解说,顺,指的是舜的父亲最终受到感化,转变了态度,结局圆满之事;逆,指的是"桀放、纣诛",即,商汤放桀,周武诛纣,都是采用暴力,不是感化方式,所以是逆。顺与逆,都是为了解决弊政,然其差异之大如此,然而,无论圆满还是遗憾,其实他们都</div>
                    </li>
                    <li><a href="/article/1773451453554294784.htm"
                           title="如何把大象装进冰箱的思考 210624" target="_blank">如何把大象装进冰箱的思考 210624</a>
                        <span class="text-muted">大丁_初心生活</span>

                        <div>央视春晚曾有个小品,宋丹丹问赵本山,如何把大象装进冰箱。赵本山说不知道。宋丹丹得意地说,第一步,把冰箱门打开。第二步,把大象装进去。第三步,把冰箱门关上。我相信很多人看了都是一笑了之,我们也从潜意识上给出了否定的答案。樊登老师给出了不一样的解读,他认为这其实是解决问题的经典方法。“把大象装冰箱”,目标十分明确,宋丹丹给出的执行步骤也十分清晰。可赵本山却不会?难道赵本山不知道这三个步骤吗?肯定知道。</div>
                    </li>
                    <li><a href="/article/1773387157499609088.htm"
                           title="道德经第三十四章" target="_blank">道德经第三十四章</a>
                        <span class="text-muted">套马地汉纸</span>

                        <div>道德经第三十四章原文:大道泛兮,其可左右!万物恃之而生,而不辞,功成不名有。爱养万物而不为主,常无欲,可名于小;万物归焉而不为主,可名为大。是以圣人终不为大,故能成其大。译文:大道广泛流行,无所不到。万物依赖它生长而不推辞,有所成就而不自以为有功。养育万物而不自以为主,永远没有私欲,可以说是渺小得很,所以可称它为“小”;物归附于它而它不自以为主宰,可以说极其伟大,所以可称它为“大”。由于它不自以为</div>
                    </li>
                    <li><a href="/article/1773377374927257600.htm"
                           title="Linux学习系列之vim编辑器(一)" target="_blank">Linux学习系列之vim编辑器(一)</a>
                        <span class="text-muted">llibertyll</span>
<a class="tag" taget="_blank" href="/search/linux/1.htm">linux</a><a class="tag" taget="_blank" href="/search/%E5%AD%A6%E4%B9%A0/1.htm">学习</a>
                        <div>vi编辑器的操作模式输入模式—aio等—>命令模式<—:键—末行模式从输入/末行模式切换到命令模式都是需要按ESC键注:a光标后输入,i光标前输入,o直接向下加一行输入,O向上加一行输入在vi编辑器中光标的移动(命令行模式下)键组合(命令)光标的移动$光标移动到当前行的结尾0(零)光标移动到当前行的开始GG光标移动到最后一行gg光标移动到第一行在命令行模式下删除与复制的操作键组合(命令)含义dd删</div>
                    </li>
                    <li><a href="/article/1773374180901847040.htm"
                           title="6/21 最后的编织之观后感" target="_blank">6/21 最后的编织之观后感</a>
                        <span class="text-muted">ChereAlicia</span>

                        <div>今天看了7分钟震撼人心之短篇有种当局者迷旁观者清的感觉在外人眼里看来已经足够但自有人深陷其中无法自拔而那个人很有可能是自己超脱物质寻求自己真正想要的方能得到内心的平息和安宁图片发自App</div>
                    </li>
                    <li><a href="/article/1773367246526152704.htm"
                           title="每天学点成语" target="_blank">每天学点成语</a>
                        <span class="text-muted">余子烨</span>

                        <div>1、口诀:功败垂成未成功,不以为然不赞同;火中取栗被利用,罚不当罪处罚重;2、解释:功败垂成:事情接近成功的时候却遭到了失败。出自:《三国志·杨阜传》:“弃垂成之功,陷不义之名,阜以死守之。”不以为然:不认为是对的。表示不同意或否定。出自:宋·苏轼《再乞罢详定役法状》:“右臣先曾奏论前衙一役,只当招募,不当定差,执政不以为然。”火中取栗:一只猴子和一只猫看见炉火中烤着栗子,猴子叫猫去偷,猫用爪子从</div>
                    </li>
                    <li><a href="/article/1773366674838323200.htm"
                           title="k8s入门到实战(十)—— CronJob详细介绍及使用示例" target="_blank">k8s入门到实战(十)—— CronJob详细介绍及使用示例</a>
                        <span class="text-muted">一弓虽</span>
<a class="tag" taget="_blank" href="/search/k8s%E5%AD%A6%E4%B9%A0/1.htm">k8s学习</a><a class="tag" taget="_blank" href="/search/kubernetes/1.htm">kubernetes</a><a class="tag" taget="_blank" href="/search/%E5%AE%B9%E5%99%A8/1.htm">容器</a><a class="tag" taget="_blank" href="/search/%E4%BA%91%E5%8E%9F%E7%94%9F/1.htm">云原生</a>
                        <div>CronJob什么是CronJob在k8s中,CronJob是一种用于定期执行任务的资源对象。它基于Cron表达式,允许您在指定的时间间隔内自动运行容器化的任务。CronJob可以定义以下属性:schedule:指定任务执行的时间表,使用标准的Cron表达式语法。例如,“0****”表示每小时执行一次任务。jobTemplate:定义要执行的任务的模板,通常是一个Pod模板。这个模板包含了任务所需</div>
                    </li>
                    <li><a href="/article/1773349852445474816.htm"
                           title="2023-11-07" target="_blank">2023-11-07</a>
                        <span class="text-muted">碎碎念念的每一天</span>

                        <div>忽然开始自卑了,感觉自己无论各个方面都太平平,读书不多,家务做不好,体重控制的不好,早起也困难,起来又不知道该做什么,没有一个可以为之奋斗的目标,也没有很好的交际圈,感觉自己就像一只在笼子里的鸟,想飞也飞不高……我想改变这样的现状,可该如何呢?其实都是我不够努力,没有争分夺秒,没有自律,这些都是我未来跟人拉开差距的原因,所以,我不应该只是自卑,自卑过头会变成焦虑,我应该努力去改变,哪怕改变一小点,</div>
                    </li>
                    <li><a href="/article/1773342128299114496.htm"
                           title="stm32之GPIO寄存器" target="_blank">stm32之GPIO寄存器</a>
                        <span class="text-muted">luofengmacheng</span>
<a class="tag" taget="_blank" href="/search/%E5%B5%8C%E5%85%A5%E5%BC%8F/1.htm">嵌入式</a><a class="tag" taget="_blank" href="/search/stm32/1.htm">stm32</a><a class="tag" taget="_blank" href="/search/%E5%B5%8C%E5%85%A5%E5%BC%8F%E7%A1%AC%E4%BB%B6/1.htm">嵌入式硬件</a><a class="tag" taget="_blank" href="/search/%E5%8D%95%E7%89%87%E6%9C%BA/1.htm">单片机</a>
                        <div>文章目录1背景2GPIO寄存器的类型2.1端口配置寄存器2.2设置/清除寄存器和位清除寄存器3总结1背景C51单片机在进行数据的输入输出时,是直接操作与外部引脚关联的内部寄存器,例如,当设置P2_1为0时,就是将外部引脚的P21引脚设置为低电平,当读取P2_1时,就是读取P21的电平。与之类似,stm32芯片内部也有很多用于输入输出的寄存器,这些寄存器也是用于操作外部引脚,但是比C51单片机复杂很</div>
                    </li>
                    <li><a href="/article/1773340139582455808.htm"
                           title="《老子》笔记19 2018-10-28" target="_blank">《老子》笔记19 2018-10-28</a>
                        <span class="text-muted">海上明月共</span>

                        <div>第二十二章[原文]曲则全,枉则直,洼则盈,敝则新,少则得,多则惑。是以圣人抱一为天下式。不自见,故明;不自是,故彰,不自伐,故有功;不自矜,故长。夫唯不争,故天下莫能与之争。古之所谓"曲则全"者,岂虚言哉?诚全而归之。[译文]委曲便会保全,屈枉便会直伸;低洼便会充盈,陈旧便会更新;少取便会获得,贪多便会迷惑。所以有道的人坚守这一原则作为天下事理的范式,不自我表扬,反能显明;不自以为是,反能是非彰明</div>
                    </li>
                    <li><a href="/article/1773338753864105984.htm"
                           title="爱心账户之存钱日记11.20" target="_blank">爱心账户之存钱日记11.20</a>
                        <span class="text-muted">静_d595</span>

                        <div>新沙发终于到了,满心欢喜,可旧沙发的处理又是个头疼的问题,免费送都没有人要。临时想起来公司那个收垃圾的,看看付费能不能拉走。对方说随便给点都行,我又拿不准,于是就想说88把。对方说给两包烟就行了,我说还是给钱吧,也不想欠人情。如果请专门搬东西的估计得200加了,还算是比较顺利,刚好想到他。选了12元存进我的爱心账户!</div>
                    </li>
                    <li><a href="/article/1773256662182854656.htm"
                           title="大话设计模式之工厂模式" target="_blank">大话设计模式之工厂模式</a>
                        <span class="text-muted">码农客栈</span>
<a class="tag" taget="_blank" href="/search/%E8%AE%BE%E8%AE%A1%E6%A8%A1%E5%BC%8F/1.htm">设计模式</a><a class="tag" taget="_blank" href="/search/%E5%BC%80%E5%8F%91%E8%AF%AD%E8%A8%80/1.htm">开发语言</a>
                        <div>工厂模式(FactoryPattern)是一种创建型设计模式,它提供了一种创建对象的最佳方式,而无需指定将要创建的对象的确切类。通过使用工厂模式,我们可以将对象的创建和使用分离,从而使代码更具灵活性和可维护性。在工厂模式中,通常会有三种角色:工厂接口(FactoryInterface):声明了一个用于创建对象的接口,这样所有具体工厂类都必须实现该接口。具体工厂(ConcreteFactory):实</div>
                    </li>
                    <li><a href="/article/1772792265987653632.htm"
                           title="大话设计模式之代理模式" target="_blank">大话设计模式之代理模式</a>
                        <span class="text-muted">码农客栈</span>
<a class="tag" taget="_blank" href="/search/%E8%AE%BE%E8%AE%A1%E6%A8%A1%E5%BC%8F/1.htm">设计模式</a><a class="tag" taget="_blank" href="/search/%E4%BB%A3%E7%90%86%E6%A8%A1%E5%BC%8F/1.htm">代理模式</a>
                        <div>代理模式(ProxyPattern)是一种结构型设计模式,它允许通过代理对象控制对另一个对象的访问。代理对象充当客户端和实际对象之间的中介,客户端通过代理对象间接访问实际对象,从而可以在访问控制、缓存、延迟加载等方面提供额外的功能。在代理模式中,通常会有三种角色:Subject(抽象主题):声明了真实对象和代理对象的共同接口,这样在任何使用真实对象的地方都可以使用代理对象。RealSubject(</div>
                    </li>
                    <li><a href="/article/1772789495305928704.htm"
                           title="【1.1 编程基础之输入输出】09. 字符菱形" target="_blank">【1.1 编程基础之输入输出】09. 字符菱形</a>
                        <span class="text-muted">青少年编程小助手_Python</span>
<a class="tag" taget="_blank" href="/search/Openjudge%E9%A2%98%E7%9B%AE%E8%A7%A3%E6%9E%90/1.htm">Openjudge题目解析</a><a class="tag" taget="_blank" href="/search/%E7%AE%97%E6%B3%95/1.htm">算法</a><a class="tag" taget="_blank" href="/search/%E9%9D%92%E5%B0%91%E5%B9%B4%E7%BC%96%E7%A8%8B/1.htm">青少年编程</a><a class="tag" taget="_blank" href="/search/%E7%94%B5%E5%AD%90%E5%AD%A6%E4%BC%9A/1.htm">电子学会</a><a class="tag" taget="_blank" href="/search/%E7%AD%89%E7%BA%A7%E8%80%83%E8%AF%95/1.htm">等级考试</a><a class="tag" taget="_blank" href="/search/gesp/1.htm">gesp</a>
                        <div>09:字符菱形总时间限制:1000ms内存限制:65536kB描述给定一个字符,用它构造一个对角线长5个字符,倾斜放置的菱形。输入输入只有一行,包含一个字符。输出该字符构成的菱形。样例输入*样例输出*************参考程序(1)C语言#includeintmain(){charc;scanf("%c",&c);printf("%c\n",c);printf("%c%c%c\n",c,c,</div>
                    </li>
                    <li><a href="/article/1772769102092369920.htm"
                           title="剩28天19年12月3日星期二 晴朗" target="_blank">剩28天19年12月3日星期二 晴朗</a>
                        <span class="text-muted">岁月流转足迹可循</span>

                        <div>图片发自App图片发自App图片发自App图片发自App昨天升完旗,去操场上走走,才发现霜是如此之重,脚踩上去,都有凝粒!天是越来越冷了!不过晴空净爽,空气清冽,一呼一吸之间很是舒服!中午写了几个大字,童爸说笔画还有些软,一语中的。需用心提升,贵在坚持坚持吧!下午做了张丽华的顺车,捎我到陈仓元,再坐了3路到西关,费时50分钟!晚上吃完饭送童去学校,到八点半下楼运动半小时,跳了500个绳,弯弯腰!感</div>
                    </li>
                    <li><a href="/article/1772765062977617920.htm"
                           title="很久以前的故事 138" target="_blank">很久以前的故事 138</a>
                        <span class="text-muted">余间上古</span>

                        <div>偶遇机械人大公子镇静下来继续说:“你可知所谓的拍卖会是要杀掉一些人,某些人斗不过一个映山城,天下虽大,从此后恐无容身之地。”他语气一缓:“如果你肯联手,我不但不计较你今日之举,杀手阁随时恭候大驾光临,从此多了一个方便之门。”“我只是寻找路彦而已......”薛莹笑道直呼其名:“剑二,用尽心思糊弄别人,独行者经常内斗再正常不过!你不过是一个道修,踏入神明之境而已,理解不到独行者的真谛……”薛莹言尽意</div>
                    </li>
                    <li><a href="/article/1772714654930370560.htm"
                           title="因你" target="_blank">因你</a>
                        <span class="text-muted">秋之枫520</span>

                        <div>空气中,咖啡味弥散那是,一种我爱的香醇与你分别后,我的咖啡再没加过冰糖粉与牛奶品着苦涩,如同我生命的味道你若来了,倒给你半杯加糖加奶,空气多了两份香甜若你走了,我又会是一满杯一切如初,黑夜依旧宁静只有醇香陪着,久久不散因你,咖啡才会有苦有甜因你,我的小诗才有了主角因你,黑夜冲了杯喝不尽的咖啡因你,黑夜有了写不完的零碎小诗因你,我爱上了黑夜的这份宁静秋之枫。(原作)</div>
                    </li>
                    <li><a href="/article/1772702560600260608.htm"
                           title="湘容儿成长日记第744天之只要不放弃,终究会遇见" target="_blank">湘容儿成长日记第744天之只要不放弃,终究会遇见</a>
                        <span class="text-muted">墨mo晴</span>

                        <div>2020年10月30号亲子共读地點:床边亲子共读时间:晚上亲子共读时长:35分钟亲子共读内容:芭比姐妹与小马、芭比之完美圣诞、三字经045-047今天收获还是很大,因为我解决了一个一直没有解决的问题。那就是我的手机内存。我的手机是64G的,界面上看到的软件又没有太多的东西被我删了,又删手机里面的图片,竟然没有50张。可就是经常的提醒我内存不足,内存不足。我和同事一起探讨过,我和我先生说过,可是终究</div>
                    </li>
                    <li><a href="/article/1772693453457653760.htm"
                           title="零基础机器学习(5)之线性回归模型的性能评估" target="_blank">零基础机器学习(5)之线性回归模型的性能评估</a>
                        <span class="text-muted">一只特立独行猪</span>
<a class="tag" taget="_blank" href="/search/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0/1.htm">机器学习</a><a class="tag" taget="_blank" href="/search/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0/1.htm">机器学习</a><a class="tag" taget="_blank" href="/search/%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92/1.htm">线性回归</a><a class="tag" taget="_blank" href="/search/%E4%BA%BA%E5%B7%A5%E6%99%BA%E8%83%BD/1.htm">人工智能</a>
                        <div>文章目录线性回归模型的性能评估1.举例1-单一特征2.举例2-多特征线性回归模型的性能评估评估线性回归模型时,首先要建立评估的测试数据集(测试集不能与训练集相同),然后选择合适的评估方法,实现对线性回归模型的评估。回归任务中最常用的评估方法有均方误差、均方根误差和预测准确率(确定系数)。1.举例1-单一特征分别对两个模型进行评估,输入的测试集如表所示。面积/(m2)售价/(万元)面积/(m2)售价</div>
                    </li>
                                <li><a href="/article/17.htm"
                                       title="分享100个最新免费的高匿HTTP代理IP" target="_blank">分享100个最新免费的高匿HTTP代理IP</a>
                                    <span class="text-muted">mcj8089</span>
<a class="tag" taget="_blank" href="/search/%E4%BB%A3%E7%90%86IP/1.htm">代理IP</a><a class="tag" taget="_blank" href="/search/%E4%BB%A3%E7%90%86%E6%9C%8D%E5%8A%A1%E5%99%A8/1.htm">代理服务器</a><a class="tag" taget="_blank" href="/search/%E5%8C%BF%E5%90%8D%E4%BB%A3%E7%90%86/1.htm">匿名代理</a><a class="tag" taget="_blank" href="/search/%E5%85%8D%E8%B4%B9%E4%BB%A3%E7%90%86IP/1.htm">免费代理IP</a><a class="tag" taget="_blank" href="/search/%E6%9C%80%E6%96%B0%E4%BB%A3%E7%90%86IP/1.htm">最新代理IP</a>
                                    <div>  
推荐两个代理IP网站: 
  
1. 全网代理IP:http://proxy.goubanjia.com/ 
  
2. 敲代码免费IP:http://ip.qiaodm.com/ 
  
  
120.198.243.130:80,中国/广东省
58.251.78.71:8088,中国/广东省
183.207.228.22:83,中国/</div>
                                </li>
                                <li><a href="/article/144.htm"
                                       title="mysql高级特性之数据分区" target="_blank">mysql高级特性之数据分区</a>
                                    <span class="text-muted">annan211</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a><a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84/1.htm">数据结构</a><a class="tag" taget="_blank" href="/search/mongodb/1.htm">mongodb</a><a class="tag" taget="_blank" href="/search/%E5%88%86%E5%8C%BA/1.htm">分区</a><a class="tag" taget="_blank" href="/search/mysql/1.htm">mysql</a>
                                    <div>


mysql高级特性
  1 以存储引擎的角度分析,分区表和物理表没有区别。是按照一定的规则将数据分别存储的逻辑设计。器底层是由多个物理字表组成。
  
  2 分区的原理
     分区表由多个相关的底层表实现,这些底层表也是由句柄对象表示,所以我们可以直接访问各个分区。存储引擎管理分区的各个底层
	 表和管理普通表一样(所有底层表都必须使用相同的存储引擎),分区表的索引只是</div>
                                </li>
                                <li><a href="/article/271.htm"
                                       title="JS采用正则表达式简单获取URL地址栏参数" target="_blank">JS采用正则表达式简单获取URL地址栏参数</a>
                                    <span class="text-muted">chiangfai</span>
<a class="tag" taget="_blank" href="/search/js/1.htm">js</a><a class="tag" taget="_blank" href="/search/%E5%9C%B0%E5%9D%80%E6%A0%8F%E5%8F%82%E6%95%B0%E8%8E%B7%E5%8F%96/1.htm">地址栏参数获取</a>
                                    <div>GetUrlParam:function GetUrlParam(param){
		var reg = new RegExp("(^|&)"+ param +"=([^&]*)(&|$)");
		var r = window.location.search.substr(1).match(reg);
		if(r!=null</div>
                                </li>
                                <li><a href="/article/398.htm"
                                       title="怎样将数据表拷贝到powerdesigner (本地数据库表)" target="_blank">怎样将数据表拷贝到powerdesigner (本地数据库表)</a>
                                    <span class="text-muted">Array_06</span>
<a class="tag" taget="_blank" href="/search/powerDesigner/1.htm">powerDesigner</a>
                                    <div>================================================== 
 
1、打开PowerDesigner12,在菜单中按照如下方式进行操作 
file->Reverse Engineer->DataBase 
点击后,弹出 New Physical Data Model 的对话框 
2、在General选项卡中 
Model name:模板名字,自</div>
                                </li>
                                <li><a href="/article/525.htm"
                                       title="logbackのhelloworld" target="_blank">logbackのhelloworld</a>
                                    <span class="text-muted">飞翔的马甲</span>
<a class="tag" taget="_blank" href="/search/%E6%97%A5%E5%BF%97/1.htm">日志</a><a class="tag" taget="_blank" href="/search/logback/1.htm">logback</a>
                                    <div>一、概述 
 
1.日志是啥? 
当我是个逗比的时候我是这么理解的:log.debug()代替了system.out.print(); 
当我项目工作时,以为是一堆得.log文件。 
这两天项目发布新版本,比较轻松,决定好好地研究下日志以及logback。 
传送门1:日志的作用与方法: 
http://www.infoq.com/cn/articles/why-and-how-log 
上面的作</div>
                                </li>
                                <li><a href="/article/652.htm"
                                       title="新浪微博爬虫模拟登陆" target="_blank">新浪微博爬虫模拟登陆</a>
                                    <span class="text-muted">随意而生</span>
<a class="tag" taget="_blank" href="/search/%E6%96%B0%E6%B5%AA%E5%BE%AE%E5%8D%9A/1.htm">新浪微博</a>
                                    <div>转载自:http://hi.baidu.com/erliang20088/item/251db4b040b8ce58ba0e1235 
  
  
近来由于毕设需要,重新修改了新浪微博爬虫废了不少劲,希望下边的总结能够帮助后来的同学们。 
     现行版的模拟登陆与以前相比,最大的改动在于cookie获取时候的模拟url的请求</div>
                                </li>
                                <li><a href="/article/779.htm"
                                       title="synchronized" target="_blank">synchronized</a>
                                    <span class="text-muted">香水浓</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a><a class="tag" taget="_blank" href="/search/thread/1.htm">thread</a>
                                    <div>    Java语言的关键字,可用来给对象和方法或者代码块加锁,当它锁定一个方法或者一个代码块的时候,同一时刻最多只有一个线程执行这段代码。当两个并发线程访问同一个对象object中的这个加锁同步代码块时,一个时间内只能有一个线程得到执行。另一个线程必须等待当前线程执行完这个代码块以后才能执行该代码块。然而,当一个线程访问object的一个加锁代码块时,另一个线程仍然</div>
                                </li>
                                <li><a href="/article/906.htm"
                                       title="maven 简单实用教程" target="_blank">maven 简单实用教程</a>
                                    <span class="text-muted">AdyZhang</span>
<a class="tag" taget="_blank" href="/search/maven/1.htm">maven</a>
                                    <div>1. Maven介绍  1.1. 简介 java编写的用于构建系统的自动化工具。目前版本是2.0.9,注意maven2和maven1有很大区别,阅读第三方文档时需要区分版本。 1.2. Maven资源 见官方网站;The 5 minute test,官方简易入门文档;Getting Started Tutorial,官方入门文档;Build Coo</div>
                                </li>
                                <li><a href="/article/1033.htm"
                                       title="Android 通过 intent传值获得null" target="_blank">Android 通过 intent传值获得null</a>
                                    <span class="text-muted">aijuans</span>
<a class="tag" taget="_blank" href="/search/android/1.htm">android</a>
                                    <div>我在通过intent 获得传递兑现过的时候报错,空指针,我是getMap方法进行传值,代码如下        1   2   3   4   5   6   7   8   9      
public 
void 
getMap(View view){       
         
Intent i =</div>
                                </li>
                                <li><a href="/article/1160.htm"
                                       title="apache 做代理 报如下错误:The proxy server received an invalid response from an upstream" target="_blank">apache 做代理 报如下错误:The proxy server received an invalid response from an upstream</a>
                                    <span class="text-muted">baalwolf</span>
<a class="tag" taget="_blank" href="/search/response/1.htm">response</a>
                                    <div>网站配置是apache+tomcat,tomcat没有报错,apache报错是: 
The proxy server received an invalid response from an upstream server. The proxy server could not handle the request GET /. Reason: Error reading fr</div>
                                </li>
                                <li><a href="/article/1287.htm"
                                       title="Tomcat6 内存和线程配置" target="_blank">Tomcat6 内存和线程配置</a>
                                    <span class="text-muted">BigBird2012</span>
<a class="tag" taget="_blank" href="/search/tomcat6/1.htm">tomcat6</a>
                                    <div>1、修改启动时内存参数、并指定JVM时区 (在windows server 2008 下时间少了8个小时) 
在Tomcat上运行j2ee项目代码时,经常会出现内存溢出的情况,解决办法是在系统参数中增加系统参数:  
window下, 在catalina.bat最前面 
  set JAVA_OPTS=-XX:PermSize=64M -XX:MaxPermSize=128m -Xms5</div>
                                </li>
                                <li><a href="/article/1414.htm"
                                       title="Karam与TDD" target="_blank">Karam与TDD</a>
                                    <span class="text-muted">bijian1013</span>
<a class="tag" taget="_blank" href="/search/Karam/1.htm">Karam</a><a class="tag" taget="_blank" href="/search/TDD/1.htm">TDD</a>
                                    <div>一.TDD 
        测试驱动开发(Test-Driven Development,TDD)是一种敏捷(AGILE)开发方法论,它把开发流程倒转了过来,在进行代码实现之前,首先保证编写测试用例,从而用测试来驱动开发(而不是把测试作为一项验证工具来使用)。 
        TDD的原则很简单: 
a.只有当某个</div>
                                </li>
                                <li><a href="/article/1541.htm"
                                       title="[Zookeeper学习笔记之七]Zookeeper源代码分析之Zookeeper.States" target="_blank">[Zookeeper学习笔记之七]Zookeeper源代码分析之Zookeeper.States</a>
                                    <span class="text-muted">bit1129</span>
<a class="tag" taget="_blank" href="/search/zookeeper/1.htm">zookeeper</a>
                                    <div>    public enum States {
        CONNECTING, //Zookeeper服务器不可用,客户端处于尝试链接状态
        ASSOCIATING, //???
        CONNECTED, //链接建立,可以与Zookeeper服务器正常通信
        CONNECTEDREADONLY, //处于只读状态的链接状态,只读模式可以在</div>
                                </li>
                                <li><a href="/article/1668.htm"
                                       title="【Scala十四】Scala核心八:闭包" target="_blank">【Scala十四】Scala核心八:闭包</a>
                                    <span class="text-muted">bit1129</span>
<a class="tag" taget="_blank" href="/search/scala/1.htm">scala</a>
                                    <div>Free variable A free variable of an expression is a variable that’s used inside the expression but not defined inside the expression. For instance, in the function literal expression (x: Int) => (x</div>
                                </li>
                                <li><a href="/article/1795.htm"
                                       title="android发送json并解析返回json" target="_blank">android发送json并解析返回json</a>
                                    <span class="text-muted">ronin47</span>
<a class="tag" taget="_blank" href="/search/android/1.htm">android</a>
                                    <div>package com.http.test; 
 
 
import org.apache.http.HttpResponse; 
import org.apache.http.HttpStatus; 
import org.apache.http.client.HttpClient; 
import org.apache.http.client.methods.HttpGet; 
import </div>
                                </li>
                                <li><a href="/article/1922.htm"
                                       title="一份IT实习生的总结" target="_blank">一份IT实习生的总结</a>
                                    <span class="text-muted">brotherlamp</span>
<a class="tag" taget="_blank" href="/search/PHP/1.htm">PHP</a><a class="tag" taget="_blank" href="/search/php%E8%B5%84%E6%96%99/1.htm">php资料</a><a class="tag" taget="_blank" href="/search/php%E6%95%99%E7%A8%8B/1.htm">php教程</a><a class="tag" taget="_blank" href="/search/php%E5%9F%B9%E8%AE%AD/1.htm">php培训</a><a class="tag" taget="_blank" href="/search/php%E8%A7%86%E9%A2%91/1.htm">php视频</a>
                                    <div>今天突然发现在不知不觉中自己已经实习了 3 个月了,现在可能不算是真正意义上的实习吧,因为现在自己才大三,在这边撸代码的同时还要考虑到学校的功课跟期末考试。让我震惊的是,我完全想不到在这 3 个月里我到底学到了什么,这是一件多么悲催的事情啊。同时我对我应该 get 到什么新技能也很迷茫。所以今晚还是总结下把,让自己在接下来的实习生活有更加明确的方向。最后感谢工作室给我们几个人这个机会让我们提前出来</div>
                                </li>
                                <li><a href="/article/2049.htm"
                                       title="据说是2012年10月人人网校招的一道笔试题-给出一个重物重量为X,另外提供的小砝码重量分别为1,3,9。。。3^N。 将重物放到天平左侧,问在两边如何添加砝码" target="_blank">据说是2012年10月人人网校招的一道笔试题-给出一个重物重量为X,另外提供的小砝码重量分别为1,3,9。。。3^N。 将重物放到天平左侧,问在两边如何添加砝码</a>
                                    <span class="text-muted">bylijinnan</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a>
                                    <div>
public class ScalesBalance {

	/**
	 * 题目:
	 * 给出一个重物重量为X,另外提供的小砝码重量分别为1,3,9。。。3^N。 (假设N无限大,但一种重量的砝码只有一个)
	 * 将重物放到天平左侧,问在两边如何添加砝码使两边平衡
	 * 
	 * 分析:
	 * 三进制
	 * 我们约定括号表示里面的数是三进制,例如 47=(1202</div>
                                </li>
                                <li><a href="/article/2176.htm"
                                       title="dom4j最常用最简单的方法" target="_blank">dom4j最常用最简单的方法</a>
                                    <span class="text-muted">chiangfai</span>
<a class="tag" taget="_blank" href="/search/dom4j/1.htm">dom4j</a>
                                    <div>要使用dom4j读写XML文档,需要先下载dom4j包,dom4j官方网站在 http://www.dom4j.org/目前最新dom4j包下载地址:http://nchc.dl.sourceforge.net/sourceforge/dom4j/dom4j-1.6.1.zip 
解开后有两个包,仅操作XML文档的话把dom4j-1.6.1.jar加入工程就可以了,如果需要使用XPath的话还需要</div>
                                </li>
                                <li><a href="/article/2303.htm"
                                       title="简单HBase笔记" target="_blank">简单HBase笔记</a>
                                    <span class="text-muted">chenchao051</span>
<a class="tag" taget="_blank" href="/search/hbase/1.htm">hbase</a>
                                    <div> 一、Client-side write buffer 客户端缓存请求   描述:可以缓存客户端的请求,以此来减少RPC的次数,但是缓存只是被存在一个ArrayList中,所以多线程访问时不安全的。   可以使用getWriteBuffer()方法来取得客户端缓存中的数据。 默认关闭。      二、Scan的Caching   描述: next( )方法请求一行就要使用一次RPC,即使</div>
                                </li>
                                <li><a href="/article/2430.htm"
                                       title="mysqldump导出时出现when doing LOCK TABLES" target="_blank">mysqldump导出时出现when doing LOCK TABLES</a>
                                    <span class="text-muted">daizj</span>
<a class="tag" taget="_blank" href="/search/mysql/1.htm">mysql</a><a class="tag" taget="_blank" href="/search/mysqdump/1.htm">mysqdump</a><a class="tag" taget="_blank" href="/search/%E5%AF%BC%E6%95%B0%E6%8D%AE/1.htm">导数据</a>
                                    <div>  执行 mysqldump -uxxx -pxxx -hxxx -Pxxxx database tablename > tablename.sql  
导出表时,会报 
 
mysqldump: Got error: 1044: Access denied for user 'xxx'@'xxx' to database 'xxx' when doing LOCK TABLES 
 
解决</div>
                                </li>
                                <li><a href="/article/2557.htm"
                                       title="CSS渲染原理" target="_blank">CSS渲染原理</a>
                                    <span class="text-muted">dcj3sjt126com</span>
<a class="tag" taget="_blank" href="/search/Web/1.htm">Web</a>
                                    <div>   从事Web前端开发的人都与CSS打交道很多,有的人也许不知道css是怎么去工作的,写出来的css浏览器是怎么样去解析的呢?当这个成为我们提高css水平的一个瓶颈时,是否应该多了解一下呢?   
  
       一、浏览器的发展与CSS      
  
   </div>
                                </li>
                                <li><a href="/article/2684.htm"
                                       title="《阿甘正传》台词" target="_blank">《阿甘正传》台词</a>
                                    <span class="text-muted">dcj3sjt126com</span>

                                    <div>Part Ⅰ: 
《阿甘正传》Forrest Gump经典中英文对白 
Forrest: Hello! My names Forrest. Forrest Gump. You wanna Chocolate? I could eat about a million and a half othese. My momma always said life was like a box ochocol</div>
                                </li>
                                <li><a href="/article/2811.htm"
                                       title="Java处理JSON" target="_blank">Java处理JSON</a>
                                    <span class="text-muted">dyy_gusi</span>
<a class="tag" taget="_blank" href="/search/json/1.htm">json</a>
                                    <div>Json在数据传输中很好用,原因是JSON 比 XML 更小、更快,更易解析。 
在Java程序中,如何使用处理JSON,现在有很多工具可以处理,比较流行常用的是google的gson和alibaba的fastjson,具体使用如下: 
1、读取json然后处理 
class ReadJSON
{
    public static void main(String[] args)
    </div>
                                </li>
                                <li><a href="/article/2938.htm"
                                       title="win7下nginx和php的配置" target="_blank">win7下nginx和php的配置</a>
                                    <span class="text-muted">geeksun</span>
<a class="tag" taget="_blank" href="/search/nginx/1.htm">nginx</a>
                                    <div>1.  安装包准备 
nginx :  从nginx.org下载nginx-1.8.0.zip 
php: 从php.net下载php-5.6.10-Win32-VC11-x64.zip, php是免安装文件。 
RunHiddenConsole: 用于隐藏命令行窗口 
  
2. 配置 
# java用8080端口做应用服务器,nginx反向代理到这个端口即可 
p</div>
                                </li>
                                <li><a href="/article/3065.htm"
                                       title="基于2.8版本redis配置文件中文解释" target="_blank">基于2.8版本redis配置文件中文解释</a>
                                    <span class="text-muted">hongtoushizi</span>
<a class="tag" taget="_blank" href="/search/redis/1.htm">redis</a>
                                    <div>转载自: http://wangwei007.blog.51cto.com/68019/1548167 
       在Redis中直接启动redis-server服务时, 采用的是默认的配置文件。采用redis-server   xxx.conf 这样的方式可以按照指定的配置文件来运行Redis服务。下面是Redis2.8.9的配置文</div>
                                </li>
                                <li><a href="/article/3192.htm"
                                       title="第五章 常用Lua开发库3-模板渲染" target="_blank">第五章 常用Lua开发库3-模板渲染</a>
                                    <span class="text-muted">jinnianshilongnian</span>
<a class="tag" taget="_blank" href="/search/nginx/1.htm">nginx</a><a class="tag" taget="_blank" href="/search/lua/1.htm">lua</a>
                                    <div>动态web网页开发是Web开发中一个常见的场景,比如像京东商品详情页,其页面逻辑是非常复杂的,需要使用模板技术来实现。而Lua中也有许多模板引擎,如目前我在使用的lua-resty-template,可以渲染很复杂的页面,借助LuaJIT其性能也是可以接受的。 
  
如果学习过JavaEE中的servlet和JSP的话,应该知道JSP模板最终会被翻译成Servlet来执行;而lua-r</div>
                                </li>
                                <li><a href="/article/3319.htm"
                                       title="JZSearch大数据搜索引擎" target="_blank">JZSearch大数据搜索引擎</a>
                                    <span class="text-muted">颠覆者</span>
<a class="tag" taget="_blank" href="/search/JavaScript/1.htm">JavaScript</a>
                                    <div>系统简介: 
 
 大数据的特点有四个层面:第一,数据体量巨大。从TB级别,跃升到PB级别;第二,数据类型繁多。网络日志、视频、图片、地理位置信息等等。第三,价值密度低。以视频为例,连续不间断监控过程中,可能有用的数据仅仅有一两秒。第四,处理速度快。最后这一点也是和传统的数据挖掘技术有着本质的不同。业界将其归纳为4个“V”——Volume,Variety,Value,Velocity。大数据搜索引</div>
                                </li>
                                <li><a href="/article/3446.htm"
                                       title="10招让你成为杰出的Java程序员" target="_blank">10招让你成为杰出的Java程序员</a>
                                    <span class="text-muted">pda158</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a><a class="tag" taget="_blank" href="/search/%E7%BC%96%E7%A8%8B/1.htm">编程</a><a class="tag" taget="_blank" href="/search/%E6%A1%86%E6%9E%B6/1.htm">框架</a>
                                    <div>如果你是一个热衷于技术的  
Java 程序员, 那么下面的 10 个要点可以让你在众多 Java 开发人员中脱颖而出。      
1. 拥有扎实的基础和深刻理解 OO 原则     对于 Java 程序员,深刻理解 Object Oriented Programming(面向对象编程)这一概念是必须的。没有 OOPS 的坚实基础,就领会不了像 Java 这些面向对象编程语言</div>
                                </li>
                                <li><a href="/article/3573.htm"
                                       title="tomcat之oracle连接池配置" target="_blank">tomcat之oracle连接池配置</a>
                                    <span class="text-muted">小网客</span>
<a class="tag" taget="_blank" href="/search/oracle/1.htm">oracle</a>
                                    <div>tomcat版本7.0 
配置oracle连接池方式: 
修改tomcat的server.xml配置文件: 
 <GlobalNamingResources>
 
			  <Resource name="utermdatasource" auth="Container"  
 type="javax.sql.DataSou</div>
                                </li>
                                <li><a href="/article/3700.htm"
                                       title="Oracle 分页算法汇总" target="_blank">Oracle 分页算法汇总</a>
                                    <span class="text-muted">vipbooks</span>
<a class="tag" taget="_blank" href="/search/oracle/1.htm">oracle</a><a class="tag" taget="_blank" href="/search/sql/1.htm">sql</a><a class="tag" taget="_blank" href="/search/%E7%AE%97%E6%B3%95/1.htm">算法</a><a class="tag" taget="_blank" href="/search/.net/1.htm">.net</a>
                                    <div>    这是我找到的一些关于Oracle分页的算法,大家那里还有没有其他好的算法没?我们大家一起分享一下! 
 
 

-- Oracle 分页算法一
select * from (
       select page.*,rownum rn from (select * from help) page 
       -- 20 = (currentPag</div>
                                </li>
                </ul>
            </div>
        </div>
    </div>

<div>
    <div class="container">
        <div class="indexes">
            <strong>按字母分类:</strong>
            <a href="/tags/A/1.htm" target="_blank">A</a><a href="/tags/B/1.htm" target="_blank">B</a><a href="/tags/C/1.htm" target="_blank">C</a><a
                href="/tags/D/1.htm" target="_blank">D</a><a href="/tags/E/1.htm" target="_blank">E</a><a href="/tags/F/1.htm" target="_blank">F</a><a
                href="/tags/G/1.htm" target="_blank">G</a><a href="/tags/H/1.htm" target="_blank">H</a><a href="/tags/I/1.htm" target="_blank">I</a><a
                href="/tags/J/1.htm" target="_blank">J</a><a href="/tags/K/1.htm" target="_blank">K</a><a href="/tags/L/1.htm" target="_blank">L</a><a
                href="/tags/M/1.htm" target="_blank">M</a><a href="/tags/N/1.htm" target="_blank">N</a><a href="/tags/O/1.htm" target="_blank">O</a><a
                href="/tags/P/1.htm" target="_blank">P</a><a href="/tags/Q/1.htm" target="_blank">Q</a><a href="/tags/R/1.htm" target="_blank">R</a><a
                href="/tags/S/1.htm" target="_blank">S</a><a href="/tags/T/1.htm" target="_blank">T</a><a href="/tags/U/1.htm" target="_blank">U</a><a
                href="/tags/V/1.htm" target="_blank">V</a><a href="/tags/W/1.htm" target="_blank">W</a><a href="/tags/X/1.htm" target="_blank">X</a><a
                href="/tags/Y/1.htm" target="_blank">Y</a><a href="/tags/Z/1.htm" target="_blank">Z</a><a href="/tags/0/1.htm" target="_blank">其他</a>
        </div>
    </div>
</div>
<footer id="footer" class="mb30 mt30">
    <div class="container">
        <div class="footBglm">
            <a target="_blank" href="/">首页</a> -
            <a target="_blank" href="/custom/about.htm">关于我们</a> -
            <a target="_blank" href="/search/Java/1.htm">站内搜索</a> -
            <a target="_blank" href="/sitemap.txt">Sitemap</a> -
            <a target="_blank" href="/custom/delete.htm">侵权投诉</a>
        </div>
        <div class="copyright">版权所有 IT知识库 CopyRight © 2000-2050 E-COM-NET.COM , All Rights Reserved.
<!--            <a href="https://beian.miit.gov.cn/" rel="nofollow" target="_blank">京ICP备09083238号</a><br>-->
        </div>
    </div>
</footer>
<!-- 代码高亮 -->
<script type="text/javascript" src="/static/syntaxhighlighter/scripts/shCore.js"></script>
<script type="text/javascript" src="/static/syntaxhighlighter/scripts/shLegacy.js"></script>
<script type="text/javascript" src="/static/syntaxhighlighter/scripts/shAutoloader.js"></script>
<link type="text/css" rel="stylesheet" href="/static/syntaxhighlighter/styles/shCoreDefault.css"/>
<script type="text/javascript" src="/static/syntaxhighlighter/src/my_start_1.js"></script>





</body>

</html>