某企业计划搭建私有云平台,以实现资源的池化弹性管理、企业应用的集中管理、统一安全认证和授权管理。按照给出的云平台架构进行IaaS、PaaS、大数据系统部署及运维管理,并进行云存储网盘(包括web及安卓客户端)开发、大数据分析应用开发和微信小程序开发,最后提交工程文档。
赛项所采用的云计算系统架构如图1所示,IP地址规划如表1所示。
图****1 云计算系统架构图
表1 IP地址规划表
设备名称 | 接口 | IP****地址 | 说明 |
---|---|---|---|
控制节点服务器 | eth0 | 192.168.100.10/24 | Vlan 100 |
eth1 | 192.168.200.10/24(初始IP) | Vlan 200 | |
计算节点服务器 | eth0 | 192.168.100.20/24 | Vlan 100 |
eth1 | 192.168.200.20/24(初始IP) | Vlan 200 | |
PC-1 | 本地连接 | 172.16.x.2/16 | Vlan 1 |
PC-2 | 本地连接 | 172.16.x.3/16 | Vlan 1 |
PC-3 | 本地连接 | 172.16.x.4/16 | Vlan 1 |
交换机 | Vlan 1 | 172.16.x.1/16 | x为考位号 |
Vlan 100 | 192.168.100.1/24 | ||
Vlan 200 | 192.168.200.1/24 |
注:根据以上信息,检查硬件连线及网络设备配置,确保网络连接正常。
修改云平台IaaS各节点的系统配置,按以下步骤搭建云平台,并完成相应的答题。
按以下要求设置主机名、防火墙及SELinux:
(1)设置控制节点主机名为controller,计算节点主机名为compute;
(2)关闭控制节点和计算节点的防火墙,设置开机不启动;
(3)设置控制节点和计算节点的SELinux为Permissive模式;
(4)退出SecureCRT,重新通过ssh连接各节点服务器;
使用命令查询控制节点和计算节点的主机名、防火墙是否处于关闭状态及SELinux的状态。以文本形式依次将命令行及查询信息提交到答题框。
[root@controller ~]# hostname
controller
[root@controller ~]#systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
Active: inactive (dead)
Docs: man:firewalld(1)
[root@controller ~]# getenforce
Permissive
通过SecureFX上传两个镜像文件CentOS-7-x86_64-DVD-1511.iso和 XianDian-IaaS-v2.2.iso到控制节点的opt目录下;在opt目录下创建两个子目录centos和iaas,并将镜像文件对应挂载到上述两个目录下;使用df命令查看挂载的信息(需显示挂载的文件系统类型)。依次将操作命令及执行结果以文本形式提交到答题框。
[root@controller ~]# mkdir /opt/centos
[root@controller ~]# mkdir /opt/iaas
[root@controller opt]# mount -o loop CentOS-7-x86_64-DVD-1511.iso centos/
mount: /dev/loop0 is write-protected, mounting read-only
[root@controller opt]# mount -o loop XianDian-IaaS-v2.2.iso iaas/
mount: /dev/loop1 is write-protected, mounting read-only
配置控制节点本地yum源文件local.repo,搭建ftp服务并配置根目录为指向存放yum源的路径;配置计算节点yum源文件ftp.repo,使用控制节点ftp服务作为yum源,其中节点的地址以主机名表示;使用cat命令查看控制节点和计算节点的yum源全路径配置文件。依次将操作命令及返回结果以文本形式提交到答题框。
[root@controller ~]# cat /etc/yum.repos.d/local.repo
[centos]
name=centos
baseurl=file:///opt/centos
enabled=1
gpgcheck=0
[iaas]
name=iaas
baseurl=file:///opt/iaas/iaas-repo
enabled=1
gpgcheck=0
[root@compute ~]# cat /etc/yum.repos.d/ftp.repo
[centos]
name=centos
baseurl=ftp://controller/centos
enabled=1
gpgcheck=0
[iaas]
name=iaas
baseurl=ftp://controller/iaas/iaas-repo
enabled=1
gpgcheck=0
在控制节点和计算节点分别安装iaas-xiandian软件包,首先完成脚本文件中基本变量的配置,然后根据表2完成脚本文件openrc.sh。以文本形式提交脚本文件的内容到答题框中。
表2 变量配置表
服务 | 变量 | 参数/密码 |
---|---|---|
Mysql | root | 000000 |
Keystone | 000000 | |
Glance | 000000 | |
Nova | 000000 | |
Neutron | 000000 | |
Heat | 000000 | |
Keystone | DOMAIN_NAME | demo |
Admin | 000000 | |
rabbit | 000000 | |
Glance | 000000 | |
Nova | 000000 | |
Neutron | 000000 | |
Heat | 000000 | |
Neutron | Metadata | 000000 |
External Network | enp9s0 |
[root@controller opt]# cat /etc/xiandian/openrc.sh
##--------------------system config--------------------##
##Controller Server Manager IP. example:x.x.x.x
HOST_IP=192.168.100.10
##Controller Server hostname. example:controller
HOST_NAME=controller
##Compute Node Manager IP. example:x.x.x.x
HOST_IP_NODE=192.168.100.20
##Compute Node hostname. example:compute
HOST_NAME_NODE=compute
##--------------------Rabbit config ------------------##
##user for rabbit. example:openstack
RABBIT_USER=openstack
##Password for rabbit user .example:000000
RABBIT_PASS=000000
##--------------------MySQL config---------------------##
##Password for MySQL root user . exmaple:000000
DB_PASS=000000
##--------------------Keystone config------------------##
##Password for Keystore admin user. exmaple:000000
DOMAIN_NAME=demo
ADMIN_PASS=000000
DEMO_PASS=000000
##Password for Mysql keystore user. exmaple:000000
KEYSTONE_DBPASS=000000
##--------------------Glance config--------------------##
##Password for Mysql glance user. exmaple:000000
GLANCE_DBPASS=000000
##Password for Keystore glance user. exmaple:000000
GLANCE_PASS=000000
##--------------------Nova config----------------------##
##Password for Mysql nova user. exmaple:000000
NOVA_DBPASS=000000
##Password for Keystore nova user. exmaple:000000
NOVA_PASS=000000
##--------------------Neturon config-------------------##
##Password for Mysql neutron user. exmaple:000000
NEUTRON_DBPASS=000000
##Password for Keystore neutron user. exmaple:000000
NEUTRON_PASS=000000
##metadata secret for neutron. exmaple:000000
METADATA_SECRET=000000
##External Network Interface. example:eth1
INTERFACE_NAME=enp8s0
##First Vlan ID in VLAN RANGE for VLAN Network. exmaple:101
minvlan=101
##Last Vlan ID in VLAN RANGE for VLAN Network. example:200
maxvlan=200
##--------------------Cinder config--------------------##
##Password for Mysql cinder user. exmaple:000000
CINDER_DBPASS=000000
##Password for Keystore cinder user. exmaple:000000
CINDER_PASS=000000
##Cinder Block Disk. example:md126p3
BLOCK_DISK=md126p4
##--------------------Swift config---------------------##
##Password for Keystore swift user. exmaple:000000
SWIFT_PASS=000000
##The NO1. NODE Object Disk for Swift. example:md126p4. The 2nd will be OBJECT_DISK_2
OBJECT_DISK=md126p5
##The NO1. NODE IP for Swift Storage Network. example:x.x.x.x. The 2nd will be STORAGE_LOCAL_NET_IP_2
STORAGE_LOCAL_NET_IP=192.168.100.20
##--------------------Heat config----------------------##
##Password for Mysql heat user. exmaple:000000
HEAT_DBPASS=000000
##Password for Keystore heat user. exmaple:000000
HEAT_PASS=000000
##--------------------Ceilometer config----------------##
##Password for Mysql ceilometer user. exmaple:000000
CEILOMETER_DBPASS=000000
##Password for Keystore ceilometer user. exmaple:000000
CEILOMETER_PASS=000000
##--------------------AODH config----------------##
##Password for Mysql AODH user. exmaple:000000
AODH_DBPASS=000000
##Password for Keystore AODH user. exmaple:000000
AODH_PASS=000000
使用脚本安装数据库服务,使用root用户登录数据库,查询数据库mysql中所有表的信息,并查询表user中的host,user,paassword信息。依次将操作命令和返回结果以文本形式提交到答题框。
[root@cloud-controller ~]# mysql -uroot -p000000
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 59411
Server version: 5.1.73-log Source distribution
Copyright © 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.
mysql> show databases;
±-------------------+
| Database |
±-------------------+
| information_schema |
| chinaskills |
| cinder |
| glance |
| heat |
| keystone |
| mymonitor |
| mysql |
| neutron |
| nova |
| sahara |
| trove |
| xd_databases |
±-------------------+
13 rows in set (0.00 sec)
mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
±--------------------------+
| Tables_in_mysql |
±--------------------------+
| columns_priv |
| db |
| event |
| func |
| general_log |
| help_category |
| help_keyword |
| help_relation |
| help_topic |
| host |
| ndb_binlog_index |
| plugin |
| proc |
| procs_priv |
| servers |
| slow_log |
| tables_priv |
| time_zone |
| time_zone_leap_second |
| time_zone_name |
| time_zone_transition |
| time_zone_transition_type |
| user |
±--------------------------+
23 rows in set (0.00 sec)
mysql> select host,user,password from user;
±-----------------±---------±------------------------------------------+
| host | user | password |
±-----------------±---------±------------------------------------------+
| localhost | root | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| cloud-controller | root | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| 127.0.0.1 | root | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| localhost | keystone | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| % | keystone | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| localhost | glance | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| % | glance | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| localhost | nova | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| % | nova | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| localhost | neutron | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| % | neutron | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| localhost | cinder | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| localhost | heat | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| % | heat | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| localhost | sahara | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| % | sahara | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| % | cinder | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| % | root | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| localhost | trove | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
| % | trove | *032197AE5731D4664921A6CCAC7CFCE6A0698693 |
±-----------------±---------±------------------------------------------+
20 rows in set (0.00 sec)
使用脚本安装keystone服务。将keystone的数据库导出到当前路径下并命名为keystone.sql,查询文件keystone.sql的前五行,依次将操作命令和返回而结果以文本形式提交到答题框。
[root@controller ~]# mysqldump -uroot -p000000 keystone > keystone.sql
[root@controller ~]# head -n 5 keystone.sql
– MySQL dump 10.16 Distrib 10.1.17-MariaDB, for Linux (x86_64)
–
– Host: localhost Database: keystone
– Server version 10.1.17-MariaDB
使用脚本安装glance服务,使用systemctl命令查询glance组件中所有服务的状态,依次将操作命令和返回结果以文本形式提交到答题框。
[root@controller ~]# systemctl status openstack-glance*
● openstack-glance-api.service - OpenStack Image Service (code-named Glance) API server
Loaded: loaded (/usr/lib/systemd/system/openstack-glance-api.service; enabled; vendor preset: disabled)
Active: active (running) since Fri 2018-01-12 23:28:31 EST; 3 days ago
Main PID: 10326 (glance-api)
CGroup: /system.slice/openstack-glance-api.service
├─10326 /usr/bin/python2 /usr/bin/glance-api
├─10460 /usr/bin/python2 /usr/bin/glance-api
├─10461 /usr/bin/python2 /usr/bin/glance-api
├─10462 /usr/bin/python2 /usr/bin/glance-api
├─10463 /usr/bin/python2 /usr/bin/glance-api
├─10464 /usr/bin/python2 /usr/bin/glance-api
├─10465 /usr/bin/python2 /usr/bin/glance-api
├─10466 /usr/bin/python2 /usr/bin/glance-api
├─10467 /usr/bin/python2 /usr/bin/glance-api
├─10468 /usr/bin/python2 /usr/bin/glance-api
├─10469 /usr/bin/python2 /usr/bin/glance-api
├─10470 /usr/bin/python2 /usr/bin/glance-api
├─10471 /usr/bin/python2 /usr/bin/glance-api
├─10472 /usr/bin/python2 /usr/bin/glance-api
├─10473 /usr/bin/python2 /usr/bin/glance-api
├─10474 /usr/bin/python2 /usr/bin/glance-api
└─10475 /usr/bin/python2 /usr/bin/glance-api
Jan 15 04:45:24 node-1 glance-api[10326]: /usr/lib/python2.7/site-packages/glance/location.py:428: DeprecationWarning: Using function/method 'ImageProxy.verify_signature_if…the N cycle.
Jan 15 04:45:24 node-1 glance-api[10326]: self._verify_signature_if_needed(checksum)
Jan 15 04:45:24 node-1 glance-api[10326]: /usr/lib/python2.7/site-packages/glance/location.py:452: DeprecationWarning: Using function/method 'glance.common.signature_utils.sh…the N cycle.
Jan 15 04:45:24 node-1 glance-api[10326]: self.image.extra_properties)):
Jan 15 22:32:26 node-1 glance-api[10326]: /usr/lib/python2.7/site-packages/glance/location.py:428: DeprecationWarning: Using function/method 'ImageProxy.verify_signature_if…the N cycle.
Jan 15 22:32:26 node-1 glance-api[10326]: self._verify_signature_if_needed(checksum)
Jan 15 22:32:26 node-1 glance-api[10326]: /usr/lib/python2.7/site-packages/glance/location.py:452: DeprecationWarning: Using function/method 'glance.common.signature_utils.sh…the N cycle.
Jan 15 22:32:26 node-1 glance-api[10326]: self.image.extra_properties)):
Jan 16 00:33:59 controller glance-api[10326]: /usr/lib/python2.7/site-packages/glance/common/wsgi.py:941: DeprecationWarning: Using function/method ‘TasksController.index()’ is deprecate…
Jan 16 00:33:59 controller glance-api[10326]: return method(*args, **kwargs)
● openstack-glance-registry.service - OpenStack Image Service (code-named Glance) Registry server
Loaded: loaded (/usr/lib/systemd/system/openstack-glance-registry.service; enabled; vendor preset: disabled)
Active: active (running) since Fri 2018-01-12 23:28:31 EST; 3 days ago
Main PID: 10327 (glance-registry)
CGroup: /system.slice/openstack-glance-registry.service
├─10327 /usr/bin/python2 /usr/bin/glance-registry
├─10419 /usr/bin/python2 /usr/bin/glance-registry
├─10420 /usr/bin/python2 /usr/bin/glance-registry
├─10421 /usr/bin/python2 /usr/bin/glance-registry
├─10422 /usr/bin/python2 /usr/bin/glance-registry
├─10423 /usr/bin/python2 /usr/bin/glance-registry
├─10424 /usr/bin/python2 /usr/bin/glance-registry
├─10425 /usr/bin/python2 /usr/bin/glance-registry
├─10426 /usr/bin/python2 /usr/bin/glance-registry
├─10427 /usr/bin/python2 /usr/bin/glance-registry
├─10428 /usr/bin/python2 /usr/bin/glance-registry
├─10429 /usr/bin/python2 /usr/bin/glance-registry
├─10430 /usr/bin/python2 /usr/bin/glance-registry
├─10431 /usr/bin/python2 /usr/bin/glance-registry
├─10432 /usr/bin/python2 /usr/bin/glance-registry
├─10433 /usr/bin/python2 /usr/bin/glance-registry
└─10434 /usr/bin/python2 /usr/bin/glance-registry
Jan 12 23:45:32 node-1 glance-registry[10327]: /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:1056: OsloDBDeprecationWarning: EngineFacade is deprecated;…enginefacade
Jan 12 23:45:32 node-1 glance-registry[10327]: expire_on_commit=expire_on_commit, _conf=conf)
Jan 12 23:45:32 node-1 glance-registry[10327]: /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:1056: OsloDBDeprecationWarning: EngineFacade is deprecated;…enginefacade
Jan 12 23:45:32 node-1 glance-registry[10327]: expire_on_commit=expire_on_commit, _conf=conf)
Jan 12 23:45:32 node-1 glance-registry[10327]: /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:1056: OsloDBDeprecationWarning: EngineFacade is deprecated;…enginefacade
Jan 12 23:45:32 node-1 glance-registry[10327]: expire_on_commit=expire_on_commit, _conf=conf)
Jan 12 23:45:32 node-1 glance-registry[10327]: /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:1056: OsloDBDeprecationWarning: EngineFacade is deprecated;…enginefacade
Jan 12 23:45:32 node-1 glance-registry[10327]: expire_on_commit=expire_on_commit, _conf=conf)
Jan 12 23:45:34 node-1 glance-registry[10327]: /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:1056: OsloDBDeprecationWarning: EngineFacade is deprecated;…enginefacade
Jan 12 23:45:34 node-1 glance-registry[10327]: expire_on_commit=expire_on_commit, _conf=conf)
Hint: Some lines were ellipsized, use -l to show in full.
使用脚本安装nova服务。使用nova相关命令查询计算节点虚拟机监控器状态,依次将操作命令和返回结果以文本形式提交到答题框。
[root@controller ~]# nova hypervisor-list
±—±--------------------±------±--------+
| ID | Hypervisor hostname | State | Status |
±—±--------------------±------±--------+
| 1 | controller | up | enabled |
±—±--------------------±------±--------+
使用脚本安装neutron服务,并配置为GRE网络:
(1)创建云主机外部网络为ext-net,子网为ext-subnet,虚拟机浮动IP网段为192.168.200.0/24,网关为192.168.200.1;
(2)创建云主机隧道网络int-net1,子网为int-subnet1,虚拟机子网IP网段为10.0.0.0/24,网关为10.0.0.1;
(3)创建云主机隧道网络int-net2,子网为int-subnet2,虚拟机子网IP网段为10.0.1.0/24,网关为10.0.1.1;
(4)添加名为ext-router的路由器,配置路由接口地址,完成隧道网络int-net1和外部网络ext-net的连通。
使用neutron相关命令查询所创建路由的详细信息,依次将操作命令和返回结果以文本形式提交到答题框。
[root@controller ~]# neutron router-show ext-router
±------------------------±----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
±------------------------±----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | True |
| availability_zone_hints | |
| availability_zones | nova |
| description | |
| distributed | False |
| external_gateway_info | {“network_id”: “0266da86-9296-4813-99eb-a6aba8336b91”, “enable_snat”: true, “external_fixed_ips”: [{“subnet_id”: “915291d8-63f4-4296-b60b-0254f93d8c2b”, |
| | “ip_address”: “172.24.13.101”}]} |
| ha | False |
| id | b7a1d373-f6d1-4750-9dba-bd0c56e66202 |
| name | ext-router |
| routes | |
| status | ACTIVE |
| tenant_id | fdb6961908f64d42b87bf1f575391074 |
±------------------------±----------------------------------------------------------------------------------------------------------------------------------------------------------------+
使用脚本安装dashboard服务,使用curl命令查询网址http://192.168.100.10/dashboard,依次将操作命令和返回结果以文本形式提交到答题框
[root@controller ~]# curl http://192.168.100.10/dashboard/auth/login/
Login - XianDian Dashboard
云计算基础架构服务平台
容器平台架构如图2所示。
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-BHGcVqdC-1622509137149)(file:///C:/Users/ADMINI~1/AppData/Local/Temp/msohtmlclip1/01/clip_image002.jpg)]
图2 容器平台架构
根据 PaaS 平台的部署架构,PaaS 平台部署在 IaaS 平台的 2 台云主机上。其中Server 云主机上部署 Server 节点和Registry节点,Client 云主机上部署 Client 节点。每个云主机配置如下:
(1)系统配置
Server/Registry节点:2CPU,4G内存,60G硬盘;
Client节点:2CPU,4G内存,60G硬盘;
(2)操作系统:centos_7-x86_64;
(3)IP:Server和Client云主机ip动态分配;
(4)Server节点的主机名:Server;
(5)Client节点的主机名:Client。
根据配置要求,完成配置文件的自定义与修改,搭建容器平台。登录容器平台,通过curl命令查询Rancher管理平台首页。依次将操作命令及返回结果以文本形式提交到答题框。
[root@server images]# curl http://10.0.6.126:8080/env/1a5/apps/stacks
Loading
根据提供的软件包,通过“应用商店”部署Grafana,修改网页访问端口为9090,通过curl命令访问服务首页。依次将操作命令及返回结果以文本形式提交到答题框。
PrometheusTimeSeriesCollectionandProcessingServer
Togglenavigation
Alerts
Graph
Targets
Help
按以下配置在云平台中创建云主机:
(1)名称:iaas_all;
(2)镜像文件:iaas-all;
(3)云主机类型:m1.large;
(4)网络1:int-net1,绑定浮动IP;
(5)网络2:int-net2。
登录http://192.168.100.10/dashboard。申请两台centos7系统的云主机,构建mariadb高可用数据库集群,通过命令查询集群状态。依次将操作命令和返回结果以文本形式提交到答题框。
MariaDB [(none)]> show status like ‘wsrep_%’;
±-----------------------------±-------------------------------------+
| Variable_name | Value |
±-----------------------------±-------------------------------------+
| wsrep_apply_oooe | 0.000000 |
| wsrep_apply_oool | 0.000000 |
| wsrep_apply_window | 1.000000 |
| wsrep_causal_reads | 0 |
| wsrep_cert_deps_distance | 1.000000 |
| wsrep_cert_index_size | 2 |
| wsrep_cert_interval | 0.000000 |
| wsrep_cluster_conf_id | 16 |
| wsrep_cluster_size | 2 |
| wsrep_cluster_state_uuid | 5a9e502d-0144-11e8-a557-372bcf5c813f |
| wsrep_cluster_status | Primary |
| wsrep_commit_oooe | 0.000000 |
| wsrep_commit_oool | 0.000000 |
| wsrep_commit_window | 1.000000 |
| wsrep_connected | ON |
| wsrep_desync_count | 0 |
| wsrep_evs_delayed | |
| wsrep_evs_evict_list | |
| wsrep_evs_repl_latency | 0/0/0/0/0 |
| wsrep_evs_state | OPERATIONAL |
| wsrep_flow_control_paused | 0.000000 |
| wsrep_flow_control_paused_ns | 0 |
| wsrep_flow_control_recv | 0 |
| wsrep_flow_control_sent | 0 |
| wsrep_gcomm_uuid | 5a9d6abc-0144-11e8-a747-c288e5a37eb3 |
| wsrep_incoming_addresses | 172.26.3.13:3306,172.26.3.11:3306 |
| wsrep_last_committed | 2 |
| wsrep_local_bf_aborts | 0 |
| wsrep_local_cached_downto | 1 |
| wsrep_local_cert_failures | 0 |
| wsrep_local_commits | 0 |
| wsrep_local_index | 1 |
| wsrep_local_recv_queue | 0 |
| wsrep_local_recv_queue_avg | 0.100000 |
| wsrep_local_recv_queue_max | 2 |
| wsrep_local_recv_queue_min | 0 |
| wsrep_local_replays | 0 |
| wsrep_local_send_queue | 0 |
| wsrep_local_send_queue_avg | 0.000000 |
| wsrep_local_send_queue_max | 1 |
| wsrep_local_send_queue_min | 0 |
| wsrep_local_state | 4 |
| wsrep_local_state_comment | Synced |
| wsrep_local_state_uuid | 5a9e502d-0144-11e8-a557-372bcf5c813f |
| wsrep_protocol_version | 7 |
| wsrep_provider_name | Galera |
| wsrep_provider_vendor | Codership Oy [email protected] |
| wsrep_provider_version | 25.3.22(r3764) |
| wsrep_ready | ON |
| wsrep_received | 30 |
| wsrep_received_bytes | 3751 |
| wsrep_repl_data_bytes | 355 |
| wsrep_repl_keys | 1 |
| wsrep_repl_keys_bytes | 31 |
| wsrep_repl_other_bytes | 0 |
| wsrep_replicated | 1 |
| wsrep_replicated_bytes | 450 |
| wsrep_thread_count | 2 |
±-----------------------------±-------------------------------------+
58 rows in set (0.00 sec)
登录“iaas_all”云主机。
(1)使用rabbitmqctl命令创建用户xiandian-admin,密码为admin;
(2)给xiandian-admin用户创建administrator角色并查询;
(3)授予用户xiandian-admin对本机所有资源可写可读可执行的权限,查询xiandian-admin用户的授权信息。
依次将操作命令及返回结果以文本形式提交到答题框。
[root@xiandian ~]# rabbitmqctl add_user xiandian-admin admin
Creating user “xiandian-admin” …
[root@xiandian ~]# rabbitmqctl set_user_tags xiandian-admin administrator
Setting tags for user “xiandian-admin” to [administrator] …
[root@xiandian ~]# rabbitmqctl list_users
Listing users …
xiandian []
xiandian-admin [administrator]
guest [administrator]
[root@xiandian ~]# rabbitmqctl set_permissions xiandian-admin “." ".” “.*”
Setting permissions for user “xiandian-admin” in vhost “/” …
[root@xiandian ~]# rabbitmqctl list_user_permissions xiandian-admin
Listing permissions for user “xiandian-admin” …
/ .* .* .*
登录compute节点,使用命令将KVM进程绑定到特定的0-5号cpu上。依次将操作命令及返回结果以文本形式提交到答题框。
[root@controller ~]# ps -e |grep kvm
765 ? 00:00:00 kvm-irqfd-clean
7490 ? 00:00:12 qemu-kvm
7497 ? 00:00:00 kvm-pit/7490
[root@controller ~]# taskset -cp 0-5 7490
pid 7490 's current affinity list: 0-15
pid 7490 's new affinity list: 0-5
前提:按要求配置静态fixed_key,使cinder和nova组件可以使用加密过的Block Storage卷服务,配置好之后,创建一个卷类型叫luks,并把这个类型配置为加密类型,cipher使用“aes-xts-plain64”,key_size使用“512”,control-location使用“front-end”,Provider使用“nova.volume.encryptors.luks.LuksEncryptor”。
(1)登录“iaas_all”云主机,创建两块云硬盘,分别叫“unencrypted volume”和“encrypted volume”大小都是1G,前者不加密,后者使用luks加密卷类型;
(2)使用nova命令,创建一个云主机vm-test,使用cirros镜像,自行连接网络;
(3)将创建的两块云硬盘attach到云主机vm-test上,使用cinder list查看。
依次将操作命令和返回结果以文本形式提交到答题框。
[root@xiandian ~]# cinder create --display-name ‘unencrypted volume’ 1
±-------------------------------±-------------------------------------+
| Property | Value |
±-------------------------------±-------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2018-01-25T00:43:42.000000 |
| description | None |
| encrypted | False |
| id | 348199b8-c2b4-45ed-a03c-7e542e4f5488 |
| metadata | {‘readonly’: ‘False’} |
| migration_status | None |
| multiattach | False |
| name | unencrypted volume |
| os-vol-host-attr:host | xiandian@lvm#LVM |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 0ab2dbde4f754b699e22461426cd0774 |
| replication_status | disabled |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | available |
| updated_at | 2018-01-25T05:10:07.000000 |
| user_id | 53a1cf0ad2924532aa4b7b0750dec282 |
| volume_type | None |
±-------------------------------±-------------------------------------+
[root@xiandian ~]# cinder create --display-name ‘encrypted volume’ --volume-type luks 1
±-------------------------------±-------------------------------------+
| Property | Value |
±-------------------------------±-------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2018-01-25T02:48:13.000000 |
| description | None |
| encrypted | True |
| id | 928f4eab-69da-4bdb-9a30-1edbb2cbe6ee |
| metadata | {} |
| migration_status | None |
| multiattach | False |
| name | encrypted volume |
| os-vol-host-attr:host | xiandian@lvm#LVM |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 0ab2dbde4f754b699e22461426cd0774 |
| replication_status | disabled |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | creating |
| updated_at | 2018-01-25T02:48:14.000000 |
| user_id | 53a1cf0ad2924532aa4b7b0750dec282 |
| volume_type | luks |
±-------------------------------±-------------------------------------+
[root@xiandian ~]# cinder list
±-------------------------------------±----------±-------------------±-----±------------±---------±------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
±-------------------------------------±----------±-------------------±-----±------------±---------±------------+
| 348199b8-c2b4-45ed-a03c-7e542e4f5488 | available | unencrypted volume | 1 | - | false | |
| 928f4eab-69da-4bdb-9a30-1edbb2cbe6ee | available | encrypted volume | 1 | luks | false | |
±-------------------------------------±----------±-------------------±-----±------------±---------±------------+
[root@xiandian ~]# nova boot --flavor m1.tiny --image “cirros” vm-test
[root@xiandian ~]# cinder list
±-------------------------------------±----------±-------------------±-----±------------±---------±------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
±-------------------------------------±----------±-------------------±-----±------------±---------±------------+
| 348199b8-c2b4-45ed-a03c-7e542e4f5488 | available | unencrypted volume | 1 | - | false | |
| 928f4eab-69da-4bdb-9a30-1edbb2cbe6ee | available | encrypted volume | 1 | luks | false | |
±-------------------------------------±----------±-------------------±-----±------------±---------±------------+
[root@xiandian ~]# nova volume-attach vm-test 348199b8-c2b4-45ed-a03c-7e542e4f5488
±---------±-------------------------------------+
| Property | Value |
±---------±-------------------------------------+
| device | /dev/vdb |
| id | 348199b8-c2b4-45ed-a03c-7e542e4f5488 |
| serverId | fce35460-2653-4d2a-ad18-e0522a8dbfd4 |
| volumeId | 348199b8-c2b4-45ed-a03c-7e542e4f5488 |
±---------±-------------------------------------+
[root@xiandian ~]# nova volume-attach vm-test 928f4eab-69da-4bdb-9a30-1edbb2cbe6ee
±---------±-------------------------------------+
| Property | Value |
±---------±-------------------------------------+
| device | /dev/vdc |
| id | 928f4eab-69da-4bdb-9a30-1edbb2cbe6ee |
| serverId | fce35460-2653-4d2a-ad18-e0522a8dbfd4 |
| volumeId | 928f4eab-69da-4bdb-9a30-1edbb2cbe6ee |
±---------±-------------------------------------+
[root@xiandian ~]# cinder list
±-------------------------------------±-------±-------------------±-----±------------±---------±-------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
±-------------------------------------±-------±-------------------±-----±------------±---------±-------------------------------------+
| 348199b8-c2b4-45ed-a03c-7e542e4f5488 | in-use | unencrypted volume | 1 | - | false | fce35460-2653-4d2a-ad18-e0522a8dbfd4 |
| 928f4eab-69da-4bdb-9a30-1edbb2cbe6ee | in-use | encrypted volume | 1 | luks | false | fce35460-2653-4d2a-ad18-e0522a8dbfd4 |
±-------------------------------------±-------±-------------------±-----±------------±---------±-------------------------------------+
在云平台后台管理的过程中块存储功能无法使用,错误现象为创建的云硬盘状态为error。找出错误原因,进行修复,修复之后创建云硬盘,查询云硬盘的详细信息。依次将错误的内容、修复的操作以及查询到信息以文本形式提交到答题框。
无
在云平台后台管理的过程中无法上传镜像,找出错误原因,并进行修复,修复之后使用cirros-0.3.4-x86_64-disk.img创建名为cirros的镜像,查询镜像的详细信息。依次将错误的内容、修复的操作以及查询到的信息以文本形式提交到答题框。
无
在容器server节点使用nginx:latest镜像创建一个名为yidaoyun的容器,只能使用0号CPU。启动容器后查看容器CPU的使用情况,依次将操作命令及返回结果以文本形式提交到答题框。
[root@server images]# docker run -dit --name yidaoyun --cpuset-cpus=“0” nginx:latest /bin/bash
6f377e734d407649f8c2703eb336a145a88cd78bdedc077ad1714872b6406514
[root@server images]# cat /sys/fs/cgroup/cpuset/system.slice/docker-6f377e734d407649f8c2703eb336a145a88cd78bdedc077ad1714872b6406514.scope/cpuset.cpus
0
在容器server节点使用mysql:8.0镜像创建名为mysqldb的容器、使用镜像nginx:latest创建名为nginxweb的容器,将nginxweb容器内数据库连接至mysqldb容器。使用inspect -f查看链接内容的信息,依次将以上操作命令及返回结果填入答题框。
[root@server ~]# docker run -d -it --name mysqldb -P mysql:8.0 /bin/bash
2db3a90ee5ce5502fa36e22ab8cb8237c5ac353ab6549b6ec7ac1fab0fccbb38
[root@server ~]# docker run -d -it --name nginxweb -P --link mysqldb:db nginx:latest /bin/bash
a23b55c18ad1d003fe38cc0bfc1b2d6cf6c92f06f7cd923d55b96e902211c003
[root@server ~]# docker inspect --format {{.HostConfig.Links}} nginxweb
[/mysqldb:/nginxweb/db]
在容器server节点使用docker命令创建名为xd_net的网络,网络网段为192.168.3.0/24,网关为192.168.3.1,创建完成后查询网络列表。依次将以上操作命令及返回结果填入答题框。
[root@server ~]# docker network create --subnet=192.168.3.0/24 --ip-range=192.168.3.0/24 --gateway=192.168.3.1 xd_net
b8199203dbfeeee03956ab851ccaaa9753b83955a82abf91114260fc43715ad2
[root@server ~]# docker network ls
NETWORK ID NAME DRIVER SCOPE
472e57668c5e bridge bridge local
96c6462537eb host host local
30c76b9daa4d none null local
b8199203dbfe xd_net bridge local
在容器server节点,使用supermin5命令(若命令不存在,则自己安装)构建名为centos-7的centos7系统docker镜像,镜像预装yum、net-tools、initscripts和vi命令。构建完成后提交镜像至容器仓库,并查看此镜像。依次将操作命令及返回结果以文本形式提交到答题框。
[root@server ~]# supermin5 -v --prepare bash yum net-tools initscripts vi coreutils -o supermin.d
[root@server ~]# supermin5 -v --build --format chroot supermin.d -o appliance.d
[root@server ~]# echo 7 > appliance.d/etc/yum/vars/releasever
[root@server ~]# tar --numeric-owner -cpf centos-7.tar -C appliance.d .
[root@server ~]# cat centos-7.tar | docker import - 10.0.6.126:5000/centos-7
sha256:75b27ee7851e297faa2a085c7dac3c0f25bc4ac63949b04616fa3d247e52d007
[root@server ~]# docker images centos-7
REPOSITORY TAG IMAGE ID CREATED SIZE
10.0.6.126:5000/centos-7 latest 75b27ee7851e 50 seconds ago 261.5 MB
以上题构建的centos-7镜像为基础,按以下要求构建http服务镜像http:v1.0:
(1)删除镜像的yum源,使用当前系统的yum源文件;
(2)完成后安装http服务;
(3)暴露80端口。
使用cat命令查看Dockerfile文件并构建镜像。依次将操作命令及返回结果以文本形式提交到答题框。
[root@server nginx]# cat Dockerfile
FROM 10.0.6.126:5000/centos-7:latest
MAINTAINER Xiandian
RUN rm -fv /etc/yum.repos.d/*
ADD local.repo /etc/yum.repos.d/
RUN yum install -y httpd
EXPOSE 80
[root@server nginx]# docker build -t 10.0.6.126:5000/httpd:v1.0 .
[root@server nginx]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
10.0.6.126:5000/httpd v1.0 a41a37cb9467 6 minutes ago 554.2 MB
(1)在容器server节点,创建/root/www1目录,在www1目录下编写index.jsp文件; (2)使用tomcat镜像创建容器,容器的默认项目地址连接到/root/www1目录,要求访问tomcat的时候输出语句为“this is Tomcat1”;
(3)进入容器启动tomcat服务,使用curl命令查询tomcat的首页。
依次将操作命令及返回结果以文本形式提交到答题框。
[root@server ~]# docker run -it -d -P -h tomcat1 -v /root/www1:/usr/local/tomcat/webapps/ROOT 10.0.6.126:5000/tomcat:latest /bin/bash
f3f8c24dfb0adf112e8a6fdcb49fbd584fd5f9fbbf454146d369e97c3366985c
[root@server ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f3f8c24dfb0a 10.0.6.126:5000/tomcat:latest “/bin/bash” 14 seconds ago Up 11 seconds 0.0.0.0:32768->8080/tcp adoring_bose
[root@server ~]# docker exec -it f3f8c24dfb0a /bin/bash
root@tomcat1:/usr/local/tomcat# cd bin/
root@tomcat1:/usr/local/tomcat/bin# startup.sh
Tomcat started.
[root@server ~]# curl http://10.0.6.126:32768
Tomcat1
this is Tomcat1
大数据平台的搭建采用分布式部署,部署在云平台的两台虚拟机上,在云主机1上部署大数据平台master节点,在云主机2上部署大数据平台slaver节点:
云主机1:
(1)名称:master;
(2)镜像文件:hadoop_master_centos7_x86_xiandian_images-v05.qcow2;
(3)类型:4CPU、8G内存、100G硬盘;
(4)网络1:int-net1,绑定浮动IP。
云主机2:
(1)名称:slaver;
(2)镜像文件:hadoop_slaver1_centos7_x86_xiandian_images-v05.qcow2;
(3)类型:4CPU、8G内存、100G硬盘;
(4)网络1:int-net1,绑定浮动IP。
使用cat命令查看云主机master和slaver的hosts文件。依次将操作命令及返回结果以文本形式提交到答题框。
[root@master ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.0.10 master.hadoop master
10.0.0.11 slaver1.hadoop
[root@slaver1 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.0.10 master.hadoop
10.0.0.11 slaver1.hadoop slaver1
检查云主机master和slaver的内存配置文件,查看Transparent Huge Pages状态。依次将操作命令及返回结果以文本形式提交到答题框。
# echo never > /sys/kernel/mm/transparent_hugepage/enabled
# echo never > /sys/kernel/mm/transparent_hugepage/defrag
# cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
在大数据master节点查询ambari-server的运行状态信息,依次将操作命令和返回结果以文本形式提交到答题框。
[root@master ~]# ambari-server status
Using python /usr/bin/python
Ambari-server status
Ambari Server running
Found Ambari Server PID: 1854 at: /var/run/ambari-server/ambari-server.pid
(1)启动大数据平台并登录http://{master-ip}:8080,用户名:admin,密码:admin; (2)平台中已经安装了以下服务组件:HDFS、MapReduce2、YARN、Tez、Hive、HBase、Pig、Zookeeper、Mahout。其中master节点Mariadb数据库用户密码配置如表3所示。
表3 数据库用户密码配置表
用户名 | 密码 |
---|---|
root | bigdata |
ambari | bigdata |
hive | bigdata |
启动平台中安装的HDFS、MapReduce2、YARN、Zookeeper等服务;
(3)先后在master节点和slaver节点的Linux Shell中查看Hadoop集群的服务进程信息。
依次将操作命令及返回结果以文本形式提交到答题框。
[root@master ~]# jps
16145 Jps
18195 Bootstrap
9668 DataNode
23796 QuorumPeerMain
10053 NameNode
11285 NodeManager
3340 HMaster
1374 AmbariServer
3375 ApplicationHistoryServer
[root@slaver1 ~]# jps
20529 RunJar
8242 NodeManager
7907 ApplicationHistoryServer
19509 RunJar
5830 SecondaryNameNode
17494 Jps
5464 DataNode
21129 RunJar
9513 QuorumPeerMain
7117 JobHistoryServer
8637 ResourceManager
(1)在HDFS文件系统的根目录下创建目录yidaoyun/file;
(2)将附件中的BigDataSkills.txt文件上传到yidaoyun/file目录中;
(3)查看HDFS中yidaoyun/file目录的文件列表信息;
(4)查询HDFS文件系统中/tmp目录下的目录个数、文件个数和文件总大小。
依次将操作命令及返回结果以文本形式提交到答题框。
[root@master ~]# hadoop fs -mkdir -p /1daoyun/file
[root@master ~]# hadoop fs -put BigDataSkills.txt /1daoyun/file
[root@master ~]# hadoop fs -ls /1daoyun/file
Found 1 items
-rw-r–r-- 3 root hdfs 49 2017-05-06 09:57 /1daoyun/file/BigDataSkills.txt
[root@master ~]# hadoop fs -count /tmp
21 6 4336 /tmp
登录大数据master节点,新建hbasetest.txt文件,编写内容,要求如下:
(1)新建一张表为’test’, 列族为’cf’;
(2)向这张表批量插入如下数据;
‘row1’, ‘cf:a’, ‘value1’
‘row2’, ‘cf:b’, ‘value2’
‘row3’, ‘cf:c’, ‘value3’
‘row4’, ‘cf:d’, ‘value4’
(3)插入数据完毕后用scan命令查询表内容;
(4)用get命令只查询row1的内容;
(5)最后退出hbase shell。
使用命令运行hbasetest.txt。依次将hbasetest.txt的内容、执行命令和返回结果以文本形式提交到答题框。
[root@exam1 ~]# cat hbasetest.txt
create ‘test’, ‘cf’
list ‘test’
put ‘test’, ‘row1’, ‘cf:a’, ‘value1’
put ‘test’, ‘row2’, ‘cf:b’, ‘value2’
put ‘test’, ‘row3’, ‘cf:c’, ‘value3’
put ‘test’, ‘row4’, ‘cf:d’, ‘value4’
scan ‘test’
get ‘test’, ‘row1’
exit
[root@exam1 ~]# hbase shell hbasetest.txt
0 row(s) in 1.5010 seconds
TABLE
test
1 row(s) in 0.0120 seconds
0 row(s) in 0.1380 seconds
0 row(s) in 0.0090 seconds
0 row(s) in 0.0050 seconds
0 row(s) in 0.0050 seconds
ROW COLUMN+CELL
row1 column=cf:a, timestamp=1522314428726, value=value1
row2 column=cf:b, timestamp=1522314428746, value=value2
row3 column=cf:c, timestamp=1522314428752, value=value3
row4 column=cf:d, timestamp=1522314428758, value=value4
4 row(s) in 0.0350 seconds
COLUMN CELL
cf:a timestamp=1522314428726, value=value1
1 row(s) in 0.0190 seconds
登录hbase数据库,创建一张名为member,列族为’address’,'info’的表,向该表插入数据为:
‘xiandianA’,‘info:age’,‘24’
‘xiandianA’,‘info:birthday’,‘1990-07-17’
‘xiandianA’,‘info:company’,‘alibaba’
‘xiandianA’,‘address:contry’,‘china’
‘xiandianA’,‘address:province’,‘zhejiang’
‘xiandianA’,‘address:city’,‘hangzhou’
使用命令查询member表中xiandianA的所有info信息,将’info:age’的值改为99,并只查询’info:age’信息。依次将操作命令和返回结果以文本形式提交到答题框。
hbase(main):001:0> create ‘member’,‘address’,‘info’
0 row(s) in 1.5730 seconds
=> Hbase::Table - member
hbase(main):002:0> list
TABLE
emp
member
2 row(s) in 0.0240 seconds
hbase(main):007:0> put’member’,‘xiandianA’,‘info:age’,‘24’
0 row(s) in 0.1000 seconds
hbase(main):008:0> put’member’,‘xiandianA’,‘info:birthday’,‘1990-07-17’
0 row(s) in 0.0130 seconds
hbase(main):010:0> put’member’,‘xiandianA’,‘info:company’,‘alibaba’
0 row(s) in 0.0080 seconds
hbase(main):011:0> put’member’,‘xiandianA’,‘address:contry’,‘china’
0 row(s) in 0.0080 seconds
hbase(main):012:0> put’member’,‘xiandianA’,‘address:province’,‘zhejiang’
0 row(s) in 0.0070 seconds
hbase(main):013:0> put’member’,‘xiandianA’,‘address:city’,‘hangzhou’
0 row(s) in 0.0090 seconds
hbase(main):014:0> get ‘member’,‘xiandianA’,‘info’
COLUMN CELL
info:age timestamp=1522140592336, value=24
info:birthday timestamp=1522140643072, value=1990-07-17
info:company timestamp=1522140745172, value=alibaba
3 row(s) in 0.0170 seconds
hbase(main):015:0>
hbase(main):016:0* put ‘member’,‘xiandianA’,‘info:age’,‘99’
0 row(s) in 0.0080 seconds
hbase(main):018:0> get ‘member’,‘xiandianA’,‘info:age’
COLUMN CELL
info:age timestamp=1522141564423, value=99
1 row(s) in 0.0140 seconds
登录大数据master节点,在root目录下新建一个abc.txt,里面的内容为:
hadoop hive
solr redis
kafka hadoop
storm flume
sqoop docker
spark spark
hadoop spark
elasticsearch hbase
hadoop hive
spark hive
hadoop spark
登录spark-shell,首先使用命令统计abc.txt的行数,接着对abc.txt文档中的单词进行计数,并按照单词首字母的升序进行排序,最后统计结果行数。依次将操作命令及返回结果以文本形式提交到答题框。
scala> val words=sc.textFile(“file:///root/abc.txt”).count
words: Long = 11
scala> val words=sc.textFile(“file:///root/abc.txt”).flatMap(.split("\W+")).map(x=>(x,1)).reduceByKey(+_).sortByKey().collect
words: Array[(String, Int)] = Array((docker,1), (elasticsearch,1), (flume,1), (hadoop,5), (hbase,1), (hive,3), (kafka,1), (redis,1), (solr,1), (spark,5), (sqoop,1), (storm,1))
scala> val words=sc.textFile(“file:///root/abc.txt”).flatMap(.split("\W+")).map(x=>(x,1)).reduceByKey(+_).count
n 0.0080 seconds
hbase(main):011:0> put’member’,‘xiandianA’,‘address:contry’,‘china’
0 row(s) in 0.0080 seconds
hbase(main):012:0> put’member’,‘xiandianA’,‘address:province’,‘zhejiang’
0 row(s) in 0.0070 seconds
hbase(main):013:0> put’member’,‘xiandianA’,‘address:city’,‘hangzhou’
0 row(s) in 0.0090 seconds
hbase(main):014:0> get ‘member’,‘xiandianA’,‘info’
COLUMN CELL
info:age timestamp=1522140592336, value=24
info:birthday timestamp=1522140643072, value=1990-07-17
info:company timestamp=1522140745172, value=alibaba
3 row(s) in 0.0170 seconds
hbase(main):015:0>
hbase(main):016:0* put ‘member’,‘xiandianA’,‘info:age’,‘99’
0 row(s) in 0.0080 seconds
hbase(main):018:0> get ‘member’,‘xiandianA’,‘info:age’
COLUMN CELL
info:age timestamp=1522141564423, value=99
1 row(s) in 0.0140 seconds
登录大数据master节点,在root目录下新建一个abc.txt,里面的内容为:
hadoop hive
solr redis
kafka hadoop
storm flume
sqoop docker
spark spark
hadoop spark
elasticsearch hbase
hadoop hive
spark hive
hadoop spark
登录spark-shell,首先使用命令统计abc.txt的行数,接着对abc.txt文档中的单词进行计数,并按照单词首字母的升序进行排序,最后统计结果行数。依次将操作命令及返回结果以文本形式提交到答题框。
scala> val words=sc.textFile(“file:///root/abc.txt”).count
words: Long = 11
scala> val words=sc.textFile(“file:///root/abc.txt”).flatMap(.split("\W+")).map(x=>(x,1)).reduceByKey(+_).sortByKey().collect
words: Array[(String, Int)] = Array((docker,1), (elasticsearch,1), (flume,1), (hadoop,5), (hbase,1), (hive,3), (kafka,1), (redis,1), (solr,1), (spark,5), (sqoop,1), (storm,1))
scala> val words=sc.textFile(“file:///root/abc.txt”).flatMap(.split("\W+")).map(x=>(x,1)).reduceByKey(+_).count
words: Long = 12