百度PY-Day5理论课课堂笔记

声明：本文首发于soarli博客（https://blog.soarli.top/archives/383.html

）转载请注明来源。

目录：

EasyDL概述

EasyDL体验

创建、上传数据集：

图片演示：

结果反馈：

EasyDL应用案例

大作业：数据爬取、分析与内容审核

爬取数据的要点：

即：所有评论不会直接显示到默认页面元素中，每次点击更多评论按钮时就会新加载一个get_comments文件请求，我们要从此处着手分析。

请求该资源，可得下图所示资源：


老师介绍了last_id的作用是记录当前评论加载的位置，即作为下一个请求中的一个参数，是得下一个请求从当前位置继续加载。

因此，我们可尝试不断获取last_id，从而生成新的URL，再去获取新的评论。

反复如此，便可模拟用户点击“查看更多评论”的操作。

大作业参考结果

实操：

第一步：爱奇艺《青春有你2》评论数据爬取(参考链接：https://www.iqiyi.com/v_19ryfkiv8w.html#curid=15068699100_9f9bab7e0d1e30c494622af777f4ba39)

• 爬取任意一期正片视频下评论
• 评论条数不少于1000条

第二步：词频统计并可视化展示

• 数据预处理：清理清洗评论中特殊字符（如：@#￥%、emoji表情符）,清洗后结果存储为txt文档
• 中文分词：添加新增词（如：青你、奥利给、冲鸭），去除停用词（如：哦、因此、不然、也好、但是）
• 统计top10高频词
• 可视化展示高频词

第三步：绘制词云

• 根据词频生成词云
• 可选项-添加背景图片，根据背景图片轮廓生成词云

第四步：结合PaddleHub，对评论进行内容审核

需要的配置和准备

• 中文分词需要jieba
• 词云绘制需要wordcloud
• 可视化展示中需要的中文字体
• 网上公开资源中找一个中文停用词表
• 根据分词结果自己制作新增词表
• 准备一张词云背景图（附加项，不做要求，可用hub抠图实现）
• paddlehub配置

!pip install jieba
!pip install wordcloud
Looking in indexes: https://pypi.mirrors.ustc.edu.cn/simple/
Requirement already satisfied: jieba in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (0.42.1)
Looking in indexes: https://pypi.mirrors.ustc.edu.cn/simple/
Requirement already satisfied: wordcloud in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (1.6.0)
Requirement already satisfied: numpy>=1.6.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from wordcloud) (1.16.4)
Requirement already satisfied: matplotlib in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from wordcloud) (2.2.3)
Requirement already satisfied: pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from wordcloud) (6.2.0)
Requirement already satisfied: pytz in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->wordcloud) (2019.3)
Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->wordcloud) (2.8.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->wordcloud) (2.4.2)
Requirement already satisfied: six>=1.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->wordcloud) (1.12.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->wordcloud) (1.1.0)
Requirement already satisfied: cycler>=0.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->wordcloud) (0.10.0)
Requirement already satisfied: setuptools in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from kiwisolver>=1.0.1->matplotlib->wordcloud) (41.4.0)

# Linux系统默认字体文件路径
# !ls /usr/share/fonts/
# 查看系统可用的ttf格式中文字体
!fc-list :lang=zh | grep ".ttf"
/home/aistudio/.fonts/simhei.ttf: SimHei,黑体:style=Regular,Normal,obyčejné,Standard,Κανονικά,Normaali,Normál,Normale,Standaard,Normalny,Обычный,Normálne,Navadno,Arrunta

# !wget https://mydueros.cdn.bcebos.com/font/simhei.ttf # 下载中文字体
# #创建字体目录fonts
!mkdir .fonts
# # 复制字体文件到该路径
!cp simhei.ttf .fonts/
mkdir: cannot create directory ‘.fonts’: File exists

#安装模型
!hub install porn_detection_lstm==1.1.0
!pip install --upgrade paddlehub
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
Module porn_detection_lstm-1.1.0 already installed in /home/aistudio/.paddlehub/modules/porn_detection_lstm
Looking in indexes: https://pypi.mirrors.ustc.edu.cn/simple/
Requirement already up-to-date: paddlehub in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (1.6.1)
Requirement already satisfied, skipping upgrade: gunicorn>=19.10.0; sys_platform != "win32" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (20.0.4)
Requirement already satisfied, skipping upgrade: flake8 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (3.7.9)
Requirement already satisfied, skipping upgrade: numpy; python_version >= "3" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (1.16.4)
Requirement already satisfied, skipping upgrade: cma==2.7.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (2.7.0)
Requirement already satisfied, skipping upgrade: colorlog in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (4.1.0)
Requirement already satisfied, skipping upgrade: flask>=1.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (1.1.1)
Requirement already satisfied, skipping upgrade: Pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (6.2.0)
Requirement already satisfied, skipping upgrade: opencv-python in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (4.1.1.26)
Requirement already satisfied, skipping upgrade: chardet==3.0.4 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (3.0.4)
Requirement already satisfied, skipping upgrade: pandas; python_version >= "3" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (0.23.4)
Requirement already satisfied, skipping upgrade: protobuf>=3.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (3.10.0)
Requirement already satisfied, skipping upgrade: requests in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (2.22.0)
Requirement already satisfied, skipping upgrade: tb-paddle in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (0.3.6)
Requirement already satisfied, skipping upgrade: yapf==0.26.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (0.26.0)
Requirement already satisfied, skipping upgrade: nltk in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (3.4.5)
Requirement already satisfied, skipping upgrade: sentencepiece in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (0.1.85)
Requirement already satisfied, skipping upgrade: tensorboard>=1.15 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (2.1.0)
Requirement already satisfied, skipping upgrade: pre-commit in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (1.21.0)
Requirement already satisfied, skipping upgrade: pyyaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (5.1.2)
Requirement already satisfied, skipping upgrade: six>=1.10.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (1.12.0)
Requirement already satisfied, skipping upgrade: setuptools>=3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from gunicorn>=19.10.0; sys_platform != "win32"->paddlehub) (41.4.0)
Requirement already satisfied, skipping upgrade: entrypoints<0.4.0,>=0.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8->paddlehub) (0.3)
Requirement already satisfied, skipping upgrade: mccabe<0.7.0,>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8->paddlehub) (0.6.1)
Requirement already satisfied, skipping upgrade: pycodestyle<2.6.0,>=2.5.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8->paddlehub) (2.5.0)
Requirement already satisfied, skipping upgrade: pyflakes<2.2.0,>=2.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8->paddlehub) (2.1.1)
Requirement already satisfied, skipping upgrade: Werkzeug>=0.15 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.0->paddlehub) (0.16.0)
Requirement already satisfied, skipping upgrade: click>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.0->paddlehub) (7.0)
Requirement already satisfied, skipping upgrade: itsdangerous>=0.24 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.0->paddlehub) (1.1.0)
Requirement already satisfied, skipping upgrade: Jinja2>=2.10.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.0->paddlehub) (2.10.3)
Requirement already satisfied, skipping upgrade: python-dateutil>=2.5.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pandas; python_version >= "3"->paddlehub) (2.8.0)
Requirement already satisfied, skipping upgrade: pytz>=2011k in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pandas; python_version >= "3"->paddlehub) (2019.3)
Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->paddlehub) (2.8)
Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->paddlehub) (1.25.6)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->paddlehub) (2019.9.11)
Requirement already satisfied, skipping upgrade: moviepy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from tb-paddle->paddlehub) (1.0.1)
Requirement already satisfied, skipping upgrade: grpcio>=1.24.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from tensorboard>=1.15->paddlehub) (1.26.0)
Requirement already satisfied, skipping upgrade: google-auth-oauthlib<0.5,>=0.4.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from tensorboard>=1.15->paddlehub) (0.4.1)
Requirement already satisfied, skipping upgrade: google-auth<2,>=1.6.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from tensorboard>=1.15->paddlehub) (1.10.0)
Requirement already satisfied, skipping upgrade: markdown>=2.6.8 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from tensorboard>=1.15->paddlehub) (3.1.1)
Requirement already satisfied, skipping upgrade: wheel>=0.26; python_version >= "3" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from tensorboard>=1.15->paddlehub) (0.33.6)
Requirement already satisfied, skipping upgrade: absl-py>=0.4 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from tensorboard>=1.15->paddlehub) (0.8.1)
Requirement already satisfied, skipping upgrade: toml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->paddlehub) (0.10.0)
Requirement already satisfied, skipping upgrade: aspy.yaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->paddlehub) (1.3.0)
Requirement already satisfied, skipping upgrade: identify>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->paddlehub) (1.4.10)
Requirement already satisfied, skipping upgrade: cfgv>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->paddlehub) (2.0.1)
Requirement already satisfied, skipping upgrade: nodeenv>=0.11.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->paddlehub) (1.3.4)
Requirement already satisfied, skipping upgrade: virtualenv>=15.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->paddlehub) (16.7.9)
Requirement already satisfied, skipping upgrade: importlib-metadata; python_version < "3.8" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->paddlehub) (0.23)
Requirement already satisfied, skipping upgrade: MarkupSafe>=0.23 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask>=1.1.0->paddlehub) (1.1.1)
Requirement already satisfied, skipping upgrade: decorator<5.0,>=4.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from moviepy->tb-paddle->paddlehub) (4.4.0)
Requirement already satisfied, skipping upgrade: imageio-ffmpeg>=0.2.0; python_version >= "3.4" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from moviepy->tb-paddle->paddlehub) (0.3.0)
Requirement already satisfied, skipping upgrade: proglog<=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from moviepy->tb-paddle->paddlehub) (0.1.9)
Requirement already satisfied, skipping upgrade: imageio<3.0,>=2.5; python_version >= "3.4" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from moviepy->tb-paddle->paddlehub) (2.6.1)
Requirement already satisfied, skipping upgrade: tqdm<5.0,>=4.11.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from moviepy->tb-paddle->paddlehub) (4.36.1)
Requirement already satisfied, skipping upgrade: requests-oauthlib>=0.7.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=1.15->paddlehub) (1.3.0)
Requirement already satisfied, skipping upgrade: cachetools<5.0,>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard>=1.15->paddlehub) (4.0.0)
Requirement already satisfied, skipping upgrade: rsa<4.1,>=3.1.4 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard>=1.15->paddlehub) (4.0)
Requirement already satisfied, skipping upgrade: pyasn1-modules>=0.2.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard>=1.15->paddlehub) (0.2.7)
Requirement already satisfied, skipping upgrade: zipp>=0.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from importlib-metadata; python_version < "3.8"->pre-commit->paddlehub) (0.6.0)
Requirement already satisfied, skipping upgrade: oauthlib>=3.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=1.15->paddlehub) (3.1.0)
Requirement already satisfied, skipping upgrade: pyasn1>=0.1.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from rsa<4.1,>=3.1.4->google-auth<2,>=1.6.3->tensorboard>=1.15->paddlehub) (0.4.8)
Requirement already satisfied, skipping upgrade: more-itertools in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from zipp>=0.5->importlib-metadata; python_version < "3.8"->pre-commit->paddlehub) (7.2.0)

from __future__ import print_function
import requests
import json
import re #正则匹配
import time #时间处理模块
import jieba #中文分词
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
import matplotlib.font_manager as font_manager
from PIL import Image
from wordcloud import WordCloud  #绘制词云模块
import paddlehub as hub

#请求爱奇艺评论接口，返回response信息
def getMoveinfo(url):
    '''
    请求爱奇艺评论接口，返回response信息
    参数  url: 评论的url
    :return: response信息
    '''
    session = requests.Session()
    headers = {
        "User-Agent": "Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/11.0 Mobile/15A372 Safari/604.1",
        "Accept": "application/json",
        "Referer": "http://m.iqiyi.com/v_19rqriflzg.html",
        "Origin": "http://m.iqiyi.com",
        "Host": "sns-comment.iqiyi.com",
        "Connection": "keep-alive",
        "Accept-Language": "en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7,zh-TW;q=0.6",
        "Accept-Encoding": "gzip, deflate"
    }
    response = session.get(url, headers=headers)
    if response.status_code == 200:
        return response.text
    return None

#解析json数据，获取评论
    '''
    解析json数据，获取评论
    参数  lastId:最后一条评论ID  arr:存放文本的list
    :return: 新的lastId
    '''
def saveMovieInfoToFile(lastId, arr):
    url='https://sns-comment.iqiyi.com/v3/comment/get_comments.action?agent_type=118&agent_version=9.11.5&authcookie=null&business_type=17&content_id=15068699100&page=&page_size=10&types=time&last_id='
    url+=str(lastId)
    responseTxt = getMoveinfo(url)
    responseJson=json.loads(responseTxt)
    comments=responseJson['data']['comments']
    for val in comments:
        # print(val.keys())
        if 'content' in val.keys():
            print(val['content'])
            arr.append(val['content'])
        lastId = str(val['id'])
    return lastId

#去除文本中特殊字符

def clear_special_char(content):
    '''
    正则处理特殊字符
    参数 content:原文本
    return: 清除后的文本
    '''
    comp = re.compile('[^A-Z^a-z^0-9^\u4e00-\u9fa5]')
    return comp.sub('', content)

# text_zh = '$你好！我是个程%序^猿，标注!!#码农￥'
# print(clear_special_char(text_zh))

def fenci(text):
    '''
    利用jieba进行分词
    参数 text:需要分词的句子或文本
    return：分词结果
    '''
    # 添加自定义字典 add_words.txt
    # jieba.load_userdict('') 
    seg=jieba.lcut(text)
    return seg

def stopwordslist(file_path):
    '''
    创建停用词表
    参数 file_path:停用词文本路径
    return：停用词list
    '''
    # f= open(file_path, 'r') 
    # my_data = [i.strip('\n') for i in f]

    stopwords= [line.strip() for line in open(file_path,encoding='UTF-8').readline()]
    return stopwords

# file_path=r'/home/aistudio/stopwords/中文停用词表.txt'
# list=stopwordslist(file_path)
# print(list)

def movestopwords(sentence, stopwords, counts):
    '''
    去除停用词,统计词频
    参数 file_path:停用词文本路径 stopwords:停用词list counts: 词频统计结果
    return：None
    '''
    # out=[]
    for word in sentence:
        if word not in stopwords:
            if len(word) !=1:
                counts[word]=counts.get(word,0)+1
    return None

    

def drawcounts(counts, num):
    '''
    绘制词频统计表
    参数 counts: 词频统计结果 num:绘制topN
    return：none
    '''
    x_aixs=[]
    y_aixs=[]
    c_order=sorted(counts.items(), key=lambda x:x[1],reverse=True)
    for c in c_order[:num]:
        x_aixs.append(c[0])
        y_aixs.append(c[1])
    
    matplotlib.rcParams['font.sans-serif']=['SimHei']
    matplotlib.rcParams['axes.unicode_minus']=False
    plt.bar(x_aixs, y_aixs)
    plt.title('词频统计结果')
    plt.show()

def drawcloud(word_f):
    '''
    根据词频绘制词云图
    参数 word_f:统计出的词频结果
    return：none
    '''    
    cloud_mask=np.array(Image.open('cloud.jpg'))
    st=set(['东西', '这是'])
    wc=WordCloud(background_color='white',
    mask=cloud_mask,
    max_words=150,
    font_path='simhei.ttf',
    min_font_size=10,
    max_font_size=100,
    width=400,
    relative_scaling=0.3,
    stopwords=st)
    wc.fit_words(word_f)
    wc.to_file('pic.png')

def text_detection(text, file_path):
    '''
    使用hub对评论进行内容分析
    return：分析结果

    '''
    porn_detection_lstm=hub.Module(name='porn_detection_lstm')
    f=open('aqy.txt', 'r', encoding='utf-8')
    for line in f:
        if len(line.strip())==1:
            continue
        else:
            test_text.append(line)
    f.close()

    input_dict={'text':test_text}
    results=porn_detection_lstm.detection(data=input_dict,use_gpu=True,batch_size=1)
    for index, item in enumerate(results):
        if item['porn_detection_key'] =='porn':
            print(item['text'],':', item['porn_probs'])

#评论是多分页的，得多次请求爱奇艺的评论接口才能获取多页评论,有些评论含有表情、特殊字符之类的
#num 是页数，一页10条评论，假如爬取1000条评论，设置num=100
## 转换数据
if __name__ == '__main__':
    num=110
    lastId='0'
    arr=[]
    with open('aqy.txt', 'a', encoding='utf-8') as f:
        for i in range(num):
            lastId=saveMovieInfoToFile(lastId, arr)
            time.sleep(0.5)
        for item in arr:
            item=clear_special_char(item)
            if item.strip()!='':
                try:
                    f.write(item+'\n')
                except  e:
                    print('含有特殊字符')
    print("共获取评论：", len(arr))
    f=open('aqy.txt', 'r', encoding='utf-8')
    counts={}
    for line in f:
        words=fenci(line)
        stopwords=stopwordslist(r'./stopwords/中文停用词表.txt')
        movestopwords(words, stopwords, counts)
    drawcounts(counts, 10)
    drawcloud(counts)
    f.close()

    file_path='aqy.txt'
    test_text=[]
    text_detection(test_text, file_path)
孔雪儿有点像张嘉倪，又有像唐嫣
虞书欣真的是太顽皮了，竟然能说出谁比她丑就坐在旁边，毫不疑问肯定是坐在赵小棠旁边了，真的给我们快乐啊。
青春有你2乃万  实力rapper乃万 原创歌手乃万   可盐可甜乃万  可奶可酷乃万 美貌杀人乃万  绝美嗓音乃万 心动狙击乃万   魅力爆表乃万  舞台王者乃万
你看到这一期之后，就知道虞书欣的造型是多么的好笑呢，这会让人家的心情会变得好起来哟。
那个说只有你一个人不喜欢陈珏的，告诉你只有你一个人。
刘雨昕小姐姐
你是最棒的呦!!!
哈哈
演员
如果排名全部都只算制作人的助力值，那分班的作用气势不大
上官呢
大虞海棠真的是给我们带来了很多的快乐呢。比如人家哭的时候，她就说别哭了哟，而且还摆了造型，讨人家可爱呢。
戴燕妮，金子涵，戴燕妮，金子涵，戴燕妮。金子涵
戴燕妮戴燕妮，金子涵金子涵，戴燕妮戴燕妮，金子涵金子涵，戴燕妮金子涵戴燕妮金子涵，戴燕妮金子涵，戴燕妮金子涵
戴燕妮姐姐戴燕妮姐姐我爱你我爱你
戴燕妮姐姐
喻言冲鸭!
别在这里说青你学员差好吗，去看看创造营的那帮孩子吧，要多没实力多没实力
[呲牙][呲牙][呲牙][呲牙]
那个爆炸头好丑。[得意][得意][得意][得意][得意]
你的包扎头好丑哦
永远支持大鱼海棠
虞书欣
真的，喻言真的好像一个男班，稳住方向、活跃氛围。经常说点逗比话让大家情绪积极一点，看着她们真好！
看起来很高冷的可妮，私底下其实是关心队友，愿意把外套分给别人穿的女孩，一路走来，看着她们成长。
虞书欣的眨眨眼果然是很撩人啊，竟然把赵小棠都撩了，然后就帮她拧盖了，她们两个说话还有意思啊。
虞书欣说话还真的太有魔性了，竟然宁瓶盖都这么可爱，真的是把我给笑哭了呢。
乃万加油
来看一下这一期吧，真的是给人家的快乐的，如果你不开心的话，那就快带你来看哟。
赵小棠还蛮有意思的啊，看到虞书欣就说了一个张三丰，蛮搞笑的，都不知道张三丰这个称号是怎么得来的呢？
欣欣你最棒
要笑死谁虞书欣
许馨文真的是我看着变好的！唱歌实力真的强，她看上去憨憨的，希望她越走越好吧！
虞书欣❤️
刘雨昕
嗯？
刘雨昕刘雨昕刘雨昕[抓狂][抓狂][抓狂]
今天的怎么没看见
她们的努力与懈怠、前进与落后、优点与缺陷，都理应被共同正视。加油吧！
刘雨昕给我冲！！！！！！！
9.23分了
蔡PD我来了
蔡徐坤
我想看虞书欣，关键时刻咋还没更新呢[流泪]......慢慢等待
虞书欣你很好，你真实，做好你自己加油
今天为什么没有更新
今天怎么回事，不是8点正播放吗？现在都8点18了怎么回事？
#.................................................
#精简博文体量需要，此处手动删去了1000多行爬取到的评论信息。
#.................................................
尝试不一样的风格让你有不一样的自己！
1
加油
孔雪儿，冲呀！
奥利给！
加
昕昕爱你
共获取评论： 1097

[2020-04-28 02:02:29,061] [    INFO] - Installing porn_detection_lstm module
[2020-04-28 02:02:29,064] [    INFO] - Module porn_detection_lstm already installed in /home/aistudio/.paddlehub/modules/porn_detection_lstm
让我康下我昕姐的粉在哪里吖刘总真的太了好爱好爱色
 : 0.7552
刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣啊啊啊啊妈妈爱你嗷嗷嗷嗷嗷么么哒亲亲亲亲亲亲亲亲加油喔
 : 0.8556
喻言刘雨欣曾可妮谢可寅刘令姿许佳琪这几个比较适合色色
 : 0.9986
让我康下我昕姐的粉在哪里吖刘总真的太了好爱好爱色
 : 0.7552
刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣啊啊啊啊妈妈爱你嗷嗷嗷嗷嗷么么哒亲亲亲亲亲亲亲亲加油喔
 : 0.8556
喻言刘雨欣曾可妮谢可寅刘令姿许佳琪这几个比较适合色色
 : 0.9986
让我康下我昕姐的粉在哪里吖刘总真的太了好爱好爱色
 : 0.7552
刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣啊啊啊啊妈妈爱你嗷嗷嗷嗷嗷么么哒亲亲亲亲亲亲亲亲加油喔
 : 0.8556
喻言刘雨欣曾可妮谢可寅刘令姿许佳琪这几个比较适合色色
 : 0.9986
让我康下我昕姐的粉在哪里吖刘总真的太了好爱好爱色
 : 0.7552
刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣啊啊啊啊妈妈爱你嗷嗷嗷嗷嗷么么哒亲亲亲亲亲亲亲亲加油喔
 : 0.8556
喻言刘雨欣曾可妮谢可寅刘令姿许佳琪这几个比较适合色色
 : 0.9986
让我康下我昕姐的粉在哪里吖刘总真的太了好爱好爱色
 : 0.7552
刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣啊啊啊啊妈妈爱你嗷嗷嗷嗷嗷么么哒亲亲亲亲亲亲亲亲加油喔
 : 0.8556
喻言刘雨欣曾可妮谢可寅刘令姿许佳琪这几个比较适合色色
 : 0.9986
让我康下我昕姐的粉在哪里吖刘总真的太了好爱好爱色
 : 0.7552
刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣啊啊啊啊妈妈爱你嗷嗷嗷嗷嗷么么哒亲亲亲亲亲亲亲亲加油喔
 : 0.8556
喻言刘雨欣曾可妮谢可寅刘令姿许佳琪这几个比较适合色色
 : 0.9986
让我康下我昕姐的粉在哪里吖刘总真的太了好爱好爱色
 : 0.7552
刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣啊啊啊啊妈妈爱你嗷嗷嗷嗷嗷么么哒亲亲亲亲亲亲亲亲加油喔
 : 0.8556
喻言刘雨欣曾可妮谢可寅刘令姿许佳琪这几个比较适合色色
 : 0.9986
让我康下我昕姐的粉在哪里吖刘总真的太了好爱好爱色
 : 0.7552
刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣啊啊啊啊妈妈爱你嗷嗷嗷嗷嗷么么哒亲亲亲亲亲亲亲亲加油喔
 : 0.8556
喻言刘雨欣曾可妮谢可寅刘令姿许佳琪这几个比较适合色色
 : 0.9986
让我康下我昕姐的粉在哪里吖刘总真的太了好爱好爱色
 : 0.7552
刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣啊啊啊啊妈妈爱你嗷嗷嗷嗷嗷么么哒亲亲亲亲亲亲亲亲加油喔
 : 0.8556
喻言刘雨欣曾可妮谢可寅刘令姿许佳琪这几个比较适合色色
 : 0.9986
让我康下我昕姐的粉在哪里吖刘总真的太了好爱好爱色
 : 0.7552
刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣啊啊啊啊妈妈爱你嗷嗷嗷嗷嗷么么哒亲亲亲亲亲亲亲亲加油喔
 : 0.8556
喻言刘雨欣曾可妮谢可寅刘令姿许佳琪这几个比较适合色色
 : 0.9986
让我康下我昕姐的粉在哪里吖刘总真的太了好爱好爱色
 : 0.7552
刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣啊啊啊啊妈妈爱你嗷嗷嗷嗷嗷么么哒亲亲亲亲亲亲亲亲加油喔
 : 0.8556
喻言刘雨欣曾可妮谢可寅刘令姿许佳琪这几个比较适合色色
 : 0.9986
让我康下我昕姐的粉在哪里吖刘总真的太了好爱好爱色
 : 0.7552
刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣刘雨欣啊啊啊啊妈妈爱你嗷嗷嗷嗷嗷么么哒亲亲亲亲亲亲亲亲加油喔
 : 0.8556
喻言刘雨欣曾可妮谢可寅刘令姿许佳琪这几个比较适合色色
 : 0.9986
我也喜欢虞书欣色色色
 : 0.9947
虞书欣棒棒哒
 : 0.6694
虞书欣是最好的一开始的评级并不算什么欣欣已经很努力了加油石榴们挺你色色色
 : 0.9955
我怎么觉得没必要继续下去了直接出道吧这九人是最理想的了色色色
 : 0.9974
许佳琪宝贝棒棒的加油我会永远支持你哒么么哒亲亲亲亲亲亲
 : 0.8274
大家帮忙投一投许佳琪吧她真的是一个非常棒的姐姐虽然声乐不行但是她也说过自己会努力的她的舞蹈也是棒棒的她都掉排名了希望大家可以多多支持kiki
 : 0.8494
色色色
 : 0.9998
色色色色色
 : 0.9998
坤坤色色
 : 0.902
坤学长好师啊啊啊啊啊啊啊啊啊色
 : 0.5859
虞书欣赵小棠孔雪儿色色
 : 0.971
欣欣子冲冲冲色色
 : 0.8359
谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅谢可寅色色色色色色色色色色色色
 : 0.9795
我要喻言高位出道色色色色
 : 0.9993
赵小棠赵小棠虞书欣虞书欣色色色
 : 0.9788
雪儿姐姐加油你真的好漂亮待你成团出道我要去看你的演出加油
 : 0.5526

display(Image.open('pic.png')) #显示生成的词云图像

顺便贴出原始背景图像（cloud.jpg）：