当我们遇到一大段文本该如何获取文本中热词,首先我先对文本进行序列化(序列化过程就不说了),然后对其进行统计:
words = [
'look','into','my','eyes','look','into','my','eyes','the',
'eyes','the','eyes','the','eyes','not','around','the','eyes',
"don't",'look','around','the','eyes','look','into','my','eyes',
"you're",'under'
]
# 集合对列表去重,防止重复统计
words_set = set(words)
words_dict = {}
# 遍历集合中的每个单词,并统计每一个单词出现次数,存入字典
for i in words_set:
words_dict[i] = words.count(i)
print(words_dict)
# 按字典的值对字典进行降序排序
sorted(words_dict.items(),key = lambda x:x[1],reverse=True)
输出结果如下:
{'eyes': 8, 'not': 1, 'look': 4, 'into': 3, "you're": 1, "don't": 1, 'around': 2, 'the': 5, 'under': 1, 'my': 3}
[('eyes', 8), ('the', 5), ('look', 4),('into', 3), ('my', 3),
('around', 2), ('not', 1), ("you're", 1), ("don't", 1), ('under', 1)]
words = [
'look','into','my','eyes','look','into','my','eyes','the',
'eyes','the','eyes','the','eyes','not','around','the','eyes',
"don't",'look','around','the','eyes','look','into','my','eyes',
"you're",'under'
]
from collections import Counter
# 直接对每一个单词进行次数统计,返回Counterl类(就是一个字典),里面元素默认降序排序
word_counts = Counter(words)
print(word_counts)
# Counter类中most_common方法返回出现频率最高的4个单词
top_four = word_counts.most_common(4)
print(top_four)
输出结果如下:
Counter({'eyes': 8, 'the': 5, 'look': 4, 'into': 3, 'my': 3, 'around': 2, 'not': 1, "don't": 1, "you're": 1, 'under': 1})
[('eyes', 8), ('the', 5), ('look', 4), ('into', 3)]
Counter实例还很容易和数学运算操作相结合,比如:
from collections import Counter
words = [
'look','into','my','eyes','look','into','my','eyes','the',
'eyes','the','eyes','the','eyes','not','around','the','eyes',
"don't",'look','around','the','eyes','look','into','my','eyes',
"you're",'under'
]
words1 = ['why','are','you','not','looking','in','my','eyes']
a = Counter(words)
print('a:',a)
b = Counter(words1)
print('b:',b)
print("a+b:",a+b)
输出结果如下:
a: Counter({'eyes': 8, 'the': 5, 'look': 4, 'into': 3, 'my': 3, 'around': 2, 'not': 1, "don't": 1, "you're": 1, 'under': 1})
b: Counter({'why': 1, 'are': 1, 'you': 1, 'not': 1, 'looking': 1, 'in': 1, 'my': 1, 'eyes': 1})
a+b: Counter({'eyes': 9, 'the': 5, 'look': 4, 'my': 4, 'into': 3, 'not': 2, 'around': 2, "don't": 1, "you're": 1, 'under': 1, 'why': 1, 'are': 1, 'you': 1, 'looking': 1, 'in': 1})
使用该方法将带来的便利显而易见。