世界而世界
世界而世界

【NLP入门】赛题1-新闻文本分类-Task5-基于深度学习的文本分类2-1Word2Vec

赛题1-新闻文本分类-Task5-基于深度学习的文本分类2-1Word2Vec

本次任务四天完成

赛题

  • 赛题讲解
  • 赛题
  • 数据讲解
  • 机器学习讲解
  • 深度学习-fastText
  • 深度学习-word2vec

安装 pyTorch

  • 官网
  • 下载地址及安装说明

跑程序

  1. 先导入包
import logging
import random

import numpy as np
import torch

logging.basicConfig(level=logging.INFO, format='%(asctime)-15s %(levelname)s: %(message)s')

# set seed 
# 这个暂时没看懂
seed = 666
random.seed(seed)
np.random.seed(seed)
torch.cuda.manual_seed(seed)
torch.manual_seed(seed)
  1. 读取少量数据
# split data to 10 fold
fold_num = 10
data_file = './data/1/train_set.csv'
import pandas as pd

train_set = pd.read_csv(data_file, sep='\t', nrows=1000)
train_set
  1. data to fold
def all_data2fold(fold_num, num=10000):
    fold_data = []
    f = pd.read_csv(data_file, sep='\t', encoding='UTF-8')
    texts = f['text'].tolist()[:num]
    labels = f['label'].tolist()[:num]

    total = len(labels)

    index = list(range(total))
    np.random.shuffle(index)

    all_texts = []
    all_labels = []
    for i in index:
        all_texts.append(texts[i])
        all_labels.append(labels[i])

    label2id = {}
    for i in range(total):
        label = str(all_labels[i])
        if label not in label2id:
            label2id[label] = [i]
        else:
            label2id[label].append(i)
            
    all_index = [[] for _ in range(fold_num)]
    for label, data in label2id.items():
        # print(label, len(data))
        batch_size = int(len(data) / fold_num)
        other = len(data) - batch_size * fold_num
        for i in range(fold_num):
            cur_batch_size = batch_size + 1 if i < other else batch_size
            # print(cur_batch_size)
            batch_data = [data[i * batch_size + b] for b in range(cur_batch_size)]
            all_index[i].extend(batch_data)

    batch_size = int(total / fold_num)
    other_texts = []
    other_labels = []
    other_num = 0
    start = 0
    
    for fold in range(fold_num):
        num = len(all_index[fold])
        texts = [all_texts[i] for i in all_index[fold]]
        labels = [all_labels[i] for i in all_index[fold]]

        if num > batch_size:
            fold_texts = texts[:batch_size]
            other_texts.extend(texts[batch_size:])
            fold_labels = labels[:batch_size]
            other_labels.extend(labels[batch_size:])
            other_num += num - batch_size
        elif num < batch_size:
            end = start + batch_size - num
            fold_texts = texts + other_texts[start: end]
            fold_labels = labels + other_labels[start: end]
            start = end
        else:
            fold_texts = texts
            fold_labels = labels

        assert batch_size == len(fold_labels)

        # shuffle
        index = list(range(batch_size))
        np.random.shuffle(index)

        shuffle_fold_texts = []
        shuffle_fold_labels = []
        for i in index:
            shuffle_fold_texts.append(fold_texts[i])
            shuffle_fold_labels.append(fold_labels[i])

        data = {'label': shuffle_fold_labels, 'text': shuffle_fold_texts}
        fold_data.append(data)
        
    logging.info("Fold lens %s", str([len(data['label']) for data in fold_data]))

    return fold_data


fold_data = all_data2fold(10)
  1. build train data for word2vec
# build train data for word2vec
fold_id = 9

train_texts = []
for i in range(0, fold_id):
    data = fold_data[i]
    train_texts.extend(data['text'])
    
logging.info('Total %d docs.' % len(train_texts))
  1. 训练并保存模型
logging.info('Start training...')
from gensim.models.word2vec import Word2Vec

num_features = 100     # Word vector dimensionality
num_workers = 8       # Number of threads to run in parallel

train_texts = list(map(lambda x: list(x.split()), train_texts))
model = Word2Vec(train_texts, workers=num_workers, size=num_features)
model.init_sims(replace=True)

# save model
model.save("./data/1/word2vec.bin")
  1. 加载模型并格式化
# load model
model = Word2Vec.load("./data/1/word2vec.bin")

# convert format
model.wv.save_word2vec_format('./data/1/word2vec.txt', binary=False)
  1. 跑全量
# 跑全量

train_set = pd.read_csv(data_file, sep='\t')
train_set

fold_data = all_data2fold(10, 250000)


# build train data for word2vec
fold_id = 9

train_texts = []
for i in range(0, fold_id):
    data = fold_data[i]
    train_texts.extend(data['text'])
    
logging.info('Total %d docs.' % len(train_texts))

logging.info('Start training...')
from gensim.models.word2vec import Word2Vec

num_features = 100     # Word vector dimensionality
num_workers = 16       # Number of threads to run in parallel

train_texts = list(map(lambda x: list(x.split()), train_texts))
model = Word2Vec(train_texts, workers=num_workers, size=num_features)
model.init_sims(replace=True)

# save model
model.save("./data/1/word2vec_full.bin")

# load model
model = Word2Vec.load("./data/1/word2vec_full.bin")

# convert format
model.wv.save_word2vec_format('./data/1/word2vec_full.txt', binary=False)

# 日志
2020-07-31 13:20:18,167 INFO: Fold lens [20000, 20000, 20000, 20000, 20000, 20000, 20000, 20000, 20000, 20000]
2020-07-31 13:20:18,324 INFO: Total 180000 docs.
2020-07-31 13:20:18,325 INFO: Start training...
2020-07-31 13:21:12,556 INFO: collecting all words and their counts
2020-07-31 13:21:12,583 INFO: PROGRESS: at sentence #0, processed 0 words, keeping 0 word types
2020-07-31 13:21:14,992 INFO: PROGRESS: at sentence #10000, processed 9130106 words, keeping 5270 word types
2020-07-31 13:21:17,295 INFO: PROGRESS: at sentence #20000, processed 18164877 words, keeping 5657 word types
2020-07-31 13:21:19,485 INFO: PROGRESS: at sentence #30000, processed 27276746 words, keeping 5869 word types
2020-07-31 13:21:21,800 INFO: PROGRESS: at sentence #40000, processed 36152077 words, keeping 6032 word types
2020-07-31 13:21:24,144 INFO: PROGRESS: at sentence #50000, processed 45041581 words, keeping 6141 word types
2020-07-31 13:21:26,447 INFO: PROGRESS: at sentence #60000, processed 54232818 words, keeping 6241 word types
2020-07-31 13:21:28,571 INFO: PROGRESS: at sentence #70000, processed 63209681 words, keeping 6309 word types
2020-07-31 13:21:30,462 INFO: PROGRESS: at sentence #80000, processed 72425114 words, keeping 6394 word types
2020-07-31 13:21:32,251 INFO: PROGRESS: at sentence #90000, processed 81546121 words, keeping 6455 word types
2020-07-31 13:21:34,020 INFO: PROGRESS: at sentence #100000, processed 90528591 words, keeping 6503 word types
2020-07-31 13:21:35,957 INFO: PROGRESS: at sentence #110000, processed 99636366 words, keeping 6546 word types
2020-07-31 13:21:37,771 INFO: PROGRESS: at sentence #120000, processed 108810893 words, keeping 6609 word types
2020-07-31 13:21:39,586 INFO: PROGRESS: at sentence #130000, processed 117956062 words, keeping 6649 word types
2020-07-31 13:21:41,520 INFO: PROGRESS: at sentence #140000, processed 127001041 words, keeping 6683 word types
2020-07-31 13:21:43,417 INFO: PROGRESS: at sentence #150000, processed 136142501 words, keeping 6709 word types
2020-07-31 13:21:45,430 INFO: PROGRESS: at sentence #160000, processed 145281408 words, keeping 6761 word types
2020-07-31 13:21:47,425 INFO: PROGRESS: at sentence #170000, processed 154309124 words, keeping 6797 word types
2020-07-31 13:21:49,944 INFO: collected 6819 word types from a corpus of 163437638 raw words and 180000 sentences
2020-07-31 13:21:49,952 INFO: Loading a fresh vocabulary
2020-07-31 13:21:50,036 INFO: effective_min_count=5 retains 5981 unique words (87% of original 6819, drops 838)
2020-07-31 13:21:50,036 INFO: effective_min_count=5 leaves 163435977 word corpus (99% of original 163437638, drops 1661)
2020-07-31 13:21:50,060 INFO: deleting the raw counts dictionary of 6819 items
2020-07-31 13:21:50,070 INFO: sample=0.001 downsamples 62 most-common words
2020-07-31 13:21:50,070 INFO: downsampling leaves estimated 141079171 word corpus (86.3% of prior 163435977)
2020-07-31 13:21:50,087 INFO: estimated required memory for 5981 words and 100 dimensions: 7775300 bytes
2020-07-31 13:21:50,088 INFO: resetting layer weights
2020-07-31 13:21:51,172 INFO: training model with 16 workers on 5981 vocabulary and 100 features, using sg=0 hs=0 sample=0.001 negative=5 window=5
2020-07-31 13:21:52,213 INFO: EPOCH 1 - PROGRESS: at 1.15% examples, 1632418 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:21:53,218 INFO: EPOCH 1 - PROGRESS: at 2.33% examples, 1632715 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:21:54,224 INFO: EPOCH 1 - PROGRESS: at 3.34% examples, 1565865 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:21:55,230 INFO: EPOCH 1 - PROGRESS: at 4.17% examples, 1457888 words/s, in_qsize 32, out_qsize 2
2020-07-31 13:21:56,233 INFO: EPOCH 1 - PROGRESS: at 5.04% examples, 1413364 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:21:57,239 INFO: EPOCH 1 - PROGRESS: at 5.99% examples, 1402917 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:21:58,285 INFO: EPOCH 1 - PROGRESS: at 6.66% examples, 1328337 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:21:59,307 INFO: EPOCH 1 - PROGRESS: at 7.58% examples, 1319193 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:00,312 INFO: EPOCH 1 - PROGRESS: at 8.59% examples, 1328107 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:22:01,315 INFO: EPOCH 1 - PROGRESS: at 9.73% examples, 1354074 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:22:02,317 INFO: EPOCH 1 - PROGRESS: at 10.77% examples, 1365465 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:03,323 INFO: EPOCH 1 - PROGRESS: at 11.77% examples, 1367189 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:04,328 INFO: EPOCH 1 - PROGRESS: at 12.88% examples, 1380325 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:22:05,344 INFO: EPOCH 1 - PROGRESS: at 13.96% examples, 1386061 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:22:06,347 INFO: EPOCH 1 - PROGRESS: at 15.00% examples, 1394157 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:07,356 INFO: EPOCH 1 - PROGRESS: at 16.00% examples, 1395515 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:08,368 INFO: EPOCH 1 - PROGRESS: at 17.05% examples, 1399388 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:22:09,379 INFO: EPOCH 1 - PROGRESS: at 18.13% examples, 1400447 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:22:10,388 INFO: EPOCH 1 - PROGRESS: at 19.27% examples, 1406652 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:11,392 INFO: EPOCH 1 - PROGRESS: at 20.35% examples, 1412626 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:22:12,395 INFO: EPOCH 1 - PROGRESS: at 21.54% examples, 1424993 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:22:13,399 INFO: EPOCH 1 - PROGRESS: at 22.79% examples, 1437218 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:14,400 INFO: EPOCH 1 - PROGRESS: at 23.99% examples, 1446832 words/s, in_qsize 28, out_qsize 4
2020-07-31 13:22:15,405 INFO: EPOCH 1 - PROGRESS: at 25.16% examples, 1455819 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:22:16,406 INFO: EPOCH 1 - PROGRESS: at 26.53% examples, 1469288 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:22:17,409 INFO: EPOCH 1 - PROGRESS: at 27.84% examples, 1481901 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:22:18,409 INFO: EPOCH 1 - PROGRESS: at 29.08% examples, 1492260 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:19,415 INFO: EPOCH 1 - PROGRESS: at 30.32% examples, 1499888 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:22:20,418 INFO: EPOCH 1 - PROGRESS: at 31.51% examples, 1506514 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:22:21,418 INFO: EPOCH 1 - PROGRESS: at 32.70% examples, 1513681 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:22:22,419 INFO: EPOCH 1 - PROGRESS: at 33.88% examples, 1520697 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:23,421 INFO: EPOCH 1 - PROGRESS: at 35.11% examples, 1525848 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:22:24,430 INFO: EPOCH 1 - PROGRESS: at 36.29% examples, 1529542 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:22:25,433 INFO: EPOCH 1 - PROGRESS: at 37.56% examples, 1535647 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:26,433 INFO: EPOCH 1 - PROGRESS: at 38.69% examples, 1535730 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:27,437 INFO: EPOCH 1 - PROGRESS: at 39.82% examples, 1536726 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:28,442 INFO: EPOCH 1 - PROGRESS: at 40.83% examples, 1533979 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:22:29,449 INFO: EPOCH 1 - PROGRESS: at 41.91% examples, 1534758 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:30,455 INFO: EPOCH 1 - PROGRESS: at 43.08% examples, 1536308 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:22:31,472 INFO: EPOCH 1 - PROGRESS: at 44.24% examples, 1538663 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:32,476 INFO: EPOCH 1 - PROGRESS: at 45.46% examples, 1542211 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:22:33,477 INFO: EPOCH 1 - PROGRESS: at 46.65% examples, 1544562 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:34,486 INFO: EPOCH 1 - PROGRESS: at 47.79% examples, 1546760 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:35,489 INFO: EPOCH 1 - PROGRESS: at 48.92% examples, 1547774 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:36,499 INFO: EPOCH 1 - PROGRESS: at 49.78% examples, 1539974 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:22:37,520 INFO: EPOCH 1 - PROGRESS: at 50.99% examples, 1542646 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:22:38,533 INFO: EPOCH 1 - PROGRESS: at 52.19% examples, 1543733 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:22:39,550 INFO: EPOCH 1 - PROGRESS: at 53.28% examples, 1541904 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:22:40,550 INFO: EPOCH 1 - PROGRESS: at 54.42% examples, 1542848 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:22:41,553 INFO: EPOCH 1 - PROGRESS: at 55.45% examples, 1541626 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:42,558 INFO: EPOCH 1 - PROGRESS: at 56.61% examples, 1542230 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:22:43,559 INFO: EPOCH 1 - PROGRESS: at 57.84% examples, 1546282 words/s, in_qsize 31, out_qsize 1
2020-07-31 13:22:44,565 INFO: EPOCH 1 - PROGRESS: at 58.97% examples, 1547196 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:45,570 INFO: EPOCH 1 - PROGRESS: at 60.08% examples, 1547264 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:22:46,586 INFO: EPOCH 1 - PROGRESS: at 61.31% examples, 1549707 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:22:47,590 INFO: EPOCH 1 - PROGRESS: at 62.47% examples, 1551058 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:22:48,595 INFO: EPOCH 1 - PROGRESS: at 63.65% examples, 1553694 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:22:49,608 INFO: EPOCH 1 - PROGRESS: at 64.82% examples, 1555700 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:50,614 INFO: EPOCH 1 - PROGRESS: at 66.04% examples, 1557343 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:51,631 INFO: EPOCH 1 - PROGRESS: at 66.98% examples, 1553587 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:52,641 INFO: EPOCH 1 - PROGRESS: at 68.03% examples, 1552013 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:53,645 INFO: EPOCH 1 - PROGRESS: at 69.00% examples, 1548429 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:22:54,652 INFO: EPOCH 1 - PROGRESS: at 69.83% examples, 1542826 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:22:55,652 INFO: EPOCH 1 - PROGRESS: at 70.72% examples, 1538669 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:56,655 INFO: EPOCH 1 - PROGRESS: at 71.65% examples, 1534758 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:22:57,658 INFO: EPOCH 1 - PROGRESS: at 72.60% examples, 1532066 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:22:58,661 INFO: EPOCH 1 - PROGRESS: at 73.51% examples, 1527684 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:22:59,672 INFO: EPOCH 1 - PROGRESS: at 74.48% examples, 1525946 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:00,674 INFO: EPOCH 1 - PROGRESS: at 75.44% examples, 1523053 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:01,682 INFO: EPOCH 1 - PROGRESS: at 76.42% examples, 1520543 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:02,701 INFO: EPOCH 1 - PROGRESS: at 77.35% examples, 1516918 words/s, in_qsize 32, out_qsize 4
2020-07-31 13:23:03,716 INFO: EPOCH 1 - PROGRESS: at 78.42% examples, 1516322 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:04,723 INFO: EPOCH 1 - PROGRESS: at 79.51% examples, 1516176 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:05,728 INFO: EPOCH 1 - PROGRESS: at 80.63% examples, 1516853 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:06,744 INFO: EPOCH 1 - PROGRESS: at 81.76% examples, 1517444 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:07,759 INFO: EPOCH 1 - PROGRESS: at 82.81% examples, 1517244 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:23:08,762 INFO: EPOCH 1 - PROGRESS: at 83.77% examples, 1515463 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:09,782 INFO: EPOCH 1 - PROGRESS: at 84.79% examples, 1514157 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:10,783 INFO: EPOCH 1 - PROGRESS: at 85.76% examples, 1512559 words/s, in_qsize 32, out_qsize 1
2020-07-31 13:23:11,796 INFO: EPOCH 1 - PROGRESS: at 86.84% examples, 1512420 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:12,803 INFO: EPOCH 1 - PROGRESS: at 87.93% examples, 1512111 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:13,805 INFO: EPOCH 1 - PROGRESS: at 89.03% examples, 1512788 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:23:14,809 INFO: EPOCH 1 - PROGRESS: at 90.09% examples, 1512060 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:15,822 INFO: EPOCH 1 - PROGRESS: at 91.12% examples, 1510881 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:16,832 INFO: EPOCH 1 - PROGRESS: at 92.20% examples, 1510524 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:17,845 INFO: EPOCH 1 - PROGRESS: at 93.28% examples, 1509743 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:18,849 INFO: EPOCH 1 - PROGRESS: at 94.02% examples, 1504712 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:23:19,854 INFO: EPOCH 1 - PROGRESS: at 95.03% examples, 1503291 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:20,860 INFO: EPOCH 1 - PROGRESS: at 96.25% examples, 1505576 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:21,869 INFO: EPOCH 1 - PROGRESS: at 97.33% examples, 1505733 words/s, in_qsize 30, out_qsize 2
2020-07-31 13:23:22,878 INFO: EPOCH 1 - PROGRESS: at 98.39% examples, 1505741 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:23:23,879 INFO: EPOCH 1 - PROGRESS: at 99.50% examples, 1506413 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:24,236 INFO: worker thread finished; awaiting finish of 15 more threads
2020-07-31 13:23:24,260 INFO: worker thread finished; awaiting finish of 14 more threads
2020-07-31 13:23:24,266 INFO: worker thread finished; awaiting finish of 13 more threads
2020-07-31 13:23:24,271 INFO: worker thread finished; awaiting finish of 12 more threads
2020-07-31 13:23:24,282 INFO: worker thread finished; awaiting finish of 11 more threads
2020-07-31 13:23:24,287 INFO: worker thread finished; awaiting finish of 10 more threads
2020-07-31 13:23:24,290 INFO: worker thread finished; awaiting finish of 9 more threads
2020-07-31 13:23:24,291 INFO: worker thread finished; awaiting finish of 8 more threads
2020-07-31 13:23:24,294 INFO: worker thread finished; awaiting finish of 7 more threads
2020-07-31 13:23:24,295 INFO: worker thread finished; awaiting finish of 6 more threads
2020-07-31 13:23:24,295 INFO: worker thread finished; awaiting finish of 5 more threads
2020-07-31 13:23:24,296 INFO: worker thread finished; awaiting finish of 4 more threads
2020-07-31 13:23:24,297 INFO: worker thread finished; awaiting finish of 3 more threads
2020-07-31 13:23:24,298 INFO: worker thread finished; awaiting finish of 2 more threads
2020-07-31 13:23:24,298 INFO: worker thread finished; awaiting finish of 1 more threads
2020-07-31 13:23:24,299 INFO: worker thread finished; awaiting finish of 0 more threads
2020-07-31 13:23:24,300 INFO: EPOCH - 1 : training on 163437638 raw words (140293279 effective words) took 93.1s, 1507020 effective words/s
2020-07-31 13:23:25,322 INFO: EPOCH 2 - PROGRESS: at 1.17% examples, 1650214 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:23:26,334 INFO: EPOCH 2 - PROGRESS: at 2.38% examples, 1658085 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:23:27,351 INFO: EPOCH 2 - PROGRESS: at 3.51% examples, 1637239 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:28,352 INFO: EPOCH 2 - PROGRESS: at 4.70% examples, 1640433 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:29,367 INFO: EPOCH 2 - PROGRESS: at 5.86% examples, 1639116 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:30,381 INFO: EPOCH 2 - PROGRESS: at 6.98% examples, 1626626 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:31,381 INFO: EPOCH 2 - PROGRESS: at 8.26% examples, 1650298 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:32,381 INFO: EPOCH 2 - PROGRESS: at 9.55% examples, 1664371 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:33,395 INFO: EPOCH 2 - PROGRESS: at 10.73% examples, 1665971 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:34,400 INFO: EPOCH 2 - PROGRESS: at 11.89% examples, 1659153 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:35,408 INFO: EPOCH 2 - PROGRESS: at 13.10% examples, 1661147 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:36,420 INFO: EPOCH 2 - PROGRESS: at 14.42% examples, 1673743 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:37,430 INFO: EPOCH 2 - PROGRESS: at 15.69% examples, 1685828 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:38,438 INFO: EPOCH 2 - PROGRESS: at 16.98% examples, 1695905 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:39,440 INFO: EPOCH 2 - PROGRESS: at 18.38% examples, 1706249 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:40,455 INFO: EPOCH 2 - PROGRESS: at 19.73% examples, 1712029 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:23:41,459 INFO: EPOCH 2 - PROGRESS: at 21.04% examples, 1718315 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:42,471 INFO: EPOCH 2 - PROGRESS: at 22.37% examples, 1724715 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:43,474 INFO: EPOCH 2 - PROGRESS: at 23.66% examples, 1729277 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:44,485 INFO: EPOCH 2 - PROGRESS: at 24.93% examples, 1730849 words/s, in_qsize 32, out_qsize 1
2020-07-31 13:23:45,488 INFO: EPOCH 2 - PROGRESS: at 26.28% examples, 1733434 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:46,491 INFO: EPOCH 2 - PROGRESS: at 27.55% examples, 1733922 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:47,496 INFO: EPOCH 2 - PROGRESS: at 28.70% examples, 1728944 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:48,506 INFO: EPOCH 2 - PROGRESS: at 29.95% examples, 1726919 words/s, in_qsize 32, out_qsize 2
2020-07-31 13:23:49,524 INFO: EPOCH 2 - PROGRESS: at 31.10% examples, 1723348 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:23:50,525 INFO: EPOCH 2 - PROGRESS: at 32.22% examples, 1719481 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:23:51,527 INFO: EPOCH 2 - PROGRESS: at 33.37% examples, 1716408 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:52,538 INFO: EPOCH 2 - PROGRESS: at 34.48% examples, 1711412 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:53,543 INFO: EPOCH 2 - PROGRESS: at 35.64% examples, 1707006 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:54,546 INFO: EPOCH 2 - PROGRESS: at 36.78% examples, 1704449 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:55,555 INFO: EPOCH 2 - PROGRESS: at 38.02% examples, 1701747 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:23:56,563 INFO: EPOCH 2 - PROGRESS: at 39.27% examples, 1701576 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:57,577 INFO: EPOCH 2 - PROGRESS: at 40.39% examples, 1698487 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:58,577 INFO: EPOCH 2 - PROGRESS: at 41.47% examples, 1694500 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:23:59,597 INFO: EPOCH 2 - PROGRESS: at 42.67% examples, 1692694 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:24:00,604 INFO: EPOCH 2 - PROGRESS: at 43.92% examples, 1695005 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:24:01,605 INFO: EPOCH 2 - PROGRESS: at 45.22% examples, 1698444 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:24:02,614 INFO: EPOCH 2 - PROGRESS: at 46.50% examples, 1699895 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:03,619 INFO: EPOCH 2 - PROGRESS: at 47.68% examples, 1699270 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:04,622 INFO: EPOCH 2 - PROGRESS: at 48.78% examples, 1695742 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:24:05,624 INFO: EPOCH 2 - PROGRESS: at 49.96% examples, 1694786 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:06,635 INFO: EPOCH 2 - PROGRESS: at 51.23% examples, 1696169 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:07,642 INFO: EPOCH 2 - PROGRESS: at 52.55% examples, 1698102 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:08,643 INFO: EPOCH 2 - PROGRESS: at 53.78% examples, 1698317 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:09,643 INFO: EPOCH 2 - PROGRESS: at 55.04% examples, 1699605 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:24:10,647 INFO: EPOCH 2 - PROGRESS: at 56.33% examples, 1701703 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:11,653 INFO: EPOCH 2 - PROGRESS: at 57.62% examples, 1703298 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:12,662 INFO: EPOCH 2 - PROGRESS: at 58.91% examples, 1706055 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:13,670 INFO: EPOCH 2 - PROGRESS: at 60.24% examples, 1709031 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:24:14,671 INFO: EPOCH 2 - PROGRESS: at 61.60% examples, 1711997 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:24:15,676 INFO: EPOCH 2 - PROGRESS: at 62.82% examples, 1712622 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:24:16,685 INFO: EPOCH 2 - PROGRESS: at 63.99% examples, 1712838 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:24:17,687 INFO: EPOCH 2 - PROGRESS: at 65.31% examples, 1715124 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:24:18,701 INFO: EPOCH 2 - PROGRESS: at 66.55% examples, 1714921 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:24:19,708 INFO: EPOCH 2 - PROGRESS: at 67.80% examples, 1715513 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:20,727 INFO: EPOCH 2 - PROGRESS: at 69.03% examples, 1714670 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:24:21,728 INFO: EPOCH 2 - PROGRESS: at 70.12% examples, 1712983 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:22,736 INFO: EPOCH 2 - PROGRESS: at 71.40% examples, 1713502 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:24:23,746 INFO: EPOCH 2 - PROGRESS: at 72.66% examples, 1714313 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:24:24,753 INFO: EPOCH 2 - PROGRESS: at 73.88% examples, 1714385 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:24:25,755 INFO: EPOCH 2 - PROGRESS: at 75.13% examples, 1715171 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:26,760 INFO: EPOCH 2 - PROGRESS: at 76.31% examples, 1713672 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:27,761 INFO: EPOCH 2 - PROGRESS: at 77.53% examples, 1713213 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:28,776 INFO: EPOCH 2 - PROGRESS: at 78.70% examples, 1711536 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:29,786 INFO: EPOCH 2 - PROGRESS: at 79.83% examples, 1709369 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:30,791 INFO: EPOCH 2 - PROGRESS: at 80.99% examples, 1708386 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:31,797 INFO: EPOCH 2 - PROGRESS: at 82.22% examples, 1708489 words/s, in_qsize 28, out_qsize 6
2020-07-31 13:24:32,812 INFO: EPOCH 2 - PROGRESS: at 83.39% examples, 1707727 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:24:33,825 INFO: EPOCH 2 - PROGRESS: at 84.54% examples, 1706240 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:34,832 INFO: EPOCH 2 - PROGRESS: at 85.62% examples, 1704300 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:24:35,844 INFO: EPOCH 2 - PROGRESS: at 86.74% examples, 1701606 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:24:36,848 INFO: EPOCH 2 - PROGRESS: at 87.95% examples, 1701441 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:24:37,850 INFO: EPOCH 2 - PROGRESS: at 89.08% examples, 1700244 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:38,852 INFO: EPOCH 2 - PROGRESS: at 90.32% examples, 1700242 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:39,864 INFO: EPOCH 2 - PROGRESS: at 91.52% examples, 1699570 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:24:40,866 INFO: EPOCH 2 - PROGRESS: at 92.78% examples, 1699750 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:41,870 INFO: EPOCH 2 - PROGRESS: at 94.01% examples, 1700398 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:24:42,880 INFO: EPOCH 2 - PROGRESS: at 95.12% examples, 1698031 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:24:43,891 INFO: EPOCH 2 - PROGRESS: at 96.27% examples, 1696513 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:44,897 INFO: EPOCH 2 - PROGRESS: at 97.04% examples, 1689082 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:24:45,905 INFO: EPOCH 2 - PROGRESS: at 97.90% examples, 1682927 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:24:46,910 INFO: EPOCH 2 - PROGRESS: at 99.08% examples, 1683055 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:24:47,605 INFO: worker thread finished; awaiting finish of 15 more threads
2020-07-31 13:24:47,610 INFO: worker thread finished; awaiting finish of 14 more threads
2020-07-31 13:24:47,613 INFO: worker thread finished; awaiting finish of 13 more threads
2020-07-31 13:24:47,629 INFO: worker thread finished; awaiting finish of 12 more threads
2020-07-31 13:24:47,640 INFO: worker thread finished; awaiting finish of 11 more threads
2020-07-31 13:24:47,647 INFO: worker thread finished; awaiting finish of 10 more threads
2020-07-31 13:24:47,651 INFO: worker thread finished; awaiting finish of 9 more threads
2020-07-31 13:24:47,667 INFO: worker thread finished; awaiting finish of 8 more threads
2020-07-31 13:24:47,668 INFO: worker thread finished; awaiting finish of 7 more threads
2020-07-31 13:24:47,669 INFO: worker thread finished; awaiting finish of 6 more threads
2020-07-31 13:24:47,670 INFO: worker thread finished; awaiting finish of 5 more threads
2020-07-31 13:24:47,671 INFO: worker thread finished; awaiting finish of 4 more threads
2020-07-31 13:24:47,672 INFO: worker thread finished; awaiting finish of 3 more threads
2020-07-31 13:24:47,674 INFO: worker thread finished; awaiting finish of 2 more threads
2020-07-31 13:24:47,675 INFO: worker thread finished; awaiting finish of 1 more threads
2020-07-31 13:24:47,677 INFO: worker thread finished; awaiting finish of 0 more threads
2020-07-31 13:24:47,677 INFO: EPOCH - 2 : training on 163437638 raw words (140292959 effective words) took 83.4s, 1683004 effective words/s
2020-07-31 13:24:48,696 INFO: EPOCH 3 - PROGRESS: at 1.21% examples, 1717039 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:24:49,703 INFO: EPOCH 3 - PROGRESS: at 2.39% examples, 1675230 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:24:50,722 INFO: EPOCH 3 - PROGRESS: at 3.58% examples, 1672249 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:51,730 INFO: EPOCH 3 - PROGRESS: at 4.60% examples, 1605726 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:24:52,733 INFO: EPOCH 3 - PROGRESS: at 5.74% examples, 1613014 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:24:53,755 INFO: EPOCH 3 - PROGRESS: at 6.75% examples, 1572750 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:24:54,758 INFO: EPOCH 3 - PROGRESS: at 7.94% examples, 1581005 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:24:55,764 INFO: EPOCH 3 - PROGRESS: at 9.06% examples, 1580854 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:56,784 INFO: EPOCH 3 - PROGRESS: at 10.23% examples, 1583095 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:24:57,786 INFO: EPOCH 3 - PROGRESS: at 11.51% examples, 1608816 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:58,815 INFO: EPOCH 3 - PROGRESS: at 12.85% examples, 1625336 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:24:59,836 INFO: EPOCH 3 - PROGRESS: at 14.12% examples, 1634376 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:25:00,836 INFO: EPOCH 3 - PROGRESS: at 15.27% examples, 1635116 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:25:01,839 INFO: EPOCH 3 - PROGRESS: at 16.49% examples, 1642325 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:02,859 INFO: EPOCH 3 - PROGRESS: at 17.81% examples, 1650928 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:25:03,861 INFO: EPOCH 3 - PROGRESS: at 19.16% examples, 1659545 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:04,865 INFO: EPOCH 3 - PROGRESS: at 20.40% examples, 1664191 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:25:05,868 INFO: EPOCH 3 - PROGRESS: at 21.61% examples, 1667195 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:06,869 INFO: EPOCH 3 - PROGRESS: at 22.76% examples, 1661060 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:07,878 INFO: EPOCH 3 - PROGRESS: at 24.02% examples, 1664651 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:08,881 INFO: EPOCH 3 - PROGRESS: at 25.23% examples, 1667567 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:09,887 INFO: EPOCH 3 - PROGRESS: at 26.62% examples, 1674029 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:10,887 INFO: EPOCH 3 - PROGRESS: at 27.96% examples, 1681519 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:25:11,891 INFO: EPOCH 3 - PROGRESS: at 29.00% examples, 1672459 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:25:12,892 INFO: EPOCH 3 - PROGRESS: at 30.13% examples, 1668219 words/s, in_qsize 32, out_qsize 3
2020-07-31 13:25:13,898 INFO: EPOCH 3 - PROGRESS: at 31.37% examples, 1671416 words/s, in_qsize 31, out_qsize 1
2020-07-31 13:25:14,905 INFO: EPOCH 3 - PROGRESS: at 32.53% examples, 1672403 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:25:15,911 INFO: EPOCH 3 - PROGRESS: at 33.72% examples, 1673848 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:25:16,912 INFO: EPOCH 3 - PROGRESS: at 34.90% examples, 1672206 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:17,930 INFO: EPOCH 3 - PROGRESS: at 36.08% examples, 1670606 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:25:18,932 INFO: EPOCH 3 - PROGRESS: at 37.23% examples, 1668672 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:25:19,933 INFO: EPOCH 3 - PROGRESS: at 38.41% examples, 1666219 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:25:20,939 INFO: EPOCH 3 - PROGRESS: at 39.58% examples, 1663980 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:25:21,943 INFO: EPOCH 3 - PROGRESS: at 40.72% examples, 1663338 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:22,959 INFO: EPOCH 3 - PROGRESS: at 41.84% examples, 1661496 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:25:23,963 INFO: EPOCH 3 - PROGRESS: at 43.00% examples, 1659545 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:25:24,964 INFO: EPOCH 3 - PROGRESS: at 43.99% examples, 1653435 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:25:25,969 INFO: EPOCH 3 - PROGRESS: at 45.22% examples, 1654382 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:25:26,970 INFO: EPOCH 3 - PROGRESS: at 46.53% examples, 1658505 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:25:27,979 INFO: EPOCH 3 - PROGRESS: at 47.82% examples, 1662773 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:25:28,995 INFO: EPOCH 3 - PROGRESS: at 49.19% examples, 1668185 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:25:30,004 INFO: EPOCH 3 - PROGRESS: at 50.57% examples, 1674115 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:25:31,011 INFO: EPOCH 3 - PROGRESS: at 51.96% examples, 1679141 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:32,017 INFO: EPOCH 3 - PROGRESS: at 53.36% examples, 1684778 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:25:33,017 INFO: EPOCH 3 - PROGRESS: at 54.56% examples, 1684252 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:34,028 INFO: EPOCH 3 - PROGRESS: at 55.85% examples, 1686937 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:35,039 INFO: EPOCH 3 - PROGRESS: at 57.01% examples, 1685317 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:36,046 INFO: EPOCH 3 - PROGRESS: at 58.25% examples, 1686657 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:37,049 INFO: EPOCH 3 - PROGRESS: at 59.68% examples, 1692476 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:25:38,050 INFO: EPOCH 3 - PROGRESS: at 61.05% examples, 1697254 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:25:39,056 INFO: EPOCH 3 - PROGRESS: at 62.34% examples, 1699172 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:25:40,062 INFO: EPOCH 3 - PROGRESS: at 63.66% examples, 1702986 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:25:41,063 INFO: EPOCH 3 - PROGRESS: at 64.95% examples, 1705884 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:42,072 INFO: EPOCH 3 - PROGRESS: at 66.32% examples, 1708664 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:43,073 INFO: EPOCH 3 - PROGRESS: at 67.69% examples, 1713123 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:44,077 INFO: EPOCH 3 - PROGRESS: at 68.94% examples, 1713278 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:25:45,080 INFO: EPOCH 3 - PROGRESS: at 70.05% examples, 1711671 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:46,080 INFO: EPOCH 3 - PROGRESS: at 71.32% examples, 1712599 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:25:47,092 INFO: EPOCH 3 - PROGRESS: at 72.55% examples, 1712854 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:25:48,095 INFO: EPOCH 3 - PROGRESS: at 73.77% examples, 1712531 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:49,102 INFO: EPOCH 3 - PROGRESS: at 74.51% examples, 1702095 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:50,103 INFO: EPOCH 3 - PROGRESS: at 75.66% examples, 1700382 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:51,106 INFO: EPOCH 3 - PROGRESS: at 76.82% examples, 1698327 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:25:52,114 INFO: EPOCH 3 - PROGRESS: at 77.95% examples, 1696682 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:53,119 INFO: EPOCH 3 - PROGRESS: at 79.14% examples, 1695559 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:25:54,132 INFO: EPOCH 3 - PROGRESS: at 80.28% examples, 1694051 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:55,148 INFO: EPOCH 3 - PROGRESS: at 81.49% examples, 1693943 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:56,150 INFO: EPOCH 3 - PROGRESS: at 82.78% examples, 1695741 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:25:57,150 INFO: EPOCH 3 - PROGRESS: at 84.05% examples, 1697627 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:25:58,151 INFO: EPOCH 3 - PROGRESS: at 85.25% examples, 1697961 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:25:59,163 INFO: EPOCH 3 - PROGRESS: at 86.54% examples, 1699201 words/s, in_qsize 32, out_qsize 4
2020-07-31 13:26:00,178 INFO: EPOCH 3 - PROGRESS: at 87.65% examples, 1697018 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:01,180 INFO: EPOCH 3 - PROGRESS: at 88.91% examples, 1697865 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:26:02,186 INFO: EPOCH 3 - PROGRESS: at 90.03% examples, 1695844 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:03,192 INFO: EPOCH 3 - PROGRESS: at 91.32% examples, 1697088 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:04,195 INFO: EPOCH 3 - PROGRESS: at 92.62% examples, 1697967 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:26:05,199 INFO: EPOCH 3 - PROGRESS: at 93.90% examples, 1699375 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:26:06,212 INFO: EPOCH 3 - PROGRESS: at 95.08% examples, 1698112 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:26:07,230 INFO: EPOCH 3 - PROGRESS: at 96.38% examples, 1699267 words/s, in_qsize 25, out_qsize 6
2020-07-31 13:26:08,256 INFO: EPOCH 3 - PROGRESS: at 97.71% examples, 1700848 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:09,264 INFO: EPOCH 3 - PROGRESS: at 99.04% examples, 1703530 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:09,874 INFO: worker thread finished; awaiting finish of 15 more threads
2020-07-31 13:26:09,886 INFO: worker thread finished; awaiting finish of 14 more threads
2020-07-31 13:26:09,893 INFO: worker thread finished; awaiting finish of 13 more threads
2020-07-31 13:26:09,899 INFO: worker thread finished; awaiting finish of 12 more threads
2020-07-31 13:26:09,915 INFO: worker thread finished; awaiting finish of 11 more threads
2020-07-31 13:26:09,919 INFO: worker thread finished; awaiting finish of 10 more threads
2020-07-31 13:26:09,923 INFO: worker thread finished; awaiting finish of 9 more threads
2020-07-31 13:26:09,923 INFO: worker thread finished; awaiting finish of 8 more threads
2020-07-31 13:26:09,925 INFO: worker thread finished; awaiting finish of 7 more threads
2020-07-31 13:26:09,925 INFO: worker thread finished; awaiting finish of 6 more threads
2020-07-31 13:26:09,927 INFO: worker thread finished; awaiting finish of 5 more threads
2020-07-31 13:26:09,928 INFO: worker thread finished; awaiting finish of 4 more threads
2020-07-31 13:26:09,933 INFO: worker thread finished; awaiting finish of 3 more threads
2020-07-31 13:26:09,936 INFO: worker thread finished; awaiting finish of 2 more threads
2020-07-31 13:26:09,937 INFO: worker thread finished; awaiting finish of 1 more threads
2020-07-31 13:26:09,938 INFO: worker thread finished; awaiting finish of 0 more threads
2020-07-31 13:26:09,939 INFO: EPOCH - 3 : training on 163437638 raw words (140288813 effective words) took 82.2s, 1705776 effective words/s
2020-07-31 13:26:10,950 INFO: EPOCH 4 - PROGRESS: at 1.30% examples, 1841276 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:26:11,954 INFO: EPOCH 4 - PROGRESS: at 2.61% examples, 1843639 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:26:12,962 INFO: EPOCH 4 - PROGRESS: at 4.01% examples, 1880543 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:26:13,966 INFO: EPOCH 4 - PROGRESS: at 5.32% examples, 1870706 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:14,972 INFO: EPOCH 4 - PROGRESS: at 6.51% examples, 1831565 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:26:15,975 INFO: EPOCH 4 - PROGRESS: at 7.66% examples, 1791141 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:16,988 INFO: EPOCH 4 - PROGRESS: at 8.87% examples, 1774787 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:18,000 INFO: EPOCH 4 - PROGRESS: at 10.16% examples, 1773211 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:26:19,024 INFO: EPOCH 4 - PROGRESS: at 11.31% examples, 1755988 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:20,024 INFO: EPOCH 4 - PROGRESS: at 12.56% examples, 1752894 words/s, in_qsize 32, out_qsize 5
2020-07-31 13:26:21,031 INFO: EPOCH 4 - PROGRESS: at 13.77% examples, 1744869 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:26:22,034 INFO: EPOCH 4 - PROGRESS: at 14.94% examples, 1738055 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:26:23,035 INFO: EPOCH 4 - PROGRESS: at 16.09% examples, 1733726 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:26:24,040 INFO: EPOCH 4 - PROGRESS: at 17.32% examples, 1729917 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:25,040 INFO: EPOCH 4 - PROGRESS: at 18.59% examples, 1726058 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:26,046 INFO: EPOCH 4 - PROGRESS: at 19.67% examples, 1710623 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:26:27,065 INFO: EPOCH 4 - PROGRESS: at 20.83% examples, 1704397 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:26:28,067 INFO: EPOCH 4 - PROGRESS: at 22.00% examples, 1701065 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:26:29,068 INFO: EPOCH 4 - PROGRESS: at 23.22% examples, 1700836 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:30,080 INFO: EPOCH 4 - PROGRESS: at 24.40% examples, 1696083 words/s, in_qsize 26, out_qsize 5
2020-07-31 13:26:31,139 INFO: EPOCH 4 - PROGRESS: at 25.63% examples, 1692629 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:32,144 INFO: EPOCH 4 - PROGRESS: at 26.89% examples, 1689654 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:26:33,147 INFO: EPOCH 4 - PROGRESS: at 28.13% examples, 1691400 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:26:34,150 INFO: EPOCH 4 - PROGRESS: at 29.47% examples, 1698701 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:26:35,156 INFO: EPOCH 4 - PROGRESS: at 30.79% examples, 1705169 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:26:36,158 INFO: EPOCH 4 - PROGRESS: at 32.06% examples, 1710270 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:26:37,179 INFO: EPOCH 4 - PROGRESS: at 33.41% examples, 1717469 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:26:38,203 INFO: EPOCH 4 - PROGRESS: at 34.79% examples, 1724042 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:39,211 INFO: EPOCH 4 - PROGRESS: at 35.98% examples, 1721174 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:26:40,216 INFO: EPOCH 4 - PROGRESS: at 37.32% examples, 1726267 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:26:41,218 INFO: EPOCH 4 - PROGRESS: at 38.70% examples, 1731155 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:42,218 INFO: EPOCH 4 - PROGRESS: at 40.01% examples, 1733752 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:26:43,226 INFO: EPOCH 4 - PROGRESS: at 41.31% examples, 1737930 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:44,238 INFO: EPOCH 4 - PROGRESS: at 42.64% examples, 1740447 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:26:45,242 INFO: EPOCH 4 - PROGRESS: at 43.80% examples, 1738204 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:26:46,247 INFO: EPOCH 4 - PROGRESS: at 45.07% examples, 1738638 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:47,249 INFO: EPOCH 4 - PROGRESS: at 46.33% examples, 1738972 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:26:48,256 INFO: EPOCH 4 - PROGRESS: at 47.63% examples, 1741741 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:49,260 INFO: EPOCH 4 - PROGRESS: at 48.92% examples, 1743924 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:26:50,264 INFO: EPOCH 4 - PROGRESS: at 50.25% examples, 1746550 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:26:51,267 INFO: EPOCH 4 - PROGRESS: at 51.52% examples, 1746502 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:52,270 INFO: EPOCH 4 - PROGRESS: at 52.88% examples, 1748729 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:26:53,288 INFO: EPOCH 4 - PROGRESS: at 54.14% examples, 1748282 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:54,297 INFO: EPOCH 4 - PROGRESS: at 55.33% examples, 1746719 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:55,317 INFO: EPOCH 4 - PROGRESS: at 56.65% examples, 1746828 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:26:56,323 INFO: EPOCH 4 - PROGRESS: at 57.95% examples, 1749126 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:26:57,328 INFO: EPOCH 4 - PROGRESS: at 59.21% examples, 1750079 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:26:58,329 INFO: EPOCH 4 - PROGRESS: at 60.45% examples, 1749422 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:26:59,336 INFO: EPOCH 4 - PROGRESS: at 61.74% examples, 1749374 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:27:00,339 INFO: EPOCH 4 - PROGRESS: at 62.97% examples, 1750102 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:01,349 INFO: EPOCH 4 - PROGRESS: at 64.16% examples, 1750224 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:27:02,353 INFO: EPOCH 4 - PROGRESS: at 65.49% examples, 1751642 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:27:03,353 INFO: EPOCH 4 - PROGRESS: at 66.81% examples, 1753328 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:27:04,365 INFO: EPOCH 4 - PROGRESS: at 68.16% examples, 1755543 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:05,372 INFO: EPOCH 4 - PROGRESS: at 69.50% examples, 1757789 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:27:06,393 INFO: EPOCH 4 - PROGRESS: at 70.80% examples, 1758544 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:27:07,396 INFO: EPOCH 4 - PROGRESS: at 72.15% examples, 1761068 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:27:08,411 INFO: EPOCH 4 - PROGRESS: at 73.47% examples, 1761805 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:27:09,413 INFO: EPOCH 4 - PROGRESS: at 74.77% examples, 1763819 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:10,417 INFO: EPOCH 4 - PROGRESS: at 76.16% examples, 1766230 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:11,419 INFO: EPOCH 4 - PROGRESS: at 77.48% examples, 1767267 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:12,419 INFO: EPOCH 4 - PROGRESS: at 78.80% examples, 1768357 words/s, in_qsize 32, out_qsize 2
2020-07-31 13:27:13,436 INFO: EPOCH 4 - PROGRESS: at 80.13% examples, 1769727 words/s, in_qsize 31, out_qsize 3
2020-07-31 13:27:14,444 INFO: EPOCH 4 - PROGRESS: at 81.48% examples, 1771635 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:15,449 INFO: EPOCH 4 - PROGRESS: at 82.75% examples, 1771917 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:16,453 INFO: EPOCH 4 - PROGRESS: at 84.03% examples, 1772853 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:27:17,456 INFO: EPOCH 4 - PROGRESS: at 85.34% examples, 1774233 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:27:18,462 INFO: EPOCH 4 - PROGRESS: at 86.71% examples, 1776030 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:19,470 INFO: EPOCH 4 - PROGRESS: at 88.02% examples, 1776669 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:27:20,477 INFO: EPOCH 4 - PROGRESS: at 89.32% examples, 1777505 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:27:21,480 INFO: EPOCH 4 - PROGRESS: at 90.66% examples, 1778299 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:27:22,485 INFO: EPOCH 4 - PROGRESS: at 92.01% examples, 1779400 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:23,489 INFO: EPOCH 4 - PROGRESS: at 93.28% examples, 1778750 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:27:24,495 INFO: EPOCH 4 - PROGRESS: at 94.53% examples, 1778470 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:27:25,500 INFO: EPOCH 4 - PROGRESS: at 95.81% examples, 1779189 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:26,511 INFO: EPOCH 4 - PROGRESS: at 97.11% examples, 1779419 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:27:27,511 INFO: EPOCH 4 - PROGRESS: at 98.42% examples, 1780305 words/s, in_qsize 32, out_qsize 1
2020-07-31 13:27:28,522 INFO: EPOCH 4 - PROGRESS: at 99.69% examples, 1779974 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:27:28,703 INFO: worker thread finished; awaiting finish of 15 more threads
2020-07-31 13:27:28,715 INFO: worker thread finished; awaiting finish of 14 more threads
2020-07-31 13:27:28,723 INFO: worker thread finished; awaiting finish of 13 more threads
2020-07-31 13:27:28,751 INFO: worker thread finished; awaiting finish of 12 more threads
2020-07-31 13:27:28,764 INFO: worker thread finished; awaiting finish of 11 more threads
2020-07-31 13:27:28,765 INFO: worker thread finished; awaiting finish of 10 more threads
2020-07-31 13:27:28,765 INFO: worker thread finished; awaiting finish of 9 more threads
2020-07-31 13:27:28,766 INFO: worker thread finished; awaiting finish of 8 more threads
2020-07-31 13:27:28,767 INFO: worker thread finished; awaiting finish of 7 more threads
2020-07-31 13:27:28,767 INFO: worker thread finished; awaiting finish of 6 more threads
2020-07-31 13:27:28,768 INFO: worker thread finished; awaiting finish of 5 more threads
2020-07-31 13:27:28,769 INFO: worker thread finished; awaiting finish of 4 more threads
2020-07-31 13:27:28,769 INFO: worker thread finished; awaiting finish of 3 more threads
2020-07-31 13:27:28,776 INFO: worker thread finished; awaiting finish of 2 more threads
2020-07-31 13:27:28,777 INFO: worker thread finished; awaiting finish of 1 more threads
2020-07-31 13:27:28,777 INFO: worker thread finished; awaiting finish of 0 more threads
2020-07-31 13:27:28,779 INFO: EPOCH - 4 : training on 163437638 raw words (140297511 effective words) took 78.8s, 1779793 effective words/s
2020-07-31 13:27:29,800 INFO: EPOCH 5 - PROGRESS: at 1.25% examples, 1757441 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:27:30,802 INFO: EPOCH 5 - PROGRESS: at 2.58% examples, 1820269 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:27:31,810 INFO: EPOCH 5 - PROGRESS: at 3.89% examples, 1823721 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:27:32,813 INFO: EPOCH 5 - PROGRESS: at 5.23% examples, 1839671 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:27:33,817 INFO: EPOCH 5 - PROGRESS: at 6.52% examples, 1833797 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:27:34,824 INFO: EPOCH 5 - PROGRESS: at 7.79% examples, 1819459 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:27:35,829 INFO: EPOCH 5 - PROGRESS: at 9.06% examples, 1812443 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:27:36,829 INFO: EPOCH 5 - PROGRESS: at 10.37% examples, 1815165 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:37,835 INFO: EPOCH 5 - PROGRESS: at 11.66% examples, 1817169 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:38,860 INFO: EPOCH 5 - PROGRESS: at 13.05% examples, 1822238 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:27:39,863 INFO: EPOCH 5 - PROGRESS: at 14.37% examples, 1824829 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:40,864 INFO: EPOCH 5 - PROGRESS: at 15.63% examples, 1823211 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:27:41,868 INFO: EPOCH 5 - PROGRESS: at 16.93% examples, 1824726 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:27:42,880 INFO: EPOCH 5 - PROGRESS: at 18.22% examples, 1816567 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:43,895 INFO: EPOCH 5 - PROGRESS: at 19.39% examples, 1797829 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:27:44,900 INFO: EPOCH 5 - PROGRESS: at 20.52% examples, 1784611 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:45,922 INFO: EPOCH 5 - PROGRESS: at 21.66% examples, 1773634 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:27:46,926 INFO: EPOCH 5 - PROGRESS: at 22.94% examples, 1769955 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:27:47,927 INFO: EPOCH 5 - PROGRESS: at 24.10% examples, 1762182 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:27:48,931 INFO: EPOCH 5 - PROGRESS: at 25.30% examples, 1759685 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:49,934 INFO: EPOCH 5 - PROGRESS: at 26.55% examples, 1753221 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:50,949 INFO: EPOCH 5 - PROGRESS: at 27.82% examples, 1751359 words/s, in_qsize 29, out_qsize 3
2020-07-31 13:27:51,954 INFO: EPOCH 5 - PROGRESS: at 29.00% examples, 1747763 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:27:52,970 INFO: EPOCH 5 - PROGRESS: at 30.22% examples, 1744273 words/s, in_qsize 31, out_qsize 6
2020-07-31 13:27:53,975 INFO: EPOCH 5 - PROGRESS: at 31.45% examples, 1744382 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:54,990 INFO: EPOCH 5 - PROGRESS: at 32.54% examples, 1737508 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:27:55,996 INFO: EPOCH 5 - PROGRESS: at 33.73% examples, 1736649 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:57,001 INFO: EPOCH 5 - PROGRESS: at 35.05% examples, 1739376 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:27:58,014 INFO: EPOCH 5 - PROGRESS: at 36.32% examples, 1740666 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:27:59,015 INFO: EPOCH 5 - PROGRESS: at 37.59% examples, 1740708 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:28:00,026 INFO: EPOCH 5 - PROGRESS: at 38.81% examples, 1737190 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:28:01,032 INFO: EPOCH 5 - PROGRESS: at 40.08% examples, 1738816 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:02,034 INFO: EPOCH 5 - PROGRESS: at 41.32% examples, 1740186 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:28:03,041 INFO: EPOCH 5 - PROGRESS: at 42.60% examples, 1740772 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:28:04,054 INFO: EPOCH 5 - PROGRESS: at 43.84% examples, 1741274 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:05,066 INFO: EPOCH 5 - PROGRESS: at 45.22% examples, 1745717 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:06,069 INFO: EPOCH 5 - PROGRESS: at 46.53% examples, 1747300 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:07,070 INFO: EPOCH 5 - PROGRESS: at 47.81% examples, 1749858 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:28:08,078 INFO: EPOCH 5 - PROGRESS: at 49.08% examples, 1750114 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:09,103 INFO: EPOCH 5 - PROGRESS: at 49.90% examples, 1734648 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:28:10,106 INFO: EPOCH 5 - PROGRESS: at 50.90% examples, 1726782 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:11,114 INFO: EPOCH 5 - PROGRESS: at 52.16% examples, 1725429 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:28:12,131 INFO: EPOCH 5 - PROGRESS: at 53.37% examples, 1723547 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:13,136 INFO: EPOCH 5 - PROGRESS: at 54.55% examples, 1721356 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:28:14,146 INFO: EPOCH 5 - PROGRESS: at 55.67% examples, 1718182 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:28:15,147 INFO: EPOCH 5 - PROGRESS: at 56.85% examples, 1716084 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:28:16,147 INFO: EPOCH 5 - PROGRESS: at 57.96% examples, 1713058 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:17,167 INFO: EPOCH 5 - PROGRESS: at 59.04% examples, 1708829 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:28:18,167 INFO: EPOCH 5 - PROGRESS: at 60.18% examples, 1706423 words/s, in_qsize 28, out_qsize 4
2020-07-31 13:28:19,172 INFO: EPOCH 5 - PROGRESS: at 61.36% examples, 1704745 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:28:20,180 INFO: EPOCH 5 - PROGRESS: at 62.62% examples, 1706564 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:21,185 INFO: EPOCH 5 - PROGRESS: at 63.85% examples, 1707717 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:22,189 INFO: EPOCH 5 - PROGRESS: at 65.07% examples, 1708137 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:28:23,189 INFO: EPOCH 5 - PROGRESS: at 66.43% examples, 1710990 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:24,190 INFO: EPOCH 5 - PROGRESS: at 67.66% examples, 1711896 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:25,198 INFO: EPOCH 5 - PROGRESS: at 68.96% examples, 1713392 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:26,206 INFO: EPOCH 5 - PROGRESS: at 70.12% examples, 1712813 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:27,212 INFO: EPOCH 5 - PROGRESS: at 71.44% examples, 1714430 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:28:28,212 INFO: EPOCH 5 - PROGRESS: at 72.81% examples, 1717697 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:28:29,213 INFO: EPOCH 5 - PROGRESS: at 74.09% examples, 1719564 words/s, in_qsize 29, out_qsize 2
2020-07-31 13:28:30,223 INFO: EPOCH 5 - PROGRESS: at 75.30% examples, 1719220 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:31,233 INFO: EPOCH 5 - PROGRESS: at 76.59% examples, 1719986 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:32,233 INFO: EPOCH 5 - PROGRESS: at 77.91% examples, 1721866 words/s, in_qsize 27, out_qsize 4
2020-07-31 13:28:33,236 INFO: EPOCH 5 - PROGRESS: at 79.14% examples, 1721467 words/s, in_qsize 32, out_qsize 0
2020-07-31 13:28:34,240 INFO: EPOCH 5 - PROGRESS: at 80.14% examples, 1716754 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:28:35,241 INFO: EPOCH 5 - PROGRESS: at 81.38% examples, 1717347 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:36,242 INFO: EPOCH 5 - PROGRESS: at 82.62% examples, 1717836 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:28:37,248 INFO: EPOCH 5 - PROGRESS: at 83.84% examples, 1718410 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:38,257 INFO: EPOCH 5 - PROGRESS: at 85.07% examples, 1718784 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:39,265 INFO: EPOCH 5 - PROGRESS: at 86.25% examples, 1717536 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:28:40,284 INFO: EPOCH 5 - PROGRESS: at 87.38% examples, 1715125 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:28:41,297 INFO: EPOCH 5 - PROGRESS: at 88.54% examples, 1713816 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:28:42,303 INFO: EPOCH 5 - PROGRESS: at 89.80% examples, 1714405 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:43,311 INFO: EPOCH 5 - PROGRESS: at 91.03% examples, 1713912 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:28:44,316 INFO: EPOCH 5 - PROGRESS: at 92.31% examples, 1714748 words/s, in_qsize 28, out_qsize 3
2020-07-31 13:28:45,322 INFO: EPOCH 5 - PROGRESS: at 93.56% examples, 1715147 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:28:46,328 INFO: EPOCH 5 - PROGRESS: at 94.78% examples, 1714159 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:28:47,342 INFO: EPOCH 5 - PROGRESS: at 95.93% examples, 1713256 words/s, in_qsize 32, out_qsize 1
2020-07-31 13:28:48,343 INFO: EPOCH 5 - PROGRESS: at 97.20% examples, 1713913 words/s, in_qsize 31, out_qsize 0
2020-07-31 13:28:49,349 INFO: EPOCH 5 - PROGRESS: at 98.40% examples, 1713743 words/s, in_qsize 30, out_qsize 2
2020-07-31 13:28:50,376 INFO: EPOCH 5 - PROGRESS: at 99.43% examples, 1709996 words/s, in_qsize 30, out_qsize 1
2020-07-31 13:28:50,879 INFO: worker thread finished; awaiting finish of 15 more threads
2020-07-31 13:28:50,886 INFO: worker thread finished; awaiting finish of 14 more threads
2020-07-31 13:28:50,899 INFO: worker thread finished; awaiting finish of 13 more threads
2020-07-31 13:28:50,917 INFO: worker thread finished; awaiting finish of 12 more threads
2020-07-31 13:28:50,928 INFO: worker thread finished; awaiting finish of 11 more threads
2020-07-31 13:28:50,932 INFO: worker thread finished; awaiting finish of 10 more threads
2020-07-31 13:28:50,935 INFO: worker thread finished; awaiting finish of 9 more threads
2020-07-31 13:28:50,937 INFO: worker thread finished; awaiting finish of 8 more threads
2020-07-31 13:28:50,939 INFO: worker thread finished; awaiting finish of 7 more threads
2020-07-31 13:28:50,941 INFO: worker thread finished; awaiting finish of 6 more threads
2020-07-31 13:28:50,958 INFO: worker thread finished; awaiting finish of 5 more threads
2020-07-31 13:28:50,958 INFO: worker thread finished; awaiting finish of 4 more threads
2020-07-31 13:28:50,967 INFO: worker thread finished; awaiting finish of 3 more threads
2020-07-31 13:28:50,968 INFO: worker thread finished; awaiting finish of 2 more threads
2020-07-31 13:28:50,969 INFO: worker thread finished; awaiting finish of 1 more threads
2020-07-31 13:28:50,970 INFO: worker thread finished; awaiting finish of 0 more threads
2020-07-31 13:28:50,971 INFO: EPOCH - 5 : training on 163437638 raw words (140291758 effective words) took 82.2s, 1707217 effective words/s
2020-07-31 13:28:50,973 INFO: training on a 817188190 raw words (701464320 effective words) took 419.8s, 1670976 effective words/s
2020-07-31 13:28:51,040 INFO: precomputing L2-norms of word weight vectors
2020-07-31 13:28:51,057 INFO: saving Word2Vec object under ./data/1/word2vec_full.bin, separately None
2020-07-31 13:28:51,072 INFO: not storing attribute vectors_norm
2020-07-31 13:28:51,077 INFO: not storing attribute cum_table
2020-07-31 13:28:51,185 INFO: saved ./data/1/word2vec_full.bin
2020-07-31 13:28:51,186 INFO: loading Word2Vec object from ./data/1/word2vec_full.bin
2020-07-31 13:28:51,262 INFO: loading wv recursively from ./data/1/word2vec_full.bin.wv.* with mmap=None
2020-07-31 13:28:51,263 INFO: setting ignored attribute vectors_norm to None
2020-07-31 13:28:51,263 INFO: loading vocabulary recursively from ./data/1/word2vec_full.bin.vocabulary.* with mmap=None
2020-07-31 13:28:51,264 INFO: loading trainables recursively from ./data/1/word2vec_full.bin.trainables.* with mmap=None
2020-07-31 13:28:51,264 INFO: setting ignored attribute cum_table to None
2020-07-31 13:28:51,265 INFO: loaded ./data/1/word2vec_full.bin
2020-07-31 13:28:51,282 INFO: storing 5981x100 projection weights into ./data/1/word2vec_full.txt

pyTorch

  • 官网
  1. 什么是 pyTorch
  • github
    • Tensor computation (like NumPy) with strong GPU acceleration
      • 具有强大的GPU加速的张量计算（如Numpy）
    • Deep neural networks built on a tape-based autograd system
      • 包含自动求导系统的深度神经网络
  • 中文教程1
    • 由Facebookd的人工智能小组开发
  • 中文教程2
    • 作为NumPy的替代品，可以利用GPU的性能进行计算
    • 作为一个高灵活性、速度快的深度学习平台

名词解析

  • Tensor 张量
    • 类似于 NumPy 的n darray
    • 知乎讲解
      • 最笨的解答是：量是有单位的，数没有
      • 数字上 5 ≠ 500, 但是 5 米和 500 厘米对应的物理量是一样的
      • A tensor is something that transforms like a tensor! 一个量, 在不同的参考系下按照某种特定的法则进行变换, 就是张量
  • gensim
    • 官网
      • topic modeling for humans
  • word2vec 词向量
    • word to vector, 单词转换成向量形式的工具
    • [NLP] 秒懂词向量Word2vec的本质
      • 语言模型, f(x)->y 词语 x 和词语 y 放在一起, 是不是人话
      • Skip-gram 模型 用一个词语作为输入, 来预测它周围的上下文
      • CBOW 模型 拿一个词语的上下文作为输入, 来预测这个词语本身
      • one-hot encoder, 词语转数值类型向量

你可能感兴趣的:(NLP,知识图谱)

  • 免费的GPT可在线直接使用（一键收藏） kkai人工智能 gpt
    1、LuminAI（https://kk.zlrxjh.top）LuminAI标志着一款融合了星辰大数据模型与文脉深度模型的先进知识增强型语言处理系统，旨在自然语言处理（NLP）的技术开发领域发光发热。此系统展现了卓越的语义把握与内容生成能力，轻松驾驭多样化的自然语言处理任务。VisionAI在NLP界的应用领域广泛，能够胜任从机器翻译、文本概要撰写、情绪分析到问答等众多任务。通过对大量文本数据的
  • 机器学习-聚类算法 不良人龍木木 机器学习机器学习算法聚类
    机器学习-聚类算法1.AHC2.K-means3.SC4.MCL仅个人笔记，感谢点赞关注！1.AHC2.K-means3.SC传统谱聚类：个人对谱聚类算法的理解以及改进4.MCL目前仅专注于NLP的技术学习和分享感谢大家的关注与支持！
  • 轻量级模型解读——轻量transformer系列 lishanlu136 #图像分类轻量级模型transformer图像分类
    先占坑，持续更新。。。文章目录1、DeiT2、ConViT3、Mobile-Former4、MobileViTTransformer是2017谷歌提出的一篇论文，最早应用于NLP领域的机器翻译工作，Transformer解读，但随着2020年DETR和ViT的出现(DETR解读，ViT解读)，其在视觉领域的应用也如雨后春笋般渐渐出现，其特有的全局注意力机制给图像识别领域带来了重要参考。但是tran
  • FlagEmbedding 吉小雨 python库python
    FlagEmbedding教程FlagEmbedding是一个用于生成文本嵌入（textembeddings）的库，适合处理自然语言处理（NLP）中的各种任务。嵌入（embeddings）是将文本表示为连续向量，能够捕捉语义上的相似性，常用于文本分类、聚类、信息检索等场景。官方文档链接：FlagEmbedding官方GitHub一、FlagEmbedding库概述1.1什么是FlagEmbeddi
  • 【NumPy】深入解析numpy.zeros()函数 二七830 numpy
    欢迎莅临我的个人主页这里是我深耕Python编程、机器学习和自然语言处理（NLP）领域，并乐于分享知识与经验的小天地！博主简介：我是二七830，一名对技术充满热情的探索者。多年的Python编程和机器学习实践，使我深入理解了这些技术的核心原理，并能够在实际项目中灵活应用。尤其是在NLP领域，我积累了丰富的经验，能够处理各种复杂的自然语言任务。技术专长：我熟练掌握Python编程语言，并深入研究了机
  • 音视频知识图谱 2022.04 关键帧Keyframe
    前些时间，我在知识星球上创建了一个音视频技术社群：关键帧的音视频开发圈，在这里群友们会一起做一些打卡任务。比如：周期性地整理音视频相关的面试题，汇集一份音视频面试题集锦，你可以看看《音视频面试题集锦2022.04》。再比如：循序渐进地归纳总结音视频技术知识，绘制一幅音视频知识图谱。下面是2022.04月知识图谱新增的内容节选：1）图谱路径：**采集/音频采集/声音三要素/响度******主观计量响
  • CV、NLP、数据控掘推荐、量化 海的那边- AI算法自然语言处理人工智能
    下面是对CV（计算机视觉）、NLP（自然语言处理）、数据挖掘推荐和量化的简要概述及其应用领域的介绍：1.CV（计算机视觉，ComputerVision）定义：计算机视觉是一门让计算机能够从图像或视频中提取有用信息，并做出决策的学科。它通过模拟人类的视觉系统来识别、处理和理解视觉信息。主要任务：图像分类：识别图像中的物体并分类，比如猫、狗、车等。目标检测：在图像或视频中定位并识别多个对象，如人脸检测
  • 深度解析：如何使用输出解析器将大型语言模型（LLM）的响应解析为结构化JSON格式 m0_57781768 语言模型json人工智能
    深度解析：如何使用输出解析器将大型语言模型（LLM）的响应解析为结构化JSON格式在现代自然语言处理（NLP）的应用中，大型语言模型（LLM）已经成为了重要的工具。这些模型能够生成丰富的自然语言文本，适用于各种应用场景。然而，在某些应用中，开发者不仅仅需要生成文本，还需要将这些生成的文本转换为结构化的数据格式，例如JSON。这种结构化的数据格式在数据传输、存储以及进一步处理时具有显著优势。本文将深
  • 使用LangChain和OpenAI实现高效文本标注 aehrutktrjk langchainpython
    使用LangChain和OpenAI实现高效文本标注引言在自然语言处理(NLP)领域，文本标注是一项重要且常见的任务。它涉及为文本分配标签，如情感、语言、风格等。本文将介绍如何使用LangChain和OpenAI的API来实现高效的文本标注系统。我们将探讨如何设置环境、定义标注模式，以及如何使用OpenAI的模型来执行标注任务。环境准备首先，我们需要安装必要的库并设置API密钥：%pipinsta
  • 大数据毕业设计hadoop+spark+hive知识图谱租房数据分析可视化大屏 租房推荐系统 58同城租房爬虫 房源推荐系统 房价预测系统 计算机毕业设计 机器学习 深度学习 人工智能 2401_84572577 程序员大数据hadoop人工智能
    做了那么多年开发，自学了很多门编程语言，我很明白学习资源对于学一门新语言的重要性，这些年也收藏了不少的Python干货，对我来说这些东西确实已经用不到了，但对于准备自学Python的人来说，或许它就是一个宝藏，可以给你省去很多的时间和精力。别在网上瞎学了，我最近也做了一些资源的更新，只要你是我的粉丝，这期福利你都可拿走。我先来介绍一下这些东西怎么用，文末抱走。（1）Python所有方向的学习路线（
  • 【NLP5-RNN模型、LSTM模型和GRU模型】 一蓑烟雨紫洛 nlprnnlstmgrunlp
    RNN模型、LSTM模型和GRU模型1、什么是RNN模型RNN（RecurrentNeuralNetwork)中文称为循环神经网络，它一般以序列数据为输入，通过网络内部的结构设计有效捕捉序列之间的关系特征，一般也是以序列形式进行输出RNN的循环机制使模型隐层上一时间步产生的结果，能够作为当下时间步输入的一部分（当下时间步的输入除了正常的输入外还包括上一步的隐层输出）对当下时间步的输出产生影响2、R
  • 基于深度学习的文本引导的图像编辑 SEU-WYL 深度学习dnn深度学习人工智能
    基于深度学习的文本引导的图像编辑（Text-GuidedImageEditing）是一种通过自然语言文本指令对图像进行编辑或修改的技术。它结合了图像生成和自然语言处理（NLP）的最新进展，使用户能够通过描述性文本对图像内容进行精确的调整和操控。1.文本引导的图像编辑的挑战文本和图像之间的对齐：如何将文本中的语义信息准确地映射到图像中的特定区域或元素是一个关键挑战。这涉及到多模态数据的对齐和理解。编
  • 甘超波：NLP婚姻中如何与老人相处 甘超波
    哈喽，大家好我是甘超波，是一名NLP爱好者，每天一篇原创文章或视频，分享我的实战经验和案例，希望给你些启发和帮助看一下，在家庭中子女与老人观念不一致时案例1：在教育孩子方面，老人习惯用老一套教育方式教育孙子，子女受不了老人这种习惯，从而发生口舌之争？2：在生活习惯方面，老人喜欢吃剩菜剩饭，子女受不了老人这种习惯，从而发生口舌之争？.....这样的事情，我相信你或多或少都听过和看过，甚至了深有感悟。
  • transformer架构(Transformer Architecture)原理与代码实战案例讲解 AI架构设计之禅 大数据AI人工智能Python入门实战计算科学神经计算深度学习神经网络大数据人工智能大型语言模型AIAGILLMJavaPython架构设计AgentRPA
    transformer架构(TransformerArchitecture)原理与代码实战案例讲解关键词：Transformer,自注意力机制,编码器-解码器,预训练,微调,NLP,机器翻译作者：禅与计算机程序设计艺术/ZenandtheArtofComputerProgramming1.背景介绍1.1问题的由来自然语言处理（NLP）领域的发展经历了从规则驱动到统计驱动再到深度学习驱动的三个阶段。
  • 英伟达（NVIDIA）B200架构解读 weixin_41205263 芯际争霸GPGPU架构gpu算力人工智能硬件架构
    H100芯片是一款高性能AI芯片，其中的TransformerEngine是专门用于加速Transformer模型计算的核心部件。Transformer模型是一种自然语言处理（NLP）模型，广泛应用于机器翻译、文本生成等任务。TransformerEngine的电路设计原理主要包括以下几个方面：
  • 《昇思 25 天学习打卡营第 25 天 | 基于 MindSpore 实现 BERT 对话情绪识别 》 Sam9029 Mindscope模型学习深度学习
    《昇思25天学习打卡营第25天|基于MindSpore实现BERT对话情绪识别》活动地址：https://xihe.mindspore.cn/events/mindspore-training-camp签名：Sam9029环境配置确保安装了正确版本的MindSpore和MindNLP库。!pipuninstallmindspore-y!pipinstall-ihttps://pypi.mirror
  • 基于人工智能的智能语音助手 人工智能发烧友 人工智能
    语音助手的自然语言处理模块是语音助手系统的关键组成部分。通过这个模块，系统能够识别用户的意图并做出相应的回应。我们可以使用NLP技术来解析文本输入，并将其转换为系统可以理解的命令或指令。在本项目中，我们将结合语音识别、自然语言处理和语音合成技术，构建一个功能简化的语音助手。一、项目背景与需求分析1.1项目目标本项目旨在创建一个语音助手系统，它可以：1.语音识别：从用户的语音输入中提取文本信息。2.
  • NLP_jieba中文分词的常用模块 Hiweir · NLP_jieba的使用自然语言处理中文分词人工智能nlp
    1.jieba分词模式（1）精确模式:把句子最精确的切分开,比较适合文本分析.默认精确模式.（2）全模式:把句子中所有可能成词的词都扫描出来,cut_all=True,缺点:速度快,不能解决歧义（3）paddle:利用百度的paddlepaddle深度学习框架.简单来说就是使用百度提供的分词模型.use_paddle=True.（4）搜索引擎模式:在精确模式的基础上,对长词再进行切分,提高召回率,
  • Linux如何查看端口 lanhuazui10 linux操作系统linux
    方法一：lsof-i:端口号用于查看某一端口的占用情况，比如查看9092端口使用情况，lsof-i:9095可以看到9095端口已经被nginx占用方法二：netstat-tunlp|grep端口号，用于查看指定的端口号的进程情况，如查看5050端口的情况，netstat-tunlp|grep5050-t(tcp)仅显示tcp相关选项-u(udp)仅显示udp相关选项-n拒绝显示别名，能显示数字的
  • 【笔记】自然语言处理NLP---概论 xhanZ NLP相关
    （from人文学院开设课程）目录1.自然语言处理概论1.1自然语言处理研究的意义、历史与现状1.1.1自然语言的特点1.1.2自然语言处理研究的意义1.1.3国外研究现状1.2NLP的方法、特点和规律1.2.1理性主义与经验主义1.2.2语料库语言学：经验主义研究方法1.2.3汉语语言处理的方法1.2.4基于知识图谱的深度学习1.自然语言处理概论1.1自然语言处理研究的意义、历史与现状1.1.1自
  • 【笔记与idea】——ACL2017论文报告会 胖胖的飞象 深度学习人工智能笔记idea
    这篇是2017年我有幸参加了中文信息学会组织的ACL2017论文报告会记的笔记，当时还是研一新生，对NLP感兴趣，偶然通过老师知晓了这次报告会，所以想去现场听听大牛们的idea、和大牛们交流（然而由于当时没有入门，啥也不懂，交流失败。。。）但是总的来说，非常感谢组织这次报告会的老师们，尽管没能和大牛们有效的交流，但是这次报告会相当于在最短的时间内读懂了数十篇精彩论文的核心内容，对我后面的学习起到了
  • 如何利用AI技术来提升用户的个性化体验和社区参与度？ Itfuture03 AI前沿技术人工智能
    要利用AI技术提升用户的个性化体验和社区参与度，可以采取以下几种策略：个性化推荐系统：通过AI算法分析用户的行为和偏好，提供定制化的服务和内容推荐，如智能推荐活动、健康管理等，让居民感受到社区的温暖和关怀。智能助手与聊天机器人：引入AI驱动的虚拟助手，提供实时帮助、个性化建议和交互式对话，改善客户体验。自然语言处理（NLP）：实现具有AI能力的NLP，创建对用户友好的应用程序，简化用户体验，如客服
  • 【Python】成功解决IndexError: list index out of range 高斯小哥 BUG解决方案合集pythonlist新手入门学习debug
    【Python】成功解决IndexError:listindexoutofrange下滑查看解决方法欢迎莅临我的个人主页这里是我静心耕耘深度学习领域、真诚分享知识与智慧的小天地！博主简介：985高校的普通本硕，曾有幸发表过人工智能领域的中科院顶刊一作论文，熟练掌握PyTorch框架。技术专长：在CV、NLP及多模态等领域有丰富的项目实战经验。已累计一对一为数百位用户提供近千次专业服务，助力他们少走
  • 使用Python和Jieba库进行中文情感分析：从文本预处理到模型训练的完整指南 快撑死的鱼 Python算法精解python人工智能开发语言
    使用Python和Jieba库进行中文情感分析：从文本预处理到模型训练的完整指南情感分析（SentimentAnalysis）是自然语言处理（NLP）领域中的一个重要分支，旨在从文本中识别出情绪、态度或意见等主观信息。在中文文本处理中，由于语言特性不同于英语，如何高效、准确地分词和提取关键词成为情感分析的关键步骤之一。在这篇文章中，我们将深入探讨如何使用Python和Jieba库进行中文情感分析，
  • 论文阅读笔记: DINOv2: Learning Robust Visual Features without Supervision 小夏refresh 论文计算机视觉深度学习论文阅读笔记深度学习计算机视觉人工智能
    DINOv2:LearningRobustVisualFeatureswithoutSupervision论文地址:https://arxiv.org/abs/2304.07193代码地址:https://github.com/facebookresearch/dinov2摘要大量数据上的预训练模型在NLP方面取得突破，为计算机视觉中的类似基础模型开辟了道路。这些模型可以通过生成通用视觉特征(即无
  • 第3篇：LangChain的架构总览与设计理念 Gemini技术窝 langchain架构大数据人工智能AIGCnlp
    LangChain库是一个专为自然语言处理（NLP）设计的强大工具包，致力于简化复杂语言模型链的构建和执行。在本文中，我们将深入解析LangChain库的架构，详细列出其核心组件、设计理念及其在不同场景中的应用，并讨论其优缺点。文章目录1.LangChain库简介2.核心组件2.1数据输入模块作用2.2数据预处理模块作用2.3数据增强模块作用2.4数据加载与批处理模块作用2.5模型训练模块作用2.
  • 读李中莹先生论“阿Q精神" 猫咪06
    这阵子重读《重塑心灵》，对“阿Q精神"一段很有感慨，在我们从小的信念里，阿Q的精神胜利法是被贬低的，是对无能力改变自己的境遇时，似手只能采用自我安慰的人的讽刺。李中莹先生在他的书中结合对话者的认可，定义阿Q精神“只求精神胜利，罔顾真实情况"，他就针对这两句话，解析阿Q精神，并进行了肯定‘，。首先“精神胜利"指的是自己内心有成功的感觉，这很符合NLP!如果所有人都认为你成功，而你自己没有成功的喜悦，
  • 书单 用户5521
    提高思维（13本）：影响力逻辑思维（理查德·尼斯贝特）离经叛道:不按常理出牌的人如何改变世界（只看最后一章总结即可）改变:问题形成和解决的原则语言的魔力:谈笑间转变信念之NLP技巧（意识到语言顺序的重要性）改变心理学的40项研究对伪心理学说不你的误区:如何摆脱负面思维掌控你的生活战胜拖拉你的灯亮着吗?别做正常的傻瓜学会提问:批判性思维指南不确定世界的理性选择小说（5本）：霍乱时期的爱情那些回不去的
  • 【Python】解决AttributeError: ‘NoneType‘ object has no attribute ‘xxxx‘ 云天徽上 Pandaspython开发语言pandas机器学习numpy
    【Python】解决AttributeError:'NoneType'objecthasnoattribute'xxxx'报错欢迎莅临我的个人主页这里是我深耕Python编程、机器学习和自然语言处理（NLP）领域，并乐于分享知识与经验的小天地！博主简介：我是云天徽上，一名对技术充满热情的探索者。多年的Python编程和机器学习实践，使我深入理解了这些技术的核心原理，并能够在实际项目中灵活应用。尤其
  • 【自然语言处理】自然语言处理NLP概述及应用 @我们的天空 人工智能技术nlp人工智能深度学习python机器学习自然语言处理scikit-learn
    自然语言处理（NaturalLanguageProcessing，简称NLP）是一门集计算机科学、人工智能以及语言学于一体的交叉学科，致力于让计算机能够理解、解析、生成和处理人类的自然语言。它是人工智能领域的一个关键分支，旨在缩小人与机器之间的交流障碍，使得机器能够更有效地识别并响应人类的自然语言指令或内容。自然语言处理NLP概述基本任务：文本分类：将文本划分为预定义的类别，如情感分析、主题分类等
  • scala的option和some 矮蛋蛋 编程scala
    原文地址： http://blog.sina.com.cn/s/blog_68af3f090100qkt8.html 对于学习 Scala 的 Java™ 开发人员来说，对象是一个比较自然、简单的入口点。在 本系列 前几期文章中，我介绍了 Scala 中一些面向对象的编程方法，这些方法实际上与 Java 编程的区别不是很大。我还向您展示了 Scala 如何重新应用传统的面向对象概念，找到其缺点
  • NullPointerException Cb123456 androidBaseAdapter
        java.lang.NullPointerException: Attempt to invoke virtual method 'int android.view.View.getImportantForAccessibility()' on a null object reference     出现以上异常.然后就在baidu上
  • PHP使用文件和目录 天子之骄 php文件和目录读取和写入php验证文件php锁定文件
    PHP使用文件和目录 1.使用include()包含文件 (1)：使用include()从一个被包含文档返回一个值 (2)：在控制结构中使用include()   include_once()函数需要一个包含文件的路径，此外，第一次调用它的情况和include()一样，如果在脚本执行中再次对同一个文件调用，那么这个文件不会再次包含。   在php.ini文件中设置
  • SQL SELECT DISTINCT 语句 何必如此 sql
    SELECT DISTINCT 语句用于返回唯一不同的值。 SQL SELECT DISTINCT 语句 在表中，一个列可能会包含多个重复值，有时您也许希望仅仅列出不同（distinct）的值。 DISTINCT 关键词用于返回唯一不同的值。 SQL SELECT DISTINCT 语法 SELECT DISTINCT column_name,column_name F
  • java冒泡排序 3213213333332132 java冒泡排序
    package com.algorithm; /** * @Description 冒泡 * @author FuJianyong * 2015-1-22上午09:58:39 */ public class MaoPao { public static void main(String[] args) { int[] mao = {17,50,26,18,9,10
  • struts2.18 +json,struts2-json-plugin-2.1.8.1.jar配置及问题！ 7454103 DAOspringAjaxjsonqq
    struts2.18  出来有段时间了！ （貌似是 稳定版）   闲时研究下下！  貌似 sruts2 搭配 json 做 ajax 很吃香！   实践了下下！ 不当之处请绕过！ 呵呵   网上一大堆 struts2+json  不过大多的json 插件 都是 jsonplugin.34.jar   strut
  • struts2 数据标签说明 darkranger jspbeanstrutsservletScheme
    数据标签主要用于提供各种数据访问相关的功能，包括显示一个Action里的属性，以及生成国际化输出等功能 数据标签主要包括： action ：该标签用于在JSP页面中直接调用一个Action，通过指定executeResult参数，还可将该Action的处理结果包含到本页面来。 bean ：该标签用于创建一个javabean实例。如果指定了id属性，则可以将创建的javabean实例放入Sta
  • 链表.简单的链表节点构建 aijuans 编程技巧
    /*编程环境WIN-TC*/ #include "stdio.h" #include "conio.h" #define NODE(name, key_word, help) \  Node name[1]={{NULL, NULL, NULL, key_word, help}} typedef struct node {  &nbs
  • tomcat下jndi的三种配置方式 avords tomcat
    jndi(Java Naming and Directory Interface，Java命名和目录接口)是一组在Java应用中访问命名和目录服务的API。命名服务将名称和对象联系起来，使得我们可以用名称 访问对象。目录服务是一种命名服务，在这种服务里，对象不但有名称，还有属性。          tomcat配置
  • 关于敏捷的一些想法 houxinyou 敏捷
    从网上看到这样一句话：“敏捷开发的最重要目标就是：满足用户多变的需求，说白了就是最大程度的让客户满意。” 感觉表达的不太清楚。 感觉容易被人误解的地方主要在“用户多变的需求”上。 第一种多变，实际上就是没有从根本上了解了用户的需求。用户的需求实际是稳定的，只是比较多，也比较混乱，用户一般只能了解自己的那一小部分，所以没有用户能清楚的表达出整体需求。而由于各种条件的，用户表达自己那一部分时也有
  • 富养还是穷养，决定孩子的一生 bijian1013 教育人生
    是什么决定孩子未来物质能否丰盛？为什么说寒门很难出贵子，三代才能出贵族？真的是父母必须有钱，才能大概率保证孩子未来富有吗？-----作者：@李雪爱与自由 事实并非由物质决定，而是由心灵决定。一朋友富有而且修养气质很好，兄弟姐妹也都如此。她的童年时代，物质上大家都很贫乏，但妈妈总是保持生活中的美感，时不时给孩子们带回一些美好小玩意，从来不对孩子传递生活艰辛、金钱来之不易、要懂得珍惜
  • oracle 日期时间格式转化 征客丶 oracle
    oracle 系统时间有 SYSDATE 与 SYSTIMESTAMP； SYSDATE：不支持毫秒，取的是系统时间； SYSTIMESTAMP：支持毫秒，日期，时间是给时区转换的，秒和毫秒是取的系统的。 日期转字符窜： 一、不取毫秒： TO_CHAR(SYSDATE, 'YYYY-MM-DD HH24:MI:SS') 简要说明， YYYY 年 MM   月
  • 【Scala六】分析Spark源代码总结的Scala语法四 bit1129 scala
    1. apply语法   FileShuffleBlockManager中定义的类ShuffleFileGroup，定义：   private class ShuffleFileGroup(val shuffleId: Int, val fileId: Int, val files: Array[File]) { ... def apply(bucketId
  • Erlang中有意思的bug bookjovi erlang
      代码中常有一些很搞笑的bug，如下面的一行代码被调用两次（Erlang beam） commit f667e4a47b07b07ed035073b94d699ff5fe0ba9b Author: Jovi Zhang <[email protected]> Date: Fri Dec 2 16:19:22 2011 +0100 erts:
  • 移位打印10进制数转16进制-2008-08-18 ljy325 java基础
    /** * Description 移位打印10进制的16进制形式 * Creation Date 15-08-2008 9:00 * @author 卢俊宇 * @version 1.0 * */ public class PrintHex { // 备选字符 static final char di
  • 读《研磨设计模式》-代码笔记-组合模式 bylijinnan java设计模式
    声明： 本文只为方便我个人查阅和理解，详细的分析以及源代码请移步 原作者的博客http://chjavach.iteye.com/ import java.util.ArrayList; import java.util.List; abstract class Component { public abstract void printStruct(Str
  • 利用cmd命令将.class文件打包成jar chenyu19891124 cmdjar
    cmd命令打jar是如下实现： 在运行里输入cmd，利用cmd命令进入到本地的工作盘符。(如我的是D盘下的文件有此路径 D:\workspace\prpall\WEB-INF\classes) 现在是想把D:\workspace\prpall\WEB-INF\classes路径下所有的文件打包成prpall.jar。然后继续如下操作： cd D: 回车 cd workspace/prpal
  • [原创]JWFD v0.96 工作流系统二次开发包 for Eclipse 简要说明 comsci eclipse设计模式算法工作swing
                           JWFD v0.96 工作流系统二次开发包 for Eclipse 简要说明     &nb
  • SecureCRT右键粘贴的设置 daizj secureCRT右键粘贴
    一般都习惯鼠标右键自动粘贴的功能，对于SecureCRT6.7.5 ，这个功能也已经是默认配置了。 老版本的SecureCRT其实也有这个功能，只是不是默认设置，很多人不知道罢了。 菜单： Options->Global Options ...->Terminal 右边有个Mouse的选项块。 Copy on Select Paste on Right/Middle
  • Linux 软链接和硬链接 dongwei_6688 linux
    1.Linux链接概念Linux链接分两种，一种被称为硬链接（Hard Link），另一种被称为符号链接（Symbolic Link）。默认情况下，ln命令产生硬链接。 【硬连接】硬连接指通过索引节点来进行连接。在Linux的文件系统中，保存在磁盘分区中的文件不管是什么类型都给它分配一个编号，称为索引节点号(Inode Index)。在Linux中，多个文件名指向同一索引节点是存在的。一般这种连
  • DIV底部自适应 dcj3sjt126com JavaScript
    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml&q
  • Centos6.5使用yum安装mysql——快速上手必备 dcj3sjt126com mysql
    第1步、yum安装mysql [root@stonex ~]#  yum -y install mysql-server 安装结果： Installed:     mysql-server.x86_64 0:5.1.73-3.el6_5                   &nb
  • 如何调试JDK源码 frank1234 jdk
    相信各位小伙伴们跟我一样，想通过JDK源码来学习Java，比如collections包，java.util.concurrent包。 可惜的是sun提供的jdk并不能查看运行中的局部变量，需要重新编译一下rt.jar。 下面是编译jdk的具体步骤：         1.把C:\java\jdk1.6.0_26\sr
  • Maximal Rectangle hcx2013 max
    Given a 2D binary matrix filled with 0's and 1's, find the largest rectangle containing all ones and return its area.   public class Solution { public int maximalRectangle(char[][] matrix)
  • Spring MVC测试框架详解——服务端测试 jinnianshilongnian spring mvc test
    随着RESTful Web Service的流行，测试对外的Service是否满足期望也变的必要的。从Spring 3.2开始Spring了Spring Web测试框架，如果版本低于3.2，请使用spring-test-mvc项目（合并到spring3.2中了）。   Spring MVC测试框架提供了对服务器端和客户端（基于RestTemplate的客户端）提供了支持。 &nbs
  • Linux64位操作系统（CentOS6.6）上如何编译hadoop2.4.0 liyong0802 hadoop
    一、准备编译软件   1.在官网下载jdk1.7、maven3.2.1、ant1.9.4，解压设置好环境变量就可以用。     环境变量设置如下：   （1）执行vim /etc/profile （2）在文件尾部加入: export JAVA_HOME=/home/spark/jdk1.7 export MAVEN_HOME=/ho
  • StatusBar 字体白色 pangyulei status
    [[UIApplication sharedApplication] setStatusBarStyle:UIStatusBarStyleLightContent]; /*you'll also need to set UIViewControllerBasedStatusBarAppearance to NO in the plist file if you use this method
  • 如何分析Java虚拟机死锁 sesame javathreadoracle虚拟机jdbc
    英文资料： Thread Dump and Concurrency Locks   Thread dumps are very useful for diagnosing synchronization related problems such as deadlocks on object monitors. Ctrl-\ on Solaris/Linux or Ctrl-B
  • 位运算简介及实用技巧（一）：基础篇 tw_wangzhengquan 位运算
    http://www.matrix67.com/blog/archives/263    去年年底写的关于位运算的日志是这个Blog里少数大受欢迎的文章之一，很多人都希望我能不断完善那篇文章。后来我看到了不少其它的资料，学习到了更多关于位运算的知识，有了重新整理位运算技巧的想法。从今天起我就开始写这一系列位运算讲解文章，与其说是原来那篇文章的follow-up，不如说是一个r
  • jsearch的索引文件结构 yangshangchuan 搜索引擎jsearch全文检索信息检索word分词
    jsearch是一个高性能的全文检索工具包，基于倒排索引，基于java8，类似于lucene，但更轻量级。   jsearch的索引文件结构定义如下：     1、一个词的索引由=分割的三部分组成：        第一部分是词        第二部分是这个词在多少
按字母分类： ABCDEFGHIJKLMNOPQRSTUVWXYZ其他