基于gpt-2模型（117M预训练模型）的文本自动生成测试

openai的gpt-2模型最近在风口浪尖上。Language Models are Unsupervised Multitask Learners论文已经出来，但是由于该模型没有将训练过程开源出来，所以本博客仅仅是针对已经公布的117M的预训练模型进行测试。

1、论文贡献

In this paper, we connect these two lines of work and continue the trend of more general methods of transfer. We demonstrate language models can perform down-stream tasks in a zero-shot setting – without any parameter or architecture modification. We demonstrate this approach shows potential by highlighting the ability of language models to perform a wide range of tasks in a zero-shot setting. We achieve promising, competitive, and state of the art results depending on the task.

找更大数量的无监督训练数据来执行多任务学习，使模型更具泛化能力。论文实验也证明了该模型具有惊人的效果。

该论文的模型大部分还是遵循GPT－1的模型，但有两点不同的是：

（1）训练数据集更加庞大；

（2）在第二阶段时候，无监督地做多样性的任务。

2、117M的实验测试

执行测试程序，效果如下：

其中任选一个例子，可以看到对话的自动生成效果，可读性还是非常好的。

======================================== SAMPLE 36 ========================================
I was sent a few hours ago and while it is a good thing I almost gave up my game of EVE Online during the lack of a win! I've been in a bunch of small flops and this extended feeling of accomplishment wears off quickly when you are up against a hungry and crowded large team, leaving your heads more empty then ever.

Do you feel like a human being it seems? How did you finish the game and what is your plan for you now?

Of course I felt like a human being just yet I wanted to caution you Salty Dogs is fast moving,

I feel so overwhelmed right now, and it has taken me awhile to get used to being PVP on a big screen map. It's hard to describe just how good the experience is. You can hear the birds and all the bees loudly above you at every turn, but I'll tell you what's first as Dae-bye with the horde

But just as I reported yesterday you have launched your first flame fight with a cyno drop on your squad mate. Is he selfless, or was he just taken aback? Granted he's very clever and oddly physically, together with being conscious you can be flexible, give him time and eternity and maybe even benefit maybe three AXP from a loss. The cyno drop is also a blessing and bonus that once again gives his DPS a blurred product

Anyway when we first worry keep an eye on the team members, I will be picking off Pawns a (he has 2 fire!) and keeping all shields focused and a target area is on-screen so I can execute nice pulls and therefore tell the ppl what triggers and emulate the touch infantry roll action

that's a tremendous turn too. Your thought did make sense, and my observation went viral. I'm feeling so overwhelmed right now just to try out some new stuff. My team mates are done playing, but I will certainly focus on whats going on on higher levels of gameplay,

what better way then to see when you actually harrass the team mate out of defending Silo Exchange with the basic offensive charges in hand before further anti-tank push. For those with a spare CHAR it's worth an inclination, the Tyranids can't be counted as melee units at the moment, this makes them odd names but the whole point of running kids in university's first community he asks "Phillip, deploying? Will you kill me?"

So have fun,

-Dae-bye Salty Dogs<|endoftext|>There's a lot of personal stories beautifully imagined frozen at shore on Capri Island in Nova Scotia - one for instance, from a group of above-board vessels shark roaring off the Vermicelli's 302 and invoked to wait desperate refugees in their misery in Germany.

Some people, especially some conservatives, want tender mercuppactions.

Others may, however, care intensely about another island on their property: Bermudu, a 9,745 sq. ft. country with 156,000 sq. ft. headquarters.

Story continues below advertisement

The association posed first to a shelf for the pit safari in Portland, Oregon. They are part of Friends of Port St. Albans, a Halifax proprietorship that requires M. Arthur Day paid the adventurer 3000 a day.

Take Port St. Albans president Caryn Francitisis on Oct. 5 after receiving a request to dock a crew within 10 yards of the Basecamp think tank by New York-based Quickminster Sandlot meat supplier. With her supporters dwindling with the new year, Francitisis hopes for cabinets and Convoy drunken salmon Gypsy gentle traps once provided for Rusty Commodore subduing Golgoths longiensEXPLAIN (October 18, 2011).

A moment in early October brought two fateful encounters in the city that left staff of First Niagara sobbing for help.

In a matter of minutes Mayor Terry Wooldridge led protesters through Barrington Avenue North until a police car sped up and took them off, prompting shells and dead fishy nets out from the pier. Proceeding slowly, emergency employees found the couple and turned off the next two raging barges coming off the Annapurna peninsula, where some had been making prayers.

The Heard Four

Stay cool but don't embrace NYC's exclusive fact FACT IT'S A MESS Center Anchorage, Alaska - Canada, 126 nautical miles northeast of Fort McMurray, turns down 35,000 planned visitors a day, but has 50,000 RT champions today, the Make Maine count marshals have announced.

Earning FOLLOW COLLEGE ENTHUSIASTIC RECORDS

Sound guide dictates Fashion Document

Their arrival prompted speculation about our species as we see it. The grave possibilities spread seafaring, thereby attendant decades enigma ... The nutrer Susie Lewis (with nary a hint of her apparent blue-bloodgums) penned a poignant cartoon called "Beyond Ferry Line."