LLM基础

LLM:
AGI 通用人工智能
计算不可约性原理
监督微调、奖励建模
饶毅关于gpt https://mp.weixin.qq.com/s/Govj_KD_afL_nK9T16nE0w
gpt2 windows :https://matters.town/@tianci/63096-%E6%95%99%E4%BD%A0%E5%A6%82%E4%BD%95%E5%9C%A8%E8%87%AA%E5%B7%B1%E7%9A%84%E7%94%B5%E8%84%91%E4%B8%8A%E8%BF%90%E8%A1%8C%E4%BD%BF%E7%94%A8gpt-2-2020-07-bafyreihimyaazpasxf6m5nb2f52pkvvv7rkz3eb6uoze4u42kka7qd7a4e
this is chat gpt
https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/#its-just-adding-one-word-at-a-time
1 adding one word each time
(1) ask for the following word
chatGPT is always asking given the text,whats the next word(token which could be part of word )
chatgpt 在做的是对文本合理的延续， 合理指的是 全网数据学习后，要能知道接下来应该是什么字词，
eg： The best thing about AI is its ability to …
chatgpt looks for meaningful things, and generate a ranked list of words that might follow, together with “probabilities”:
(2)随机选词
randomly choose a word in ranked list ,instead of the word with highest posibility;if gpt choose the best one everytime then it lose the creativity and generate massive repeated content
randomness means the same prompt produces different result
（3） “language model” neural net:
Image
i generate next word with probability
ii randomly choose a word for producing different result
2 Where Do the Probabilities Come From?
calculate how often every letter in a content text