gpt 引导驱动器
Nowadays, everybody is talking about GPT-3, San Francisco based OpenAI’s new language model. GPT stands for “generative pre-training transformer” and GPT-3 is the third iteration of this model which has 175 billion parameters — a big jump from its predecessor GPT-2 that has 1.5 billion parameters. Beta users are exploring use-cases to understand GPT-3’s capabilities and many use-cases have already gone viral on social media.
如今，每个人都在谈论基于旧金山的OpenAI新语言模型GPT-3。 GPT代表“发电式预训练变压器”，而GPT-3是该模型的第三次迭代，具有1750亿个参数，这比其具有15亿个参数的前身GPT-2有了很大的进步。 Beta用户正在探索用例以了解GPT-3的功能，并且许多用例已经在社交媒体上风行一时。
Duygu Oktem Clark, our Venture Partner and founder of DO Venture Partners spoke with Yigit Ihlamur, cofounder and General Partner of Vela Partners about GPT-3 including its effects on our lives and the worries about harmful biases.
我们的风险合伙人兼DO Venture Partners创始人Duygu Oktem Clark与Vela Partners的联合创始人兼普通合伙人Yigit Ihlamur谈了GPT-3，包括对我们生活的影响以及对有害偏见的担忧。
Duygu Oktem Clark: I know that you specialized in Machine Learning during your graduate study in 2009. How do you see the evolution of AI and machine learning since then?
Duygu Oktem Clark：我知道您在2009年的研究生学习期间专攻机器学习。从那时起，您如何看待AI和机器学习的发展？
Yigit Ihlamur: When I started my studies in 2009, academia was mainly concentrated on math-focused algorithms. Towards the end of 2010, access to storage and compute-power improved the performance of data-intensive algorithms. Academia started experimenting with those algorithms, specifically neural networks, and testing them against benchmarks such as object recognition accuracy in images. In 2012, neural networks outperformed every other algorithm in object recognition. This success motivated more people to experiment with 30+ years old neural network algorithms and iterate on them. Machine learning experts realized that more data and compute-power improve accuracy without changing the fundamentals of neural networks. Hence, the machine learning expertise moved towards data science, statistics, and custom chip design making data consumption and computation faster and easier.
Yigit Ihlamur：当我于2009年开始学习时，学术界主要集中在以数学为中心的算法上。 到2010年年底，对存储和计算能力的访问提高了数据密集型算法的性能。 学术界开始尝试这些算法，特别是神经网络，并针对基准进行测试，例如图像中的对象识别准确性。 在2012年，神经网络在对象识别方面的表现优于其他所有算法。 这项成功促使更多的人尝试使用30多年的神经网络算法并对其进行迭代。 机器学习专家意识到，更多的数据和计算能力可以提高准确性，而无需更改神经网络的基础知识。 因此，机器学习专业知识已转向数据科学，统计和定制芯片设计，从而使数据消耗和计算变得更快，更容易。
Over time, neural networks became more sophisticated. Easy-to-use developer libraries emerged. More models were trained with large datasets with cheap compute power. Thanks to the concept of ‘Transfer Learning’, experts started to build on top of other experts’ models. This spiraled an exponential growth and high collaboration within the community.
随着时间的流逝，神经网络变得越来越复杂。 易于使用的开发人员库应运而生。 使用具有低计算能力的大型数据集训练了更多模型。 得益于“转移学习”的概念，专家们开始在其他专家模型的基础上进行构建。 这导致社区内的指数级增长和高度协作。
Around 2016, neural networks were able to detect objects in images better than humans. This was similar to the time when computers were able to do math better than humans. Since this revolution, computers can see as good as humans, if not better in some use-cases. Unlike people, computers can scale and never get tired.
在2016年左右，神经网络比人类能够更好地检测图像中的物体。 这类似于计算机能够比人类做得更好的数学时代。 自从这场革命以来，即使在某些用例中，计算机也可以像人类一样好。 与人不同，计算机可以扩展并且永不疲劳。
Around that time, I got my hands on a computer vision algorithm. I was blown away as I was able to train my algorithm to detect objects in less than 2 hours. This innovation caused large industries such as automotive, robotics, security, and manufacturing to evolve rapidly.
在那段时间，我接触了计算机视觉算法。 能够训练我的算法在不到2小时的时间内检测到物体，这让我感到震惊。 这项创新使汽车，机器人，安全和制造业等大型行业Swift发展。
While computer vision reached an important milestone, other popular applications of machine learning such as speech recognition, text-to-speech, and natural language understanding were also making significant progress. However, they did not reach the same level of human-level accuracy as computer vision algorithms until recently.
尽管计算机视觉达到了一个重要的里程碑，但机器学习的其他流行应用(例如语音识别，文本到语音转换和自然语言理解)也取得了重大进展。 但是，直到最近，它们还没有达到与计算机视觉算法相同的人类水平精度。
In 2018, a new generation of neural networks, called Transformers, emerged and triggered the next growth area for machine learning, specifically in natural language understanding. GPT-3 (Generative Pretrained Transformers) is one of the iterations that helped the developer community to see the same thing they saw with computer vision. Now, machines are close to surpassing human-level accuracy in some areas of natural language understanding. For example, early results indicate that GPT-3 writes news articles and performs better in the SAT.
在2018年，新一代的神经网络被称为Transformers，并引发了机器学习的下一个增长领域，特别是在自然语言理解方面。 GPT-3(生成式预训练变压器)是帮助开发人员社区使用计算机视觉看到相同内容的一种迭代方式。 现在，在自然语言理解的某些领域，机器已接近超越人类水平的准确性。 例如，早期结果表明GPT-3撰写新闻文章并在SAT中表现更好。
The next five years will be the same as what we have seen in computer vision. Academia and technologists will iterate on GPT-3, mix and match with other algorithms, and build innovative applications to help computers understand and generate text better than humans in various areas.
未来五年将与我们在计算机视觉中看到的相同。 学术界和技术专家将迭代GPT-3，将其与其他算法混合并匹配，并构建创新的应用程序，以帮助计算机在各个领域比人类更好地理解和生成文本。
Duygu: GPT-3 has been a very popular topic since it was released in June. Beta users have been publishing about their experiences. How has been your experience with GPT-3 so far?
Duygu：自6月份发布以来，GPT-3一直是一个非常受欢迎的话题。 Beta用户一直在发布有关其体验的信息。 到目前为止，您对GPT-3的体验如何？
Yigit: The experience is no different than when one gets her first PC, connects to the internet, accesses her email from a beach on a mobile phone or social networks spotting her face on images.
Yigit：体验与第一台PC，连接到Internet，从海滩上的手机或社交网络上看到她的脸时看到她的电子邮件一样，都是如此。
We’re currently in the era of pattern matching thanks to the immense production of data by billions of devices and people. At Vela, we have a variety of in-house machine learning algorithms to facilitate sourcing and evaluating startups. One of our algorithms helps us extract entities such as companies from news articles. We built this algorithm in a week as we have significant in-house deep learning expertise. Thanks to GPT-3, we built the same entity extraction algorithm much faster.
由于数十亿设备和人员的大量数据产生，我们目前处于模式匹配时代。 在Vela，我们有多种内部机器学习算法，可促进对初创企业的采购和评估。 我们的一种算法可帮助我们从新闻文章中提取实体，例如公司。 由于我们拥有大量的内部深度学习专业知识，因此我们在一周内构建了该算法。 感谢GPT-3，我们更快地构建了相同的实体提取算法。
Duygu: What is the most surprising thing about GPT-3 for you?
Duygu：对您而言，GPT-3最令人惊讶的是什么？
Yigit: Any developer will have the capacity to extract and find information independently and efficiently in the near future. Any developer will soon have a search power similar to Google’s.
Yigit：任何开发人员都将有能力在不久的将来独立，有效地提取和查找信息。 任何开发人员都将很快拥有与Google类似的搜索能力。
Duygu: It seems like use-cases are endless with GPT-3. What are the use-cases that you have seen that you find interesting?
Duygu： GPT-3似乎用例不胜枚举 。 您发现有趣的用例是什么？
Yigit: Search and auto-complete is coming to coding, which will speed things up exponentially.
Yigit：搜索和自动完成功能即将用于编码，这将使处理速度成倍提高。
Due to our interest in the market, we extended Vela Partners’ AI algorithm to categorize tweets using GPT-3. You can see all use-cases compiled in this sheet.
由于我们对市场感兴趣，我们扩展了Vela Partners的AI算法，以使用GPT-3对推文进行分类。 您可以在此工作表中查看所有已编译的用例。
Duygu: We have seen use-cases that lead us to think about the future of coding. As a Computer Engineer and investor, I’m excited to explore how GPT-3 will impact programming. Do you think GPT-3 will replace the coders?
Duygu：我们已经看到了用例 ，这些用例使我们思考编码的未来。 作为计算机工程师和投资者，我很高兴探索GPT-3将如何影响编程。 您认为GPT-3将取代编码器吗？
Yigit: No, the work that most engineers don’t want to do will be replaced. When I copy and paste code from Stackoverflow to build an authentication flow for Google accounts, I always question why I am doing this in 2020. Millions of people do the same thing.
Yigit：不，大多数工程师不愿做的工作将被替换。 当我从Stackoverflow复制并粘贴代码以建立Google帐户的身份验证流程时，我总是质疑为什么我要在2020年这样做。成千上万的人都在做同样的事情。
What will happen is that 90% of the boring tasks will be done very fast. However, like any engineer, who reads this, knows that the devil is in the details and in the last 10%.
将会发生的事情是90％的无聊任务将很快完成。 但是，就像任何读过这篇文章的工程师一样，他知道魔鬼在细节中，并且是最后10％。
Soon engineers will focus on the real details and do more creative work. As a result of this trend, general software will be further commoditized and subfields will continue to emerge.
很快，工程师将专注于真实的细节并进行更多的创造性工作。 由于这一趋势，通用软件将进一步商品化，子领域将继续出现。
This trend of developers concentrating on more value-added work started about a decade ago. The number of low-code and no-code products have increased significantly in recent years. Their software production capability has also improved. We expect GPT-3 to accelerate this trend.
开发人员专注于更多增值工作的趋势始于大约十年前。 近年来，低码和无码产品的数量已大大增加。 他们的软件生产能力也得到了提高。 我们预计GPT-3将加速这一趋势。
We, as Vela Partners, have invested in low-code startups and this field is an important component of ‘maker tools’ investment thesis.
作为Vela Partners，我们已经投资了低代码的初创公司，这一领域是“制造商工具”投资论题的重要组成部分。
Duygu: Some people including Jeremo Pesenti, Head of AI at Facebook, and Prof. Anima Anandkumar, a Bren Professor at Caltech, raised concerns about GPT-3 due to harmful biases. What do you think about this? Do you think this issue is solvable?
Duygu：一些人，包括Facebook的AI负责人Jeremo Pesenti和Caltech的Bren教授Anima Anandkumar教授，由于有害偏见而对GPT-3表示了担忧。 你怎么看待这件事？ 您认为这个问题可以解决吗？
Yigit: I’m deeply worried about this for our society and for my family as a father of a daughter.
Yigit：对于我们的社会以及我作为女儿父亲的家庭，我深为担忧。
Machine learning is all about data. If you put garbage in, you’ll get garbage out. If you put biased data in, then algorithms will make biased decisions.
机器学习与数据有关。 如果您放入垃圾，您将得到垃圾。 如果放入有偏见的数据，则算法将做出有偏见的决策。
I am not sure if the bias overall is solvable, but obvious biases that we can clearly articulate are solvable.
我不确定总体上的偏差是否可以解决，但是我们可以明确表达的明显偏差是否可以解决。
I have never met a person in my life that is not biased. And I am not surprised about that. Our brains are pattern matchers like computers. We’re biologically coded to avoid danger. Our biases exist because we’re afraid of the unknown. Our brains must make quick decisions to offload energy through biases, and spare the rest for compute-intensive tasks. For more detailed thoughts on this subject, I suggest reading Daniel Kahneman’s famous ‘Thinking, Fast and Slow’.
我一生中从未遇到没有偏见的人。 我对此并不感到惊讶。 我们的大脑就像计算机一样是模式匹配器。 我们已进行生物编码以避免危险。 我们存在偏见是因为我们害怕未知。 我们的大脑必须做出快速决策，以通过偏差来转移能量，其余的则用于计算量大的任务。 有关此主题的更详细的想法，建议阅读Daniel Kahneman着名的“思考，快速和慢速”。
That being said, since we haven’t solved the bias among humans, how can we solve that with algorithms? Whose values are right? Isn’t ethics subjective and change from community to community?
话虽如此，既然我们还没有解决人类之间的偏见，那么如何用算法解决呢？ 谁的价值观正确？ 伦理不是主观的，而是在社区之间变化吗？
On the other hand, some common biases of our communities are clearly solvable. I still can’t believe that we’re living in a world where some people think some races should be incentivized more than others or women are not given the same chances as men. This is unacceptable. What we can do is to solve these obvious problems and use algorithms as augmentation tools for humans to overcome these biases.
另一方面，我们社区的一些普遍偏见显然可以解决。 我仍然无法相信我们生活的世界上有些人认为某些种族比其他种族更应该受到激励，或者女性没有与男性一样的机会。 这是无法接受的。 我们可以做的是解决这些明显的问题，并使用算法作为人类克服这些偏差的增强工具。
Academia, innovative companies, entrepreneurs, and venture capitalists are constantly thinking about this problem. The key question is how governments can scale these policies to the whole society. We need proper processes, laws, and regulations to address this. This is yesterday’s problem and we are already late to solve it.
学术界，创新型公司，企业家和风险资本家一直在思考这个问题。 关键问题是政府如何将这些政策扩展到整个社会。 我们需要适当的流程，法律和法规来解决此问题。 这是昨天的问题，我们已经来不及解决。
Duygu: Have you tried to use GPT-3 in other languages? Maybe in Turkish?
Duygu：您是否尝试过使用其他语言的GPT-3？ 也许用土耳其语？
Yigit: I have tried. It works, but in many cases, the quality is not as good as English yet. Since GPT-3 doesn’t have any semantic logic behind the algorithm, it depends on which web content of other languages OpenAI feeds. Apparently, the OpenAI team feeds data that they can easily find through open-source and their own crawling algorithms.
Yigit：我已经尝试过了。 它可以工作，但是在许多情况下，质量还不如英语。 由于GPT-3在该算法后面没有任何语义逻辑，因此它取决于OpenAI提要提供的其他语言的Web内容。 显然，OpenAI团队提供了可以通过开源和自己的爬网算法轻松找到的数据。
The right step for fellow non-English speaking engineers is to contribute to the open-source data crawling projects that academia is using. One such example is http://commoncrawl.org.
对于非英语系工程师来说，正确的步骤是为学术界正在使用的开源数据爬网项目做出贡献。 这样的示例之一是http://commoncrawl.org 。
Duygu: At this point, it is obvious that every area of work and life will be affected by the latest generation of AI. In your opinion which areas will be transformed primarily?
Duygu：在这一点上，很明显，工作和生活的每个领域都将受到最新一代AI的影响。 您认为哪些领域将主要转型？
Yigit: Technology touches all aspects of our lives. And, we’ve been in this transition for many years since the introduction of the PC. This is similar to the industrialization process that took many decades and changed how people lived.
Yigit：技术触及我们生活的方方面面。 而且，自从PC推出以来，我们已经进行了很多年的过渡。 这类似于耗时数十年并改变了人们生活方式的工业化过程。
Things are just going at a much more rapid pace now. The art is in the sequence. Selling pet food online in 1998 was a good idea, but consumers were not ready to make that shift. However, starting with selling books online worked really well. The value was much clearer. It unleashed the benefit of buying long-tail books, which were not available in physical stores before.
现在事情正在以更快的速度发展。 艺术是按顺序进行的。 1998年在网上销售宠物食品是个好主意，但消费者还没有准备好转变。 但是，从在线销售书籍开始确实效果很好。 价值更加清晰。 它释放了购买长尾书的好处，而长尾书是以前在实体商店中无法买到的。
The key principle is not only to provide efficiency but also to provide access to opportunities that were not economically feasible before. Constructing a framework around this idea would help us think more methodically. For example, let’s reflect on history. Many people in many countries live a better life now than a king or a sultan two centuries ago thanks to the efficiency and accessibility to education, healthcare, products, and services provided or facilitated by innovation and technology. How can we make products and services more affordable and accessible thanks to the power of personalization through data and machine learning algorithms?
关键原则不仅是提供效率，而且是提供以前在经济上不可行的机会。 围绕这个想法构建一个框架将有助于我们更加系统地思考。 例如，让我们回顾一下历史。 由于创新和技术所提供或促进的教育，医疗保健，产品和服务的效率和可及性，许多国家中的许多人现在的生活比两个世纪前的国王或苏丹更好。 借助数据和机器学习算法的个性化功能，我们如何使产品和服务更实惠，更易获得？
Products and services will be cheaper, better, and faster. Any enterprise product that costs millions of dollars will be cheaper. Any personal service that costs hundreds of dollars per hour will be more affordable. These products and services have been expensive because they require human experts to do customization and fit them for each customer. For example, wealthy individuals and large companies have accountants, doctors, and lawyers, and the rest of the population and firms are under-served and over-pay for these services. Thanks to machine learning and data, I expect that most people will have a personalized lawyer, doctor, and accountant in the coming years.
产品和服务将更便宜，更好，更快。 任何花费数百万美元的企业产品都会更便宜。 每小时花费数百美元的任何个人服务将更加实惠。 这些产品和服务之所以昂贵，是因为它们需要人类专家进行定制，并使其适合每个客户。 例如，富裕的个人和大公司都有会计师，医生和律师，而其余的人口和公司服务不足，而且为这些服务支付的费用过高。 借助机器学习和数据，我希望在未来几年中，大多数人将拥有个性化的律师，医生和会计师。
Duygu: You are investing in AI startups. Has your investment thesis changed due to GPT-3? What kind of AI startups do you expect to see in the mid to long term?
Duygu：您正在投资AI初创公司。 由于GPT-3，您的投资论点是否有所改变？ 您期望中长期看到什么样的AI初创公司？
Yigit: It made our investment thesis stronger. GPT-3 (natural language understanding, adjacent technologies, and its future iterations) is now a sub-thesis of our AI-focus.
Yigit：这使我们的投资论点更加强大。 GPT-3(自然语言理解，相邻技术及其未来迭代)现在是我们关注AI的副主题。
I expect that we’ll interact with strangers a lot less and live outside the cities more. There will be more robots, self-driving vehicles, new living and work environments, no-touch physical experiences, and personalized assistants.
我希望我们将减少与陌生人的互动，并更多地生活在城市之外。 将会有更多的机器人，自动驾驶汽车，新的生活和工作环境，无接触的身体体验以及个性化的助手。
AI is changing how we find, filter, and process information. The most important disruption will be to information aggregators such as Google, Facebook, and Twitter. Millions of developers will have the search power of Google.
人工智能正在改变我们寻找，过滤和处理信息的方式。 最重要的破坏将是对信息聚合器，例如Google，Facebook和Twitter。 数百万的开发人员将拥有Google的搜索能力。
Duygu: Thank you Yigit! It was a pleasure to talk to you about GPT-3.
Duygu：谢谢Yigit！ 很高兴 和你谈谈 GPT-3。
Yigit: Pleasure is mine. Thank you for having me.
Yigit：快乐是我的。 谢谢你有我
— —
— —
We thank Duygu and Yigit for this interesting and enlightening conversation. As MaxiTech, we are keeping a close eye on GPT-3 and looking forward to exploring its capabilities to transform our lives (hopefully) for the better :)
我们感谢Duygu和Yigit进行的有趣且启发性的对话。 作为MaxiTech，我们一直密切关注GPT-3，并期待探索其能力(希望)使我们的生活变得更好：)
翻译自: https://medium.com/@maxitech/will-gpt-3-replace-the-coders-783bf5adbfa2
gpt 引导驱动器