MEPPP 热点收录
@meppp_hot
收录
原帖正文
Andrej Karpathy 说，人们对 AI 能力的理解差距在扩大，这种认知差距正在变成一个漏斗。约 60 亿人几乎没接触过大模型；10 亿到 20 亿人偶尔用免费版 ChatGPT 查东西、写东西，觉得用处不大，很多数学和编程以外的专业人士也在这一层，对 AI 的进步只是道理上懂。约 2000 万人（Karpathy 按 0.2% 估算的，我觉得偏高，至少除以 10）在编程和数学里重度使用最前沿的模型，亲眼看着要做几周、几个月的项目，一段提示词发给 Agent 几天就能做完；还有约 5000 人能在公司内部用上更强的系统。差距主要有三个原因：一是问题够不够大，一段话就能答完的问题用不到 AI 的真本事。二是能力只在部分领域冒尖：编程和数学的结果能自动验证，比如代码测试通没通过，模型很适合用强化学习（做对了就给奖励）来练，这类能力卖给企业又最赚钱，厂商投入最多；文章写得好不好很难判定，进步就慢。三是能用到哪一档，免费版、付费版、内部版看到的东西差别很大。所以，如果你只偶尔问问免费的聊天机器人，看到的离最前沿差了好几层；如果你主要拿 AI 写东西、查资料，觉得这一年没怎么进步，这个感觉也没错。
引用自 X
原帖地址：https://x.com/i/status/2109401538279977114
原帖：
由于对法学硕士能力的共同理解的差距正在*扩大* ，因此从4月份的这条推文中除去了灰尘。 现在，它不再是“两组人过去说话” ，而是一个尖锐的漏斗。-大约60亿人（约占人口的75% ）几乎没有接触过LLM。-大约1-2B （约20 ％ ）是免费ChatGPT类产品的临时和不常用用户。这个群体看到了derpy聊天机器人，并把它们当作更好的谷歌搜索、写作辅助工具等。也许智能体会尝试为您预订航班。我有一些非技术朋友（合理地说，国际海事组织）说他们根本没有什么用处。即使是math&code之外的许多专业人士也属于此等级。 例如，高管和许多其他职能部门花费大量时间与其他人交谈，因此，虽然他们在智力上了解正在发生的事情，但它仍然是二手的，有点抽象。
-现在，我们开始在数学和代码方面专业使用前沿级LLM。大约2000万人（ 0.2 ％ ）亲眼看到，过去需要数周/数月的大型复杂项目现在可以通过智能体提示完成。构建应用程序、复制应用程序、翻译应用程序、从二进制文件反编译应用程序...这一切都发生得很快，而且是最近发生的--不到一年前，我手动编写代码，逐个将记忆的计算机代码命令键入代码编辑器字符，偶尔按Tab键自动完成一小段代码。
-最后，我们接触到了约5000人（约0.00006% ） ，他们可以在内部使用前沿级系统。外界已经看到了预览。画面显示，成千上万个智能体机器人如同蜂拥而至的蜂群，在数周内协同完成大型软件项目：制造零日漏洞、以机器速度运行网络攻击和防御、探索新科学、推进数学前沿。这些工作，如果由业内顶尖专业人士完成，则需要数年时间。与此同时，人类的审查和理解开始落后。 例如，人们仍然参与了几个月前OpenAI-HF事件的“考古学”。 数学家可能会仔细研究前沿数学的722份手稿一段时间。漏斗由多种因素共同驱动：
1.影响随着抱负、问题规模和视野而变化。带有段落答案的问题几乎不会强调系统。 你需要一大堆你真正关心的大难题。 这一方面推动了渠道的消费者/专业维度。
2.系统的锯齿（我已经单独写了很多）。 在数字化、可验证且具有经济价值的领域中，能力达到顶峰。这是因为LLM能力来自于由收入潜力驱动的策划环境混合的可验证奖励的强化学习。 这个方面主要驱动漏斗的区域（例如数学和代码）维度。
3.访问。免费层、付费层、内部。这就是当下的奇怪之处。 公众大多没有与这些系统进行交互。当他们真的这么做时，看起来就像一个笨拙的聊天机器人。大多数专业人士仍然只看到轻微的提升。而只有一小部分专业人士正在经历曲线垂直上升带来的眩晕感。这一切都在同时发生。
-现在，我们开始在数学和代码方面专业使用前沿级LLM。大约2000万人（ 0.2 ％ ）亲眼看到，过去需要数周/数月的大型复杂项目现在可以通过智能体提示完成。构建应用程序、复制应用程序、翻译应用程序、从二进制文件反编译应用程序...这一切都发生得很快，而且是最近发生的--不到一年前，我手动编写代码，逐个将记忆的计算机代码命令键入代码编辑器字符，偶尔按Tab键自动完成一小段代码。
-最后，我们接触到了约5000人（约0.00006% ） ，他们可以在内部使用前沿级系统。外界已经看到了预览。画面显示，成千上万个智能体机器人如同蜂拥而至的蜂群，在数周内协同完成大型软件项目：制造零日漏洞、以机器速度运行网络攻击和防御、探索新科学、推进数学前沿。这些工作，如果由业内顶尖专业人士完成，则需要数年时间。与此同时，人类的审查和理解开始落后。 例如，人们仍然参与了几个月前OpenAI-HF事件的“考古学”。 数学家可能会仔细研究前沿数学的722份手稿一段时间。漏斗由多种因素共同驱动：
1.影响随着抱负、问题规模和视野而变化。带有段落答案的问题几乎不会强调系统。 你需要一大堆你真正关心的大难题。 这一方面推动了渠道的消费者/专业维度。
2.系统的锯齿（我已经单独写了很多）。 在数字化、可验证且具有经济价值的领域中，能力达到顶峰。这是因为LLM能力来自于由收入潜力驱动的策划环境混合的可验证奖励的强化学习。 这个方面主要驱动漏斗的区域（例如数学和代码）维度。
3.访问。免费层、付费层、内部。这就是当下的奇怪之处。 公众大多没有与这些系统进行交互。当他们真的这么做时，看起来就像一个笨拙的聊天机器人。大多数专业人士仍然只看到轻微的提升。而只有一小部分专业人士正在经历曲线垂直上升带来的眩晕感。这一切都在同时发生。
查看引用原文
Dusting off this tweet from April because this gap in shared understanding of LLM capability is *widening*. It's now less "two groups of people speaking past each other" and more a sharp funnel.- Napkin math somewhere around 6B people (~75% of the population) have barely come in contact with LLMs at all.
- Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an agent tries to book you a flight. I have non-tech friends who (reasonably, imo) say they have not much use for it at all. Even many professionals outside of math&code are in this tier. For example, execs and many other functions spend a lot of their time talking to other people, so while they understand what is happening intellectually, it is still second-hand and a bit abstract.
- Now we get to professional use of frontier-grade LLMs in math&code. Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt. Building apps, copying apps, translating apps, decompiling apps from binaries... This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code.
- And finally we get to the ~5,000 people (~0.00006%) with access to frontier-grade systems internally. The external world has seen the preview. It looks like swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work. Meanwhile, human review and comprehension are starting to fall behind. For example, people are still involved in the "archeology" of the OpenAI-HF incident from many months ago. Mathematicians may be poring over the 722 manuscripts on frontier mathematics for a while.The funnel is driven by a combination of factors:
1. The impact scales with ambition, problem size, and horizon. A question with a paragraph answer barely stresses the system. You need a reservoir of big, difficult problems that you really care about. This aspect drives the consumer / professional dimension of the funnel.
2. The jaggedness of the system (which I have written about a lot separately). Capability peaks in domains that are digital, verifiable and economically valuable. This is because LLM capability emerges from reinforcement learning on verifiable rewards on a curated environment mixture driven by revenue potential. This aspect primarily drives the area (e.g. math&code) dimension of the funnel.
3. Access. Free-tier, paid-tier, internal.So this is the weirdness of the moment. The general public has mostly not interacted with these systems. When they have, it looks like a derpy chatbot. The majority of professionals still see only a modest uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the same time.
- Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an agent tries to book you a flight. I have non-tech friends who (reasonably, imo) say they have not much use for it at all. Even many professionals outside of math&code are in this tier. For example, execs and many other functions spend a lot of their time talking to other people, so while they understand what is happening intellectually, it is still second-hand and a bit abstract.
- Now we get to professional use of frontier-grade LLMs in math&code. Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt. Building apps, copying apps, translating apps, decompiling apps from binaries... This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code.
- And finally we get to the ~5,000 people (~0.00006%) with access to frontier-grade systems internally. The external world has seen the preview. It looks like swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work. Meanwhile, human review and comprehension are starting to fall behind. For example, people are still involved in the "archeology" of the OpenAI-HF incident from many months ago. Mathematicians may be poring over the 722 manuscripts on frontier mathematics for a while.The funnel is driven by a combination of factors:
1. The impact scales with ambition, problem size, and horizon. A question with a paragraph answer barely stresses the system. You need a reservoir of big, difficult problems that you really care about. This aspect drives the consumer / professional dimension of the funnel.
2. The jaggedness of the system (which I have written about a lot separately). Capability peaks in domains that are digital, verifiable and economically valuable. This is because LLM capability emerges from reinforcement learning on verifiable rewards on a curated environment mixture driven by revenue potential. This aspect primarily drives the area (e.g. math&code) dimension of the funnel.
3. Access. Free-tier, paid-tier, internal.So this is the weirdness of the moment. The general public has mostly not interacted with these systems. When they have, it looks like a derpy chatbot. The majority of professionals still see only a modest uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the same time.
0 条评论
登录后可以参与讨论。
成员登录还没有评论。第一条认真回应会很重要。