AI 学习中心
按入门与选型、游戏开发、图像与3D、视频与音频、智能体与自动化、模型与工程六个主题学习。站内文章提供示例与检查方法，外部资料保留原文入口。
- Token 是什么？为什么不等于字数？
用一笔多轮对话账单，分清字数、上下文、输入输出和缓存；附可复制的用量记录模板。
- 模型、平台和 API，有什么区别？
以“批量整理用户反馈”为例，决定用聊天网页、工作流还是 API，避免买错入口和套餐。
- 推理强度越高，就一定越合适吗？
用可复测的小实验选择推理强度：先定义正确答案，再比较通过率、等待时间与单次成功成本。
- 30 秒视频，为什么会生成 90 秒素材？
从六镜头短片反推素材量、重试上限与现金预算，附按次收费和成功率的算例。
- 从想法到一个可玩的 2D 游戏原型
先交付移动、收集与重开三个可检查的玩法，再衔接带源码的Godot、Cocos或Phaser完整实战。
- 把一张图片做成游戏里的 3D 道具
用一个桌面木箱练习图转3D：检查轮廓、比例、法线、材质与目标引擎表现，保留可验收交付清单。
- 用已有图片制作一支 30 秒短片
围绕一支30秒桌面小夜灯短片，完成六镜分镜、参考素材、逐镜生成、配音与最终导出检查。
- 怎样理解节点式 AI 图像工作流？
跑通官方基础文生图工作流，再固定种子做单变量对照，学习读连接、修依赖和保存可复用流程。
- 把重复工作变成可检查的自动化流程
从“反馈文本转待审任务”搭一条可检查工作流，先验证字段、去重与失败恢复，再接自动触发。
- 为游戏资产建立一份可复用的视觉规范
以背包里的钥匙、药水和金币为例，建立能真正复用的素材规范与三张资产验收表。
- 本地大模型上手：先做一轮小规模验证
先估内存，再用同一组请求验证本地模型的速度与质量；附 Ollama 用量字段和测速公式。
- 用小型资料库搭建可核对来源的问答
用三份产品资料完成一次可核对来源的问答实验，定位“找错资料”和“看见资料仍答错”的区别。
- AI 配音到成片：一份可执行的检查清单
把一段约20秒旁白做成可剪辑素材：分句、校读、多音字、停顿、响度和字幕逐项检查。
- 给 AI 看图之前，怎样描述你的问题
用一张产品界面截图练习看图提问：限定可见证据、坐标区域和输出格式，减少凭空补细节。
- 给游戏 NPC 接入大模型前，先划定行为边界
为商店NPC设计可验证的对话接口：模型负责台词，库存、金币与交易结果由游戏代码决定。
- 把一段故事拆成可生成、可剪辑的镜头表
把“小夜灯的一晚”写成六镜头拍摄表，每镜只安排一个动作，并预设生成失败时的替代方案。
- 怎样用自己的任务比较两个 AI 模型
用20条真实任务搭一份小型评测集，识别编造、格式失败和版本退化，而非凭榜单选模型。
- Blender 官方基础视频：从界面到第一个对象
跟随基础操作完成一个简单物体，保存工程并导出预览图。
- Unity Essentials：建立第一个交互作品
完成编辑器基础、场景搭建与交互任务，运行并检查自己的小作品。
- Unreal Engine：第一个小时与第一个小游戏
完成安装、基础场景与打包流程，检查分享后的工程能否运行。
- Runway 官方视频：用 Chat Mode 组织创作
用简短描述生成一组候选，比较反馈修改前后的差异。
- Runway Academy：从生成片段到创作流程
选择一个与项目匹配的视频教程，完成一个短镜头并保存输入和配置。
- DaVinci Resolve 官方视频：剪辑、调色与声音
下载原站练习素材，完成剪辑、基础调色和人声检查，导出测试片。
- 让炼丹更科学一些（十）：单调性假设的拆与补
上文《让炼丹更科学一些（九）：经典自适应梯度算法》中，我们为形如$\boldsymbol{\theta}_{t+1} = \boldsymbol{\theta}_t - \eta_t \bold...
- 让炼丹更科学一些（九）：经典自适应梯度算法
本系列前面八篇文章，都是在围绕SGD及其学习率讨论。而从本文开始，我们将正式进入自适应梯度算法的世界。可以说，现在所有的自适应梯度算法，都有一个共同的源头，那就是2011年的经典之作《Adapt...
- 让炼丹更科学一些（八）：多阶段训练的学习率
上篇文章《让炼丹更科学一些（七）：步长调度与权重平均》我们简单介绍了无调度（Schedule-Free）学习率的工作，它试图通过某种权重平均来替代学习率调度，实现常数学习率就可以训出最优模型的效...
- 动量的新理解：逼近特征层面的梯度下降
一个以动量为状态变量的优化器，基本形式如下：\begin{equation}\begin{aligned}\boldsymbol{M}_t =&\, \beta \boldsymbol{M}_{...
- 除了交叉熵，LM Loss还有什么选择？
一直以来，交叉熵（Cross Entropy）都是LLM预训练和微调的标准损失函数。那这个“标准”可以改吗？如果想改，那又有哪些选择呢？改完之后又会带来什么影响呢？可能很多读者从未认真推敲过这些...
- 简单谈谈K3的MoE和Attention
上个月，我们发布了迄今为止最大的开源模型K3。作为K2的继任者，K3并不是一次从零开始的重新设计，而是沿着我们过去一系列工作自然演化而来，并融合了我们对效果、效率、稳定性的一些最新理解和改进。可...
- 解构Scaling Law：优化、架构、数据的三重奏
训练一个大型的神经网络，最终效果会受到非常多因素的影响，换个优化器，换个模型架构，或者换一个训练集，结果都可能截然不同。在工程实践中，我们将调试这些因素的经验结果，戏称为“炼丹”。但如何从经验上...
- Harness Engineering for Self-Improvement
The concept of recursive self-improvement (RSI) dates back to I. J. Good (1965), where he defined an “ultraintelligent machine” as a system that can surpass humans in all intellectual activities and d…
- Scaling Laws, Carefully
Scaling laws are one of the most critical empirical findings in deep learning. The observation is simple in form: the training loss $L$ decreases predictably as we scale up model size $N$, dataset siz…
- Why We Think
Special thanks to John Schulman for a lot of super valuable feedback and direct edits on this post. Test time compute (Graves et al. 2016, Ling, et al. 2017, Cobbe et al. 2021) and Chain-of-thought (C…
- Reward Hacking in Reinforcement Learning
Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the reward function to achieve high rewards, without genuinely learning or completing the intended task.…
- Extrinsic Hallucinations in LLMs
Hallucination in large language models usually refers to the model generating unfaithful, fabricated, inconsistent, or nonsensical content. As a term, hallucination has been somewhat generalized to ca…
- Diffusion Models for Video Generation
Diffusion models have demonstrated strong results on image synthesis in past years. Now the research community has started working on a harder task—using it for video generation. The task itself is a …
- Thinking about High-Quality Human Data
[Special thank you to Ian Kivlichan for many useful pointers (E.g. the 100+ year old Nature paper “Vox populi”) and nice feedback. 🙏 ] High-quality data is the fuel for modern data deep learning mode…
- Adversarial Attacks on LLMs
The use of large language models in the real world has strongly accelerated by the launch of ChatGPT. We (including my team at OpenAI, shoutout to them) have invested a lot of effort to build default …
- LLM Powered Autonomous Agents
Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT, GPT-Engineer and BabyAGI, serve as inspiring examples. The p…
- Prompt Engineering
Prompt Engineering, also known as In-Context Prompting, refers to methods for how to communicate with LLM to steer its behavior for desired outcomes without updating the model weights. It is an empiri…
- How To Write With An LLM
How To Write With An LLM Thomas Ptacek on using LLMs as copyeditors, not as writing assistants: Rule Number One: You may not use a single word an LLM suggests to you. [...] I think that as a form of i…
- Self-generated prompt injections in compaction summaries
Self-generated prompt injections in compaction summaries In Our framework for reporting model misalignment OpenAI provide "six reports on unexpected or concerning model behavior we’ve observed in the …
- OpenAI agents attacked RubyGems back in May
OpenAI agents carried out an undisclosed attack on RubyGems is a new bombshell report from Spencer Kitts, Thomas Larsen, and Sydney Von Arx - three of the four authors of the report on the agent attac…
- AI眼里的你怎么测？豆包、DeepSeek 与 Codex 使用指南
AI眼里的你根据你与AI互动的方式生成趣味画像和插画封面。可以选择电脑端Codex，或在豆包、DeepSeek中完成行为统计后回来领取。结果是趣味解读，不是心理诊断。
- AI人格怎么使用：临时对话与长期默认
人格提示词调整AI的表达风格，不改变模型本身的知识、权限或事实判断。先选一种风格试用，再决定是否保存为长期偏好。
- Token费用怎么算：输入、输出与推理预算
先用参考编码统计输入词元，再设置预计输出和推理用量，最后选择有明确渠道报价的模型。估算用于比较预算，实际账单以服务商记录为准。
- The Transformer Family Version 2.0
Many new Transformer architecture improvements have been proposed since my last post on “The Transformer Family” about three years ago. Here I did a big refactoring and enrichment of that 2020 post — …
- Large Transformer Model Inference Optimization
[Updated on 2023-01-24: add a small section on Distillation.] Large transformer models are mainstream nowadays, creating SoTA results for a variety of tasks. They are powerful but very expensive to tr…
- Some Math behind Neural Tangent Kernel
Neural networks are well known to be over-parameterized and can often easily fit data with near-zero training loss with decent generalization performance on test dataset. Although all these parameters…
- Generalized Visual Language Models
Processing images to generate text, such as image captioning and visual question-answering, has been studied for years. Traditionally such systems rely on an object detection network as a vision encod…
- Learning with not Enough Data Part 3: Data Generation
Here comes the Part 3 on learning with not enough data (Previous: Part 1 and Part 2). Let’s consider two approaches for generating synthetic data for training. Augmented data. Given a set of existing …
- How to Train Really Large Models on Many GPUs?
[Updated on 2022-03-13: add expert choice routing.] [Updated on 2022-06-10]: Greg and I wrote a shorted and upgraded version of this post, published on OpenAI Blog: “Techniques for Training Large Neur…
- What are Diffusion Models?
[Updated on 2021-09-19: Highly recommend this blog post on score-based generative modeling by Yang Song (author of several key papers in the references)]. [Updated on 2022-08-27: Added classifier-free…
- Contrastive Representation Learning
The goal of contrastive representation learning is to learn such an embedding space in which similar sample pairs stay close to each other while dissimilar ones are far apart. Contrastive learning can…
- Reducing Toxicity in Language Models
Large pretrained language models are trained over a sizable collection of online data. They unavoidably acquire certain toxic behavior and biases from the Internet. Pretrained language models are very…
- Controllable Neural Text Generation
[Updated on 2021-02-01: Updated to version 2.0 with several work added and many typos fixed.] [Updated on 2021-05-26: Add P-tuning and Prompt Tuning in the “prompt design” section.] [Updated on 2021-0…
- How to Build an Open-Domain Question Answering System?
[Updated on 2020-11-12: add an example on closed-book factual QA using OpenAI API (beta). A model that can answer any question with regard to factual knowledge can lead to many useful and practical ap…
- Neural Architecture Search
Although most popular and successful model architectures are designed by human experts, it doesn’t mean we have explored the entire network architecture space and settled down with the best option. We…
- Exploration Strategies in Deep Reinforcement Learning
[Updated on 2020-06-17: Add “exploration via disagreement” in the “Forward Dynamics” section. Exploitation versus exploration is a critical topic in Reinforcement Learning. We’d like the RL agent to f…
- The Transformer Family
[Updated on 2023-01-27: After almost three years, I did a big refactoring update of this post to incorporate a bunch of new Transformer models since 2020. The enhanced version of this post is here: Th…
- Evolution Strategies
Stochastic gradient descent is a universal choice for optimizing deep learning models. However, it is not the only option. With black-box optimization algorithms, you can evaluate a target function $f…
- Meta Reinforcement Learning
In my earlier post on meta-learning, the problem is mainly defined in the context of few-shot classification. Here I would like to explore more into cases when we try to “meta-learn” Reinforcement Lea…
- Domain Randomization for Sim2Real Transfer
In Robotics, one of the hardest problems is how to make your model transfer to the real world. Due to the sample inefficiency of deep RL algorithms and the cost of data collection on real robots, we o…
- Are Deep Neural Networks Dramatically Overfitted?
[Updated on 2019-05-27: add the section on Lottery Ticket Hypothesis.] If you are like me, entering into the field of deep learning with experience in traditional machine learning, you may often ponde…
- Generalized Language Models
[Updated on 2019-02-14: add ULMFiT and GPT-2.] [Updated on 2020-02-29: add ALBERT.] [Updated on 2020-10-25: add RoBERTa.] [Updated on 2020-12-13: add T5.] [Updated on 2020-12-30: add GPT-3.] [Updated …
- Object Detection Part 4: Fast Detection Models
In Part 3, we have reviewed models in the R-CNN family. All of them are region-based object detection algorithms. They can achieve high accuracy but could be too slow for certain applications such as …
- Godot + AI 实战：从零做一个可重开的躲避游戏
中文逐步实战：建工程、画角色、实现移动与敌人、检测碰撞、计时并重开。附可导入工程与完整GDScript。
- Cocos Creator + AI 实战：制作30秒点击计分游戏
中文逐步搭建：创建Canvas与Game节点、挂载完整TypeScript组件、生成目标、处理点击、计分倒计时和重开。
- LayaAir 中文资料：编辑器入门与AI协同文档
官方中文文档索引，区分基础工程搭建和AI工具接入；这是一份资料导读，不是本站完整项目教程。
- Phaser + AI 实战：制作30秒接金币H5游戏
从一个HTML页面到可玩的接金币游戏：键盘与触摸控制、生成与回收、计分倒计时、结束重开，附源码和在线试玩。
- Web 3D 官方资料：Three.js、Babylon.js 与 PlayCanvas
资料索引，帮助定位框架手册与开发入口；不再把选型提纲标为完整3D实战。
- Flow-based Deep Generative Models
So far, I’ve written about two types of generative models, GAN and VAE. Neither of them explicitly learns the probability density function of real data, $p(\mathbf{x})$ (where $\mathbf{x} \in \mathcal…
- From Autoencoder to Beta-VAE
[Updated on 2019-07-18: add a section on VQ-VAE & VQ-VAE-2.] [Updated on 2019-07-26: add a section on TD-VAE.] Autocoder is invented to reconstruct high-dimensional data using a neural network model w…
- Attention? Attention!
[Updated on 2018-10-28: Add Pointer Network and the link to my implementation of Transformer.] [Updated on 2018-11-06: Add a link to the implementation of Transformer model.] [Updated on 2018-11-18: A…
- Implementing Deep Reinforcement Learning Models with Tensorflow + OpenAI Gym
The full implementation is available in lilianweng/deep-reinforcement-learning-gym In the previous two posts, I have introduced the algorithms of many deep reinforcement learning models. Now it is the…
- Object Detection for Dummies Part 3: R-CNN Family
[Updated on 2018-12-20: Remove YOLO here. Part 4 will cover multiple fast object detection algorithms, including YOLO.] [Updated on 2018-12-27: Add bbox regression and tricks sections for R-CNN.] In t…
- Learning Word Embedding
Human vocabulary comes in free text. In order to make a machine learning model understand and process the natural language, we need to transform the free-text words into numeric values. One of the sim…
- Anatomize Deep Learning with Information Theory
Professor Naftali Tishby passed away in 2021. Hope the post can introduce his cool idea of information bottleneck to more people. Recently I watched the talk “Information Theory in Deep Learning” by P…
- From GAN to WGAN
[Updated on 2018-09-30: thanks to Yoonju, we have this post translated in Korean!] [Updated on 2019-04-18: this post is also available on arXiv.] Generative adversarial network (GAN) has shown great r…
- How to Explain the Prediction of a Machine Learning Model?
The machine learning models have started penetrating into critical areas like health care, justice systems, and financial industry. Thus to figure out how the models make the decisions and make sure t…
- Predict Stock Prices Using RNN: Part 2
In the Part 2 tutorial, I would like to continue the topic on stock price prediction and to endow the recurrent neural network that I have built in Part 1 with the capability of responding to multiple…
- An Overview of Deep Learning for Curious People
(The post was originated from my talk for WiMLDS x Fintech meetup hosted by Affirm.) I believe many of you have watched or heard of the games between AlphaGo and professional Go player Lee Sedol in 20…
- llm-keys-ui 0.1
Release: llm-keys-ui 0.1 This plugin solves a very specific problem. I've started using Codex Remote to run coding agents on various machines while controlling them from my phone. Sometimes I use thos…
- Jev introduces a new shape of LLM - System One, aka Decision Models
Last week TypeSafe AI unveiled Jev , their first example of a new category of model that they are calling "System One models" (I'm with Maggie Appleton, I think "decision models" is a better name for …
- llm-typesafe 0.1a0
Release: llm-typesafe 0.1a0 I built this new plugin for LLM to add support for TypeSafe AI's new Jev model . Install it like this: llm install llm-typesafe Then set an API key ( get one here , the wai…
- llm 0.36
Release: llm 0.36 New OpenAI models: gpt-6-sol for GPT-6 Sol and gpt-6-luna for GPT-6 Luna . #1702 Model plugins can now declare supports_conversation = False for models that only accept single-turn p…
- llm-anthropic 0.29
Release: llm-anthropic 0.29 Adds support for Claude Opus 5.5 : llm -m claude-opus-5.5 "prompt goes here" Tags: llm , anthropic
- SF October 14th: A Birds of a Feather Session on Agentic Engineering
SF October 14th: A Birds of a Feather Session on Agentic Engineering I'm hosting an evening event with Jesse Vincent in San Francisco on Wednesday 14th October for people who are building weird and in…