AI & LLMs · Frontier
量化(Quantization)
把模型内部数字的精度调低,让文件变小、跑得更快也更省内存。代价是效果会略微下降,属于有损压缩。
Some fields on this page have no English version yet — they are shown in the original Chinese.
TL;DRIn 30 seconds
- 把模型内部那些数字记得粗一点,文件就小下来、跑起来也更省力。
- 它让原本塞不进普通电脑的大模型,能在笔记本甚至手机上跑起来。
- 这是有损的,精度压得越低,答难题时越容易出错。
OverviewIn detail
模型内部存着数以十亿计的小数,也就是它的参数。这些数字记录得越精细,模型体积越大、占的内存越多。量化做的事,就是把这些数字换成精度更低的表示,比如从精细的小数降成粗糙一些的档位。
收益很直接。原始模型动辄几十 GB,普通笔记本根本放不下;量化之后体积可能压到原版的几分之一,内存占用随之下降,个人电脑甚至手机也能跑起来。想在本机使用开源模型,这一步多半绕不开。代价是模型会变得钝一点,答难题时更容易出错。
你在开源模型的下载页里见到的 GGUF、Q4、Q8 这类名字,指的就是不同档位的量化版本。名字里的 Q 就是量化的缩写,后面的数字表示保留多少位。档位越低,文件越小、跑得越快,和原版的差距也越明显。选哪一档,就是在速度和效果之间做取舍。
容易混淆的是量化和蒸馏。量化不重新训练,只是把已经训好的模型里的数字表示变粗糙,知识本身没有换;蒸馏则是让一个小模型去模仿大模型的输出,是重新训练出来的另一个模型。另外量化并非无损,效果通常会掉一点,只是多数日常场景下人未必察觉。同一个模型往往同时放出好几档,方便你按机器条件来挑。
MythsCommon misunderstandings
- ✕以为压过之后效果没变化 ✓它是往粗里记,越压越钝,日常问题未必察觉,难题上差得出来
- ✕以为量化就是又训练了一遍 ✓它不重训,只是给训好的模型换个记法,知识本身没换
- ✕以为量化和小模型是一回事 ✓小模型是另训的一个模型,量化只是给同一个模型瘦身
In practiceWhere you will meet it
- 在开源模型下载页挑不同档位,按自己机器的条件来选
- 笔记本内存不够时,换一个更小的量化版本再试
- 同一件事既要效果好又想省机器,就拿几档实际比一比
SummaryThe key points
- 把模型内部参数的数值精度降低
- 换来更小的体积、更快的速度、更低的内存占用
- 让普通电脑和手机也能跑起大模型
- 它只压缩已训好的模型,不重新训练
FAQ
What does 量化(Quantization) mean?
把模型内部那些数字记得粗一点,文件就小下来、跑起来也更省力。它让原本塞不进普通电脑的大模型,能在笔记本甚至手机上跑起来。这是有损的,精度压得越低,答难题时越容易出错。
Where will I run into 量化(Quantization)?
在开源模型下载页挑不同档位,按自己机器的条件来选;笔记本内存不够时,换一个更小的量化版本再试;同一件事既要效果好又想省机器,就拿几档实际比一比。
What do people most often get wrong about 量化(Quantization)?
以为压过之后效果没变化 — but 它是往粗里记,越压越钝,日常问题未必察觉,难题上差得出来;以为量化就是又训练了一遍 — but 它不重训,只是给训好的模型换个记法,知识本身没换。
What are the key points about 量化(Quantization)?
把模型内部参数的数值精度降低;换来更小的体积、更快的速度、更低的内存占用;让普通电脑和手机也能跑起大模型;它只压缩已训好的模型,不重新训练。
Common pitfalls 量化是有损的,档位压得越狠效果掉得越多,别只盯着文件有多小。
Also written asYou may also see it written as
These are all the same concept — the same thing, however it is written in a skill description. Do not mistake different spellings for different ideas.
Entry 23 of 186 · All entries
Keep readingRelated reading
AI Agent
An AI program that sets its own goal, breaks it into steps and calls tools to get the job done. You hand it a task and it works out how to do it — instead of only answering one question at a time.
LLM
A very large program that learned to predict the next piece of text after reading an ocean of it. Whatever you type, it is guessing what is most likely to come next — writing, translating and answering questions are all the same thing used differently.
嵌入(Embedding)
把一段文字换算成一串数字,让电脑能比较哪两句话意思更接近。搜索和知识库能在换了说法时也找对答案,靠的就是它。
Claude
A large language model built by Anthropic, known for understanding long documents, writing well and writing code. It comes both as a chat assistant on the web and phone, and in a dedicated form for programming.
Prompt
The message you send to an AI — the instruction or question you give it. How well that passage is written decides whether the answer that comes back is on target.
MCP
A standard interface that lets AI connect to outside tools and data in one uniform way. Build a tool to it once and any AI assistant can call it, saving you from writing a separate integration for each tool.
Where to go nextExplore more
Back to the skills themselves
Most of the concepts covered here come from skills that install into your AI client with a single command.