Service & Retail AI Watch, Issue 2: LLM Training Paradigms and Enterprise Strategy

Why this issue This issue leans on Andrej Karpathy’s (a founding OpenAI team member) framing of how large language models work, translated into plain language, plus a look at the strategic paths enterprises take with LLMs. Because LLM training is such a large topic, most readers outside the AI project team need this background to understand both our internal AI projects and the model paths other companies are choosing — so Part One (model training) runs long.

Reading tip: if you already know the training basics, jump straight to Part Two (enterprise model-path selection).

Key Findings

1. Model training breaks into three main stages:

   • ① Pretraining: the model’s background-knowledge phase, forming its “worldview.” A transformer is trained on a massive dataset (typically trillions of tokens of internet text) to learn the statistical patterns of language by predicting the next token.

   • ② Supervised fine-tuning (SFT): the model’s classroom practice, learning to imitate how to solve a problem. Human-annotated data is used to fine-tune the model, using conversation sets to steer it toward compliant output. This stage typically accounts for a small share of overall training cost.

   • ③ Reinforcement learning (RL): the model’s after-class homework — try the problem first, check the answer after, and keep hunting for the best path to the correct answer. For a given prompt, thousands of independent solution attempts are sampled, and the model is pushed to improve under a reward signal. For domains without a quantifiable “best” answer (writing a joke, a poem), RLHF (reinforcement learning from human feedback) lets AI learn to substitute for a human labeling preferences.

2. What understanding these stages and paradigms reveals:

   • By training investment: pretraining is by far the most resource-intensive stage, consuming the large majority of compute and time. Post-training uses a comparatively small share of time and compute — given the steep cost, most enterprises don’t build their own pretrained LLM from scratch.

   • By stage of maturity: reinforcement learning is the field’s most exciting frontier right now. Pretraining and fine-tuning are largely mature and standardized; RL has no settled standard yet, and it’s the necessary path to pushing past current limits (as with AlphaGo) toward AGI.

   • Model limitations: the way LLMs are trained gives them inherent limits — hallucination (the model won’t admit not knowing something at its knowledge boundary), and weak computation/counting ability — because an LLM’s “cognition” isn’t grounded in true understanding, but in a statistical probability distribution learned from massive text.

   • Domestic vs. overseas build philosophy: domestic (Chinese) frontier labs are more singularly focused on reasoning ability; overseas labs focus more on multimodality and foundational architecture innovation, competing for the technical high ground. Facing compute constraints, and because inference workloads are far lighter than pretraining, domestic model makers need to strengthen reasoning to work around the compute bottleneck on complex logic — in other words, reasoning is the domestic models’ “shortcut lane.”

3. Enterprise-level LLM strategy paths: Enterprise LLM strategy today runs a spectrum from light to heavy investment, splitting into engineering optimization (prompting, RAG) and model/algorithm optimization (SFT, RL, pretraining) — these are not mutually exclusive and are often combined.

   • ① Industry giants build a proprietary “brain” (Pretrain + RL): build a competitive moat through exclusive data and compute monopoly; requires a GPU cluster at the thousand-card scale.

   • ② Vertical-domain enterprises / vertical scenarios (Prompt + RAG + SFT): reuse an open-source industry base model plus high-quality vertical dialogue data to push past the base model’s limits and amplify scenario value; requires high-quality labeled data.

   • ③ SMEs agilely adapt for knowledge Q&A (Prompt + RAG): near-zero cold-start cost, no maintenance overhead, fast access to top-tier model capability, tuned iteratively with feedback.

Related Reading

• Andrej Karpathy: Deep Dive into LLMs like ChatGPT (~3.5 hour video)
• Andrew Ng’s detailed walkthrough of how the transformer works (~1.5 hour video)
• A panoramic explainer on how reinforcement learning is reshaping 2025-era AI (Jinqiu Select, ~1 hour read, ~12,000 words)
• A detailed explainer on transformer mechanics (~1 hour read, ~12,000 words, written by an internal R&D team)
• A comprehensive survey of pretrained foundation models, from BERT to ChatGPT (~2 hour read, ~31,000 words, written by an internal R&D team)
• An Accessible Introduction to Applying LLMs (~30 min read, ~5,000 words, written by an internal R&D team)
• Internal AI briefing materials prepared for various business lines (by the foundational R&D platform’s intelligent-products team)
• An end-to-end LLM fine-tuning guide and optimization playbook

Part One: Model Training

1. Pretraining

1.1 Implementation Steps

1) Download and process internet data Using a dataset like Hugging Face’s FineWeb (accumulated at the multi-trillion-token, multi-terabyte scale) of high-quality crawled sites, HTML is converted to text and passed through several layers of base filtering before being used for pretraining:

• URL filtering: blocks pages the model shouldn’t learn from (adult content, hate speech, marketing spam), via a blocklist.
• Language filtering: filters by language — e.g. pages below a set English-content threshold are excluded, with the threshold set manually via a fastText language classifier — so the model naturally specializes in a target language.
• PII removal: strips personally identifiable information such as email addresses, home addresses, passwords, and government ID numbers.
• (Meta’s published mix of pretraining data for LLaMA gives a sense of the kinds of datasets typically blended together: Common Crawl and C4 web-crawl data, plus high-quality sources like GitHub, Wikipedia, books, ArXiv, and StackExchange, combined and sampled at set ratios to form the pretraining corpus.)

2) Tokenization The collected text is parsed into subword units (tokens) — essentially turning text into an integer sequence a computer can process, compressing sequence length while expanding vocabulary size. Current large models typically use a vocabulary on the order of 100,000+ tokens — e.g. GPT-4’s tokenizer — improving semantic representation efficiency by merging high-frequency character pairs, optimizing whitespace handling, and adapting to domain-specific terms.

3) Neural network training Training the neural network means adjusting and updating its parameters so each iteration’s predicted token moves closer to the training set. Training samples a “token window” (the context — from 0 up to, say, 8,000 tokens, and now up into the millions for frontier models) and adjusts the network’s parameters (a model like GPT-2 has on the order of tens of billions) so the correct token becomes more probable, no matter what the window contains. Massive parallel adjustment across the dataset is the training process.

Input is a variable-length token sequence; output is the predicted next token. Tokens go in at the top, pass through the network, and a predicted token comes out — each output token has a roughly fixed compute budget.

4) Inference Text generated from the model statistically resembles the training set (e.g. “read an” tends to be followed by “article”), but is better described as “inspired by” the training data rather than a verbatim copy of any sequence in it.

1.2 Base Models

A model that has only been through the above training has a few characteristics:

• Can’t natively hold a multi-turn conversation: a base model’s core ability is limited to predicting the next token given an input sequence (token-by-token completion). Through in-context learning, though, a base model can simulate basic dialogue behavior via few-shot prompting — inserting hand-written <user instruction, assistant reply> examples into the prompt to guide the model into imitating the conversational pattern.
• Random output: output is generated stochastically; the same input can produce different outputs.
• Quality-sensitive: content that appears frequently on the web is more easily “remembered” by the model; high-quality training sources (like Wikipedia) get outsized emphasis during training.
• Parameter and data scale follow a scaling law: once parameter count crosses a critical threshold (the hundred-billion range), performance improves roughly log-linearly. GPT-3, for instance, holds a conversation far better than GPT-2 given its much larger parameter count.

Table: Representative Model Parameter Evolution

Region	Model	Release	Training Tokens	Parameters
Overseas (compute-unconstrained, larger scale)	BERT	2018	Billions	Hundreds of millions
	GPT-2	2019	Tens of billions	Low billions
	GPT-3	2020	Hundreds of billions	Hundreds of billions
	Llama 3	2024	Tens of trillions	Hundreds of billions
	Gemini 1.5 Pro	2024	Tens of trillions	Undisclosed
Domestic (China)	Qwen 1.5-72B	2024	Trillions	Tens of billions
	DeepSeek-V2	2024	Trillions	Tens of billions

2. Supervised Fine-Tuning (SFT)

2.1 Implementation Steps

1) Create conversation data Human annotators write down the ideal assistant response, and the model is trained to imitate that style — a conversation dataset (on the order of 100,000 dialogues) replaces the base model’s raw internet data for further training, which is what gives the model its assistant “personality” and style during later inference. LLMs themselves can help build these datasets (LLM drafts, human edits). When you ask ChatGPT something, its output isn’t “magic” — it’s simulating the style of the human annotators whose examples it was trained on. The annotators writing these conversation datasets are typically domain experts.

2) Tokenization Conversations are encoded into tokens; the protocol is for the LLM to step in and reason/respond after the user’s turn.

2.2 What SFT Fixes

1) Hallucination (fabricated information)

• Root cause: the model won’t admit not knowing something at its knowledge boundary. Because every question in the conversation training set was given an answer, the model learns to “act confident” even on questions outside its actual knowledge.
• Mitigation: repeatedly interrogating the model to find its knowledge boundary, then adding those boundary-crossing exchanges to the training set with “I don’t know” as the correct answer. Tying uncertainty to a specific internal signal lets the model honestly say “I don’t know” when that signal fires, or trigger a tool call (like a web search) to fill the gap.
• (Note: a model can also hallucinate about itself — e.g. asked who created it, it may give its best statistical guess. This can be overridden via hard-coded SFT examples.)

2) A large model’s fuzzy memory An LLM’s memory system decomposes into two types:

• Parametric long-term memory (fuzzy memory): what it saw during pretraining, stored via weight updates as general knowledge — comparable to something you read a month ago.
• Context memory (working memory): real-time information dynamically loaded via the context window at inference time. Feeding the right context into working memory matters a great deal. Common approaches:
  • Attach the relevant context directly to the question.
  • Trigger a tool call so the LLM automatically runs a search, turning the search results into tokens pasted into working memory as network input.

3) Weak computation and counting ability (fixable via tools) LLMs are known to struggle with computation/counting (the classic examples: “how many r’s in strawberry,” or comparing 9.11 vs. 9.9). Causes: ① lack of abstract understanding of numeric logic; ② lack of abstract understanding of letters (the model reasons in tokens, humans in letters); ③ a short-token answer gets less compute, making errors more likely (the model is tempted to skip the “work”). Producing the reasoning steps before the final answer noticeably improves accuracy.

3. Reinforcement Learning (RL)

3.1 Implementation Steps

Put simply, RL is “taking the LLM through school.” Human cognition and LLM cognition differ, and human annotators may not actually know which dataset best suits how an LLM learns. Given a prompt, the model needs an exploration mechanism to dynamically discover the best solution — continually searching for the token sequence that reasons its way to the correct answer. The core RL loop: pose a question, generate an answer, score it, adjust the model. This process is extremely inference-heavy (sampling large numbers of candidate answers to filter for high-reward paths). RL works best in domains where correctness can be explicitly verified (like coding and math), which gives it a clean reward signal.

3.2 RL in Practice

DeepSeek’s emergence proved RL can produce a real capability leap in large models. The R1 model learned “chain of thought” — trying a problem from multiple angles and discovering that more attempts improve accuracy (at the cost of more tokens).

Domestic and overseas model-building philosophies currently diverge sharply:

• Domestic (China) models: more singularly focused on reasoning ability. Facing compute constraints, and with inference workloads far lighter than pretraining, model makers need to strengthen reasoning to break through the compute bottleneck on complex logic — reasoning is the domestic models’ “shortcut lane.”
• Overseas models: focus more on multimodality and foundational architecture innovation, competing for the technical high ground (Sora, GPT-5 multimodality, the open-source Llama ecosystem).

Table: Domestic vs. Overseas Model Reasoning Capability

Representative Model	Is a Reasoning Model?	Thinking-Mechanism Characteristics	Visibility of Thinking Process
DeepSeek-R1	✅ Yes	RL-optimized reasoning path, autonomously explores multi-step solutions	Full chain-of-thought (CoT) shown
Tencent Hunyuan-Turbo	✅ Yes (enterprise tier)	Invokes a slow-thinking “T1” model for deep reasoning on complex tasks	Enterprise tier exposes logic tracing
Baidu ERNIE 4.0	⚠️ Partial	Multi-step reasoning in specific domains (e.g. medical); general use relies on SFT	Process shown in the control interface
OpenAI o3-mini	⚠️ Partial	RLHF fine-tuned; generates an internal “draft” and outputs only the conclusion (hidden)	User sees only the result; process hidden
Google Gemini 1.5 Pro	⚠️ Partial	Activates a multi-step reasoning chain when enabled (requires a toggle)	User can toggle reasoning display
Alibaba Qwen 1.5	⚠️ Partial	Chat variant adds RLHF-based reasoning enhancement, supports tool-assisted chains of thought	Requires a specific prompt to trigger
OpenAI GPT-4o / o1	❌ No	SFT-dominant, relies on pattern matching rather than autonomous reasoning	Generates results directly
Anthropic Claude 3.1	❌ No	Instruction-tuned for fluent output, lacks dynamic strategy exploration	Generates results directly
Meta LLaMA-3-70B	❌ No	Pure pretraining + SFT, no reward mechanism introduced	Generates results directly
ByteDance Doubao/Skywork	❌ No	SFT-dominant, optimized for response efficiency	Generates results directly

3.3 RLHF (RL for Domains That Can’t Be Verified)

RLHF (reinforcement learning from human feedback) targets qualitative/creative domains (telling a joke, writing a poem).

• How it works: since correctness can’t be checked by rule, a “reward model” is trained instead. Humans first rank the model’s generated answers by quality; the reward model learns that human preference, then substitutes for a human in scoring the LLM’s future outputs.
• Advantage: exploits the gap between how hard it is to generate vs. to judge. Writing 100 great jokes yourself is hard (generation); ranking a set of jokes by quality is easy (discrimination) — so this approach captures accurate human preference more efficiently.
• Disadvantage: can be misleading (it’s a simulation, not ground truth); and it can be gamed — the model may find “adversarial examples,” generating nonsensical output that nonetheless fools the reward model into a high score (e.g. strings of meaningless words). For this reason, RLHF iteration is capped and can’t be over-optimized.

Part Two: Enterprise Model-Path Selection

“Pretraining is the tech arms race, SFT is the commercialization accelerator, RLHF is the user-experience moat.” — Andrej Karpathy, 2024

1. Enterprise LLM Optimization

The essence of an enterprise’s path choice is fighting for control amid a technological revolution. Strategic paths run from light to heavy (and can be combined):

• Prompt (mandatory baseline): prompt engineering, establishes a baseline and enables fast testing.
• RAG (either/or with SFT, or combined): extends the model’s knowledge base through retrieval (like an open-book exam), solving for timeliness and proprietary-knowledge gaps.
• SFT (either/or with RAG, or combined): used to replicate a specific structure, style, or format, or to raise precision in a key scenario.
• RL (optional): for more complex, dynamic scenarios, some enterprises choose to add reinforcement learning.
• Proprietary pretraining (optional): constrained by data security or foundational capability, very few top-tier enterprises choose to pretrain their own model from scratch.

Note: Prompt and RAG can run on closed-source models, while algorithmic tuning generally requires an open-source model.

2. Player Playbooks

2.1 Bloomberg — Proprietary Pretraining

• Path chosen: built on 40 years of proprietary financial data. ① A custom tokenizer that treats financial terms as distinct tokens to reduce ambiguity; ② real-time feedback into the pretraining layer to keep the model’s “worldview” current; ③ data-privacy protection.
• Build cost: assembled a large financial-heavy token corpus (roughly half financial data); took several months to build; overall build cost was comparatively low since the financial data itself required no acquisition spend.
• Capabilities delivered: intelligent financial search (high accuracy), real-time risk alerting (flagged the SVB liquidity risk well ahead of the event), and automated research-report generation (freeing up a meaningful chunk of analyst headcount).
• Key result: financial Q&A accuracy that outperformed GPT-4, alongside growth in platform subscriptions.

2.2 Xueersi/Learning Engine (MathGPT) — Proprietary Pretraining

• Path chosen: a vertical domain (math) with a high bar for solution-logic correctness, step-by-step stability, and engaging explanation — general-purpose models fall short. Built on two decades of proprietary education data.
• Build timeline: announced in mid-2023; a hundred-billion-parameter MathGPT shipped a few months later; the training dataset was open-sourced shortly after, to support chain-of-thought training industry-wide.
• Capabilities delivered: B2B customized teaching support; consumer-facing photo-based problem solving, automated grading, and tutoring Q&A.
• Key result: standout performance on Chinese-language math benchmarks; commercially, it helped its learning-tablet hardware line become a category leader, and contributed to a meaningful year-over-year revenue increase.

2.3 JD Retail — LLM + SFT + RAG

• Path chosen: general-purpose LLMs lack retail-specific expertise, and proprietary pretraining is too costly. Chose a strong open-source base model and applied SFT with e-commerce marketing and consumer-behavior data.
• Execution: evaluated roughly a dozen mainstream LLMs (Yanxi, ChatGLM, Llama, and others) side by side. Layered in RAG to improve timeliness, reduce hallucination, and safeguard data security.

2.4 Dashenlin (pharmacy retail chain) — LLM + RAG

• Path chosen: pharmacy retail lacks deep in-house technical capability, so a lighter open-source-model approach was preferred. To solve for proprietary-knowledge gaps and hallucination, adopted an LLM + RAG framework combined with OCR and other multimodal techniques.
• Build timeline: in 2025, built its first intelligent assistant system on a full-scale DeepSeek R1 deployment plus a cloud provider’s LLM knowledge engine.
• Capabilities delivered: proprietary AI knowledge Q&A, pharmacist-assistance support, and intelligent office tools (one-click generation of marketing plans/training materials).
• Key result: substantially faster drug-information query response time, and efficient handling of a very high volume of annual customer review/feedback data.

Appendix / Glossary

• “No settled standard yet”: in the RL field, how to choose the best approach, prompt distribution, and training process is all still being actively experimented with.
• RAG (Retrieval-Augmented Generation): augmenting generation with retrieved external content.
• SFT for style/format: fine-tuning lets a model learn a specific expressive style — details that are hard to define through prompt engineering alone.
• Bloomberg’s cost breakdown: roughly split across compute rental, upfront data processing, and the core engineering team.
• Dashenlin: a well-known pharmacy retail chain founded in the late 1990s, driving toward a “human-machine collaborative operations model” (e.g. intelligent site selection, replenishment, and promotions).