Talkie

who they are

A 13-billion-parameter model trained only on English text published before 1931, then taught to converse with the etiquette manuals, letter-writing guides and encyclopedias of its time. It talks like someone with no knowledge of the modern world, though its makers document leaks: it knows of Roosevelt and some of the Second World War.

in their own words

UserWho was president of the United States in 1936, and what were the most significant pieces of legislation he signed?

talkie-1930Franklin D. Roosevelt was president of the United States from 1933 to 1937. The most important legislation signed by him was the National Recovery Act of 1933; the Agricultural Adjustment Act of 1935; and the Emergency Banking Act of 1935 (amended in 1936).

Introducing talkie, figure 5: a leak past 1930

how they were made

1. 13B trained from scratch on 260B tokens of pre-1931 English: books, newspapers, journals, patents, case law
2. fine-tuned on instruction pairs drawn from period etiquette manuals, letter-writing guides, cookbooks and encyclopedias
3. online preference optimisation with Claude Sonnet 4.6 as judge, then SFT on synthetic chats with Claude Opus 4.6

variants

• talkie-1930-13b-base · the base model
• talkie-1930-13b-it · the conversation partner
• talkie-web-13b-base · its modern twin, trained on the web
• TALKIE (talkie-box) · N8Programs' 1930s thinking machine, speaking in a play transcript; SFT and KTO on the base

sources: intro · code ● · plate: Stilleven met boeken — Hercules Segers, ca. 1618-1622