Chime: methods and measurements

Luke Steuber · October 8, 2026

Return to the write-up and shareable cards.

Keyboard pilot: unequal prior learning

This was an exploratory run, not a controlled accuracy benchmark. The same 36 prediction targets were used on both phones, but earlier test attempts gave LocalType on the 9a and Gboard on the 10 more exposure to them. The table records what appeared; it does not isolate hardware or model quality.

The latest run completed 432/432 cases: 108 per keyboard per phone, on API 37. LocalType 0.5.9 (29) ran with Chime active; Gboard was 18.4.1.985164140-beta-arm64-v8a. The 36 prediction targets produced 262 prefix observations per arm. Repeating them on two phones does not create independent language samples.

Existing preferences and personalization were retained. Pixel 9a’s Chime setting was temporarily enabled and restored. Earlier diagnostic attempts exposed 9a LocalType and 10 Gboard to more of the prediction targets than their opposing arms. This unequal exposure prevents a causal model ranking. Ordinary fields could learn the invented text. No production dictionary or typing log was exported.

Why the phones differed

The app versions and Android API matched. Personalization did not, and Gboard’s internal model identity was not measured. The same fixed model with identical context and state should give consistent rankings across phones; speed alone is not an explanation for these settled suggestion counts.

Rechecking the transcripts reproduced all four next-word counts. LocalType’s five extra hits on the 9a were later, tomorrow, water, keys and afternoon, all in the third slot. LocalType can reserve that slot for a learned follower. This pattern is consistent with personalization, but the saved personal tables were not exported, so it is not a controlled attribution. Gboard’s five extra Pixel 10 hits were water, keys, file, snacks and sunlight; its internal cause remains unknown.

On Pixel 10, 10/36 versus 18/36 means LocalType missed eight more target words: 27.8% versus 50.0% top-three target coverage. That result should not be presented as a prediction win. A miss means the chosen target was absent, not necessarily that all three offered words were implausible. Future comparison needs fresh evaluation text, matched settings, equivalent prior exposure and a declared learning policy; previously used targets remain diagnostic examples.

Useful completions appeared for 35/36 targets on each LocalType phone, 34/36 on Gboard 9a and 35/36 on Gboard 10. Potential net characters saved across the 36 targets were 135/119 on 9a and 130/142 on 10 (LocalType/Gboard). These ceilings subtract one tap and assume immediate acceptance. All 12 actual completion taps per arm preserved subsequent typing and caret position.

LocalType repaired 19/24 isolated typos; Gboard repaired 21/24. Neither made another wrong replacement in that isolated set. All 48 clean controls remained correct. In separate phrase cases, LocalType changed xkcd to did on both phones, while Gboard kept it. Gboard produced Luke’s keyboard; LocalType left lukes keyboard unchanged.

LocalType offered six of eight requested selected-word repairs on each phone and inserted all offered repairs correctly. Gboard showed its selected-text toolbar, with no target word in the tested strip; Writing Tools and other manual repair interfaces were not scored. LocalType’s later selected-word fixes are excluded from this 0.5.9 comparison.

LocalType accepted hello from he in four punctuation probes: period and domain continuation worked, while comma plus an explicit space doubled the space and newline retained a preceding space. Gboard did not offer hello at that fixed prefix, so its fallback typing cannot establish post-suggestion spacing behavior.

Protected and ordinary transport preserved all 960 injected characters at requested 120/60/30 ms intervals, including overlapping contacts. Injection and editor-command timing are not physical input latency. Model-only speed, battery use, TalkBack speech, real-app acceptance and everyday savings are separate. Original IMEs, enabled lists and wake settings were restored.

Architecture and context

One CIFG-LSTM layer has 670 hidden units, a 96-dimensional projection and tied 96-dimensional input/output embeddings: 2,025,114 parameters. The vocabulary has 16,384 tokens: 16,054 words, 293 emoji, 31 punctuation marks and six special tokens. Context is the current sentence, up to 63 tokens after a start token, with a 32-token window hop. Unknown words map to an unknown token.

The architecture follows Hard et al. (2018). The 2023 Gboard paper describes larger recurrent models pretrained on public text, then trained with federated learning and differential privacy. Published sentence and device counts cannot be divided by Chime’s word count to obtain a meaningful data ratio. Current proprietary next-word models have not been evaluated on Chime’s development text.

Training data and recipe

The five licensed English sources are NUS SMS, human-written user/customer turns from Taskmaster-1 and Taskmaster-3, English OpenAssistant 2 prompts, and synthetic SODA dialogue generated with GPT-3.5. SODA supplies most of the text. No private messages, LocalType logs or personalized weights were used. Filtering removed phone numbers, URLs, email addresses and credential-like strings; filtering is not proof of anonymity. Revisions, licenses and attribution are in the source credits.

NUS SMS, Taskmaster-1 and OpenAssistant lines appear eight times per pass; Taskmaster-3 and SODA appear once. That is the ×8 label: repeated exposure, not extra parameters or unique data. Human-written lines are 12.868% of the weighted pass. The training-loop cost estimate is $5.10 at $2.50/hour, excluding setup and uploads; it is not a bill.

Prediction measurements

Potential keystroke savings counts characters skipped if the intended word is tapped immediately when it first appears among three suggestions. It is a simulated ceiling. With no personal learning, Chime scored 41.44% on 463 NUS SMS development lines and 59.50% on 567 Taskmaster-1 requests. These lines were not training text, but were used for checkpoint selection; no held-out test result is claimed. SMS sender overlap and dialogue clustering limit independence. These scores cannot be compared with another keyboard’s scores on different data or with correction counts.

Pixel runtime

Each run used 600 ordinary contexts over three passes plus 30 long stress sentences; warm timings pool passes two and three. Both phones ran API 37. Model loading took 139.090/131.655 ms and first calls 18.941/22.952 ms on the 9a/10. Thermal status stayed at 0 on the 9a and rose from 0 to 1 on the 10. These are model-only timings, not a controlled phone comparison, finger-to-editor latency or battery measurements.

Export, verification and reuse

Int8 export raised perplexity from 12.985256 to 12.998372 (+0.101%) on 5,000 development lines. Kotlin matched all 120 exported reference contexts and top-ten rankings with maximum logit error 1.27 × 10⁻⁵. NumPy and browser JavaScript matched the same references below 5 × 10⁻⁸. Numerical agreement establishes implementation consistency, not prediction quality.

The int8 binary is 2,221,879 bytes. SHA-256: 3255dbeaf6e7fc0922afdca42ef751a22e3ad28903ab4acc9eb0022b9a704318. The float32 selected checkpoint hash is 5f5b684b91be428f567e6e2bf27226fcfec4f95ab07c57f61347b14ba5ff00d7.

The model package contains the weights, vocabulary, a standalone NumPy reference and licenses. It contains no corpus or training pipeline. Weights and vocabulary are CC BY-SA 4.0; inference code is Apache-2.0. These licenses do not cover the Android app.