# When a Score Is Not a Fact | Techne Path A plain-English, interactive guide to probabilistic text classification using fictional data and worked calculations. *2026-10-26 · Scott Isaacson* [Skip to content](#main) [![Techne Path](https://t3.tigrisbucket.io/morning-wood-5868/logos/techneplan/techne-path-logo.png)From systems to signals.](#top) [Method](#method)[Worked example](#demo)[Limits](#limits) A field guide to probabilistic classification # When a score is not a fact. A likelihood can summarize patterns in training data. It cannot tell you who a person is. This interactive guide shows the math, the assumptions, and the places a tidy score can go wrong. **The example uses fictional posts about gardening and baseball.** It does not infer political views, age, gender, or any other personal trait. Nothing leaves your browser. Model notes / 001Synthetic data ## A score is a summary. Its meaning depends on the data behind it. Gardening.78 Baseball.22 Bayes in one line**posterior ∝ prior × likelihood** The boundary Building a system to infer a stranger's politics or identity from social activity can expose private traits and produce false accusations. **This page keeps the statistics and removes the profiling.** 01 / METHOD ## From words to a probability A multinomial Naive Bayes classifier counts how often words appear in labeled examples. It then compares how likely a new passage is under each topic. 01 ### Label examples Start with fictional passages already labeled gardening or baseball. Labels come with the teaching data. The model does not invent them. 02 ### Count tokens Split text into lowercase words. Count each word within each topic, then add smoothing so an unseen word does not zero out the score. 03 ### Compare evidence Multiply the prior by each token likelihood. In practice, add log probabilities to avoid tiny floating point values. 04 ### Normalize Rescale the topic scores so they sum to one. The result is a model score, not a statement about the author's identity. P(topic | words) ∝ P(topic) × ∏ P(word | topic) The prior is the topic's share before reading the passage. The likelihood asks how expected each word is within that topic. The code uses `log(prior) + Σ log(likelihood)`, then applies softmax. 02 / WORKED EXAMPLE ## Try the math on fictional text Edit the passage or choose a sample. The base rate slider changes the starting assumption before any words are read. Synthetic post history Mostly gardening Mostly baseball Mixed and ambiguous The tomato plants need soil and water. My garden is finally growing, and the compost bin is full. Use fictional text only. Do not paste another person's posts or personal information. Starting share for gardening 50% Calculate the model score Estimated gardening share 96% **Gardening favored**The token pattern leans toward gardening. Gardening 96% Baseball 4% ### Which recognized tokens moved the log odds? Bars to the right favor gardening. Bars to the left favor baseball. Repeated words add repeated weight. ``` for each topic: score = log(topic_prior) for each recognized token: count = token_count_in_topic + 1 total = all_tokens_in_topic + vocabulary_size score += log(count / total) probability = softmax(scores) ``` ### What the demo assumes Each recognized token is treated as conditionally independent given the topic. That assumption is convenient, not realistic. Phrases, sarcasm, quoted text, and context can break it. The teaching table below is deliberately tiny. It makes the arithmetic inspectable. It is not a trained language model or a benchmark of classification accuracy. Calculation updates locally in this page. 03 / EVIDENCE ## A percentage needs a receipt A useful model report separates the score from its test results. These charts use invented evaluation data to show what that report should contain. ### Calibration If many cases receive a score near 70%, about 70% should have the labeled outcome. The gap between those values is the calibration error. Synthetic calibration chartObserved outcomes fall below model scores at higher confidence levels. 0%33%67%100%0%50%100%Mean model scoreObserved 75% observedat a 100% bin Observed frequencyPerfect calibration ### Errors by group One overall accuracy number can hide uneven mistakes. Report false positive rates for each relevant group, with sample sizes and intervals. Synthetic false positive ratesFour fictional test groups have error rates of 6, 9, 14 and 19 percent. 05101520% 6%9%14%19% Group AGroup BGroup CGroup Dn=500n=500n=500n=500 Invented values for teaching. They are not measurements of Facebook users or any real population. 04 / LIMITS ## Where the tidy story breaks The formula is easy. Getting evidence that deserves trust is much harder. 01 ### Labels can be wrong A model learns the judgments in its training set. Ambiguous labels and annotator disagreement become part of the score. 02 ### Words borrow context Quoting a post is not endorsing it. Jokes and local slang can flip meaning. A bag of words throws that context away. 03 ### Behavior changes Language moves. Interests change. A model tested on last year's posts may fail on today's without any code changing. 04 ### Base rates matter Rare outcomes need strong evidence. Move the prior in the demo and watch the same words produce a different result. 05 ### Confidence can be misplaced Softmax forces the scores to sum to one. That does not make the model right, especially when the input differs from its training data. ### Before anyone acts on a model score * Define the decision and who could be harmed. * Document consent, provenance, and label quality. * Test calibration and errors on held out data. * Check performance across relevant groups. * Set an abstain rule for weak or unfamiliar evidence. * Give people a way to review and correct mistakes. ## Use the math. Keep the person's dignity. Classification can help sort public documentation, route support tickets, or label consented research data. It should not turn fragments of a person's life into an identity claim. The honest output includes uncertainty, provenance, and a reason to stop. ![Techne Path](https://t3.tigrisbucket.io/morning-wood-5868/logos/techneplan/techne-path-logo-reverse.png) Interactive teaching material. All examples and evaluation values are fictional. No Facebook data is accessed, stored, or transmitted.