Real Florida Man or Not?

How we verify every headline

A checklist with a green tick: how every headline is checked before it enters the game

A guessing game about real news only works if "real" means something. This page describes, step by step, what happens between a headline appearing on a news site and it showing up in a round - and what we do when we get it wrong.

1. Collection

Real headlines come from two places: the r/FloridaMan community, where members post links to news articles, and news search feeds for phrases like "Florida man" and "Florida woman". We take only link posts to actual articles. Questions, videos, memes, image posts, opinion pieces and listicles ("Top ten Florida Man headlines") are dropped before anything else happens, as are stories from a short list of tabloid domains that do not do their own reporting.

2. The link must still work

For every candidate we fetch the article URL. If the publisher has taken it down, we look for a copy in the Internet Archive's Wayback Machine and link that instead; if there is no copy, the headline is not used. Links that only redirect inside a news aggregator are resolved to the publisher's own page, so the source you see under a headline is the newspaper or station that wrote it, not an intermediary.

3. Safety review

Every headline is then reviewed by a language model against a fixed list of exclusions: death or serious injury, violence against people or animals, sexual content, anything involving minors, hate, self-harm, and partisan politics. The model returns a verdict and a one-line reason, and the reason is kept with the headline so a human can audit the decision. Together with the listicle filter above, this rejects almost half of the collected headlines. Rejected headlines are never shown anywhere on the site. The bar is deliberately conservative: a story that is merely dark is out, even if it is technically harmless.

4. Tagging

The same review assigns each headline one or more topics - alligators, police encounters, driving, food, Walmart and so on - which drive both the headline library and the pairing: a real alligator story is only ever matched with a fake alligator story, so that topic alone never gives the answer away.

5. Writing the fakes

Fake headlines are written by a language model that is shown a sample of real, approved headlines on one topic and asked to write new ones with the same length, tense, hedging and house style. It is told the fakes must be fiction: no real people, no real companies, no place smaller than a city, nothing that describes an event that actually happened. A separate "judge" pass reads each candidate cold and rejects anything that is obviously invented, too clever, unsafe under the same rules as step 3, or too close to a real headline we already have. A large share of the drafts is thrown away at this step.

6. Pairing and difficulty

Each pair keeps a running count of how many players answered it and how many were right. Pairs that turn out to be too easy - more than 90 % correct after a couple of hundred answers - are retired, and a fake that has been shown next to a real headline is never offered to it again. The daily set of five is chosen to prefer pairs that land in the 40-75 % range and to cover different topics; a real headline is not reused in the daily challenge within 60 days.

7. Answers and sources are public

Once a day is over, its five pairs are published in the archive with the correct answers, the share of players who got each pair right and a link to the original article for every real headline. The full set of real headlines, by topic, is in the headline library with the same links. If you think a "real" headline is wrong - the link is dead, the story was retracted, the headline was altered - the contact address on the privacy page reaches us and we will pull it.

What we do not do

Play today's challenge →