Detailed Analysis
A developer building InTruth, a real-time political fact-checking tool, has posted an update on the project's progress after a hiatus, noting that many demo videos still circulating online reflect an outdated version of the system. The most significant technical change is a shift in the underlying model powering verdict generation: InTruth now uses Claude Sonnet instead of Claude Haiku to ground its fact-check outputs, pulling roughly 500 words from each cited source to substantiate claims. This move from a smaller, faster model to a more capable mid-tier model suggests a deliberate tradeoff, prioritizing reasoning quality and nuance over raw speed or cost efficiency, which is a common pattern as builders scale side projects from proof-of-concept toward something people rely on for accuracy-sensitive tasks like political claims.
The reported accuracy metrics are notable for a community-built, open-source tool. Benchmarked against established fact-checking organizations including PolitiFact, AP, Maldita, RTVE, and FactCheck.org, InTruth achieved 89% precision when labeling a statement as false, with zero true statements incorrectly flagged as false. That last statistic is particularly important in the fact-checking domain: false negatives (missing a lie) are generally less reputationally damaging than false positives (wrongly branding a true statement as false), since the latter erodes user trust and can itself become a source of misinformation. The tool also covers 79–85% of "check-worthy" claims, meaning it identifies most statements that warrant scrutiny in the first place, a nontrivial NLP challenge distinct from verdict accuracy itself.
Beyond raw accuracy, the addition of source bias tagging (Left, Lean-Center, Center, Right-Center, Right) represents a meaningful transparency feature. Rather than presenting verdicts as authoritative black boxes, InTruth now lets users see the political leanings of the sources it draws on to reach a conclusion, an increasingly demanded feature in a media environment where trust in fact-checking institutions is itself politically contested. This kind of transparency layer addresses a common criticism of AI-driven fact-checkers: that they can launder bias through seemingly neutral technology. Making sourcing visible allows users to calibrate their own trust rather than accepting verdicts on faith.
Practically, the project has reportedly grown to 24,000 users while remaining free, open source, and functional across 16+ languages, hosted publicly on GitHub. This trajectory reflects a broader trend of independent developers using Claude and similar frontier models as infrastructure for public-interest tools rather than commercial products, leveraging API access to build applications addressing misinformation at a scale that would have required a dedicated newsroom or NLP team just a few years ago. The project also illustrates how quickly capability claims in this space become outdated: the author's explicit callout that "old demos" are still circulating underscores how fast underlying models and pipelines evolve, and how public perception of AI tools can lag behind their actual current state, a dynamic increasingly common as builders iterate on Claude-based applications faster than word-of-mouth or media coverage can keep pace.
Read original article →