Neutrality Standard
Porkbarrel.ai only works if people on every side trust it. This file lists the rules that keep it unbiased. Every workstream follows them, and they are published on the site as they are written here.
No system can promise zero bias, so we don't claim it. What we promise is that bias would be hard to introduce, easy to detect and quick to fix, and that anyone can check our work.
1. The scorer never knows who wrote the bill
- The scoring prompt receives only the bill's title, official summary and section text.
- Sponsor names, party, state, committee and any other political label are stripped before scoring, along with member names inside the text.
- Party and sponsor are added back only for display, after scoring is finished.
2. The same test for every bill
- One rubric, one prompt, one model version for all bills at any given time. Every flag records the rubric version and model it was made with.
- Which bills we score is decided by a fixed rule, not by editors. In V1 that's every bill that has been reported, passed a chamber or been scheduled for a vote. No picking bills because of who sponsored them.
- If the rubric changes, all bills in the current Congress are re-scored with the new version.
3. Facts only, in fixed wording
- A flag states a fact: "Section 412 is not described by this bill's title or official summary." The reason is one plain sentence about the text.
- No characterizing words in flags, pages or posts: no "corrupt", "sneaky", "slipped in", "giveaway", "scandal", and no adjectives about lawmakers' motives.
- The exact section text is always shown next to the flag, with a link to the official source.
4. People approve; they don't choose
- Reviewers can approve or reject flags the scorer produced, with a logged reason. They can't add flags of their own.
- Every rejection and approval is logged and auditable.
- Flags on bills with a single-party sponsor list need two reviewers to approve before publishing. As the team grows, those two reviewers should have different political leanings.
5. Measure bias continuously
- Party audit: publish flag rates by sponsor party every month. We don't force the numbers to be equal (that would be its own bias), but any gap gets investigated and explained in public.
- Counterfactual test: the test set includes matched pairs, where the same kind of provision appears in bills from each party. The scorer must treat both the same.
- Second-model check: a sample of flags is re-scored by a second model from a different provider. Disagreements go to a person.
- Framing test: the scorer is checked against sections reworded with left- and right-leaning language. The call must not change.
6. Social follows the same rules
- What gets posted is chosen by a fixed ranking (for example, size of the section or how far it is from the bill's subject), never by which side it embarrasses.
- Tag every sponsor of a bill or none of them.
- A monthly balance report covers posts by sponsor party, and is published.
- The automated posting in V4 starts only after six months of party audits that hold up.
7. Later phases, same standard
- Money and donor data (V2, V3) is shown for every member who voted, in the same format, from official sources (USAspending.gov, FEC, lobbying disclosures).
- We show records next to each other. We never claim that a donation caused a vote.
8. Open to inspection
- Published on the site: the rubric, the exact scoring prompt, model versions, the test set, the monthly party audit and every correction.
- Anyone can challenge a flag. Challenges are answered publicly, and accepted challenges become corrections.
- Funding and ownership are disclosed. No money from parties, campaigns, PACs or advocacy groups.
- An advisory board with members from across the political spectrum reviews the methodology once a year.
Checklist for every change
Before anything ships, ask:
- Could this treat two bills differently because of who sponsored them?
- Does any wording describe motive instead of text?
- Could an outsider reproduce this result from what we publish?