Scoring Methodology & Commercial Separation Policy | AMPM

How we check prices: the full process from “finding a number” to “daring to publish it”

The three iron rules above are the principles; this section is how we actually do it. Every rule corresponds to a program that is really running on the site — these are not slogans written for show.

1. Source tiers: official pricing page > official documentation > third party

For every price or quota, we first ask “who said this?”:

1. Official pricing page (e.g. replit.com/pricing, console.groq.com/docs/rate-limits): highest tier. If the pricing page and any other place disagree on the same point, the pricing page wins.
2. Official documentation, help centre, terms: second tier. Used to fill in details the pricing page omits (refund window, rate limits, credit expiry rules).
3. Third party (review sites, blogs, price-comparison sites, communities): lowest tier. A third-party figure can never be published on its own. It may only be disclosed when the official page is blocked or silent, and the text must state “third-party source, not confirmed against the official site”.

Search-engine AI summaries do not count as a source. They are someone else’s digest, not the original text.

2. Two-pass: a change only counts once it has been “seen again the next day”

Price data on this site is not kept current by a person watching every day. A price watcher (scripts/watch/watch.py) on the VPS runs a round every three hours, and its method is deliberately simple:

• It does not parse prices; it only detects whether “this page has changed”. After each pricing page is fetched, only the text related to amounts and quotas is kept and a fingerprint is computed. If the fingerprint matches the previous one, the page is skipped.
• On most days it doesn’t even need to download the content. It uses conditional requests (ETag / If-Modified-Since); a 304 from the server means nothing changed, at zero cost.
• A changed fingerprint does not immediately change the data. The first sighting of a change is recorded only as “suspected”; the page is fetched again at least 20 hours later, and only if both fetches differ from the old one and the two new fingerprints agree is it promoted to “confirmed change”. That is the two-pass: a single observation never counts. It guards against A/B-test pages, regional pricing and temporary error pages being mistaken for a price increase.
• All records are append-only. Every fetch result, every confirmed change, and a raw snapshot of the page on the day of the change are written as append-only files, so you can go back later and check “what exactly was seen that day”.

After a change is confirmed, a person (or the system, by rule) still has to apply it to the site’s data. That step has three locks: the two-pass really did pass; the current value on the site equals the “old value” in the record (guarding against someone having edited it manually in between); and when applying, the price file, the evidence file and the history file must all three be written successfully — if any one fails, everything is rolled back. Changes to free quotas are applied automatically and logged as “approved by the system under two-pass” if the new value is single and unambiguous; price changes are always left for a human, because annual-billed monthly-equivalent prices and monthly list prices are far too easy to confuse, and a machine should not decide that for people.

3. Verification stamps: stamped only on a successful fetch; otherwise we say so

A tool page shows two dates with different meanings:

• “Price last confirmed”: the date a person manually checked each item, stored in the data file’s last_verified field. This stamp is only applied after a human check.
• “Latest automatic check”: the last time the price watcher successfully read that tool’s pricing page, pulled from the watcher’s own state file — not entered by a person. If a tool has never been fetched successfully, this field is blank and the page honestly shows “no record yet” instead of filling in a fake date.

If the manual check date is more than 14 days old, the page label automatically changes to “Last manual check”, reminding you that no one has looked at this figure for a while.

4. Handling blocks: 403 does not mean “does not exist”

Many official pages return 403 or 429 to automated connections, or are rendered entirely by JavaScript so that a static fetch only gets an empty shell. Our rules:

1. 403 / 429 means “blocked”, not “not found”, and certainly not “does not exist”. Only a 404 may be used as one piece of evidence that a page does not exist.
2. Sources that plain HTTP cannot fetch are handed to the second leg: a real browser (Playwright) opens the page, waits for JavaScript to finish rendering, then reads it — likewise zero AI, zero cost. The two legs keep separate state, because the “static text” and “rendered text” fingerprints of the same page are necessarily different, and mixing them would make them cancel each other out.
3. A source that fails three times in a row backs off for 24 hours before retrying — no brute-force retries, no provoking an official block.
4. If both legs fail, the tool page is marked “official page could not be read automatically”, and the article states “official page blocked; the following is from third-party sources”. We never pass off third-party figures as official just because a fetch failed.

5. How we fix mistakes: fix the root cause, keep a record, sync every reference

We do make mistakes, and there are real cases on the site. The rules for fixing them:

• First trace it to “which sentence, which field” was wrong, and fix the root cause rather than the symptom. For example, we once wrote an “annual-billed monthly equivalent” as a “monthly list price”; the fix was not to change that one number, but to re-scan every field of the same type.
• A fix must be synced to every place that references it: tool page, comparison page, guide articles, conclusion sentences. Changing the data without changing the references is the same as not changing it.
• Every change leaves a traceable record: price changes have a public price change log; every edit to a data file carries a time and a basis; evidence files are append-only.
• Corrections are marked directly in the article. Where an official site contradicts itself (e.g. two different numbers on the same page), we list both, with dates, rather than picking one on the vendor’s behalf.

6. How you can verify us

Every article ends with “Sources” and “Last verified”; tool pages show the verification date and source links; the Scoring Charter explains how scores are computed; the Disclosure page explains our relationships with vendors. If you see any figure that differs from the official site, please tell us via the contact page; we will handle it under the process in section 5 above and leave a record.

This section was added on 2026-09-04 and corresponds to the price watcher, change-application and pre-deployment check programs actually running on the site; if the program logic changes, this page is updated in step.

How AMPM100 seats are earned and swapped: the eight gates and the replacement rules

No seat on the list is hand-picked. A program recomputes every seat every 3 hours under the rules below (data/founding_100_v1.json).

The eight gates (all must pass to be a “full member”; passing only the score threshold but not all gates makes a tool “provisional”)

1. Evidence coverage ≥ 80%: the proportion of the five dimensions backed by verifiable evidence; below 80% no seat is granted.
2. Zero unresolved major data conflicts: if sources disagree on a figure and it has not been cleared up, the tool cannot be listed.
3. At least four of five dimensions valid: tools with too little data are not scored.
4. Primary category confirmed: whether it is a chatbot or an image generator must be settled first.
5. Product identity and continuity confirmed: we confirm it is still alive, has not been renamed, and has not shut down.
6. Pricing / free-plan evidence valid: for priced tools, the price and free quota must have an official source.
7. Key facts double-verified: key figures such as price and quota must be found consistent in two independent checks.
8. Seat status legitimate: not on probation and not in the museum.

Replacement, probation, museum

• Passes all eight gates and the category has an open seat → listed.
• When a category is full, a challenger must beat the lowest-scoring full member in that category by 3 points for 7 consecutive days to take the seat.
• A full member that fails a gate or is replaced → 21 days of probation (data remains available and it can re-qualify) → still failing at the end → moved to the museum (only the reason for removal is displayed; no further checks).
• Newly listed tools get a 14-day protection period, so they are not replaced right after joining.

What a challenger is

Challenger = the “under review” list: tools that meet the score threshold but have not yet passed all eight gates. The challenger page shows only how many gates remain and which ones are blocking; it computes no separate score and has no separate ranking (data/challengers_v1.json is derived entirely from the rules above).

Cross-category fill seats

Vacancies in fixed categories are filled by candidates that have already passed the eight gates; fill seats are labelled “verified” or “provisional, under review”, and a tool under review is never labelled as verified.

Every rule in this section corresponds to a program actually running on the site; if a rule changes, this page changes with it. Last updated: 2026-09-12.