How do you build a price comparison tool with live product data from multiple stores?

Identical branded goods (electronics, beauty, toys, appliances, books) carry manufacturer identifiers, so the same item can be matched across stores and compared on price. Fashion and home goods are often store-exclusive or sold under different names, so a tool in those categories compares similar products rather than identical ones. Choose before you build: the matcher, the data you need and the interface all follow from it.

Either way, each row in your comparison needs the same fields:

• the store and the product URL a shopper will land on;
• the exact variant (size, color, capacity) the price is for;
• the current price, the original price if it is on sale, and the currency;
• stock for that variant;
• shipping and tax terms when known;
• when the price was last checked.

There are four common ways to discover where a product is sold, and most tools combine two:

• Affiliate network feeds (CJ, Rakuten Advertising, Awin, Impact). Participating merchants publish product feeds with prices and identifiers, and you earn commission on click-outs. This is how many comparison sites start. Feeds refresh about daily and cover only merchants in the network.
• Store sitemaps and product feeds. Sitemaps list product URLs but not prices, so you still fetch each page.
• Each store's own site search, queried by product name or model number. Accurate for that store, but it is one integration per store.
• A cross-store product search API, which returns candidate offers from many stores for one query.

Google's Merchant API (formerly the Content API for Shopping) manages a merchant's own listings; it doesn't return other stores' prices.

A wrong match is worse than a missing one: it tells someone a different item is cheaper. Match with a ladder, strongest key first:

• GTIN (UPC or EAN), left-padded to 14 digits with the check digit validated.
• Brand + manufacturer part number (MPN), with punctuation stripped. Model numbers in titles (WH-1000XM5, KSM150PS) are a good source when the MPN field is empty.
• Brand + title tokens + key attributes (capacity, color, pack size), scored, with a threshold tuned on labeled pairs. Below the threshold, show the offer as "similar", not "same".

Match at the variant level. Stores often put several sizes or capacities on one page, and those are different prices. See how to normalize product data across retailers for the full matching and variant model.

"Live" doesn't mean fetching every store on every page view. That's slow, and it gets you rate-limited. Use tiers:

• Background refresh of every tracked offer, daily as a baseline.
• Hot refresh of offers people are viewing or tracking, every few minutes to hours.
• Just-in-time check of the winning offer before a click-out or a price-drop alert. If it changed, update the page before sending anyone.

Write every observation to a price history table with its timestamp. History powers price-drop alerts and "lowest in 30 days" claims, and it lets you spot extraction errors: a price that drops 90% overnight is usually a parsing bug, not a deal. Respect each store's robots rules and rate limits, and cache aggressively.

• Discovery: feeds, search or sitemaps produce candidate product URLs.
• Fetch and extract: turn each URL into a normalized offer (price, currency, variant, stock).
• Match: attach each offer to a canonical product, or mark it unmatched for review.
• Store: canonical products, current offers, and price history.
• Serve: for a product, rank in-stock offers for the same variant by price.
• Refresh: the tiered loop above, plus alerts when extraction starts failing for a store.

create table products (
  id          bigint primary key,
  gtin        char(14) unique,      -- null when no identifier is known
  brand       text not null,
  mpn         text,
  title       text not null
);

create table offers (
  product_id  bigint references products(id),
  store       text not null,
  url         text not null,
  variant     jsonb not null,       -- {"size": "M", "color": "black"}
  price       numeric(12, 2) not null,
  original    numeric(12, 2),
  currency    char(3) not null,
  in_stock    boolean,              -- null = unknown
  checked_at  timestamptz not null,
  primary key (product_id, store, variant)
);

create table price_history (
  product_id  bigint,
  store       text,
  variant     jsonb,
  price       numeric(12, 2),
  currency    char(3),
  observed_at timestamptz
);

The hard part of a price comparison tool is the data: discovery, extraction, matching and freshness across many stores. These are the main ways to get it.

How to choose

• Identical branded goods from large retailers, earning commission: affiliate network feeds, with an extraction API to fill gaps, or Channel3 for search with commission-earning links.
• A fixed list of stores on your own schedule: Zyte or Firecrawl to fetch and extract; you run discovery, matching and refresh.
• Configurable collection from specific marketplaces: Apify Actors.
• Price history and analytics across merchants by UPC: Datafiniti.
• Searching many stores from a plain-language query, with current prices and per-variant stock in one schema: StoreRouter.

Prices and capabilities are from each vendor's public docs and pricing pages as of November 6, 2026. Side-by-side detail: StoreRouter comparisons.

StoreRouter covers discovery and normalized offers. search finds products across covered stores from a plain-language query, with optional price, brand and catalog filters. parse returns each offer in one schema: current and original price, currency, per-variant stock, and GTIN where the store publishes it. Your code keeps the matching rules, ranking, price history and interface.