Product Enrichment

The problem

A distributor feed gives you a part number, a brand, a cost and a line of text that reads HP 8GB DDR4-3200 SODIMM (BULK). That is not something a client will buy from. So somebody opens the manufacturer’s site in another tab, copies a paragraph, saves three JPEGs to a shared drive, and pastes it all into a quote — for one product, out of the quarter of a million in the feed.

The ones that do get written up go stale. The photo is hotlinked to a manufacturer URL that eventually 404s, so a quote sent six months ago now renders with a broken image. And nobody can tell you which products have real content and which are still a part number in a spreadsheet.

What Solidlio does about it

Solidlio holds a local copy of Icecat’s product index — around 26 million entries — and matches your catalog against it on part number, alternate part number, GTIN and brand. Matched products are enriched from Icecat’s Level-4 API: the marketing description, the full specification set grouped the way the manufacturer groups it, every gallery image, and every manual, datasheet and safety sheet. The images and PDFs are then copied out of Icecat and into Solidlio’s own object storage, so a quote sent today still renders in two years. Products Icecat will not release under your subscription are recorded rather than dropped, and become a prioritised CSV you can send Icecat to have them opened up.

Capabilities

How it works

1. Sync the reference data — Suppliers, categories, features and measures come down as Icecat’s master lists. Everything downstream references them.

2. Build the index — The full product file streams to disk, then through gunzip and a SAX parser into the database. Progress reports 0–49% for the download and 50–100% for the parse.

3. Match — Every catalog product without an Icecat link is tried against five strategies in confidence order. A match records the Icecat ID, the data quality — ICECAT for editorially standardised, SUPPLIER for as-supplied — and marks the product pending enrichment.

4. Enrich — Matched products are fetched from Icecat’s Level-4 API at five requests per second. Each outcome is recorded as enriched, forbidden, not found or errored, so a re-run never repeats work that already succeeded.

5. Bring the media home — Images and PDFs download ten products at a time, deduplicate on content hash, and land in private storage. The product’s URLs switch from Icecat’s host to Solidlio’s the moment the bytes arrive.

6. Work the refusals — Products outside the subscription are already in the access queue. Set a priority, export the CSV, send it to Icecat, re-enrich when access lands.

Operations and audit

• Every enrichment call is logged. Part number, brand, GTIN, Icecat ID, HTTP status, error message, source and timestamp — granted and refused alike. Refusals become the subscription worklist; a 404 is recorded distinctly from a 403, because “does not exist” and “not in your plan” are different problems.
• Raw responses are retained. The complete Icecat payload is stored per product per language alongside content-availability flags, so what was received can be inspected after the fact.
• Every scheduled run is recorded. The daily index sync and weekly reference sync each write a run-history row — running, then completed, failed or timed out — and a run that fails every retry notifies platform administrators.
• Media is never public at the storage layer. The container is private; the anonymous serve route is the only way in, and it will only release keys carrying a catalog path segment.
• Outbound fetches are constrained. Media downloads reject non-HTTP schemes and every private, loopback, link-local and carrier-grade-NAT address range.

Editions

Product enrichment carries no plan gate. The catalog is global and platform-operated; every tenant on every plan reads the same enriched data. Access is governed by role:

Integrations

Icecat — Full Icecat (level 4). Two channels: the OCI file repositories at data.icecat.biz for the bulk index and reference XML, and the live per-product JSON API at live.icecat.biz for on-demand detail.