UNCLESAMOC2.3 / OC3 / OC4
05
uncle_parserCompetitor Parser
Automatically import products from any website into your online store. Once configured, the module automatically parses on a schedule, downloads photos, creates categories, and updates prices.
OVERVIEW
Competitor Parser allows you to automatically import products and categories from any third-party website directly into your online store. You set it up once, and then everything runs on schedule without any intervention.
SCREENSHOTS
FEATURES
▸ What it parses
Main fields — name, description (HTML preserved), model, SKU, price, promotional price, stock quantity.
Images — main photo and full gallery; images inside HTML descriptions are also downloaded and replaced with local links; compression and resizing on download; WebP support with auto-conversion to JPG.
Attributes — parses name/value pairs from feature tables; supports grouping by sections ("Memory," "Camera," "Power") as separate attribute groups.
Options — variants like Size, Color, and any others; multiple option slots for sites with different layouts; each value can be assigned a price and quantity.
Categories — parses breadcrumbs; two modes: place all products in one category or automatically recreate the hierarchy. Categories are created in OpenCart automatically.
Images — main photo and full gallery; images inside HTML descriptions are also downloaded and replaced with local links; compression and resizing on download; WebP support with auto-conversion to JPG.
Attributes — parses name/value pairs from feature tables; supports grouping by sections ("Memory," "Camera," "Power") as separate attribute groups.
Options — variants like Size, Color, and any others; multiple option slots for sites with different layouts; each value can be assigned a price and quantity.
Categories — parses breadcrumbs; two modes: place all products in one category or automatically recreate the hierarchy. Categories are created in OpenCart automatically.
▸ Data sources
Link — catalog page with pagination; the parser navigates through pages automatically.
Sitemap — XML sitemap, takes the list of product URLs directly.
Feed / RSS / Atom — any XML feed with products.
Pagination is supported in two ways: CSS/XPath selector of the "Next Page" button or URL pattern (?page={n}).
Sitemap — XML sitemap, takes the list of product URLs directly.
Feed / RSS / Atom — any XML feed with products.
Pagination is supported in two ways: CSS/XPath selector of the "Next Page" button or URL pattern (?page={n}).
▸ Import modes
New only — existing products are not affected
Update only — updates existing products, does not create new ones
New + update — creates new products and updates existing ones
Full synchronization — additionally removes products from the store that are no longer in the source
Update only — updates existing products, does not create new ones
New + update — creates new products and updates existing ones
Full synchronization — additionally removes products from the store that are no longer in the source
▸ Background processing and scheduling
Parsing is performed via a cron request — no execution time limits, no timeouts. Each profile has its own cron link; simply add it to the server scheduler. The schedule is configured directly in the profile (cron expression).
There is a 0.5-second pause between requests to the source site, with browser headers and cookie support to bypass basic bot protection.
There is a 0.5-second pause between requests to the source site, with browser headers and cookie support to bypass basic bot protection.
▸ Registry and testing
Smart matching — the module maintains a registry of all parsed products and categories. When rerun, it does not create duplicates: it finds an existing product by URL, then by SKU, then by name. If the product URL changes, the registry is updated automatically.
Risk-free testing:
Category Page Test — shows how many products were found and example links
Product Test — parses one page and shows all found fields; nothing is saved to the database
Diagnostics — shows "found" / "not found" / "empty" / "selector not specified" for each field
Risk-free testing:
Category Page Test — shows how many products were found and example links
Product Test — parses one page and shows all found fields; nothing is saved to the database
Diagnostics — shows "found" / "not found" / "empty" / "selector not specified" for each field
▸ Service tools
SEO URL management — scans store URLs, finds products and categories without slugs, fills in missing ones, corrects duplicates.
Image cleanup — finds files no longer used in any product, shows how much space will be freed, and deletes with one click.
Registry cleanup — removes registry entries for products or categories already deleted through the admin panel.
Disable incomplete products — finds and disables products without photos or with a price of 0.
Image cleanup — finds files no longer used in any product, shows how much space will be freed, and deletes with one click.
Registry cleanup — removes registry entries for products or categories already deleted through the admin panel.
Disable incomplete products — finds and disables products without photos or with a price of 0.
▸ Interface and requirements
The entire module is managed from a single page in the OpenCart admin panel. Dark mode support. Real-time log — see what's happening right now. Multiple profiles — each with its own settings, schedule, and source. Category and product registry with pagination and quick editing links.
Requirements: cURL extension, server Cron, PHP extensions: curl, gd, dom.
Requirements: cURL extension, server Cron, PHP extensions: curl, gd, dom.
CHANGELOG
New features:
Fixed:
- OpenCart 4 support — the module now works on the current OpenCart 4 as well, with the same parsing logic as the OpenCart 2.3/3.x and ocStore version
- Quick category crawl — a separate mode: the module first builds the entire donor category tree without parsing products, then automatically queues product downloads for each leaf category
- Targeted category re-parse — re-parse a single category from the already-built tree without restarting the whole profile
- Bulk deletion and catalog maintenance — batch-delete products and categories for an entire profile or a specific category; a separate "Empty categories" tool; repeated clicks on the delete button no longer create duplicate tasks
- "Task Queue" tab — all current background tasks across all profiles in one place: status, progress, creation and start time; stop any specific task and clear finished-task history
- Profile export and import — profile settings are exported to a file and imported on another store or server
- Multi-language out of the box — the same parsed content is automatically saved to all required store languages at once, configurable per profile
- Flexible brand handling — the brand is detected automatically from donor data or falls back to the profile's default; you can prevent the module from touching a manually assigned brand when a product is re-parsed
Fixed:
- Catalog crawling now correctly handles cases where the product list or the subcategory list itself spans multiple pages, and no longer loops endlessly if the donor site serves "infinite" pagination
- Links and technical attributes (id) copied as-is from the donor site are now stripped from the product description and additional description blocks
- Batch processing is now time-limited — protection against hosting timeouts on large catalogs
Security:
Performance:
Fixed:
- Enhanced protection against changing the purpose of the request when collecting data from the donor site (checking the address before the request and on each redirect, blocking access to internal/local addresses).
- Added authorization for automatically running scheduled tasks (cron) - starting by a third-party request is no longer possible.
- Checking the donor site's SSL certificate is enabled by default; for individual profiles it can be disabled manually if necessary.
- Limited access to sections and actions of the module for users with insufficient rights.
- Eliminated the possibility of executing third-party code by viewing the product description.
Performance:
- Accelerated loading of product images during parsing due to parallel loading instead of sequential.
Fixed:
- Fixed an error due to which the processing of a sitemap containing links to other sitemaps (sitemap index) could be interrupted with an error.
- Fixed and translated module interface texts in all supported languages.
Security:
Fixed:
New features:
- cron endpoint protected with token - anonymous parsing is blocked
- SSRF protection in the image uploader - blocking private IP ranges and dangerous protocols
- access rights check (modify) for all mutating actions of the admin controller
Fixed:
- duplication of products created outside the parsing module
New features:
- AJAX pagination of the registry; Categories and Products tables are loaded via AJAX, 50 records at a time, lazy-loaded when the tab is first opened
- SEO URL management
- SEO URL management (service tab), scanning, filling in missing ones, fixing duplicates; Three suffix modes: number / language / duplicate
- Image cleaning (service tab), scanning unused files by folder, deleting and protecting images from descriptions
- Registry cleaning (service tab), deleting entries for which the OC object has already been deleted
New features:
- Full synchronization mode, deletes products from the OC that are no longer in the source
- Product options, parsing options (size, color, etc.), multiple option slots, price and quantity per option
- Grouped attributes, support for feature sections ("Memory," "Camera," etc.), customizable default group name
- Template pagination, field pagination_url_template with placeholder {n}, auto-detection of the current page number; support for full URL, path, and query parameters.
- Strip words, strip_words removes specified phrases from the title, model, SKU, and description; for HTML, only text nodes are crawled.
- Breadcrumb fallback; if a product has no breadcrumbs, the source page's breadcrumbs are used (requested once before batch processing).
- Download images from the description. ![]()
tags in the HTML description are downloaded and saved locally; the src in the description is replaced with a local path.