In order to properly prepare data for migration, data validation and testing are essential steps to provide improved efficiency during the full implementation.
Before attempting to load any data, be sure to familiarize yourself with the BigCommerce API Best Practices and Rate Limits.
To ensure a smooth execution during migration, prepare a subset of your catalog to perform a partial migration as a test. This should be around 50 to 1000 data records, depending on the size and complexity of your catalog, that cover a broad range of potential product configurations
If your catalog contains any of the following, ensure that your representative sample contains at least one product with each to test the associated BigCommerce APIs.
Select a mix of data for testing, intentionally including records with inconsistencies, missing fields, edge-case values, or legacy formatting, as these samples are most likely to surface real-world migration issues.
If you haven’t yet, configure your complete category tree and brands prior to loading your representative sample to ensure accurate assignments to each.
Understand that BigCommerce handles data deletion and archiving differently. Products and Customers can be deleted, but Orders can only be archived.
As incorrect order migration can directly impact billing, it is imperative to fully understand these distinctions and their migration implications.
Conduct all initial data migration tests in a sandbox environment if possible.
Once you’ve identified a representative sample, begin the partial migration.
If you use Postman as your primary API client, you can use the BigCommerce V3 API library.
Each Catalog API endpoint’s specification is accompanied by one or more example request bodies to clarify format and structure. For more information, review the Catalog API Reference.
Given potential datatype mismatches and platform limitations, data validation is a key step in ensuring a successful migration. After migrating your representative dataset, you should
Get All Products endpoint and comparing it programmatically to your source dataset.By combining manual and automated validation methods, you ensure that your data is complete, accurate, mapped as intended, and behaves as expected.
Once you’ve finished data validation on your sample dataset, user acceptance testing (UAT) is critical. This step ensures your data migration mapping is accurate and functions as expected in both the BigCommerce control panel and storefront.
As part of the UAT process, review the following product fields and storefront behaviors:
Record your findings and any issues in a tracking sheet or ticketing system. Plan for iterative corrections and re-tests as needed before migrating your full catalog.
To improve the overall migration experience, the following optional steps will allow more flexibility in automation.
HTTP 429 responses. Implement rest-and-retry logic to handle rate limiting effectively.HTTP 429 responses, disable any third-party API applications on your store during migration, as the BigCommerce API does not distinguish between individual clients when managing rate limits on a single store.Be aware of account-level and store-level API quotas (such as product, image, or request limits). Exceeding these can cause partial migrations or silent failures.
After migrating your test data, verify storefront display, search, filtering, checkout, and reporting to ensure all features work as expected.
Use scripting languages (such as Python or Node.js) to extract real records from your source system, and programmatically mutate copies to include edge cases (e.g., missing fields, maximum/minimum values, special characters). Consider sampling actual problematic records from production logs or validation failures.
Automate GET requests to retrieve migrated data from BigCommerce, then compare the results field-by-field with your source dataset. Log differences and use schema validation to catch datatype or structure mismatches.
Export both source and destination datasets as structured data (JSON/CSV), normalize formats, and use diff tools or custom scripts to check for field-level matches and missing/extra records.
Implement exponential backoff and retry logic in your scripts for HTTP 429 (rate limit) or 5xx errors, and monitor response headers for rate limit information to throttle requests as needed.
Track IDs of all created test resources during migration, and use API DELETE calls or batch operations to remove them once tests are complete. For production, use clear naming conventions or tagging to identify test data for cleanup.