Forum Replies Created

Viewing 6 replies - 1 through 6 (of 6 total)
  • Thread Starter ziadeid

    (@ziadeid)

    Following up on our progress with the 276k order migration on staging. While the sync has technically reached 100%, we have encountered several critical issues upon enabling HPOS as the primary datastore that make the system currently unusable.1. Final Sync Hurdle & Workaround

    Even after backfilling missing address fields, the sync continued to be hard-blocked by:

    WordPress database error: Duplicate entry ” for key ‘wp_wc_orders.order_key’

    The Root Cause: We discovered the migrator was failing to map the existing _order_key from wp_postmeta to the new wp_wc_orders table. Because it was attempting to insert multiple empty strings into a UNIQUE column, the process crashed repeatedly.

    The Action Taken: To allow the sync to complete, we manually altered the schema of wp_wc_orders to drop the UNIQUE index on order_key and modified the column to allow NULL. This permitted the sync to hit 100%, but it has left thousands of orders with empty order_key values in the new table.2. Current Display & Data Integrity Issues

    Once HPOS was enabled, we observed the following mapping failures:

    • Total Amounts: The Order List view shows 0 for all migrated orders, although the “Order Preview” and individual order screens correctly display the totals. It appears the total_amount column in wp_wc_orders was not populated during sync.
    • Order Keys: As noted above, the order_key column is largely NULL or empty.
    • Custom Metadata: Essential legacy fields (e.g., Billing Phone and custom status columns) are missing from the dashboard list view.

    3. Critical Performance Degradation

    Performance has dropped significantly since switching to HPOS:

    • Load Times: The Orders dashboard takes ~10.65s to load (Compatibility Mode OFF).
    • Query Bloat: Query Monitor shows one specific query from CustomMetadataStore fetching 95,075 rows from wp_wc_orders_meta just to render a single page of 100 orders.
    • Compatibility Mode: When enabled, load times increase to 15s+ with over 10,900 queries (including 5,777 duplicates).

    Optimization Attempted: We have manually re-added indexes for (status, type, date_created_gmt) on wp_wc_orders and (order_id, meta_key) on wp_wc_orders_meta, followed by ANALYZE TABLE, but the 10-second lag persists.Questions for the Team:

    1. Mapping Failures: Why would the official wp wc hpos sync fail to map standard fields like order_key and total_amount despite them existing in postmeta?
    2. Metadata Fetching: Why is HPOS triggering a 95k-row metadata fetch for a standard list view? Is there a specific way to re-trigger a “repair” of these metadata mappings?
    3. Official Repair: Is there a recommended “official” way to force a re-sync of these specific missing columns without resetting the entire 276k-order migration?

    We want to ensure we have a clean, performant path before we attempt this logic on the Production site. Looking forward to your guidance.

    And also all that will be help or we should just delete the old orders and start with just the last year to make the dashboard lighter

    Thread Starter ziadeid

    (@ziadeid)

    Quick update: We managed to get the sync working!

    What was happening: We found that about 250k of our legacy orders were missing billing/shipping meta (like names/phone) because of custom checkout logic we used in the past. The new HPOS wp_wc_order_addresses table is very strict and has NOT NULL constraints on those columns. The sync was crashing because it was trying to insert NULL, which triggered that “duplicate empty order_key” error.

    Our Fix: We manually backfilled those missing fields with placeholders (like ‘N/A’ or empty strings). We tested a single order using wp wc hpos backfill and it worked perfectly with no errors.

    Current Status: Based on that success, we are now running the full sync for all orders. It is currently at 30% and running smoothly!

    Question for you: Is it normal for the HPOS schema to be this strict for legacy data? We had to do a lot of manual database “repair” to get the migrator to accept our old orders. Since this is all on a staging site, if there is a better or “more official” way to handle this on the production site to avoid issues, please let me know.

    We’ll send a final update once it hits 100%.

    Thread Starter ziadeid

    (@ziadeid)

    Hi, thanks for the explanation.

    We downgraded WP-CLI PHP to 8.2.29 on staging and confirmed it via:

    wp --info
    

    (PHP binary now /usr/bin/php8.2).

    Notes:

    • wp wc hpos reset is not available in our environment (reset is not a registered subcommand).
    • HPOS status still shows ~276,749 unmigrated orders.

    We then ran:

    wp wc hpos sync --batch-size=50
    

    Result:

    • The sync starts and shows progress (0% → 1% → 5%).
    • It repeatedly throws this error:
    WordPress database error: Duplicate entry '' for key 'wp_wc_orders.order_key'
    
    • Despite the progress output, no rows are written to wp_wc_orders, and count_unmigrated does not decrease.

    Additional checks:

    • _order_key in wp_postmeta has no NULL or empty values.
    • wp_wc_orders has order_key as NOT NULL with a UNIQUE index.

    So after downgrading to PHP 8.2, the issue changed from “silent no-write” to a blocking duplicate empty order_key error during sync.

    Please advise on how to safely handle/reset this state so HPOS sync can proceed (e.g. recommended cleanup, backfill, or alternative reset approach).

    Thread Starter ziadeid

    (@ziadeid)

    Thanks for your guidance. Here’s what I tried so far:

    1. Check unmigrated orders:
    wp wc hpos count_unmigrated
    

    Output:

    PHP Deprecated: Creation of dynamic property WP_CLI\Dispatcher\Subcommand::$longdesc is deprecated in phar:///usr/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Dispatcher/CompositeCommand.php on line 38
    ليست هناك ⁦276750⁩ من الطلبات لمزامنتها.
    

    This shows 276,750 orders pending sync.

    1. Attempted HPOS sync in batches:
    wp wc hpos sync --batch-size=50
    

    Some orders appeared to start syncing (example output below):

    ,'2021-08-21 17:12:00','2021-08-21 17:28:33',0,'shop_order','','SDG',0.000000,9600.000000,0,'[email protected]','bacs','تريد شحن محفظتك ؟','197.252.201.18','Mozilla/5.0 (Linux; Android 9; JKM-LX1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Mobile Safari/537.36',NULL);
    

    However, after running this, checking the count again with:

    wp wc hpos count_unmigrated
    

    still shows:

    ليست هناك ⁦276750⁩ من الطلبات لمزامنتها.
    

    So no apparent change in the unmigrated orders.

    Database permissions:
    I’ve verified that my database user has full privileges on the site database, including INSERT and UPDATE on wp_wc_orders and wp_wc_orders_meta. The database is not in read-only mode, so write access should not be an issue.

    Could you advise on the next steps?

    Thread Starter ziadeid

    (@ziadeid)

    Thanks for the guidance.

    • I checked WooCommerce → Settings → Advanced → Features → View and manage, and no plugins or extensions are listed as blocking HPOS.

    • The staging environment has 1 GB PHP memory, MySQL 8, and sufficient CPU resources(We have a VPS host, and we just use 50% of his power.).

    What should I do next?

    Thread Starter ziadeid

    (@ziadeid)

    Thanks for the guidance. Here are the results from staging:

    1- HPOS tables

    one table is still empty after the sync:SELECT COUNT(*) FROM wp_wc_orders; SELECT COUNT(*) FROM wp_wc_orders_meta;

    SELECT COUNT(*) FROM wp_wc_orders; COUNT(*)0

    SELECT COUNT(*) FROM wp_wc_orders_meta; COUNT(*)1

    2- Plugins and custom code
    All custom plugins were developed by me. To rule out conflicts, I deactivated all plugins except WooCommerce and re-enabled compatibility mode. No change in behavior.

    3- Query Monitor DB drop-in
    The db.php drop-in was fully removed. But still the same.

    4- Error logs
    PHP error logs were checked during the sync. No errors or warnings were found.


    And this is the staging site URL, but I deactivated all plugins and enabled the default theme, so nothing will work correctly.
    https://yallanal3b.store/

    • This reply was modified 5 months, 1 week ago by ziadeid.
    • This reply was modified 5 months, 1 week ago by ziadeid.
Viewing 6 replies - 1 through 6 (of 6 total)