Elasticsearch Migration for a Video Commerce Platform

The challenge

The client, a global video commerce company spanning live TV, e-commerce, and mobile platforms, ran its Elasticsearch-powered observability stack on-prem — supporting the log ingestion, monitoring, and alerting that keep its digital storefronts running. A full migration would have meant moving 52 terabytes of historical data the platform didn’t need live.

Early attempts hit snapshot credential mismatches, ILM misconfiguration under the platform’s high ingestion volume, and API key and SAML authentication issues that were blocking dashboard access.

What we delivered

SquareShift migrated only the system indices the platform actually needed — Watcher, Kibana, and related indices — using dual writes through Logstash and Fleet, so historical data stayed in place and out of scope.

The team fixed the snapshot and SAML/API key issues, tuned ILM for the hot and frozen tiers under high ingestion, restored dashboard access, and migrated all four Fleet policies with matching index templates. Over 3,500 watchers were updated through automation rather than by hand.

The payoff

The client’s observability stack now runs its essential system indices on Elastic Cloud, with monitoring, alerting, and dashboards intact and Fleet policies aligned. Skipping the 52 terabytes of historical data kept the migration focused and avoided a costly, unnecessary data transfer.