Case study
BigQuery Optimization Cuts Cost for a Learning Platform
How SquareShift re-engineered BigQuery for a global learning and talent-management platform to cut processing cost and speed up analytics.
The expensive part of most BigQuery bills isn't the querying, it's rebuilding tables that didn't need to move. Preserve what's already correct and only reprocess what actually changed.
Data & Analytics Practice Lead, SquareShift
A global provider of cloud-based learning and talent-management software, used by organizations to recruit, train, and engage their employees.
Its analytics ran on BigQuery, and rising data volume was driving up both processing cost and query time.
- Lower processing cost — Tables no longer dropped and rebuilt on every ETL run.
- Faster queries — Incremental processing replaces full-table reprocessing.
- Efficient joins — SQL logic tuned with column pruning and CTEs.
How SquareShift delivered it.
The challenge
A global provider of cloud-based learning and talent-management software runs its analytics on BigQuery. As data volume grew, so did the bill: every ETL job dropped and recreated large tables from scratch, burning compute on work that didn’t need to happen again.
The platform needed to keep pace with growing data and stay fast for the teams querying it, without the cost climbing in lockstep with volume.
What we delivered
SquareShift re-engineered the BigQuery setup around persistent tables, so historical data no longer gets dropped and rebuilt on every run. Incremental processing now fetches only the recent data that actually changed, replacing full-table reprocessing.
On the query side, column pruning and SQL join optimization cut unnecessary data scanning, table clustering and partitioning organized the data around the queries the business actually runs, and Common Table Expressions simplified and sped up complex transformations.
The payoff
Processing cost came down because ETL jobs stopped rebuilding tables that were already correct, and queries run faster with join logic and clustering tuned to the platform’s real access patterns.
The analytics environment now scales with data volume instead of scaling cost alongside it.
Where this work sits