Case study
On-Prem Observability for a National Digital ID Program
How SquareShift deployed a secure, on-premise Elasticsearch observability platform across two data centers for a national digital identity program, sized for 1.5TB of ingestion a day.
One of the world's largest digital identity programs, a government initiative verifying and managing digital IDs for millions of residents across India.
Its infrastructure runs fully on-premises to meet strict compliance and data-privacy requirements.
Geographically separate data centers running active-active HA. TB of daily ingestion capacity delivered for the platform. TB retention capacity across hot/warm storage tiers.
Key servicesHow SquareShift delivered it.
The challenge
The client runs one of the world’s largest digital identity programs, verifying and managing digital IDs for millions of residents. Legacy tools couldn’t scale to the program’s data volumes or meet its security bar, and the in-house team needed expert support for capacity planning and deployment.
Given the sensitivity of the data involved, the solution had to be delivered on-site under strict compliance and data-privacy requirements — with no shortcuts to a faster, cloud-only build.
What we delivered
SquareShift deployed Elasticsearch with APM, machine learning, and SIEM features across two geographically separate data centers, sized for 1.5TB/day of ingestion and 270TB of retention. Kafka MirrorMaker handled geo-redundant ingestion between sites.
The team ran full high-availability configuration and stress testing to validate performance under load, delivering everything through on-site engagement rather than remote-only delivery, given the sensitivity of the data.
The payoff
The program now runs on-premise observability sized for 1.5TB of ingestion a day and 270TB of retention, with high availability spanning two data centers instead of a single point of failure.
Custom dashboards and rigorous benchmark testing give the client’s own team the tools to operate the platform going forward, backed by an architecture built and validated for compliance from day one.
At national-identity scale, on-site delivery isn't a compliance checkbox — it's the only way to stress-test high-availability failover before sensitive data ever touches the cluster.
Platform & Software Engineering Practice Lead, SquareShift
Where this work sits