Archive article - published on January 06 2022
Site Impact needed a way to modernize and scale-out their data management platform. We were tasked with architecting a push button, built to scale infrastructure to service their business operations.
The challenge
Site Impact reached out to us to discuss the ability of tackling uptime and scale problems faced with their current data management platform. Their current database and web platform would consistently go offline each month, causing $100,000's in lost revenue every quarter.
The solution
After assessing the project during a Discovery phase, we were able to extract an MVP in order to produce what was desired. We focused heavily on resiliency for their infrastructure and modernization for their application stack. The output of this was migrating from VM's to GKE for their frontend application & moving from MSSQL to BigQuery for their data anaylitics. The data management side would be handled by Composer, Dataproc and GCS.
The results
Key Technologies:
GKE - Autoscaling capabilities to allow for infinite growth and prevention of single point of failures for their frontend application.
BigQuery - Ability to execute efficient SQL queries on tables 400GB large with hundreds of millions of rows of data, some tables spanning 600+ columns.
Composer - Airflow provides a dependency-driven ETL pipeline which runs all needed manipulations and automatically presents the data up to BigQuery.
Dataproc - Pyspark code utilizing dataproc’s compute, built to handle PB's of data.
We focused heavily on resiliency for their infrastructure and modernization for their application stack. The output of this was migrating from VM's to GKE for their frontend application.
About Site Impact, LLC
Site Impact are one of the leading providers in data and marketing resources. Specializing in multi-channel direct marketing services.
Industry: Advertising & Marketing
Primary project location: United States
About WALTLabs.io, LLC
At WALT Labs, we provide professional services around application modernization and cloud strategy.
Products: Google Cloud Platform
Let’s just have a chat and see where this goes.
