Building a modern, agile data warehouse with a proven 3-Tiered Architecture.

Greenlight, a debit card for kids with parental controls, supports over 700,000 active users on its app-based service.  Greenlight is committed to helping parents raise financially savvy kids who have first-hand experience with saving, investing and money management.

Managing financial transactions and user behavior across hundreds of thousands of active users requires a strategic data infrastructure. Greenlight needed to design their data warehouse and data lake with enough flexibility to support their future growth. To this end, their data engineers had been building an internal data lake and data warehouse for several months. This was meant to replace their existing legacy mySQL database which was taking upwards of 12 hours to complete its daily ETL process.  

Unfortunately, engineers leading this effort resigned from Greenlight before the company’s new data infrastructure was complete. This left Greenlight in an untenable state, with thousands of lines of Python / Airflow and hard-coded ETLs. Adding even a small number of new models would have required considerable resources and months to complete. Greenlight leaders knew that refilling positions internally wouldn't offer the diverse skills they needed to pivot the project towards a more sustainable solution. Instead, they reached out to Mammoth Growth for support building a modern, agile data warehouse.  

After interviewing outgoing team members and internal stakeholders, Mammoth Growth deployed our proven data warehouse design process, now known as “Data Team as a Service.” 

• For Greenlight, this meant investing in key tools such as Snowflake and dbt.  Snowflake offered power and scalability which acted as a data warehouse / data lake hybrid, and dbt meant they could transform the raw data from their lake on demand. 
• This immediately gave Greenlight much more flexibility and control over their customer data, since their data warehouse could now be fully rebuilt at any time. 
• In this new setup, data models would be fluid and could be refactored or augmented with no data rebuild. And these updated data models were referentia, with a dependency tree that was automatically maintained, Greenlight was able to better manage employees’ time.  
• dbt Cloud offered fully managed scheduling, orchestration, testing, alerting, continuous integration, and continuous delivery or deployment (CI/CD). 
• And because dbt’s modeling followed a scripted SQL syntax, ongoing management of the new system would be much smoother, since hiring for SQL tends to be easier than hiring for Python data engineering.

Data modeling can quickly get messy and unmanageable even with the best intentions. To prevent this from happening, Mammoth Growth followed a 3-tier architecture which we have perfected with our Data Team as a Service offering:

• Bronze: Abstract the data lake data, clean up/normalize data types. This allowed Greenlight to migrate from AWS DMS to Fivetran in just one day with no downtime and no model rewrites.
• Silver: Codifying business rules and defining key events in a loose FactDim structure while not getting bogged down by a strict star snowflake schema. This step is designed to save time for everyone at Greenlight, since duplicating event names can lead to inaccurate results in everything your data touches downstream.
• Gold: Mart-style models which are dashboard specific and combine many silver tables to answer departmental questions. This created a more efficient system where anyone at Greenlight could quickly access the data they needed.

This 3-tier architecture also enabled smarter staffing. Data engineers could focus on loading data into Snowflake and making it available. Another team focused on ensuring integrity of business rules. Then analysts put everything together and build the necessary data sets to support their Tableau dashboards. In only 8 weeks, Mammoth Growth delivered the initial project requirements which included net new historical snapshots of key data sets.

In the following months, Mammoth Growth continued to port over necessary functionality and eventually assisted Greenlight in hiring their first full time engineer to support their updated data warehouse and data lake. Mammoth Growth’s insights enabled Greenlight’s Growth team to operate independently while delivering highly scalable, accurate, and resilient dashboards, driving business results.