JetSMART is a low-cost airline created by a private investment fund called Indigo Partners, with over 15 years of experience in developing successful ultra-low-cost airlines worldwide. Currently, JetSMART is expanding in the region with flights in Chile, Argentina, and Peru, with the main objective of connecting millions of people across South America.
Business Challenges
Data is one of the key drivers for JetSMART to make data-driven decisions, gain relevant insights, and understand the company's performance. The main objective of this project was to design a multidimensional model to support an analytical process delivered to business users using the company's widely used visualization tool.
This project started with data already available in Redshift, building a process to transform this data with a clear focus on profitability, scalability, flexibility (related to programming purposes), and the goal of creating a self-managed solution. The data is cleaned, enriched, and transformed into an existing data storage solution (Redshift) with the clear purpose of building a "single source of truth" that business users can rely on and access through a business intelligence tool (PowerBI) to support the company's analytical process.
Low-Cost Data Processing Using Serverless Computing and Cloud Data Warehouse
The solution designed by ARKHO for our client in the AWS cloud was built to provide a comprehensive serverless solution based on the following technical pillars:
Infrastructure: The provisioning infrastructure was provided by the AWS CloudFormation service to create and have control over all resources.
Process scheduling: The process needed to be scheduled to run at a specific time in the early morning without any manual intervention and with predefined parameters. CloudWatch Events was leveraged with a CRON expression to execute the process daily.
Process orchestration: Sending notifications, tracking errors, and visualizing each step of the process were achieved using AWS Step Functions, where the same process can be used with different parameters through defined tasks using a simple JSON input.
Notifications: The status of each step in the process needed to be notified to an external system. Step Functions, along with AWS SNS, made it possible to send messages to multiple subscribers.
Error tracking: Along with a notification, any errors encountered needed to be captured and easily readable for anyone who can take action and resolve the issue. DynamoDB, a NoSQL database, was used to store this type of data.
Results
A controlled infrastructure created through AWS CloudFormation templates that can be easily transported between regions, accounts, and different environments within the company.
Version control of templates using the S3 versioning feature to track all changes.
An orchestrated process to support the transformation model using Step Functions that closely resembles this design: