End-to-end Azure data engineering pipeline ingesting real-time earthquake data from the USGS API. Implements a Bronze–Silver–Gold lakehouse using Azure Data Factory, Databricks, ADLS Gen2, and Synapse Analytics, with both manual execution and fully automated daily-triggered workflows.

This repository is dedicated to showcasing projects built on Databricks, focusing on big data analytics, data engineering, and machine learning workflows.

End-to-end Azure Data Engineering project using ADF for incremental ingestion, Databricks (DLT) for Medallion Architecture, and Delta Lake for CDC (SCD Type 1). Managed via Databricks Asset Bundles (DABs) for professional CI/CD. Focuses on real-time streaming, scalability, and Star Schema modeling.

Batch ingestion and ELT pipeline on GCP using Airflow, BigQuery, and dbt

Hands-on project demonstrating how to load and query semi/unstructured data in Snowflake. Covered: Loading JSON data into a VARIANT column and extracting structured values using FLATTEN, Loading Parquet files from AWS S3 into relational tables with proper column mapping.

Automated Data Loading in Snowflake using Snowpipe (End-to-End Implementation).

Terraform module to manage Snowflake warehouses with configurable sizing, scaling, auto-suspend, and cost controls.

🌍 Ingest and transform real-time earthquake data using Azure tools to deliver reliable analytics-ready datasets for insightful decision-making.