ShyftLabs

Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, and SQL. Integrate data from multiple sources including databases, Amazon S3, files, and REST APIs. Build and manage Delta Lake tables following Medallion Architecture. Create, schedule, monitor, and optimize Databricks Jobs and Workflows. Collaborate with cross‑functional teams to deliver production‑ready data solutions.

Strong expertise in Python, PySpark, and Advanced SQL. Hands‑on experience with the Databricks Lakehouse Platform and Unity Catalog. Proficiency in data modeling (Star/Snowflake Schema, Fact & Dimension tables, SCD). Experience with REST API integrations for data ingestion and export. Knowledge of Spark performance tuning, Delta Lake optimization, and CI/CD with Git.

Experience with Auto Loader, Spark Declarative pipelines, Kafka, Airflow, or dbt. Databricks certification. Familiarity with AWS services such as S3, Glue, and Lambda.