Lead Big Data Architect for Warehouse Migration

Cascade Apps

Contract · Remote · Flexible hours

About the role

Cascade Apps is modernizing its data backbone to boost performance and reliability of its customer‑facing services. As a Big Data Architect you will lead the redesign of the "Velocity Hub" data pipeline, ensuring low‑latency, fault‑tolerant processing for millions of daily events.

Key responsibilities

• Design and implement a scalable architecture for Velocity Hub using Apache Spark on EMR.
• Refactor existing ETL jobs to run in Docker containers orchestrated by Kubernetes.
• Introduce schema evolution and data quality checks with Apache Avro and Deequ.
• Optimize query performance on PostgreSQL and Redshift by tuning indexes and partitioning strategies.
• Set up real‑time monitoring dashboards in Grafana with Prometheus metrics.
• Conduct load‑testing milestones to verify sub‑second latency under peak traffic.

Must-have skills

• Deep expertise in big data frameworks (Spark, Flink).
• Strong command of Python for data pipelines and automation.
• Extensive experience with AWS services (S3, EMR, Redshift, Kinesis).
• Proficiency in containerization (Docker) and orchestration (Kubernetes).
• Solid background in relational databases, especially PostgreSQL.

Nice to have

• Knowledge of streaming platforms like Kafka.
• Familiarity with CI/CD for data engineering (GitLab CI, Jenkins).