Lead Big Data Architect for Data Pipeline Build

BlueSky Ventures

Contract · Flexible hours

About the role

We are seeking a seasoned Big Data Architect to design and implement a scalable data platform that supports advanced analytics and real‑time processing. The project focuses on building a robust data lake and warehouse architecture to enable our data science teams to derive actionable insights.

Key responsibilities

• Design end‑to‑end big data pipelines using AWS services.
• Develop data models and schemas for structured and semi‑structured data.
• Implement data ingestion, transformation, and storage solutions with high availability.
• Optimize performance and cost of data workloads.
• Collaborate with engineers and analysts to define data requirements.
• Ensure data security, governance, and compliance across the platform.

Must-have skills

• Extensive experience with AWS data services (Redshift, S3, Glue, Kinesis).
• Strong proficiency in Python for data engineering.
• Deep knowledge of PostgreSQL and relational data modeling.
• Hands‑on expertise with Docker containerization.
• Proven ability to design scalable big data architectures.

Nice to have

• Familiarity with Spark or Flink for stream processing.
• Experience with CI/CD pipelines for data workflows.