Senior Data Engineer Needed for Analytics Feed Build

NexGen Media

Contract · Remote · Flexible hours

About the role

NexGen Media is modernizing its digital publishing platform and needs a senior Data Engineer to lead a test‑automation uplift. The project, codenamed TestPulse, focuses on building a robust regression‑risk reduction framework that will run before every major content launch.

Key responsibilities

• Design and implement an end‑to‑end data validation pipeline using Python and AWS Glue to compare pre‑ and post‑deployment datasets.
• Develop automated test suites in dbt that verify schema integrity and data quality across PostgreSQL and MySQL warehouses.
• Integrate CI/CD hooks in GitLab to trigger data tests on every code merge and generate actionable failure reports.
• Collaborate with QA engineers to embed data checks into UI‑level test scenarios, reducing manual regression effort by 40%.
• Document the TestPulse architecture, create runbooks, and train the analytics team on interpreting test outcomes.

Must-have skills

• Advanced Python programming for data pipelines.
• Deep experience with PostgreSQL and MySQL schema management.
• Proficiency in AWS services (S3, Lambda, Glue, CloudWatch).
• Strong knowledge of data testing frameworks such as dbt and Great Expectations.
• Hands‑on CI/CD integration for data workflows.

Nice to have

• Familiarity with Airflow or Prefect orchestration tools.
• Experience in performance testing of large‑scale data sets.