Prompt Engineer – Eval Harness Design

CloudWave Digital

Contract · Remote · Flexible hours

About the role

CloudWave Digital is building a next‑gen test automation uplift for its internal release pipeline. As a Prompt Engineer you will craft and fine‑tune prompts that drive AI‑assisted test case generation, helping to cut regression risk before each product launch. The work focuses on the "PulseGuard" module, a suite of automated regression checks powered by LLMs.

Key responsibilities

• Design and iterate prompt templates for generating Selenium‑compatible test scripts using Python and JavaScript.
• Integrate prompt‑driven test generation into the CI/CD pipeline on GitHub Actions.
• Collaborate with QA leads to define regression coverage metrics and validate generated tests against real‑world scenarios.
• Maintain a prompt versioning repository, documenting changes and performance impacts.
• Run benchmark experiments to measure reduction in manual test authoring time.
• Provide concise documentation and training snippets for the engineering team.

Must-have skills

• Proficiency in crafting LLM prompts for code generation.
• Hands‑on experience with Python, JavaScript, and REST API integration.
• Familiarity with test automation frameworks (e.g., Selenium, Playwright).
• Understanding of CI/CD concepts and Git workflows.
• Ability to evaluate model outputs for correctness and bias.

Nice to have

• Experience with OpenAI or Anthropic APIs.
• Background in MLOps or model deployment pipelines.