Engineering (Coding) Subject Matter Expert - AI Evaluation (US-Remote)

Job Description:

Seeking multiple Engineering (Coding) Subject Matter Experts to help design, run, and optimize data collection and evaluation workflows for GenAI research.

You’ll translate high-level research needs into scalable processes, produce and curate challenging domain problems, and ensure factual, bias-aware, high-quality datasets for LLM training.

• To Note: This is for an immediate project need. Project is approved for 3-months initially, with possibility to extend based on project/client demands.

Responsibilities:

• Partner with GenAI researchers/engineers to capture data needs and success criteria.
• Expand high-level requirements into clear, executable workflows for larger teams.
• Execute collection/evaluation workflows rapidly with minimal supervision.
• Innovate on workflows to maximize throughput and quality.
• Collaborate cross-functionally to maintain quality at scale.
• Conduct in-depth LLM-assisted research; gather reliable, up-to-date info.
• Craft original, high-quality content and hard problems for LLM eval/train.
• Perform rigorous fact-checking (precision/recall) to prevent misinformation.

Requirements:

• Education: Master’s with distinction or PhD in Engineering (Coding); top-tier institution preferred. Significant domain experience considered.
• Content development expertise across Engineering (Coding).
• Exceptional writing: clear, concise, engaging; across formats.
• Strong research & fact-checking; bias mitigation.
• Originality/creativity; distinct human voice.
• Detail orientation; precise data presentation; thorough proofreading.
• Communication: articulate complex info; strong collaboration.
• Understanding of AI/LLMs, their capabilities/limits.
• Prompt engineering and familiarity with AI writing tools.
• Ethical AI awareness and data literacy (collection, cleaning, transformation).
• Thrives in fast-paced, minimally supervised environments.