Job description
AI Evaluation Engineer (Engineering Simulation & Design) | Contractor | 40 hrs/week | Worldwide Remote
Turing is seeking experienced AI Evaluation Engineers specializing in engineering simulation and design to author and validate complex, model-breaking problems that train and evaluate state-of-the-art AI agents. This is a high-impact contractor role working at the frontier of AI research across disciplines including Electrical, Mechanical, Control Systems, Aerospace, Systems, and Robotics Engineering.
About Turing
Based in San Francisco, Turing is the world's leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing accelerates frontier research through high-quality data, advanced training pipelines, and top AI researchers specializing in coding, reasoning, STEM, multilinguality, multimodality, and agents.
Key Responsibilities
Model-Breaking Problem Design: Author original, self-contained engineering design tasks with competing constraints, explicit optimization targets, validated reference solutions, and objective autograders.
Environment & Simulation Integration: Build, run, and validate problem environments using open-source simulation tools and custom Python test benches.
Trajectory Analysis & Failure Mode Taxonomy: Evaluate coding agent outputs and execution logs across repeated trials to identify systemic failure modes such as misinterpreted simulator feedback, premature design convergence, or physically impossible geometries.
Difficulty Calibration & Benchmark Refinement: Iteratively refine problem difficulty based on empirical model performance data without introducing ambiguity.
Cross-Functional Collaboration: Partner with AI researchers, pod leads, and domain experts to integrate high-rigor benchmarks into the model evaluation pipeline.
Requirements
Education: Master's degree or PhD in Electrical, Mechanical, Aerospace, Control Systems, Systems Engineering, Robotics, or an applied science discipline.
Experience: 3+ years of hands-on engineering design experience.
Simulation Tooling: Proficiency with at least one domain-relevant open-source simulation package (e.g., ngspice, PySpice, OpenFOAM, FEniCSx, CalculiX, python-control, CadQuery, OpenModelica, Cantera, Gmsh) and strong Python scripting skills.
AI Evaluation: Hands-on experience with modern LLMs/coding agents and evaluation concepts (pass@k, failure-mode analysis, nondeterministic behavior), with the ability to audit trajectory logs and isolate core reasoning and tool-use failures.
Domain Rigor: Uncompromising attention to physical plausibility, unit consistency, boundary conditions, convergence criteria, and technical documentation.
Availability: Weekend on-call availability required (part-time engagement acceptable).
Infrastructure: Personal desktop or laptop with a stable, high-speed internet connection.
Target Engineering Domains
Electrical Engineering
Mechanical Engineering
Aerospace Engineering
Engagement Details
Commitment: 40 hours per week with 4 hours of overlap with PST
Engagement Type: Contractor
Duration: Up to 24 weeks
Evaluation Process
Shortlisted candidates will receive a Job Interest Form.
Finalized candidates will undergo a delivery review before proceeding further.
Prepare your application
Use the description above to assess your fit for Benture. This checklist is guidance from Resumize, rather than additional requirements from the employer. Imported listings can be shortened or change after collection, so confirm the complete description and current availability before sending an application.
Match requirements to evidence
Identify the responsibilities and skills the employer explicitly names. For each important requirement, choose one example from your work, education, training, or projects. Explain what you did, how you did it, and what changed as a result. Include a measurable outcome when you can support it. If you lack a requirement, describe related experience accurately instead of adding an unfamiliar skill to your resume.
Put the most relevant examples near the top of your resume. Keep job titles, dates, qualifications, and contact details consistent across your application materials. Use the employer’s terminology when it describes your experience accurately; a copied list of keywords does not explain your contribution. A short, specific bullet is easier to review than a paragraph that mixes several unrelated duties.
Check the working arrangement
Confirm the location, schedule, employment type, and any eligibility requirements on the employer’s site. A remote label does not establish worldwide eligibility or flexible hours. If compensation, benefits, equipment, travel, or contract duration are missing from this listing, write down your questions for the recruiter. Do not assume details from another opening at the same company apply to this role.
Review before submitting
Follow the employer’s requested file format and application instructions. Open your exported resume and check that its text is selectable, its sections read in order, and its links work. Proofread names, dates, and contact information. Share confidential work samples only when you have permission, and use an anonymized example when necessary.
Submit through a trusted employer or recruiting destination. If the original listing has disappeared, search the employer’s careers page for the title rather than assuming the vacancy remains open. Save the application date, job reference, and the version of your resume you sent. Those details help you prepare for a later conversation and avoid submitting conflicting information through several job boards.
Review your resume for writing and readability feedback, or browse other openings if this opportunity is unavailable.