LLM Red Team Specialist - AI Job at Mercor, New York, NY

  • Mercor
  • New York, NY

Job Description

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: LLM Red Team Specialist — Failure Modes & Edge Cases
Type: Contract
Compensation: $60–$90/hour
Location: Remote
Commitment: 35 hours/week

Role Responsibilities

  • Evaluate frontier AI models on coding, ML , and analysis tasks to identify vulnerabilities and failure modes.
  • Design complex tasks that challenge models and are fair for grading.
  • Document findings with clear evidence and reproducible steps.
  • Collaborate with task authors to close loopholes and improve grading.
  • Share insights with researchers to enhance benchmark quality.
  • Work independently and asynchronously to meet deadlines and improve AI model performance .

Qualifications

Must-Have

  • MSc or PhD in a STEM field or equivalent experience.
  • 1+ years in research, research-engineering, security, or AI evaluation.
  • Experience identifying vulnerabilities in LLMs or ML systems .
  • Proficiency in Python and Git .
  • Familiarity with LLM capabilities and evaluation techniques.
  • Ability to work 35 hours/week .

Preferred

  • Experience in AI training , model evaluation, or benchmark/task authoring.

Resources & Support

  • For details about the interview process and platform information, please check:
  • For any help or support, reach out to: support@mercor.com

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Job Tags

Contract work, Summer work, Remote work

Similar Jobs

STAMPEDE VENTURES INC

HVAC Mechanic Job at STAMPEDE VENTURES INC

 ...pneumatic direct digital control systems, and air distribution systems Operate, procure, introduce, and get back HFC, HCFC, and CFC refrigerants Perform repairs and maintenance of reach in, walk in, stand alone ice machines, central plan chillers, pumps, free standing A... 

Dunkin - Baskin Robbins

Cake Decorator Job at Dunkin - Baskin Robbins

 ...a Cake Decorator, you will assemble and prepare cakes for both in shop, display and custom orders, including building cakes with ice cream, frosting, decorating with icing, chocolates, and edible images. Training to decorate will be provided. However, any previous experience... 

Medical Solutions

Travel Ultrasound Tech - $2,048 per week in Winston Salem, NC Job at Medical Solutions

 ...Ultrasound Tech Location: Winston Salem, NC Agency: Medical Solutions Pay: $2,048 per week Shift Information: Days Start Date: 9/14/2026 About the Position A facility in Winston Salem, NC is looking for an exceptional Sonographer for a Local... 

Casino Pauma

Food Server Job at Casino Pauma

 ...making sure that they understand that they are part of our family. NATIVE AMERICAN PREFERENCE STATEMENT Pauma Band of Mission Indians, a federally recognized Indian Tribe, in accordance with section 703(i) of Title VII of the Civil Rights Act of 1964, will give... 

System One

HVAC Mechanic Job at System One

 ...plumbing and HVAC Manages and oversees mechanical operations Performs mechanical system maintenance across HVAC, plumbing, and refrigeration Troubleshoots and diagnoses issues across a range of mechanical systems Ensures knowledge and application of state and...