AI Model Evaluator

Part-time/Full-time/Contract/Temporary
Remote within US
$25-40/hr
No visa sponsorship

Benefits

Bonus
Equity / Stock Options
Paid Time Off (PTO)
Flexible Working Hours

Job description

We're looking for people to review answers written by AI models. Each task
gives you a prompt and the model's response. Your job is to judge how well it
answered, then explain why in a sentence or two that someone else could check.

Accuracy comes first, but tone and usefulness matter too. An answer can be
correct and still unhelpful if it is too vague, too long, or assumes knowledge
the person asking does not have. You will learn to separate those cases and
score them the same way each time.

The work follows a written rubric, and consistency matters more than speed. A
reviewer who scores similar answers similarly is worth more to us than a fast
one who does not.

No technical background is required. What helps is careful reading, patience
with repetitive tasks, and the willingness to say "I'm not sure" and ask rather
than guess.

The details:

  • Remote, 10 to 40 hours a week
  • Three months minimum, likely to extend
  • $25 - $40 per hour depending on experience, paid twice a month
  • Training is paid, including the first week while you calibrate

You will get feedback on your first few hundred reviews, and we expect you to
use it. That is the whole job: getting steadily more consistent.

All backgrounds welcome.

More information

Experience level

Entry-level or graduates · Junior (1-2 years) · Mid-level (3-4 years) · Senior (5-7 years) · Expert and leadership (8+years)

Job skills

LLM Evaluation

Prompt Engineering

Data Annotation

Evaluation Rubric Design

AI Safety Assessment

Error Analysis

Dataset Quality Assurance

Company overview

company-logo
Slight Edge AI