Online Search Quality Analyst - Remote - Blacksburg, Virginia, US

Outlier is an AI training platform owned and operated by Scale AI. Outlier Experts work on a variety of projects, from generating training data to evaluating the performance of models.

Are you ready to apply Make sure you understand all the responsibilities and tasks associated with this role before proceeding.
We are seeking detail-oriented English speakers to review AI-generated media. Roleplay live conversations with an AI voice model and score each exchange, compare generated images and videos to judge which follows the prompt better, and explain in writing what looked or sounded wrong, all remotely and on your own schedule.
At Outlier, your expertise helps shape how advanced AI systems reason, learn, and improve. Through Outlier’s evaluation and feedback loop, your work helps turn frontier research into AI that performs better in the real world.

Holding a live, scenario-based conversation in English with AI voice models, then ranking how natural, useful, and clear each one sounded
Comparing pairs of AI-generated images and videos and judging which one better follows the prompt
Inspecting outputs closely, including zooming in and replaying, to spot problems like distorted anatomy, flickering or unnatural motion, text errors, and audio that doesn't match the video
Rating each output across dimensions like adherence, quality, and realism, then writing a specific explanation in English that points to what you saw or heard

Compare two AI-generated images for the same text prompt and pick the better one, judging how well each follows the instructions, visual quality, and visible AI artifacts. Some tasks rank edited images against the original.
Video evaluations: Rank AI-generated videos against a text prompt, or edited videos against their source clip.
Video caption labeling: Label first-person videos of people or robots doing hands-on tasks. Contributors split each clip into individual actions and write precise captions, starting from an AI draft and correcting it.
Audio and voice evaluations: Rate AI-generated speech on naturalness, pronunciation, and how closely it matches a target voice. Fluent English speaker with strong, clear written communication and comfortable with natural, everyday conversational speech
Experience writing short, evidence-based explanations or reviews
Pursuing or holding a Bachelor's degree or above

A valid ID and mobile phone from your country of residence
A LinkedIn profile showing your educational background and work experience

Project work: Earn up to USD $15 per hour
Payments are processed weekly on Tuesdays, covering tasks completed during the previous pay period (Tuesday to Monday at midnight UTC), via PayPal, AirTM, or ACH transfer.
Rates vary based on expertise, skills assessment, location, project need, and other factors. For example, higher rates may be offered to PhDs. For non-core work, such as during initial project onboarding or project overtime phases, lower rates apply. Additional incentives data is based on payments made in the past six months and is updated quarterly. Certain projects offer incentive payments. Please review the payment terms for each project.

Collaborate with experts in your field to push the boundaries of pre-release models

Create an account on Outlier
# This is a 1099 contract opportunity on the platform. You will join Outlier as a freelancer/independent contractor, not an employee. Because this is a freelance opportunity, we do not offer internships, sponsorship, or employment. If you are an international student, you may be able to sign up for Outlier if you are on a visa. You should contact your tax and/or immigration advisor with specific questions regarding your circumstances. xklufbp
Remote working/work at home options are available for this role.

Back to blog

Other Jobs To Apply

Warehouse Associate | $40/hr Remote

Customer Service Specialist 1 at WesBanco: 100% Remote (FL, IN, KY, MD, MI, OH, PA, TN, VA, WV)

Remote Jobs $30 An Hour, Disney Wfh Jobs ( Remote Ops Producer II )

Remote Chat Support Associate (Part-Time, Work From Home, Entry-Level Opportunity with Paid Training)

Vice President, Sales, Individual and Family Markets

Remote Out of Office Position / Data Entry (Hiring Immediately)

Amazon Data Entry Clerk (Remote)

Amazon Customer Service - Work From Home $16-$24/hr

Construction Project Scheduling Specialist – Airports, Infrastructure

Remote Virtual Assistant – Client Services & Operations

Work From Home Customer Service Rep

Remote Entry-Level Role – No Experience Needed | Flexible & Beginner Friendly

Ground Support Equipment Technician - STL

Remote Junior Data Entry Specialist

Call Center Specialist [Work From Home - Night Shift - Full Time]

Legal Assistant Remote / Telecommute Jobs

Coca-Cola Data Entry Clerk (Remote)

Social Media Manager - Remote

Warehouse Receiver / Forklift Driver

Staff / Sr. ML Engineer, Indexing Infrastructure

Apple Home Advisor - [Part Time] Remote Jobs in Canada

Senior Account Manager Business Sales

Client Support Specialist - Full Time - Work From Home

Senior Manager – Learning and Development

Support Job Remote Amazon Data Entry Jobs Hiring (URGENT) – Part-Time Job ID- 1110 Job ID- 3358

Chewy Remote Jobs Customer support (Work From Home )

__ _✓ Work-From-Home Associate | Entry Level | No Experience Needed

Remote Inventory Supervisor for AI-Driven Warehouse Ops

Clinical Appeals RN – Remote – M-F working alternating Saturdays

Warehouse Associate 8am-4:30pm

Data Entry (Entry-Level, Remote, Full-time) - WFH In Nigeria

Maintenance Supervisor

Remote Social Media Writer (Work from Anywhere) (Brand)

Part-Time Store Cashier/Stocker

Remote Key Account Executive – Facility Supplies

amazon flex $15+/ Hour (Sign on Bonus)!

Entry Level - Fully Remote Position - Fully Remote + No Experience Needed

Cloud Systems Architect Remote / Telecommute Jobs

Senior Analyst Relations Leader – AI

Dock Worker-Unloader(Part-Time)@ Amazon-Tempe

Prog & Proj Mgmt-Sr Mgr

Amazon Delivery Driver

Amazon Customer Service

Flexible Hours:Equipment Coordinator-Kannapolis

Telehealth Nurse

Earn Side Money at Home Testing Products

Call Center Specialist [Work From Home - Night Shift - Full Time]

Part-Time Work From Home Amex Virtual Assistant at American Express – Job ID- 495 Job ID- 2678

Digital Ad Design Consultant - Multi Location Sales (US Remote)

Remote Business Development Manager Nevada (RENO)

Common Interview Questions And Answers

1. HOW DO YOU PLAN YOUR DAY?

This is what this question poses: When do you focus and start working seriously? What are the hours you work optimally? Are you a night owl? A morning bird? Remote teams can be made up of people working on different shifts and around the world, so you won't necessarily be stuck in the 9-5 schedule if it's not for you...

2. HOW DO YOU USE THE DIFFERENT COMMUNICATION TOOLS IN DIFFERENT SITUATIONS?

When you're working on a remote team, there's no way to chat in the hallway between meetings or catch up on the latest project during an office carpool. Therefore, virtual communication will be absolutely essential to get your work done...

3. WHAT IS "WORKING REMOTE" REALLY FOR YOU?

Many people want to work remotely because of the flexibility it allows. You can work anywhere and at any time of the day...

4. WHAT DO YOU NEED IN YOUR PHYSICAL WORKSPACE TO SUCCEED IN YOUR WORK?

With this question, companies are looking to see what equipment they may need to provide you with and to verify how aware you are of what remote working could mean for you physically and logistically...

5. HOW DO YOU PROCESS INFORMATION?

Several years ago, I was working in a team to plan a big event. My supervisor made us all work as a team before the big day. One of our activities has been to find out how each of us processes information...

6. HOW DO YOU MANAGE THE CALENDAR AND THE PROGRAM? WHICH APPLICATIONS / SYSTEM DO YOU USE?

Or you may receive even more specific questions, such as: What's on your calendar? Do you plan blocks of time to do certain types of work? Do you have an open calendar that everyone can see?...

7. HOW DO YOU ORGANIZE FILES, LINKS, AND TABS ON YOUR COMPUTER?

Just like your schedule, how you track files and other information is very important. After all, everything is digital!...

8. HOW TO PRIORITIZE WORK?

The day I watched Marie Forleo's film separating the important from the urgent, my life changed. Not all remote jobs start fast, but most of them are...

9. HOW DO YOU PREPARE FOR A MEETING AND PREPARE A MEETING? WHAT DO YOU SEE HAPPENING DURING THE MEETING?

Just as communication is essential when working remotely, so is organization. Because you won't have those opportunities in the elevator or a casual conversation in the lunchroom, you should take advantage of the little time you have in a video or phone conference...

10. HOW DO YOU USE TECHNOLOGY ON A DAILY BASIS, IN YOUR WORK AND FOR YOUR PLEASURE?

This is a great question because it shows your comfort level with technology, which is very important for a remote worker because you will be working with technology over time...