A Little Humanity

How people are training AI to be more like us

Not to anthropomorphize it too much, but generative AI tries hard to get it right. It “wants” to give us the answers we’re looking for. But the problem is, sometimes it doesn’t know the answer. And though it tries to emulate humans, well, it’s not human. There are certain things only humans know how to do. For AI to give its best approximation, it needs direct tutoring from people, with more detailed learning for specifics tasks than LLMs can gain by scraping the internet. Some enterprising companies are providing just that.

‘What Only Humans Know’

Founded in 2023, Mercor already has a $10 billion valuation. What started as a service to match companies with skilled engineers abroad has evolved into a marketplace for skilled and specialist humans to find high-paying work training AI systems. These experts in fields such as law, medicine, and finance create rubrics and evaluations that help the systems improve.

“Everyone’s been focused on what models can do,” CEO and co-founder Brendan Foody told Fortune. “But the real opportunity is teaching them what only humans know — judgment, nuance, and taste.”

Though this might seem like human experts training their digital replacements, Foody doesn’t see it that way. In his view, training AI to automate repetitive tasks and to do acquire more human-like skills allows people to move up the value chain and do more important work.

“It’s like we have this bottleneck of only so much human labor in the economy,” he said. “That shape is going to change radically over the next decade.”

Mercor’s AI Productivity Index (APEX) judges AI system on how well they perform at tasks human experts in specialized fields usually do. Foody anticipates millions of humans training AI in what he likens to a new industrial revolution.

“The challenge now is to be thoughtful about what comes next: the higher, better things humans will spend time on,” Foody said, “and how quickly we can help make that future real.”

A Trustworthy Model

Like humans, AI makes mistakes. But sometimes the mistakes AI makes are ones human would never make. For example, if you saw a photo of a dog wearing sunglasses, you would easily recognize it as a – very stylish – dog. AI probably wouldn’t because that is not what it has been taught that dogs look like. To help AI models perceive more like humans, researchers at Brown University have launched an online game called ClickMe in which players click on and draw over the parts of an image that they think help them recognize the subject. That trains the AI on what humans see and how they use their vision to identify things.

“Click Me is a game where you reveal to an AI partner what parts of an image you think are most important for classification,” student lead Jay Gopal told Providence’s WJAR. “This gives you the opportunity to draw a map that says you believe humans would focus on these areas, and then we can extract a map similar to this from AI models, and we can do a direct comparison between the brain and AI to see how brain-aligned any given deep learning model is.”

Thanks to its popularity on Reddit and Instagram, ClickMe has collected millions of “maps” information from thousands of users and how long it took them, giving the AI a large data sample with which to work. The more they behave like humans and can explain their actions a way that makes sense to humans, the more trust AI systems will engender.

“Before you can apply an AI model to healthcare, you need to be able to explain to doctors and patients that it’s a trustworthy model,” Gopal said. “You have to show how it got to the conclusions that it reached.”

Move Like a Human

You might have seen videos of humanoid robots not exactly acing simple tasks like not walking straight into things. Oftentimes, these robots are being teleoperated by humans behind the scenes, sometimes far away. But there’s also an entire industry dedicated to teaching humanoid robots how to move more like humans.

“Companies are building foundation models fit for the physical world,” Ulrik Stig Hansen, co-founder of data management platform Encord, told the Los Angeles Times. “There’s this huge resurgence in robotics.”

One company Encord contracts with is India-based Objectways, which employs people to perform relatively simple tasks such as folding towels and boxes while wearing GoPro cameras that capture their actions, creating training videos for humanoid robots.

Though LLMs aren’t perfect, they have vast amounts of training data on the internet to learn from. Data on how physical objects – and humans in particular – move is harder to come by and translate into training data. Micro1 pays people to record themselves going about their days with smart glasses. Figure AI is partnering with trillion-dollar real estate company Brookfield to train robots on footage from 100,000 homes. Scale AI says it has more than 100,000 hours of footage of human actions from its prototype lab. All this training could go a long way toward improving the performance of humanoid robots in what Nvidia projects to be a $38 billion industry in the next decade.

“Today, a mix of real and synthetic data is being used, gathered from human demonstrations, teleoperation sessions and staged environments,” Mohammad Musa, founder of Deepen AI, told the Times. “Much of this work still occurs outside the West, but automation and simulation are reducing that dependency over time.”

With enough training, humanoid robots might make all the right moves.