OpenAI Is Asking Contractors to Upload Work From Past Jobs to Evaluate the Performance of AI Agents | EUROtoday

OpenAI is asking third-party contractors to add actual assignments and duties from their present or earlier workplaces in order that it could possibly use the information to judge the efficiency of its next-generation AI fashions, in line with data from OpenAI and the coaching information firm Handshake AI obtained by WIRED.

The venture seems to be a part of OpenAI’s efforts to determine a human baseline for various duties that may then be in contrast with AI fashions. In September, the corporate launched a brand new analysis course of to measure the efficiency of its AI fashions towards human professionals throughout quite a lot of industries. OpenAI says this can be a key indicator of its progress in direction of attaining AGI, or an AI system that outperforms people at most economically precious duties.

“We’ve hired folks across occupations to help collect real-world tasks modeled off those you’ve done in your full-time jobs, so we can measure how well AI models perform on those tasks,” reads one confidential doc from OpenAI. “Take present items of long-term or advanced work (hours or days+) that you just’ve completed in your occupation and switch every right into a job.”

OpenAI is asking contractors to describe tasks they’ve done in their current job or in the past and to upload real examples of work they did, according to an OpenAI presentation about the project viewed by WIRED. Each of the examples should be “a concrete output (not a summary of the file, but the actual file), e.g., Word doc, PDF, Powerpoint, Excel, image, repo,” the presentation notes. OpenAI says people can also share fabricated work examples created to demonstrate how they would realistically respond in specific scenarios.

OpenAI and Handshake AI declined to comment.

Real-world tasks have two components, according to the OpenAI presentation. There’s the task request (what a person’s manager or colleague told them to do) and the task deliverable (the actual work they produced in response to that request). The company emphasizes multiple times in instructions that the examples contractors share should reflect “real, on-the-job work” that the person has “actually done.”

One example in the OpenAI presentation outlines a task from a “Senior Lifestyle Manager at a luxury concierge company for ultra-high-net-worth individuals.” The goal is to “Prepare a short, 2-page PDF draft of a 7-day yacht trip overview to the Bahamas for a family who will be traveling there for the first time.” It includes additional details regarding the family’s interests and what the itinerary should look like. The “experienced human deliverable” then shows what the contractor in this case would upload: a real Bahamas itinerary created for a client.

OpenAI instructs the contractors to delete corporate intellectual property and personally identifiable information from the work files they upload. Under a section labeled “Important reminders,” OpenAI tells the workers to “Remove or anonymize any: personal information, proprietary or confidential data, material nonpublic information (e.g., internal strategy, unreleased product details).”

One of the information considered by WIRED doc mentions an ChatGPT instrument referred to as “Superstar Scrubbing” that provides advice on how to delete confidential information.

Evan Brown, an intellectual property lawyer with Neal & McDevitt, tells WIRED that AI labs that receive confidential information from contractors at this scale could be subject to trade secret misappropriation claims. Contractors who offer documents from their previous workplaces to an AI company, even scrubbed, could be at risk of violating their previous employers’ non-disclosure agreements, or exposing trade secrets.

“The AI lab is putting a lot of trust in its contractors to decide what is and isn’t confidential,” says Brown. “If they do let something slip through, are the AI labs really taking the time to determine what is and isn’t a trade secret? It seems to me that the AI lab is putting itself at great risk.”