OpenAI seeks to one-up Anthropic with new customer

What happened

As AI models have become more powerful, the potential for those models to be misused has grown — as has a clamor for safety guardrails that can stop such abuse from happening. AI companies must now walk a delicate tight rope between respecting their enterprise customers’ privacy while also watching usage for possible issues.

Private Safety Processing can analyze those multiple conversations for signs of abuse without human review of a user’s conversations. In the case where the system is triggered, it may send a “narrowly defined signal” to OpenAI that warns of a specific type of activity, the company says. Based on that signal, OpenAI can then decide whether “enforcement is necessary,” it says.

If so, OpenAI will reach out to the customer for more context or to work with them on the issue and a customer may choose to share data with OpenAI at their discretion, the spokesperson said. ” Every one of those review sessions is “recorded in a tamper-proof log that reviewers cannot suppress or modify,” the company says.

Why it matters

Sensing an opportunity to one-up its rival Anthropic, OpenAI just announced a privacy-centric safety approach to monitoring for misuse. The company is previewing a new service to select customers that it calls Private Safety Processing. This is an automated system that watches for potential abuse while simultaneously retaining none of the customer’s data. This system clearly runs counter to Anthropic’s recently announced data retention policy.

” Those models include all Mythos-class models and “future models with similar capabilities,” the company says. This policy, which was announced in July, was designed for the purposes of safety allowing the lab to sift and analyze potential impropriety. However, it has deeply concerned some enterprises that handle large amounts of sensitive data and don’t want it harbored (or inspected) by the AI lab.

OpenAI — like most other AI companies — already afford customers a relative level of privacy by adhering to a policy known as Zero Data Retention. ZDR uses agents within the OpenAI API to monitor for abuse on a per session basis. In this way, customer data isn’t retained by the company but companies are still able to scan for bad activity without the need for human intervention.

It’s worth noting that Anthropic also largely abides by ZDR — except when it comes to “covered models,” like Fable. OpenAI says that Private Safety Processing is a new technology that widens ZDR’s scope. It describes it as a form of long-horizon safety monitoring that assesses the inputs and outputs of multiple conversations — not just one.

Again, the monitoring is conducted by an agent, which, if triggered, catches interactions and analyzes them across sessions for signs of potential misuse. The new tech helps OpenAI detect malicious use of AI that takes place over multiple sessions, a spokesperson told TechCrunch. A bad actor — hypothetically someone trying to engineer malware for a cyberattack — may spread out their requests to avoid detection.

What to watch

The corporate competition between OpenAI and Anthropic is tense at the moment, with both companies looking for any opportunity to gain an advantage on the other. A recent report showed that OpenAI’s Q2 grew more slowly than Anthropic. Anthropic’s annualized revenue run rate is now reportedly $65 billion. Anthropic investors have said it could IPO at $2 trillion, while OpenAI is also working on its IPO.

When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.