As AI models have become more powerful, the potential for those models to be misused has grown — as has a clamor for safety guardrails that can stop such abuse from happening. AI companies must now walk a delicate tight rope between respecting their enterprise customers’ privacy while also watching usage for possible issues.
OpenAI — like most other AI companies — already afford customers a relative level of privacy by adhering to a policy known as Zero Data Retention. ZDR uses agents within the OpenAI API to monitor for abuse on a per session basis. In this way, customer data isn’t retained by the company but companies are still able to scan for bad activity without the need for human intervention. It’s worth noting that Anthropic also largely abides by ZDR — except when it comes to “covered models,” like Fable.
OpenAI says that Private Safety Processing is a new technology that widens ZDR’s scope. It describes it as a form of long-horizon safety monitoring that assesses the inputs and outputs of multiple conversations — not just one. Again, the monitoring is conducted by an agent, which, if triggered, catches interactions and analyzes them across sessions for signs of potential misuse.
The new tech helps OpenAI detect malicious use of AI that takes place over multiple sessions, a spokesperson told TechCrunch. A bad actor — hypothetically someone trying to engineer malware for a cyberattack — may spread out their requests to avoid detection. Private Safety Processing can analyze those multiple conversations for signs of abuse without human review of a user’s conversations.
In the case where the system is triggered, it may send a “narrowly defined signal” to OpenAI that warns of a specific type of activity, the company says. Based on that signal, OpenAI can then decide whether “enforcement is necessary,” it says. If so, OpenAI will reach out to the customer for more context or to work with them on the issue and a customer may choose to share data with OpenAI at their discretion, the spokesperson said.
By contrast, Anthropic notes that human review of customer data can occur, but only “through a controlled access path” that involves “a small set of approved reviewers.” Every one of those review sessions is “recorded in a tamper-proof log that reviewers cannot suppress or modify,” the company says.