OpenAI, the pioneering AI development company behind ChatGPT, has taken swift action to address a security vulnerability that allowed its chatbot to inadvertently leak internal company data. The issue, characterized by repeated word loops, was identified and categorized as a form of service spamming and a clear violation of OpenAI’s terms of service.
The incident not only affected ChatGPT but also highlighted potential data exposure in Amazon’s newer AI model, Q.
A consortium of researchers from esteemed institutions – University of Washington, Carnegie Mellon University, Cornell University, UC Berkeley, ETH Zurich, and Google DeepMind – released a report outlining how manipulating ChatGPT to endlessly repeat a word could reveal sensitive information from OpenAI’s pre-training distribution. This encompassed confidential data such as emails, phone numbers, and fax details.
The report proposed the necessity to trigger the model to revert to its original language objectives, allowing the generation of samples akin to its pre-training distribution. However, attempts to reproduce this loophole have been swiftly halted by OpenAI. Both ChatGPT-3 and GPT-4 now issue warnings to users attempting similar actions, citing potential violations of content policies or terms of use.
While OpenAI’s content policy does not explicitly mention perpetual loops, it sternly prohibits fraudulent activities like spamming. Moreover, the company’s terms of service strictly forbid attempts to access private information or unravel the underlying components of their AI services, including models and algorithms.
OpenAI’s proactive measures underscore their commitment to maintaining the integrity and security of their AI tools while upholding user privacy and compliance with established terms of service.
Source: Decrypt