OpenAI, the pioneering AI development company behind ChatGPT, has taken swift action to address a security vulnerability that allowed its chatbot to inadvertently leak internal company data. The issue, characterized by repeated word loops, was identified and categorized as a form of service spamming and a clear violation of OpenAI’s terms of service.
The incident not only affected ChatGPT but also highlighted potential data exposure in Amazon’s newer AI model, Q.
A consortium of researchers from esteemed institutions – University of Washington, Carnegie Mellon University, Cornell University, UC Berkeley, ETH Zurich, and Google DeepMind – released a report outlining how manipulating ChatGPT to endlessly repeat a word could reveal sensitive information from OpenAI’s pre-training distribution. This encompassed confidential data such as emails, phone numbers, and fax details.
While OpenAI’s content policy does not explicitly mention perpetual loops, it sternly prohibits fraudulent activities like spamming. Moreover, the company’s terms of service strictly forbid attempts to access private information or unravel the underlying components of their AI services, including models and algorithms.
OpenAI’s proactive measures underscore their commitment to maintaining the integrity and security of their AI tools while upholding user privacy and compliance with established terms of service.