OpenAI’ll be taking several important safety steps ahead of making Sora available in OpenAI’s products. OpenAI is working with red teamers — domain experts in areas like misinformation, hateful content, and bias — who will be adversarially testing the model.
OpenAI is also building tools to help detect misleading content such as a detection classifier that can tell when a video was generated by Sora. OpenAI plan to include C2PA metadata in the future if we deploy the model in an OpenAI product.
In addition to us developing new techniques to prepare for deployment, OpenAI is leveraging the existing safety methods that we built for our products that use DALL·E 3, which are applicable to Sora as well.
For example, once in an OpenAI product, OpenAI’s text classifier will check and reject text input prompts that are in violation of our usage policies, like those that request extreme violence, sexual content, hateful imagery, celebrity likeness, or the IP of others. We’ve also developed robust image classifiers that are used to review the frames of every video generated to help ensure that it adheres to our usage policies, before it’s shown to the user.
OpenAI is be engaging policymakers, educators and artists around the world to understand their concerns and to identify positive use cases for this new technology. Despite extensive research and testing, OpenAI cannot predict all of the beneficial ways people will use our technology, nor all the ways people will abuse it. That’s why OpenAI believe that learning from real-world use is a critical component of creating and releasing increasingly safe AI systems over time.