OpenAI has showcased some new features in ChatGPT-4o. The update marks a substantial step forward for the AI-driven platform and is much faster and more responsive. OpenAI has improved ChatGPT-4o across text, vision and audio features while increasing the assistant to have up to five times the capacity than before.
OpenAI plans on rolling out ChatGPT-4o over time but so far, the company has released its image and text features on ChatGPT. Developers can also access the ChatGPT-4o API to test out various functions.
OpenAI showcased the voice mode in ChatGPT-4o. It now acts as a female-like voice assistant and can respond in real-time to speech while also observing the world around it. This is quite a substantial upgrade to the current voice mode which can only respond to one voice prompt at a time.
The real-time conversational speech feature can also listen in to conversations and directly translate the speech to various languages. Even prompting the feature is more natural than ever. Users can simply say “I will speak Italian and can you translate whatever I say into English while also providing the Italian response to the user”. The example can be seen below:
Wow, RIP 🪦 translators! GPT-4o changing the game, again! #ai #chatgpt pic.twitter.com/yn0nApspiO
— Derrest (@DerrestWilliams) May 13, 2024
OpenAI also showcased the ChatGPT-4o’s ability to process text and audio based on video. The upgraded feature can identify objects in the video and have active discussions with users. The latency between responses has also been improved.
At its highly anticipated live stream event, #OpenAI CTO Mira Murati shared that GPT-4o can process text, audio, and vision in one model. GPT-4o will be available for free to all #ChatGPT users. Now, the GPT-4o brings the modalities together, decreasing lag and making it… pic.twitter.com/7CZzZbo8wV
— Mashable (@mashable) May 13, 2024
There’s also a new real-time educational feature where ChatGPT-4o can identify lessons on a screen, ask the user questions and respond to their input. An example below shows this:
#OpenAISpringUpdate #ChatGPT #GPT4o
This demo is insane.A student shares their iPad screen with the new ChatGPT + GPT-4o, and the AI speaks with them and helps them learn in *realtime*.
Imagine giving this to every student in the world. The future is so, so bright. pic.twitter.com/p2LmUV3eeJ
— Frank Authority (@FrankAuthority) May 14, 2024
OpenAI shared a blog post related to Chat GPT-4o saying the new features imagine an exciting future.
Talking to a computer has never felt really natural for me; now it does. As we add (optional) personalization, access to your information, the ability to take actions on your behalf, and more, I can really see an exciting future where we are able to use computers to do much more than ever before.
