Sony is seemingly experimenting with AI-powered game characters. In a video that has surfaced online, the company is seen demonstrating Aloy from the Horizon video game series as she has full conversations with the player through voice prompts during gameplay.
The video shows Aloy responding to queries with an AI-powered voice while at the same time, her facial movements do a decent job replicating what she is saying. The demo is apparently taken in-game through both a cinematic cutscene camera and while Aloy moves about the world as you would usually play the game.
According to the information, this system was based on OpenAI’s Whisper for tech-to-speech and both ChatGPT-4 and Llama 3. The systems are combined together to form conversations and decision-making. The voice creation is then based on Sony’s own Emotional Voice Synthesis (EVS) that the company has developed internally. The animations are then generated using Sony’s Mockingbird technology.
The demo is reportedly running on PC, but according to the host, Sony has been experimenting on PS5 consoles using the system too. The host claims that this is all possible on PS5 with “little overhead”.
Of course, the demo is just a technical showcase of what Sony is “experimenting” with. The company hasn’t highlighted any plans to debut this tech in upcoming games, but it does seem like a lot of investment has gone into making this system possible.
Implementing a system like this in video games would create some rather fantastic immersion, but at the same time, it would have to be a needed feature. I can’t see myself having a conversation with Aloy, or any game character, tbh – unless the game is specifically designed in such a way as to create a unique experience with this tech.
It also brings up the question of ethics. With a voice generator and facial animations that are computer-generated on the go, why would Sony need to hire animators and voice artists? It is a slippery slope.
Sony has not commented on the leaked demo. In fact, the company has issued a take-down copyright claim on the video in order to get it offline. But it has been uploaded to another platform. You can watch the full demo here.
