• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Sony is experimenting with AI-powered PlayStation characters

Kvally

Kvally

Member

Sony is experimenting with AI-powered PlayStation characters​

Sony’s advanced technology group has created an AI-powered version of Aloy from PlayStation’s Horizon games.


Sony is working on a prototype AI-powered version of at least one its PlayStation game characters. An anonymous tipster has shared an internal video from Sony’s PlayStation group with The Verge that demonstrates an AI-powered version of Aloy from Horizon Forbidden West.
The video is narrated by Sharwin Raghoebardajal, a director of software engineering at Sony Interactive Entertainment who works on video game technology, AI, computer vision, and face technology for Sony’s PlayStation Studios Advanced Technology Group. Raghoebardajal demonstrates an AI-powered version of Aloy that can hold a conversation with a player through voice prompts during gameplay.
Aloy can be seen responding to queries with an AI-powered synthesized voice and facial movements, both in a demo setting and within the full Horizon Forbidden West game. Raghoebardajal makes it clear this is just a prototype that has been developed alongside Guerrilla Games to demonstrate the technology internally at Sony.




The technology demo uses OpenAI’s whisper for speech-to-text, and both GPT-4 and Llama 3 for conversations and decision making. Sony’s has its own internal Emotional Voice Synthesis (EVS) system that it uses for speech generation, according to Raghoebardajal, and audio to face animation is powered by Sony’s own Mockingbird technology.

While the demo is seen running on PC, Sony has also experimented with running parts of this technology directly on PS5 consoles with “little overhead,” according to Raghoebardajal. Sony first demonstrated this internally a year ago, before showing a more advanced version behind closed doors at its Sony Technology Exchange Fair (STEF) in Tokyo in November.
“This is just a glimpse of what is possible,” says Raghoebardajal. This tech demo doesn’t answer the more obvious questions over whether it even makes sense to talk to Aloy when you’re supposed to be playing as her in Horizon Forbidden West, or the impact such a technology could have on voice actors and game developers.

The AI-powered version of Aloy.


Nvidia has been working on its own similar technology for AI-powered NPCs in games, where you speak freely to video game characters. Ace, as Nvidia calls it, has been demonstrated several times throughout 2024, and the GPU maker has even worked with Inworld AI to create Covert Protocol, a playable tech demo of Nvidia’s AI game technology.

Microsoft has also been partnering closely with Inworld AI to eventually bring AI characters to Xbox, allowing game developers to use generative AI characters, storylines, and more. Microsoft has also created its own Muse AI model that generates gameplay, designed initially for game developers to create a game environment for prototyping and ideating.

There is already a lot of fear among game developers and studios about how AI could impact the creative process of making video games, particularly when so many layoffs are impacting the gaming industry. Almost half (49 percent) of 3,000 respondents to a Game Developers Conference (GDC) 2024 survey said that generative AI tools are currently being used in their workplace, with 31 percent saying they personally use them.
Sony’s experiments with AI-powered PlayStation characters are bound to generate more conversation around the role of AI in video game creation, just ahead of GDC next week in San Francisco. We reached out multiple times to Sony for a comment on this leaked internal video, but the company didn’t respond in time for publication.
------------------------------------------------

What are your thoughts on AI generating character voices and dialog? I am not the biggest fan personally, but it seem these experiments are growing in frequency. I already dislike AI search results in Google searches.
 
Nankatsu

Nankatsu

Gold Member
Ailoy.

Oh No Omg GIF
 
RoboFu

RoboFu

One of the green rats
I doubt it will be real time for real games any time soon ..not this gen at least. This demo even has a pause before she answers and its probably been rehearsed "modeled" many times before the video with the same questions.
 
Xtib81

Xtib81

Member
It's not quite there yet but it's obivous that this feature will be big in the future, by 2032-34 if I were to guess. It will make open world games much more believable.
 
Last edited:
hemo memo

hemo memo

You can't die before your death
So many questions!

Why is it being shared via an unofficial channel?

Why is it recorded on a phone, with poor quality footage and questionable audio?

Why is the setting supposedly an "office," yet lacking basic recording equipment like a dedicated microphone?

Why does this feel like a personal project, rather than an official PlayStation project?
 
  • Like
Reactions: Fbh
sendit

sendit

Member
Lets be real here. The most beneficial use case for AI driven characters is VR Pron. You would have to be a closet pervert to not agree.
 
Last edited:
fart town usa

fart town usa

Member
I picture an Aloy AI reminiscent of the MS Office paperclip. Aloy will help navigate the accessibility settings for people who are afraid of spiders.

It's important for Sony to invest money into cutting edge tech.
 
AJUMP23

AJUMP23

Parody of actual AJUMP23
It makes sense and it will lower development cost if you do not have to hire voice actors and can take some of the creativity away and just use the AI to make it more cost effective. That is what they want out of it I am sure.
 
Zacfoldor

Zacfoldor

Member
This seems like something that could be some real fun if a brave enough bloke decided to run with it. I'd love to grill Aloy about her motives.

Nintendo could probably do it best but they would be afraid of anything they can't control. I could really see Kojima doing some bold work with this here.

A Grok game with an established IP could be really awesome, but the IP owner would just need to know that the AI might take Simon Belmont places they don't want him to be. Save us Elon!
Elon Musk Smoking GIF
 
Fbh

Fbh

Member
Most big devs/publishers must be tinkering around with stuff like this.
That said is there any hint at this being official? It's a channel with 2 subscribers, shot in an amateurish way, with "horizon" and "sony" not even capitalized in the title
 
HRK69

HRK69

Gold Member
Fbh said:
Most big devs/publishers must be tinkering around with stuff like this.
That said is there any hint at this being official? It's a channel with 2 subscribers, shot in an amateurish way, with "horizon" and "sony" not even capitalized in the title
Click to expand...
It's a leak, not an official PlayStation release. Naturally, the leaker would share it through a random or newly created channel.
 
yurinka

yurinka

Member
hemo memo said:
So many questions!

Why is it being shared via an unofficial channel?

Why is it recorded on a phone, with poor quality footage and questionable audio?

Why is the setting supposedly an "office," yet lacking basic recording equipment like a dedicated microphone?

Why does this feel like a personal project, rather than an official PlayStation project?
Click to expand...
It's just an internal proof of content tech demo early prototype not intended to be shared publicly. They make as R&D tons of experiments like this one, which often end rejected and not being used in a final game.

And those that end being used in a final game get evolved a lot from these initial experiments.
 
Last edited:
Moochi

Moochi

Member
It is inevitable. Actors and voice actors don't want to be replaced, but we're looking at a new paradigm. The first AAA game that gets this right will be like magic, like being a real part of the fictional world. The emotional bonds people will form with these characters will be unlike anything we've ever experienced.

I think the way actors and artists perform their jobs will drastically change, but they will stilll be necessary. You can't authentically capture someone's charm or idiosyncrasies without capturing them in a thousand different situations. I actually think the character work for actors will increase thanks to this.
 
DownLikeBCPowder

DownLikeBCPowder

Member
Kvally said:

-------------------​


What are your thoughts on AI generating character voices and dialog? I am not the biggest fan personally, but it seem these experiments are growing in frequency. I already dislike AI search results in Google searches.
Click to expand...
I think it would be great for random NPCs or less important NPCs, generic enemies and things like that. It could factor in all kinds of things from the playthrough, such as what actions the player has taken that could be realistically known by the NPC for much more authentic dialogue, but the main NPCs or feature characters should likely remain human written for narrative purposes I'd think.
 
D

DeltaPolarBear

Member
The issue here is being held responsible for what an AI ended up saying that makes your company look bad. The limits here is that you don't really want an AI to say "everything". Because technically the AI also represents your company.

I feel AIs are better off used in private, where the user is responsible for what they made the AI do. Any other setting and the company would end up liable.
 
EDMIX

EDMIX

Writes a lot, says very little
RetroGamingUK said:
A interesting way of talking to NPCs in future RPG games.
Click to expand...
This.

Imagine an open world RPG where asking the NPCs questions can have a wild amount of answers and responses.
DownLikeBCPowder said:
I think it would be great for random NPCs or less important NPCs, generic enemies and things like that. It could factor in all kinds of things from the playthrough, such as what actions the player has taken that could be realistically known by the NPC for much more authentic dialogue, but the main NPCs or feature characters should likely remain human written for narrative purposes I'd think.
Click to expand...

I don't disagree, but I think they are just using Aloy as an example as like a proof of concept.

I 100% feel you will see this in the near future in games to fill out dialogue
 
Hudo

Hudo

Gold Member
Aren't there Skyrim mods that connect an LLM to NPCs as well? I mean, this is one of the lower hanging fruits, tbh.
 
You must log in or register to reply here.

Similar threads

People not AI will make games, PlayStation boss says
News Game Dev Platform
33
2K
mckmas8808 replied
Capcom is experimenting with generative AI to help generate the “hundreds of thousands of ideas needed for game development”
Game Dev
33
1K
Imtjnotu replied
Microsoft Says AI Will Lead To Job Losses, Invests $80 Billion In The Tech
2 3 4 5
229
11K
A.Romero replied
Booming British videogame industry under threat as it sleepwalks into AI disaster
25
2K
ReyBrujo replied
Satya Nadella: We want to be in gaming for gaming's sake. I hate to be in businesses where they're means to some other end.
Business 2 3
104
6K
Pasta la Vista replied
Top Bottom