Like many in the tech world, Sony is exploring the potential of artificial intelligence (AI). The company aims to leverage AI to enhance visuals and performance for PlayStation 5 Pro owners, a welcome development for those who invested in the $700 console. However, Sony’s simultaneous exploration of AI-driven NPCs has sparked concern, with early demonstrations appearing less promising.
A leaked video, initially reported by The Verge, showcased Sony’s integration of generative AI into a rendering of Aloy from Horizon: Forbidden West. In the video, Sony’s director of software engineering, Sharwin Raghoebardajal, interacts with an AI-powered Aloy using voice prompts. Rather than attempting to replicate voice actor Ashley Burch’s performance, the AI communicates with a noticeably robotic tone, accompanied by unnatural mouth movements.
When questioned about the Horizon storyline, the AI Aloy revealed key plot points from the first game. Raghoebardajal’s in-game interactions with the AI Aloy highlighted noticeable delays in response time. The AI utilized AI vision to interpret on-screen events, including identifying the time of day.
Ps5 Pro 10
“It’s the afternoon, and it’s quite hot under the sun,” the AI Aloy stated, following a brief pause after the prompt.
The AI Aloy’s speech was generated using OpenAI’s Whisper model, Meta’s Llama 3, and OpenAI’s GPT-4 large language model. The facial animations were created using Sony’s proprietary Mockingbird suite. This software was integrated into the Decima engine, the same engine used for the Horizon games developed by Guerilla Games.
Is AI Interaction What Gamers Want?
Raghoebardajal described the demo as “a glimpse of what’s possible,” but the reception has been mixed. The original video was removed, according to The Verge, by a company associated with Sony. Mirrors of the video can be found on platforms like Reddit. This exploration of AI NPCs isn’t entirely new; Nvidia has been experimenting with similar technology, allowing players to interact with AI-controlled characters in demos.
Major gaming companies are actively pursuing AI integration in future products, regardless of player demand. Microsoft, for instance, announced its development of Muse, a generative AI model for “gameplay ideation.” Muse created a hero shooter reminiscent of Overwatch, trained on assets from Ninja Theory’s Bleeding Edge. This raises questions about the potential of AI in game development and whether it can surpass human creativity.
AI’s derivative nature poses challenges for artistic mediums like gaming. While the AI Aloy may be a side project, it highlights broader industry concerns. Publishers have discussed using AI to replicate voice actors’ performances, potentially for cost-saving purposes, but the authenticity of such replications remains debatable. Deep learning models are often most effective when their presence is subtle. Ironically, game companies have been using AI for years in ways that predate the popularity of tools like ChatGPT.
AI Upscaling for PS5 Pro on the Horizon
Mark Cerny, the lead system architect of the PlayStation 5, informed Digital Foundry about Sony’s plans to significantly improve the PS5 Pro’s AI upscaling capabilities. The goal is to incorporate AMD’s FidelityFX Super Resolution (FSR) upscaler into PS5 games that already utilize PlayStation Super Resolution (PSSR). Cerny suggested that FSR 4 could be integrated into the PS5 Pro by 2026 as the “next evolution of PSSR.”
This initiative is part of “Project Amethyst,” a collaboration between Sony and AMD. Many of the initial games supporting FSR 4 are PS5-to-PC ports, such as Marvel’s Spider-Man 2. Sony intends to integrate PSSR with existing PS5 Pro-supported titles before implementing FSR 4 on the PS5. While previous FSR iterations were hardware agnostic, FSR 4 requires AMD’s RDNA 4 GPU architecture.
Integrating AMD’s machine learning technology into Sony’s console will require considerable effort. Cerny noted that the PS5 Pro’s processing power is comparable to AMD’s latest GPUs, making FSR 4 a natural progression for PlayStation’s upscaling technology.
AI upscaling enhances lower-resolution frames using algorithms, presenting them at a higher resolution while maintaining performance. FSR 4, the first machine learning-based version of FidelityFX, has shown improvements in performance and visuals in supported games. Sony’s PSSR is expected to function with games labeled “enhanced” on the PS5 Pro. While performance gains vary, titles like Call of Duty: Black Ops 6 can achieve higher frame rates on the PS5 Pro compared to the base PS5.
PC gaming increasingly relies on AI upscaling technologies like Intel XeSS, AMD FSR, and Nvidia DLSS. The PS5 Pro, while offering fewer graphics options than PC, will benefit from FSR’s background integration, potentially leading to improved frame rates and visual details.
Some gamers express reservations about the reliance on AI upscaling and generated frames. However, if these technologies enhance accessibility and gaming fidelity, they represent a positive advancement. Sony’s focus on AI should prioritize improving game performance and visuals rather than developing potentially unwanted features.
