Sony has taken a significant step towards enhancing accessibility for deaf gamers by filing a patent for an innovative in-game sign language translator. This technology, detailed in the patent titled "TRANSLATION OF SIGN LANGUAGE IN A VIRTUAL ENVIRONMENT," aims to bridge communication gaps by translating American Sign Language (ASL) into Japanese Sign Language (JSL) and vice versa within video games.

Sony Patents ASL to JSL Translator for Video Games

Proposed to Use VR Devices and Work Over Cloud Gaming

Sony's patent introduces a groundbreaking system designed to facilitate real-time sign language translation during in-game interactions. The technology captures the sign gestures of one language, converts them into text, translates the text into another language, and finally transforms the translated text into the corresponding sign gestures of the target language. This process allows for seamless communication between deaf gamers using different sign languages.

Sony emphasizes the importance of this technology, stating, "Implementations of the present disclosure relate to methods and systems for capturing sign language of one user (e.g., Japanese), and translating the sign language to another user (e.g., English)." The company acknowledges the non-universal nature of sign languages and the need for a system that can accurately interpret and generate sign language gestures tailored to the user's native sign language.

To implement this system, Sony suggests using a VR-type device or head-mounted display (HMD). "In some implementations, the HMD connects via a wired or wireless connection to a user device, such as a personal computer, game console, or other computing device," Sony explains. This setup allows for an immersive viewing experience within the virtual environment of the game.

Furthermore, Sony proposes that the system could operate over a network with a game server, enabling multiple user devices to interact within the same virtual environment. "In some implementations, the game server executes a shared session of a video game, maintaining the canonical state of the video game and its virtual environment," Sony notes, ensuring synchronization across all connected devices.

The integration of this technology with cloud gaming systems is also considered, where the game server would "render and stream video" between user devices. This approach could enhance the accessibility and efficiency of the sign language translation system, making it a valuable tool for deaf gamers worldwide.