World Model Training Data from Real VR Gameplay

EgoGaming sells VR gameplay recordings in which every video frame is paired with the player’s measured actions: where the head (the camera) moved, where both hands moved, and which buttons they pressed. That makes it action-conditioned data: you can train a model to predict the next frame from the current frames and the player’s actions, without inferring those actions from pixels.

Why VR Gameplay Works for World Models

• The camera motion is measured. In VR the player’s head is the camera. The headset’s tracked 6DoF pose for each frame tells you exactly how the viewpoint moved, so you don’t have to estimate ego-motion from the video.
• Actions are explicit. Both controllers’ 6DoF grip poses, plus button, trigger and stick input where the capture route records it, describe what the player did with their hands.
• Everything is on one clock. Video, poses, velocities and inputs share one capture clock, so you see what the player did and what the game showed in response, frame by frame.
• It is real human play. Adults playing commercial games they chose, not scripted bots. Every session is reviewed by a person, and idle or off-task sessions are rejected.
• The frame rate is the game’s own. About 70–90 frames a second at full rate, with a 15 Hz RLDS version for training.
• Consent is documented. Each session has a written consent record, which matters when your model and its training data face legal review.

What One Episode Looks Like

In RLDS, each clip of up to 60 seconds is one episode with steps at 15 Hz of the capture clock:

For full-rate training, the WebDataset version carries the 720p MP4 with one pose, velocity and flag row per video frame, plus the inputs and events for the clip. Full specification.

import tensorflow_datasets as tfds
ds = tfds.builder_from_directory(
    "rlds/egogaming_vr_sample/1.0.0").as_dataset(split="train")
for ep in ds.take(1):
    for step in ep["steps"].take(3):
        print(step["observation"]["state"], step["action"])

Ways to Get It

• Ready-made packages of existing recordings, with video, non-exclusive.
• Custom recording of the tasks and player mix you choose, in games we are cleared to record or in your own game, exclusive for 24 months. You pay only for hours that pass our checks.
• Annotated or unannotated. Take the recordings as captured and reviewed, or with labels added; what the labels cover is agreed per order.

How licensing works.

Limits to Plan Around

• The picture is a rendered game, not the real world. It suits game and VR world models; it does not replace real-world camera footage.
• Most hours so far come from a few games. For breadth across titles or genres, ask about custom recording.
• Video is silent by default.
• Inputs exist where the capture route records them; check the data card for each package.

Related: VR gameplay data vs. head-camera video · FAQ