Staff recording · EgoGaming TaskLab · Meta Quest 3S · 71.7 fps
VR Gameplay Dataset - Recorded Inside the Game
The Player’s View, Synchronized With Head and Controller Poses in Every Frame
Every clip in the free sample includes left-eye gameplay video synchronized with head and controller poses, frame by frame.
- For World Models: head and controller poses on every video frame
- For Game Studios: real players on real headsets, in your own game
- For VR Labs: several commercial games, a consent record per session
- For Robotics: head and two-controller motion, recorded to your tasks
Free 100-clip sample · RLDS and WebDataset · Pseudonymised · Players paid, consented in writingRLDS · WebDataset · Pseudonymised · Consented
The View. The Pose. One Clock.
A free sample of 100 action clips across several games, in RLDS and WebDataset, with the data card.
Explore it Yourself
This clip comes from a staff recording. The graph and the skeleton follow the video frame by frame. Play it, scrub it, or click the graph to jump.
Recorded Poses: head, left and right controller poses from the headset
Estimated from Recorded Poses: the upper body and arms drawn between them
What’s in a Recording?
Game View
The left-eye view the player saw, at the game's own frame rate.
Head and Controller Poses
Position and orientation (6DoF) for the head and both controllers for each video frame, on the same capture clock as the video, with flags where a device was lost or not reported.
Velocity and Inputs
Linear and angular velocity for each device, plus controller button, trigger and stick inputs where the capture route records them.
Tracking Quality Indicators
Device-level flags identify when tracking was lost, so you can exclude affected frames.
Hand Joint Tracking
Hand joint data is available for the subset of sessions where hand tracking was active. Other sessions include controller poses without individual hand joint data.
Dataset Documentation
Every delivery includes a data card, license terms, a session list, and checksums to verify file integrity.
Available Formats
The free sample includes the same clips in two formats. Support for LeRobot is planned—contact us if you require this format.
RLDS (TFDS)
One episode per clip (up to 60 seconds), steps at 15 Hz. Fields in observation:
image: left-eye view, 320x180 JPEGstate: head, left, right pose (px, py, pz, qx, qy, qz, qw)head_pose,left_hand_pose,right_hand_posevelocity: linear and angular, per devicetracking_flags,pose_presenthand_joints,hand_joints_validframe_index,timestamp_ns
Plus action (change to the next step), language_instruction and episode_metadata.
import tensorflow_datasets as tfds
ds = tfds.builder_from_directory(
"rlds/egogaming_vr_sample/1.0.0").as_dataset(split="train")
for ep in ds.take(1):
for step in ep["steps"].take(3):
print(step["observation"]["state"], step["action"])WebDataset
TAR archives (shards) contain one sample per clip at the original recording rate of approximately 70–90 frames per second. Each clip includes:
.mp4: H.264 video, 720 pixels high, no audio.frames.npy,.pose.npy,.vel.npy,.flags.npy: row i matches video frame i.poses.parquet,.hands.parquet,.inputs.parquet.events.json: tracking loss, recenters, state changes.json: clip metadata
LeRobot
Planned. Ask us if you need it.
How It Is Collected
- Participants are paid adults who sign up, pass a review and give written consent before anything is captured.
- They play commercial VR games they own, on PC headsets.
- Our recorder captures what the game asks the headset for, through an OpenXR API layer, or an OpenVR route for OpenVR titles.
- Every session goes through automatic checks and a human review. Very short sessions and sessions with bad tracking are rejected.
What’s Included in the Free Sample
Figures for the 100-clip sample, as of . Ask us for the current catalog.
- Clips
- Formats
Responsible Data Practices
- Privacy: Data is pseudonymized and contains no names, contact details, or locations. The license prohibits attempts to identify participants. Because VR movement data can reveal someone’s identity, all files must be treated as personal data.
- Consent and licensing: We deliver only recordings that are eligible for licensing. Each recording includes a consent record.
- Regional restrictions: Delivery follows applicable regional requirements. Accompanying documentation specifies where each recording may be distributed.
- Participant withdrawal: Participants may withdraw their consent. If they do, their files are removed from the download page. Recipients must delete all copies within 30 days and confirm deletion.
Built for Your Team
The same recordings serve different teams. Each guide explains what you get, how it fits your pipeline, and its limits.
- World model training data
Video and player actions on one clock: what the player did and how the game responded.
- VR motion data for research labs
Measured 6DoF head and controller motion from many players and games, with consent records.
- Player data for VR game studios
Real players on real headsets, recorded in your own game for bots, scoring and testing.
- Robotics and embodied AI
Head and two-controller motion recorded to your tasks, with honest limits on what game data can do.
Common Questions
What is the EgoGaming VR gameplay dataset?
Recordings of paid, consenting adults playing commercial PC VR games. Each recording has the first-person video the player saw, synchronized frame by frame with 6DoF head and controller poses, velocities, button and trigger inputs, and tracking-quality flags, all on one capture clock.
Are the head and controller poses measured or estimated?
Measured. They come from the headset's own tracking, captured by our recorder from what the game asks the headset for, not estimated from video afterwards. Every reading carries flags that show when tracking was lost.
How is the data collected, and did the players consent?
Players are adults who sign up, pass a review and agree in writing, on the website and again in the recorder, before anything is captured. They are paid for their time. Each session has a consent record, and every session is checked automatically and reviewed by a person.
Is the data anonymous?
It is pseudonymized: no names, contact details, dates of birth, locations or payment details. VR movement can identify people, so the license treats every file as personal data and forbids any attempt to identify players.
Which formats do you deliver?
RLDS (TensorFlow Datasets) and WebDataset, or plain Parquet, CSV or JSON tables with MP4 video. LeRobot is planned; tell us if you need it. Every delivery includes a data card, license terms, a session list and checksums.
Can you record our own game or specific tasks?
Yes. Custom recording lets you choose the tasks and the player mix, in your own game or in a game we are cleared to record. Custom recordings are exclusive to you for 24 months, and you pay only for hours that pass our checks.
How do I get the free sample, and what does a license cost?
Request the free sample of 100 action clips below and we email a download link with the data card. Licenses are quoted per project, depending on custom or ready-made data, annotation, volume and exclusivity.
Request Your Free Sample
Explore 100 action clips of VR gameplay data from multiple games, with an accompanying data card.
We’ll send you the download link and data card. The link remains valid for 14 days.