Is it possible to record video, depth map and 6D pose of any games? [closed]

I'm a researcher working on improving video quality in various contexts. For my current problem, I need to collect videos which have both RGB data as well as depth data. Video can be monocular or stereo. However, if there is egomotion, I'll need the camera pose (rotation and translation) as well. Is there any way to record all these information in any kind of games? Games can be on any platform: PC games, mobile games or VR games.

In short, I want to record gameplay but with additional information such as depth map and camera pose.

Games with 3D cameras can certainly provide you with position information (X,Y,Z coordinates and Angle (yaw,pitch,roll) of the camera each frame). How this is stored (format) calculated and used to render the scene to the player will be different from game engine to game engine. Thus if you want to include multiple games in your research, you may have to transform your data to be in invariant units.

For example, counter-strike-global-offensive would be a good candidate for a game that supports exposing this data to the player through the getpos console command, which you could program to run in the background in a private server context every frame, which gives you the necessary X,Y,Z,phi,psi data for the camera position and rotation.

Another good candidate is open-source games in general. Take a look at Super Tux Kart (Racing Game) or Xonotic (FPS). Since the source code is open, you can simply reprogram the game to e.g. output the information you want to a log file while it's running. Fork or Clone the game's github, modify the code to do what you want, and run your tests.

A full depth map is unlikely to be available for most games, because construcing one would essentially involve raytracing. Raytracing is very new¹ and support has only very recently been added to GPUs. I don't know of many titles that fit your criteria. Certainly GTA V and CS:GO do not have the support, they're far too old. Most open-source titles lack the manpower to implement this as it's seen as nice-to-have.

However, ID software and NVidia did a demonstrative collaboration for the recently released GeForce-2000 series cards in which quake-2 was given a Raytracing overhaul. The source code and modification is even publicly available, making it an even better fit.

Footnote

Open-source games are more conductive to research in general, because there is no corporate entity which might be interested in keeping the internal workings of a game secret, and which can go to great lengths to do so. In particular, the information you want can be useful to Cheat in various ways, so it's unlikely that any AAA game will expose it outside of a private server or Single-player context.

¹: While the technique has been known and demonstrated for a long time, hardware capabilities on desktop computers haven't been sufficient to get real-time raytracing to an acceptable performance level for gaming at high resolutions to where it is a viable alternative to the traditional render model until quite recently.