Aikouka
Lifer
- Nov 27, 2001
- 30,383
- 912
- 126
A tiny bit prior to Facebook's acquisition of Oculus, I was toying around with a VR streaming concept. It came to mind after I realized that I had missed a concert that I would have liked to attend, but given that I don't keep up with music much, I didn't even know it was going on. My solution is a bit different from what we have now as I mostly see people offerring offline "VR experiences", but what I've been postulating is the most effective way to implement streaming a VR experience.
The hardest part is the sheer amount of data. Effectively, you've got a 360 degree image, but to provide a true experience, you also want to ensure that the user is able to lift their head and see up or look down. My concept was to actually have the device be sort of "dumb" and simply provide the stitched images. The real magic would occur at the server level where a user would be logged into. The server's job would be to take a portion of the stitched image and provide it to the end user, but there'd also have to be a significant portion of overlap to avoid a user moving their head and having to wait for a buffer or something. The idea was to use adaptive rendering to provide the most quality (bitrate) at the central image, and reduce the quality as the image moves further out. Also, the biggest issue there is scale. As more users are watching, the server needs to be able to handle their image. A GPU-based rendering solution may be more effective, but what would probably be required is an aggregate serving solution. The idea is that the server tracks the user's positions and attempts to combine the jobs for users with similar views. Given that this would be used for things like concerts, plays, etc., most users would stay focused at the center as it is.
I really ought to work on this, but honestly, even though I work in software, a good portion of it is out of my realm of expertise.
The hardest part is the sheer amount of data. Effectively, you've got a 360 degree image, but to provide a true experience, you also want to ensure that the user is able to lift their head and see up or look down. My concept was to actually have the device be sort of "dumb" and simply provide the stitched images. The real magic would occur at the server level where a user would be logged into. The server's job would be to take a portion of the stitched image and provide it to the end user, but there'd also have to be a significant portion of overlap to avoid a user moving their head and having to wait for a buffer or something. The idea was to use adaptive rendering to provide the most quality (bitrate) at the central image, and reduce the quality as the image moves further out. Also, the biggest issue there is scale. As more users are watching, the server needs to be able to handle their image. A GPU-based rendering solution may be more effective, but what would probably be required is an aggregate serving solution. The idea is that the server tracks the user's positions and attempts to combine the jobs for users with similar views. Given that this would be used for things like concerts, plays, etc., most users would stay focused at the center as it is.
I really ought to work on this, but honestly, even though I work in software, a good portion of it is out of my realm of expertise.