A new approach to streaming technology may significantly improve how users experience virtual reality and augmented reality environments, according to a new study.
A new approach to streaming technology may significantly improve how users experience virtual reality and augmented reality environments, according to a new study. The research describes a method for directly predicting visible content in immersive 3D environments, potentially reducing bandwidth requirements by up to 7-fold while maintaining visual quality.
A new approach to streaming technology may significantly improve how users experience virtual reality and augmented reality environments, according to a study from NYU Tandon School of Engineering. The research -- presented in a paper at the 16th ACM Multimedia Systems Conference on April 1, 2025 -- describes a method for directly predicting visible content in immersive 3D environments, potentially reducing bandwidth requirements by up to 7-fold while maintaining visual quality. The technology is being applied in an ongoing NYU Tandon National Science Foundation-funded project to bring point cloud video to dance education, making 3D dance instruction streamable on standard devices with lower bandwidth requirements. "The fundamental challenge with streaming immersive content has always been the massive amount of data required," explained Yong Liu -- professor in the Electrical and Computer Engineering Department at NYU Tandon and faculty member at both NYU Tandon's Center for Advanced Technology in Telecommunications and NYU WIRELESS -- who led the research team."Traditional video streaming sends everything within a frame. This new approach is more like having your eyes follow you around a room -- it only processes what you're actually looking at." The technology addresses the"Field-of-View " challenge for immersive experiences. Current AR/VR applications demand high bandwidth -- a point cloud video consisting of 1 million points per frame requires more than 120 megabits per second, nearly 10 times the bandwidth of standard high-definition video. Unlike traditional approaches that first predict where a user will look and then calculate what's visible, this new method directly predicts content visibility in the 3D scene. By avoiding this two-step process, the approach reduces error accumulation and improves prediction accuracy. The system divides 3D space into"cells" and treats each cell as a node in a graph network. It uses transformer-based graph neural networks to capture spatial relationships between neighboring cells, and recurrent neural networks to analyze how visibility patterns evolve over time. For pre-recorded virtual reality experiences, the system can predict what will be visible for a user 2-5 seconds ahead, a significant improvement over previous systems that could only accurately predict a user's FoV a fraction of a second ahead. "What makes this work particularly interesting is the time horizon," said Liu."Previous systems could only accurately predict what a user would see a fraction of a second ahead. This team has extended that." The research team's approach reduces prediction errors by up to 50% compared to existing methods for long-term predictions, while maintaining real-time performance of more than 30 frames per second even for point cloud videos with over 1 million points. For consumers, this could mean more responsive AR/VR experiences with reduced data usage, while developers can create more complex environments without requiring ultra-fast internet connections. "We're seeing a transition where AR/VR is moving from specialized applications to consumer entertainment and everyday productivity tools," Liu said."Bandwidth has been a constraint. This research helps address that limitation." The researchers released their code to support continued development. Their work was supported in part by the US National Science Foundation grant 2312839. In addition to Liu, the paper's authors are Chen Li and Tongyu Zong both NYU Tandon Ph.D candidates in Electrical Engineering; Yueyu Hu, NYU Tandon Ph.D. candidate in Electrical and Electronics Engineering; and Yao Wang, NYU Tandon professor who sits in ECE, the Biomedical Engineering Department, CATT and NYU WIRELESS.Cinematography techniques can significantly increase user engagement with virtual environments and, in particular, the aesthetic appeal of what users see in virtual ... A research team recently invented a novel, wireless, skin-interfaced olfactory feedback system that can release various odours with miniaturized odor generators . The new technology integrates ... Listening to music could reduce the dizziness, nausea and headaches virtual reality users might experience after using digital devices, research suggests. Cybersickness -- a type of motion sickness ... A new report could help improve how immersive technologies such as Virtual Reality and Augmented Reality are used in healthcare education and ...
Virtual Environment Energy Technology Technology Virtual Reality Computer Graphics Video Games Computer Modeling
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
What channel is New York Yankees vs. Detroit Tigers game today? (3/21/25) | Time, TV channel, FREE LIVE STREAThe New York Yankees face the Detroit Tigers in an MLB spring training game on Friday, March 21, 2025 (3/21/25) at Ed Smith Stadium in Sarasota, Florida
Read more »
In Ohio, driver’s ed can be pricey and scarce. DeWine wants to put classes back in schools, but is that realiGov. Mike DeWine has proposed spending $50 million over two years to get driver training back in schools.
Read more »
'Gilmore Girls' Star Reveals She Gets 'No Residuals' From Netflix Streaming SuccessGilmore Girls star reveals she gets no residuals from Netflix&039;s streaming success.
Read more »
Fashion designer Rebecca Minkoff reveals ‘floozy clause' in will with husband: ‘No chance of getting anything’Rebecca Minkoff stopped by 'Virtual Reali-Tea' to fill us in on her latest endeavour: the release of her book 'Fearless' in paperback! We of course also had to chat about all things 'The Real Housewives of New York City.' where she filled us in about where her head was with the ladies after the explosive reunion.
Read more »
Eight-Year-Old Gets 3D Printer, Internet Unprepared for the Result'This hands-on approach helps develop problem-solving skills, spatial reasoning, and creativity,' an expert told Newsweek.
Read more »
This Crazy Instrument Lets Us Hear How Dinosaurs Might Have SoundedUsing 3D models of ancient skulls, Dinosaur Choir gets us closer than ever to understanding the noises that dinosaurs made.
Read more »