5/26/2023
University of Illinois launches IMMERSE to enable computing's next frontier
A workshop at the Siebel Center for Computer Science introduced IMMERSE: Center for Immersive Computing to bring together all campus activity in immersive technologies, applications, and human factors.
When the University of Illinois Urbana-Champaign announced fiscal year 2023 funding through the Investment for Growth (IFG) program, Illinois Computer Science professor Sarita Adve was excited to learn of the $4 million investment to seed development of her team’s proposal for IMMERSE: Center for Immersive Computing.
Considering the purpose that Adve has drawn out for IMMERSE, it’s easy to see why it appealed to many as an option for campus financial support through IFG – which has helped to launch several of our university’s most transformational initiatives.
Alongside three associate directors – Klara Nahrstedt, Grainger Distinguished Chair in Engineering; Eric Shaffer, Illinois CS professor; and Romit Roy Choudhury, professor with Electrical & Computer Engineering (ECE) and CS affiliate faculty member – Adve has designed IMMERSE to draw upon the collaborative and interdisciplinary work in this area already available throughout the campus.
Together, IMMERSE will help Illinois push the boundaries of what the directors believe is computing’s next frontier and its transformative impact on most human activities. Immersive computing has the potential to transform medicine, science, education, industrial design, manufacturing, maintenance, arts, retail, advertising, social interactions, and more.
“We define immersive computing as the seamless integration of the physical and the virtual,” Adve said. "We include virtual reality (VR), augmented reality (AR), mixed reality (MR), extended reality (XR), the metaverse, spatial computing, and digital twins. It is an inherently interdisciplinary field, and our center brings immersive technologies, applications, and human factors all together into one effort right here on our very own campus – including the research, education, and infrastructure activities integral to this effort.”
Not only does Adve believe that now is the right time for this effort, but she also believes this is the right university.
“Considering our history of innovation and our place in previous transformative eras of computing, it’s not surprising that we are taking this step toward immersive computing,” Adve said. “As we’ve developed IMMERSE, though, what was surprising to me was the scale at which this has been happening here already and for how long.”
Romit Roy Choudhury
ECE professor, CS affiliate faculty member
“One of my research threads is focused on synthesizing and playing audio — say from an earphone — that seems real to humans. The human brain is a sophisticated signal processing machine, so if one must ‘convince’ the brain, then the synthesized signal must model various minute aspects of reality. This is at the heart of immersive audio. But shaping and moving forward into an emerging area requires a spectrum of capabilities with breakthrough science on one end, and collaborative innovation on the other. IMMERSE appears to have many of these ingredients, which excites me because it represents a path forward to achieve the desired output.”
Klara Nahrstedt
Grainger Distinguished Chair in Engineering
“What excites me most working within IMMERSE is the rich collaboration with other researchers across the campus and the opportunity to bring immersive computing to a much higher level of ubiquity and utility. Current immersive technologies are part of the fourth wave of computing, considering the first wave started early in the 1990s, and IMMERSE has the great potential to move this wave toward higher utility for users in many domains and in a holistic manner. For my research, IMMERSE represents an opportunity to explore multi-modal immersive systems and networks, so that users experience high quality of immersion and truly active perception and actuation in immersive spaces.”
Eric Shaffer
Illinois Computer Science professor
“Immersive computing can enable people to see the formation of magnetic fields, the structural load paths in buildings, and how mass warps the shape of space. All those things are happening in classrooms on our campus right now, but there are significant questions that still need to be answered. The goal of the IMMERSE Center is to bring together people and resources to answer these questions and realize the full potential of the technology. It’s also incredibly exciting that IMMERSE is about applications as varied as teaching people to dance or surgical planning. This represents a truly multidisciplinary endeavor that spans all parts of the campus.”
At Illinois, Adve said, redefining what’s next is “in our DNA.”
This includes Adve’s own research lab, ILLIXR, that performs extended reality (XR) hardware and systems research and maintains the first of its kind end-to-end open source extended reality (XR) system used in industry and academia.
Recent hires in haptics and computer vision complement existing research in these and other immersive technologies such as graphics, audio, generative AI, robotic telepresence, networking, distributed systems, hardware, programming systems, and security and privacy.
Adve also noted that the campus offers many classes that use XR capabilities. A long running effort is a class from ECE in electromagnetism that has been using VR capabilities.
The Carle Illinois College of Medicine has enthusiastically embraced VR technology to train physicians and to help prep and deliver care in the surgical setting.
These efforts and others on campus offer real world testbeds to see the limits of today’s technology, inspiring research and evaluation methods for needed advances.
The Siebel Center for Design is poised to begin an immersive experiences certificate that will include courses based on design thinking for immersive experiences and provide prototyping facilities, complementing research from Industrial Design and the Colleges of Education, Media, and others on assessing the goodness of immersive systems.
The Grainger Engineering Library’s IDEA Lab has been providing instruction and consultation services as well as state-of-the-art facilities and equipment to faculty, students, and staff seeking to apply immersive technologies into their research and instruction.
The Studio, which stems from the Game Studies and Design undergraduate minor founded within the School of Information Sciences, is a student-driven outlet dedicated to innovative and unique methods for immersive applications development while following game industry standards.
Gies College of Business trains students to approach technology in their discipline and has an exciting “disruption lab” program embracing immersive technologies.
Adve also mentioned the National Center for Supercomputing Applications’ long history with VR through the second ever CAVE deployment and award-winning visualizations. And here at CS, professor Steve LaValle’s experience as founding chief scientist for Oculus VR led to the company’s acquisition by Meta for $2 billion and a consumer VR revolution.
IMMERSE is harnessing all these efforts, seeking to catalyze the intellectual inspiration needed to advance the field through research, education, and physical and virtual infrastructure.
That process took its next step on Monday, May 15, when IMMERSE leaders led a workshop for faculty and staff at the Siebel Center for Computer Science. University leaders like Rashid Bashir, Dean of The Grainger College of Engineering, and Susan Martinis, Vice Chancellor for Research & Innovation, joined the conversation.
“(IMMERSE) is very exciting and very timely,” Bashir said at the workshop. “In important areas, we can’t always wait for external funding to get started. Innovative problem solving is what our campus and Grainger Engineers are known for, and it’s what we do well.”
“Commitment from the university isn’t just about funding; it’s also about driving impact. I can’t wait to see the impact of immersive computing– the technologies, the education, the policies and the applications.”
About 80 faculty and staff members from across campus attended the workshop. Speakers represented campus outlets such as Applied Health Sciences, the Carle Illinois College of Medicine, the College of Education, Gies College of Business, Fine and Applied Arts, the College of Liberal Arts and Sciences, the iSchool, and more.
Adve said IMMERSE leaders’ goals for the workshop included bringing together the relevant community from across campus, identifying and enabling key stakeholders to influence the long-term direction of the effort, and begin creating the vision document central to their growth and development.
On all accounts, the workshop was deemed a huge success, energizing the community for the next steps, one of which will be a public-facing event in the fall to further collect relevant and impactful participation.
Illinois CS Department Head and Abel Bliss Professor of Engineering, Nancy M. Amato, spoke at the workshop.
Her words echoed what many believe will be possible through IMMERSE moving forward.
“Now really is the right time for IMMERSE,” Amato said. “This is exhibited so well through the technical capabilities of someone like Sarita, a computer architect, leading this charge. That’s because much of our XR work in the past hasn’t truly composed the full stack from applications to systems. This team that Sarita has built does indeed have that.
“That’s why I think IMMERSE is the future of computing, because we are going to see computing embedded in our world. We must be mindful to do it in a way that’s considered assistive to the common good. We should be thinking about that beginning today, as we gather here to envision what we can do at IMMERSE.”