$1 Million Grant From NSF Bolsters ILLIXR as Community Infrastructure for Extended Reality Research

When Sarita Adve, Richard T. Cheng Professor of Computer Science, introduced the Illinois Extended Reality (ILLIXR) testbed and consortium in April, she knew that funding for such a unique project might be a little harder to come by than traditional research projects.

Engagement with ILLIXR elicits more support

“ILLIXR will serve as the infrastructure for testing and developing algorithms and systems for XR research – a huge emerging application domain. The benefits of this effort cannot be overstated. In addition to enabling a huge application class, this will benefit systems research in general – covering all stages of application specifics co-design – languages, architectures, implementations. This co-design is critical to maximize the efficiency of such systems and deliver their full value – and this cannot be accomplished without infrastructures like ILLIXR."

“In short, our research agenda around security for augmented reality platforms would significantly benefit from the ability to leverage and build on ILLIXR as a test bed and prototyping platform, rather than reimplementing or naively abstracting away system components that are outside the scope of our own research."

“ILLIXR addresses a critical need of XR, an infrastructure by which current XR platforms can be quantitatively measured and compared and, of equal importance, infrastructure that serves to shape the future of XR via rapid assessment of new ideas and techniques. Facebook seeks to play a significant role in the development of ILLXIR infrastructure by continual sharing of industry perspective as well as tangible contributions, such as providing real world datasets in addition to guidance on state-of-the-art XR algorithms.”

Adve explained that funding goes primarily to research, not initiatives that build the infrastructure needed to produce great research projects.

But that’s exactly what makes ILLIXR groundbreaking.

It’s a first-of-its kind open source testbed and consortium for the broad Extended Reality (XR) community, which encompasses virtual (VR), augmented (AR) and mixed reality (MR). The necessity for ILLIXR exists because of the lack of an “open-source full system benchmark and testbed to drive and evaluate innovation,” according to Adve.

ILLIXR steps into this void, but it took purposeful initiative to find the right funding opportunity for the next step. That funding came through, in the form of a $1 million community research infrastructure grant from the National Science Foundation’s Directorate for Computer and Information Science and Engineering (CISE) for a period  of 36 months.

Adve remains excited about next steps, now bolstered by the NSF grant.

“This funding provides ILLIXR the credibility and resources we need to progress in the way we envisioned,” Adve said. “Part of the purpose behind these grants is to get community involvement in the infrastructure. Reaching a much broader community speaks to our hope that not only will people use ILLIXR but contribute back to it – so that it’s a growing and living infrastructure.

“Enabling the large XR systems community to do the wide variety of research that we envision requires infrastructure that is not only functional, but also available on a large number of hardware platforms, has a high level of software quality, and is well maintained and supported. This is usually difficult to do for an academic research group. The NSF funding will provide the resources for us to hire people to do this.”

In the long term, Adve has her sight set on ensuring ILLIXR becomes a self-sustaining open source XR testbed through the support and work of the growing ILLIXR consortium.

Under the stewardship of lead researcher Muhammad Huzaifa, a graduate student at Illinois CS, the ILLIXR team has already produced a full end-to-end system.

“We can now run ILLIXR on a headset and get a full XR experience that is all open source,” Huzaifa said. “Others can take this full system, replace different components with their own algorithms, innovate new hardware accelerators, or software systems. They can co-design hardware and software and co-design different parts of an XR system such as perception and graphics. They can measure the impact of their innovations on the complete system or parts of it through its latency, energy, quality of experience – everything is open source and measurable.

“The excitement of ILLIXR is that such transparency from the entire system will enable end-to-end systems research that is not possible when researchers work in their own silos.”

Collaborators through the consortium have continued their support of ILLIXR.

The initial seeds of ILLIXR were funded and later sustained by the SRC/DARPA C-FAR and ADA research centers. At a panel at the 2021 International Symposium on Computer Architecture, panelist Tim Green – Director of Innovative Research with SRC – highlighted ILLIXR in answer to the question “How do SRC and DARPA move the needle on a problem as complex as hardware-software co-design?”

Green described the development of ILLIXR, starting from a small seed project to a centerpiece of the ADA research center, as an example of work that was “high risk/high reward” and considered impossible by many when initially funded. The new NSF grant will enable ILLIXR’s next step in “making the impossible possible” for XR research.

“We are so very happy that this kind of recognition and funding came through for ILLIXR recently,” Adve said. “We definitely want the entirety of this research community to contribute and, in essence, take ownership of this project, moving forward.”