Illinois CS students lead their own research at Lapis Labs

At the University of Illinois Urbana-Champaign, “Interest in machine learning among students is growing, but the current opportunities to join labs and research groups are becoming increasingly limited.” That’s how Ron Arel describes the impetus behind Lapis Labs, a  Siebel School of Computing and Data Science at The Grainger College of Engineering student-led academic research group created to help promote machine learning and artificial intelligence research. "We're fortunate at the university to have numerous renowned labs conducting pivotal research in the field, attracting many students eager to start their research careers and contribute to solving real problems in machine learning." 

Lapis Labs collaborates with industry and academic labs across a wide range of subfields to produce conference papers and meaningful contributions to the field. The group’s latest paper, Tamper-Resistant Safeguards for Open-Weight LLMs, was produced with collaborators from the Center for AI Safety, UCLA, Carnegie Mellon, and faculty members, including Bo Li and Dawn Song from UC Berkeley. The paper recently drew the attention of Wired magazine.

Arel and Andy Zhou started an RSO on campus called AI@UIUC. As Arel puts it“We initially focused on technical work for students with less background in machine learning, working on projects using current ML tools. As research interest grew, we created several dedicated cohorts. After publishing our first few papers and traveling to Hawaii to present some of our work, we decided to establish the research team as a separate entity with a new moniker. We wanted to give our research team its own identity and differentiate it from other university labs with very similar names. And that's how Lapis Labs started.”

Arel says Lapis Labs has published 14 papers and has been able to fly its authors to machine learning conferences where papers have been accepted. One collaborative paper with IBM Research that Lapis Labs published is FIRST: Faster Improved Listwise Reranking with Single Token Decoding. The group began a collaboration with alphaXiv in July, helping to develop a reviewer program allowing anyone to comment directly on any arXiv paper.

Rishub Tamirisa was the project lead on the "Tamper-Resistant Safeguards for Open-Weight LLMs" paper, a collaborative effort with the Center for AI Safety. Work began in 2023, and Tamirisa describes, “In early to mid-March, we had what you might call a breakthrough in the problem we were solving. We got a very promising result. One of our experiments caught the attention of other more senior researchers in the field interested in helping scale up our experiments to make the results even better.” He describes the paper’s objective as making open-source or open-weight AI models, the “whole ecosystem” publicly released AI models, “robust to what we're calling tampering attacks.” Tamirisa says “we're preventing people who were trying to harmfully adapt these models from successfully doing so.”

Arel notes that although Lapis Lab has used space in NCSA with the aid of CS faculty advisor Volodymyr Kindratenko in the past, "most of our work is done remotely, and when we collaborate, we primarily use Slack. We also like to hold bi-weekly lab-wide meetings so that all students and collaborators can stay up to date with the work being done, as well as have a platform to share ideas, discuss challenges, and foster a sense of community. At the end of the day, we function as any other university lab with its own resources, computing, and research interests.”

Zhou says, “We are planning on expanding and potentially bringing on more external collaborators from universities like Stanford and Harvard. They both have clubs on campus, but those clubs typically don't publish, so we have an opportunity to collaborate with them and bring on some undergraduates from other universities.”

Arel notes that "what we discovered is that many students from different universities are eager to pursue research, but their campus organizations don’t offer those opportunities." He concludes, “Now that we've been doing this for about a year plus, we're excited to be working with other universities and students who have the skills and are seeking opportunities to publish research in the field. So that's where we're going next.”