Stewart Builds a Better Bridge from Deep Learning and Geospatial Data to Earth Science Applications

Now in his sixth and final year of the Ph.D. program with Illinois Computer Science, Adam Stewart knew he was not the conventional student entering this stage in his academic career.

Rather than having a background in CS, Stewart previously studied Earth Science at Cornell University. His primary motivation for research here during his Ph.D. centers on the impact artificial intelligence (AI) can have on environmental issues.

“We’ve seen major advancements in many areas of computing, but I don’t want to stop at technological advancements,” Stewart said. “I want to see what kind of impact they can make in other domains, which means I’m focused on a lot of different questions. Can we use techniques from computer vision and deep learning to tackle climate change and air pollution? Can we improve digital agriculture and precision farming using satellite imagery? Can we predict earthquakes, tsunamis, and volcanic eruptions before they happen and save countless lives?

“These are very different applications involving prediction and forecasting. It takes a ton of data, very complex data, which can be challenging to work with.”

The years he has spent researching items in accordance with these overarching goals produced a paper titled, “TorchGeo: Deep Learning With Geospatial Data.”

Stewart paired with his Ph.D. advisor – Illinois CS professor, Arindam Banerjee – and four other investigators from the AI for Good Research Lab at Microsoft and the University of Texas at San Antonio. Together, they addressed how the variance in data collection methods and handling of geospatial metadata make the application of deep learning methodology to remotely sensed data nontrivial.

The group developed TorchGeo – a Python library for integrating geospatial data into the PyTorch deep learning ecosystem – to solve this problem, and their paper outlined the design, implementation, and use of TorchGeo.

The impact this can have on the serious matters of Earth Science that inspire Stewart were meaningful enough for the group to earn a Best Paper Runner Up Award at the 30th ACM SIGSPATIAL Conference in November.

“The inspiration for TorchGeo was based on a notion I’ve gathered given my experience working with this data and machine learning. I have found that it is extremely difficult to work in this field unless you have a Ph.D. in both remote sensing and computer science,” Stewart said. “There are very few researchers who have expertise in both of these very different fields. So, the goal was, can we make this simpler?

“We know that small datasets are not going anywhere. They're not going to get bigger; we can't just throw money at the problem. Instead, we need to be able to train models on small data sets. Accordingly, we’ve focused on model pretraining and transfer learning.”

To get to this point – both with his paper, and bigger picture, with the confidence to take on such a topic – Stewart said that his advisor played a key role.

This is primarily because Banerjee’s research interests coincide so well with his own.

Without a pure CS background, Stewart said there were moments he succumbed to imposter syndrome. But finding a home for his unique research interests alongside Banerjee proved to be a moment that altered his path.

“It’s been great to work with someone who is passionate about the same things,” Stewart said. “Arindam has done a lot to champion my research, tell people about it, brag about it, and help connect me with interdisciplinary collaborators.”

His advisor, however, was quick to credit the student, whom he lauded for his efforts in the Ph.D. program and with this paper.

“The transition Adam has undertaken is quite remarkable. He has gone far beyond just transitioning to CS; he is among the best in software engineering and building large-scale, production-quality systems. That, combined with his expertise in AI and Geosciences, make him an asset to any team interested in research and system building at this intersection,” Banerjee said. “Some of us in the AI/ML community have been making a case for more focus on scientific and societal applications for over a decade.

“Adam gets full credit for the work he is doing, as I would have hesitated to pursue this direction because of how ambitious the TorchGeo project is. Credit goes also to his Microsoft collaborators and mentors, especially Caleb Robinson, for helping Adam bring this amazing project to fruition.”

The result proved both rewarding to Stewart while simultaneously serving as inspiration for what comes next in his academic efforts.

“Winning a runner-up award like this, and even just getting this publication out, was very validating for me,” Stewart said. “I think it really goes to show that, yes, this field is moving in a direction where people really do value the application of cutting-edge technology to real world phenomena that we desperately need to solve.”