Meet Brian, Senior Machine Learning Engineer on the AI Inference team

Brian D. is a senior machine learning (ML) engineer on our AI Inference team, which is part of the broader AI Engineering team at Red Hat. Based remotely in Chicago, Brian helps maintain LLM Compressor, a key component of vLLM (an open source inference server originally developed at UC Berkeley, and now supported by a global community). 

vLLM is designed to make AI inference—in other words, responses from models—more efficient. Through LLM Compressor, Brian and his team make it possible to optimize and deploy LLMs so they run faster, consume less energy, and operate on fewer GPUs, without compromising performance. 

The resulting impact? Lower computational barriers to working with AI—which, in turn, opens the door for more organizations, researchers, and innovators to use these models in meaningful ways. We sat down with Brian to learn more about his journey, his team, and life as a ML engineer.

Tell us about your journey to Red Hat and AI

I joined Red Hat in January through the acquisition of Neural Magic. I was actually in the middle of the interview process with Neural Magic when the acquisition was announced. It was great timing: my first week was the same week the entire Neural Magic team met in Boston for Red Hat’s new hire orientation. I’ve been in the AI/ML field for several years, but I’m still new to this role at Red Hat.

My career path has been a gradual shift toward AI. I started out as a computational physicist, then became a full-stack developer, a data scientist, a data engineer, and for the past few years, a ML software engineer. I'm drawn to AI because it's a fun and challenging mix of math and software. I’ve also been able to see firsthand how quickly the field is advancing, which has turned me from a skeptic into a believer.

I’ve worked at a 15-person startup, a 100-person startup, and a large bank with over 200,000 employees. Red Hat has been a really nice sweet spot for me so far. The team is super friendly and down-to-earth, and I’ve really enjoyed the work so far. There’s a lot of opportunity in this space, both in terms of projects and career growth. We have good direction and momentum.

This is also the first job where I’ve worked purely on open source code, and I don’t want to ever go back! Amidst all the hype and fears about AI, we have an opportunity to help democratize it by giving users the means to build their own AI applications, rather than relying on systems that only a select few can afford to create. It’s also a great way to grow our user base and learn from them about any blind spots in terms of documentation, bugs, or missing features.

Describe a typical day on the AI Inference team

We have a daily standup and a weekly meeting with stakeholders to review our sprint goals. We also have weekly deep-dive and design meetings for the engineering team. I have weekly one-on-one meetings with each team member and weekly lunch-and-learns with the research team to discuss new advances. My afternoons are usually kept open for focused work.

Right now, we’re focused on adding new compression algorithms to our suite of compression techniques, based on recent advancements in the research literature so that it is available to users and our research team. Earlier this year, I built a demo integrating LLM Compressor with Red Hat OpenShift AI. My teammates are focused on making LLM Compressor more user-friendly and compatible with the latest models and GPUs, and we recently introduced a new Speculators project that improves throughput by having a small model work in tandem with a larger model—a technique known as “speculative decoding.” I also try to set aside time to review code submissions from teammates and the community, address any issues on our GitHub repository, and check our community Slack channel.

Tell us what makes your role and team unique

It's definitely the mix of machine learning and software engineering. We build software that needs to run on many different platforms and environments. We also have to manage orchestrating data and computations across multiple GPUs. The compression algorithms are also getting more complex as research progresses. These are the kinds of unique challenges you might not encounter as a traditional application developer.

What's been your favorite thing to work on so far?

Given my physics background, I’m most interested in seeing how ideas from quantum machine learning can impact the latest advancements in AI. In a previous role at a quantum computing startup, we were exploring how quantum computing can augment machine learning. While the hardware and applications are still a long way from being fully realized, some quantum-inspired techniques that can run on GPUs are already being used to compress large language models (LLMs). I’m hoping to incorporate it into LLM Compressor as I get more familiar with the codebase.

How do you stay innovative and adaptable in such a rapidly changing field?

That's a great question, and I’m still figuring it out myself! I’m glad to be part of a talented team of engineers and researchers. I learn a lot from them in our lunch-and-learns, through new product features and demos, and by reading the occasional research paper. This is a great place (and a great field) to feel like a beginner and continue learning.

How would you like to see your career progress?

I’d like to slowly move into a product-focused role over time. Part of democratizing AI is demystifying how it works, and I think the role of a product engineer is really important in communicating what’s new and exciting, and how developers can use it to improve their AI applications.

What advice would you give to someone wanting to move into AI/ML at Red Hat?

If you’re looking to make a move into this space, find ways to contribute. All our code is open source on GitHub, and we tag "good first issues" that might be simple enough for a non-expert to tackle. At the very least, you’ll learn something and build a portfolio of contributions you can share with hiring managers to highlight your experience and coding style.

What are you interested in outside of work?

I’m usually mentally tired but physically restless at the end of the day, so you can often find me on a bike trail, at the gym or a yoga studio, or just out for a walk. I also enjoy reading; if a book is recommended to me, I’ll usually pick it up.

Our AI Engineering team is growing, and we’re looking for passionate technologists to join Brian  in making AI technology available and accessible to all. Learn more about the team and explore our open roles here.