University of Reading Chooses Red Hat Storage Server to Manage Climate Research Data

Logo - No Image

December 10, 2013

At the University of Reading, the Department of Meteorology needed a highly reliable, available, and scalable storage file system to manage data for its scientific research projects in weather, climate, and earth observation.

Customer: University of Reading

“I knew I’d get the help I needed from Red Hat to fine-tune Red Hat Storage Server for our academic, data-intensive HPC environment.” - Dan Bretherton, high performance computing manager, Department of Meteorology, University of Reading

Industry: Education
Geography: EMEA
Country: United Kingdom


Business Challenge:

At the University of Reading, the Department of Meteorology needed a highly reliable, available, and scalable storage file system to manage data for its scientific research projects in weather, climate, and earth observation.

Benefits:

With Red Hat Storage Server, the department now has an enterprise-grade product—backed by world-class service and support—that saves departmental IT staff valuable research time they used to spend on maintenance and administration tasks.

More
Background:

Established in 1892, the University of Reading is a leading force in British and international higher education and is ranked in the top 1% of universities in the world. It is the only UK university to offer a full range of undergraduate and postgraduate courses in meteorology. Its Department of Meteorology, which celebrated its 45th birthday in 2011, is world-renowned for excellence in teaching and pioneering research in atmospheric, oceanic, and climate science. The work conducted in the Department is playing a vital role in the improvement of weather forecasting and climate modeling.

Business Challenge:

The need to organize terabytes of valuable scientific data

The Department of Meteorology at the University of Reading is home to 60 postgraduate students and a thriving community of more than 200 research scientists, all conducting valuable work in the fields of atmospheric, oceanic, and climate science. This research is essential to understanding climate change and its impact on the environment.

In many cases, these scientists rely on access to high-performance computing (HPC) systems to create complex simulation models that help them predict likely outcomes with more precision, such as how weather patterns will move over hours, days, weeks, months, and years into the future.

For the most heavy-duty computing tasks, the Department’s researchers have access to supercomputer and HPC services provided by several of the UK’s research councils and the European Centre for Medium-Range Weather Forecasts. However, to process the data created by these external production runs, they use a local HPC cluster owned and operated by the Department and managed by its high-performance computing manager, Dan Bretherton.

“My priorities are to ensure not only that the Department can store hundreds of terabytes of research data efficiently and securely, but that good performance is maintained as the I/O load from our growing compute cluster increases,” said Bretherton.

Red Hat Storage Server: Scale-out NAS backed by enterprise-grade service and support

For several years, Bretherton had been using GlusterFS—an open source, scale-out file system for network-attached storage (NAS)—to organise and manage volumes of research data from different scientists and projects within the Department of Meteorology.

More recently, however, the growth of the Department’s HPC cluster and the amount of time that Bretherton was devoting to managing and maintaining GlusterFS made this approach unsustainable.

Solution:

What was needed, he decided, was a supported, scale-out NAS product backed by enterprise-grade service and support. After some exploration, he chose Red Hat Storage Server, a supported version of GlusterFS (which Red Hat purchased in 2011).

“When I started talking to Red Hat, I got the assurances I needed that the challenges I’d experienced with managing scale-out NAS could be solved,” said Bretherton. “One thing that gave me confidence was seeing that other organisations, including commercial ones that could lose a lot of money if their storage management wasn’t handled well, were using Red Hat Storage Server. Plus, while I had some familiarity with GlusterFS and knew that would be a help, I also knew I’d get the extra help I needed from Red Hat to fine-tune the product for our academic, data-intensive HPC environment.”

Bretherton reported that the implementation of Red Hat Storage Server in spring 2013 went smoothly. The product has run well ever since, and the addition of new users and new volumes of data has caused no significant problems.

Benefits:
 
Keeping pace with the Department's expanding storage needs
 
Today, the Department of Meteorology at the University of Reading is using Red Hat Storage Server to manage around 200 terabytes (TB) of research data. With the addition of two large new servers, the cluster’s capacity has recently risen to around 300TB.
 
Bretherton is gradually adding data from older, stand-alone servers within the Department to the product; in total, the Department hosts well over 1 petabyte of data. “If Red Hat Storage Server continues to work well for us, then eventually all of our data scattered around servers large and small could end up under its management,” he said.
 
Red Hat Storage Server’s global namespace capability gives Bretherton a simple and consistent way to organise data volumes by research group and project, and to allocate more storage capacity to volumes when required.
 
Another major benefit for the Department is high availability. “If a server needs taking down for maintenance, that doesn’t mean that work with a particular dataset needs to grind to a halt,” said Bretherton. “The replication feature ensures that every file exists on two separate servers, so the absence of one server for a period of time will be entirely invisible to the users.”
 
Saving time and money—with an eye toward the future
 
For Bretherton, one of the attractions of the scale-out NAS approach is that he can future-proof the Department’s HPC cluster—it lets him retire older servers that are reaching end-of-life and swap in new ones as needed, without ever having to shut the whole cluster down.
 
At the same time, the useful life of servers is extended when they’re used as part of the cluster. Plus, the utilisation rates associated with their storage capacity are higher, so the University of Reading can be sure it is getting the maximum return on its hardware investments.
 
Red Hat Storage Server has also helped to free up a lot of Bretherton’s time. “Before we implemented the product, I was spending up to 50% of my time managing and maintaining GlusterFS on the Department’s behalf,” he added. “Now, it’s more like 15% and that’s more manageable alongside my other responsibilities.”
 
As the Department of Meteorology grows, so will its use of Red Hat Storage Server. The University of Reading’s Academic Investment Project has recently led to 28 new academic appointments in the area of climate and environmental sciences, and that means 15 new faculty members will be joining the Department of Meteorology.
 
“New staff mean new research groups, new data sets and new storage requirements,” said Bretherton. “I’m confident now that our storage infrastructure is sufficiently stable and well organised to accommodate the new demands we will face.”

Contact Sales

Less