Microsoft Azure is the perfect fit for new HPC requirements at the University of Nottingham

A new Master’s Degree in Bioinformatics

Every field in life sciences has been transformed by the use of computers. In recognition of this, the University of Nottingham launched a new MSc in Bioinformatics in 2021. The course enables students with a background in biological sciences to develop skills in data, developing methods and software skills in this rapidly developing field.

When setting up the course, Dr Sian Bray, Assistant Professor of Bioinformatics at the University of Nottingham, knew that the right high-performance computing (HPC) resources would need to be available for student use. 

“Although we have this kind of massive computational capability in our research domain, it’s not typically used by our teaching domain,” explains Chris Gell, Domain Architect in the University’s Digital and Technology Services department, “and we had concerns that a spikey teaching requirement that needed lots of interactivity, like Sian’s, might reduce the experience for our research colleagues.”

Sian Bray favoured purchasing physical hardware and installing that on campus, but the timescale was too short – she needed a student-ready system in place in time for the course to start in September 2021. Luckily, the University was already familiar with Microsoft Azure, having rolled out Azure Virtual Desktops during the pandemic to support remote learning and working. Creating a HPC environment in Microsoft Azure seemed like an obvious solution to enable the team to rapidly meet their hard deadline.

“Initially, I would have been a bit more comfortable buying the physical kit but, in hindsight, it wouldn’t have been as good a solution,” says Sian Bray. 

Support from Microsoft

Given the tight timescale, the University’s team leant heavily into the resources available from Microsoft. The Azure Fast Track team gave solution architecture support and the EMEA Global Black Belt HPC team provided HPC engineering support.

“Even a large institution like ours can’t possibly have such domain-specific knowledge ready off the shelf,” acknowledges Chris Gell, “so the partnership with Microsoft is very powerful.”

Although the internal team had already stood up a Linux cluster for multicore processing jobs using CycleCloud on Microsoft Azure, they still bumped into questions: how can we easily make software available for our users? What’s the best data storage option for our purposes?

“The EMEA Global Black Belt HPC team talked us through the decisions around optioning so we could understand which approach was best for a given job,” recalls Adam Cox, “Microsoft Azure gives us so much choice – so we were experimenting with scaling, different VM series and sizes, different core counts and processor architectures.”

“The Fast Track team really helped us to understand what the minimum and maximum requirements were to make it cost effective,” agrees Robyn Ward, Digital Research Technologist at the University of Nottingham.  

Scaling up within six weeks

Over the summer of 2021, the proof of concept quickly became the solution, with the University of Nottingham team continuing to iterate. However, the size of the first cohort of students wasn’t confirmed until the course started in September. 

“What hardware would we have bought?” asks Chris Gell, “What over-capacity would we have invested in by mistake?”

This presented a risk in terms of IT expenditure, so the team created a cluster with the minimum storage and compute required, with a view to adding more resources as required.

Chris Gell adds, “the Microsoft Azure solution allowed us to get what we needed, when we needed it – no more, no less.” 

Changing requirements over the three semesters

“Bioinformatics datasets are very large,” posits Robyn Ward, “and they often require a lot of space to expand into when you do the processing on them – so it’s quite a niche set of architectures and specs.”

For this reason, Sian Bray uploads datasets only as and when needed. This means that over the three semesters of the course, as well the spikes in demand during classroom time and when homework is set, there are very different storage and compute requirements from one semester to another. 

The first semester of the Masters’ Degree focuses on the taught element of computational biology, programming and practice. Students work on common datasets of plant and bacterial genomes. The second semester offers students a chance to work on different topics in rotating groups, including with leading University of Nottingham Professor of Developmental and Computational Biology, Matt Loose. 

Working with super-long reads

Professor Loose’s team currently holds the record for the longest DNA sequence read. He is working with Oxford Nanopore tech and super-long reads.

“Prof. Loose will drop me a message on Microsoft Teams to say a student needs improved performance and within 15 minutes I can have it ready for them,” states Adam Cox.

“If that was on the physical HPC, you’d have to wait for the queue to go down,” contrasts Robyn Ward. “Because it’s fair use – everybody’s using a common resource – I would usually suggest that a researcher shapes their job slightly differently. For example, run fewer cores for a longer period so it gets through the queue more successfully.” 

By avoiding over-provisioning, there’s sustainability and cost benefits

Using Microsoft Azure and Azure CycleCloud means the team can avoid the fixed parameters of the on-premises HPC environment and dial resources up and down as needed.

“The elasticity has been amazing. It’s now clear on-premises HPC wouldn’t be the best fit for this use case,” states Chris Gell, “we would have had a resource that was dormant for much of the year. That underutilised piece of kit would still have been consuming power and cooling, so switching to Microsoft Azure improves our carbon footprint. It’s a nice benefit that our choice lines up with the University’s strategic sustainability goals.

“If we’d bought hardware, we’d have needed space in the data centre and we’d need staff to run it,” adds Robyn Ward.

“That’s one of the things I love about the Microsoft Azure HPC environment,” agrees Adam Cox, “I don’t need to spend days of my life running it. It’s quick: just drop in, do what I need to, and I’m on to something else. We’re quite a small team here, so that’s really beneficial.” 

Elasticity will continue to pay off

By the third semester, the students will all be working on their own individual research projects with different tutors and their own datasets, so the storage and compute requirements will ramp up again. 

“A real benefit for me on the academic side is that the students are isolated on the cloud HPC,” advises Sian Bray, “so they can’t go in and accidentally destroy some researcher’s data, or take up all the compute time of my research colleagues.”

In the summer, at the end of the course, the students will deactivate their accounts and delete whatever data they have. “It means we can turn it off over the summer holidays, so it’s not costing us anything,” Sian Bray says. 

A low-risk environment for learning and experimenting 

In addition to efficiencies and cost savings, using Microsoft Azure with Azure CycleCloud offers additional learning opportunities for students on the course.

“With this dedicated teaching cluster, there’s a lot more learning that can happen,” explains Robyn Ward. “Using Azure CycleCloud, the student can make changes to the numbers of cores used and see what that does to the job time and benchmarking. They can iterate through ‘Where is this code inefficient? Do I need more RAM? Do I need more cores?’ in a controlled environment. They’re getting live feedback so they can see first-hand the results of their changes on the job times – and that’s delivering greater understanding of the technologies and methods.”

Sian Bray agrees, “It’s a much smoother and quicker process and there’s more learning and playing to be done.” 

The future of the MSc in Bioinformatics at the University of Nottingham

“There are jobs I’ve run that have been pleasantly fast,” continues Sian Bray, “and it really helps that it can be set up to look and feel exactly like the on-premises environment so I can use SLURM and Linux. My experience so far is all from in-house HPCs that run SLURM, so this year I’ve taught students all about that; having a cluster and requesting resources from a cluster.”

Although the 2021 course didn’t focus on teaching students about the Microsoft Azure platform, Sian Bray thinks this might change as the course continues to evolve. 

“It would be a fun student project to give them a budget and ask them how they’d provision resources within a cluster,” says Sian Bray. “It would be a good thing to add to the course in future years.”

“And Azure CycleCloud gives us a really nice framework for doing that because it’s so user friendly,” agrees Adam Cox. 

Closer alignment between teaching and cutting-edge research

The ability to try things out extends beyond the students experimenting with Azure CycleCloud. It also frees the University IT staff to build the cluster using whatever technologies they want in the future without the need for long lead times or procurement processes.

“As bioinformatics develops and as the datasets get bigger, we want that to filter down from our research to our teaching as soon as possible,” states Robyn Ward. “Being able to flex the architecture and the different techniques means we can teach the cutting-edge immediately.”

Adam Cox agrees: “If we want to add MPI; upgrade or change the storage solution; change the VM series completely; use GPU; or different core types – we’ve got complete flexibility within Microsoft Azure to do that.”

As an example, Sian Bray adds, “GPU architecture is needed to assemble some of the long reads that Prof. Loose is working with, and now that’s something we can accommodate in the future – or even next week!”

Robyn Ward agrees, “Being able to teach a real-world example from our research that we’re working on now – that’s really powerful. And it means there’s a direct link between our research and our teaching.” 

Looking to the future

As well as being able to respond quickly to changes in research and to course topics, Sian Bray is excited about the wider opportunities of the Microsoft Azure HPC environment. 

“I definitely see it being useful in the future where we get research projects where the HPC element is only a small part,” she says, “you could spin up something like this and do it fast.”

The experience over the past year has changed the way Sian Bray will approach such requirements in future. She admits, “I went into this thinking Microsoft Azure would be a holding pattern, we would do this until we could get the course established, and we would then buy the hardware but, having worked this way for two semesters, I think Microsoft Azure will be the permanent solution for us.”