Making music with computer science

Can you imagine creating careers in computer science and music production from playing music on an iPad, or predicting viral videos on Youtube? At the Australian National University, students are using computer science music production to shape and enhance their musical ideas.

Making music with iPads

Music and apps can be a beautiful pairing in computer science music production. Just ask ANU postgrad student Charles Martin, a classical percussionist whose instrument of choice today is an iPad.

The Canberra-based computer scientist formed Ensemble Metatone – a band in which musicians perform using the iPad apps he is developing for his PhD at ANU.

“The players touch the iPad to make sounds, and we use data from their touches to understand their musical intentions,” Charles says.

“An algorithm identifies what particular gesture they’re doing at what time of the performance, and then it categorises the gestures.”

For example, the app detects whether performers are tapping, swirling or swiping their finger on the iPad, plus the frequency of their touches and what part of the screen they are focusing on.

Musicians are also asked how they felt during the performance, and all that information converges to evaluate and improve the app.

“It rewards musicians by giving them a new sound to explore.”

Predicting the future

Charles is not the only ANU researcher studying how people interact with technology in computer science music production. Lexing Xie is a research fellow using mathematical models to predict which YouTube clips will go viral.

“We use techniques similar to the ones used to spot credit card fraud or make a medical diagnosis,” she says.

“Students who learn the skills for building predictive models will have core skills that can be applied to many other data science tasks.”

Her research into designing models that predict online popularity combines computer science techniques, such as storing and analysing massive amounts of data, with the social science of how people respond and relate to videos.

Lexing is also working to improve image searches by analysing how Flickr users tag their photos. She believes that in future, computers are likely to automatically generate better tags to make images easier to find.

Media and advertisers will soon be putting images and videos through these kinds of algorithms. But they won’t be able to predict everything.

“It’s feasible to predict the top 5% of videos, but not the very top ones,” says Lexing. “Gangnam Style was one in a billion!”

Lynnette Hoffman

TO GET THERE: cecs.anu.edu.au

Which careers in computer science and music production interest you most? Take our poll or tell us in the comments!

artificial intelligence
Heather Catchpole

Author: Heather Catchpole

Heather co-founded Careers with STEM publisher Refraction Media. She loves storytelling, Asian food & dogs and has reported on science stories from live volcanoes and fossil digs