Most celestial objects—from stars and nebulas to quasars and galaxies—emit light at a range of wavelengths. Some include visible light, which is how astronomers are able to photograph them with space telescopes like Hubble. But the James Webb Space Telescope and the Chandra X-ray Observatory peer at heavenly objects in infrared and x-ray wavelengths that are invisible to the human eye. That data is often translated into visible colors to produce spectacular space images. Now, a group of astronomers is making those images accessible to a wider audience that includes visually impaired people—by turning the data into almost musical sequences of sounds.
“If you only make a visual of a Chandra image or another NASA image, you can be leaving people behind,” says Kim Arcand, a visualization scientist who collaborates with a small, independent group of astronomers and musicians on a science and art project called SYSTEM Sounds. Arcand, who describes herself as a former choir and band geek, is also the the emerging tech lead for NASA’s Chandra observatory. Until a few years ago, this meant activities like adding sound to virtual- and augmented-reality science outreach programs. Then, along with a few others who became the SYSTEM Sounds group, Arcand began converting x-ray data into audio. “We have had such a positive response from people, both sighted and blind or low vision, that it’s the project that keeps on giving,” she says. Today, the group also works with NASA’s Universe of Learning, a program that provides science education resources.
Visual images from the JWST or Chandra instruments are artificial, in a sense, because they use false colors to represent invisible frequencies. (If you actually traveled to these deep-space locations, they’d look different.) Similarly, Arcand and the SYSTEM Sounds team translate image data at infrared and x-ray wavelengths into sounds, rather than into optical colors. They call these “sonifications,” and they are meant to offer a new way to experience cosmic phenomena, like the birth of stars or the interactions between galaxies.
Translating a 2D image into sounds starts with the image’s individual pixels. Each can contain several kinds of data—like x-ray frequencies from Chandra and infrared frequencies from Webb. These can then be mapped onto sound frequencies. Anyone—even a computer program—can make a 1-to-1 conversion between pixels and simple beeps and boops. “But when you’re trying to tell a scientific story of the object,” Arcand says, “music can help tell that story.”
That’s where Matt Russo, an astrophysicist and musician, comes in. He and his colleagues pick a particular image and then feed the data into sound-editing software that they’ve written in Python. (It works a bit like GarageBand.) Like cosmic conductors, they have to make musical choices: They select instruments to represent particular wavelengths (like an oboe or flute, say, to represent the near-infrared or mid-infrared), and which objects to draw the listener’s attention to, in which order, and at which speed—similar to panning across a landscape.
They lead the listener through the image by focusing attention on one object at a time, or a selected group, so that they can be distinguished from other things in the frame. “You can’t represent everything that’s in the image through sound,” Russo says. “You have to accentuate the things that are most important.” For example, they might highlight a particular galaxy within a cluster, a spiral galaxy’s arm unfurling, or a bright star exploding. They also try to differentiate between a scene’s foreground and background: A bright Milky Way star might set off a crash cymbal, while the light from distant galaxies would trigger more muted notes.
In their latest releases, the team sonified images of a galaxy group called Stephan’s Quintet, as well as of the Sombrero Galaxy (also known as Messier 104) and the variable binary star R Aquarii, which is found in the Aquarius constellation. They used imagery from the JWST, Chandra, Hubble, and NASA’s now-defunct infrared Spitzer Space Telescope.
Stephan’s Quintet resides 290 million light-years from Earth and includes five galaxies, four of which are dancing close together. They’re in the process of flying past each other, disturbing their swirling shapes and stretching out their spiral arms. Images reveal clusters of newly forming stars, and a few spots with stars and dust clouds being pulled away from their host galaxy by the gravity of their neighbors. “We wanted to hear the five members of this quintet. We wanted to hear their relative positions and sizes, but we also wanted to give someone who’s just listening to the image an aesthetic experience of the textures and colors in it,” Russo says. Once they chose their image, he continues, “We decided to scan from the top downwards and to let the brightness in the image control the frequencies of the tones you’re hearing.”
They picked a glass marimba, with its softer sound, to represent infrared wavelengths, and a synthetic violin-like string instrument, with a harsher and brighter sound, for x-rays, so that it’s easier to distinguish the two by ear. As you listen, the first galaxy arrives on the scene, surrounded by the sounds of the distant galaxies behind it. Suddenly its neighboring galaxies emerge, and the cosmic symphony crescendos. Then it gradually tapers off, returning to the fluctuating cacophony of numerous background objects. (You can listen by clicking the player below or following this link.)
The team has sonified other astrophysical data as well, including the gravitational wave detections of merging pairs of black holes and neutron stars, and a topographic map of impact craters on the moon. (You can find all of the group’s sonifications here.)
This effort is “an amazing step toward inclusion and access,” says Christine Malec, a blind astronomy buff and longtime musician who became a consultant for SYSTEM Sounds after hearing Russo present some sonifications at a Toronto planetarium. She provides feedback to the group, like whether something in a composition is effective or confusing, or whether there’s something she’d like to hear more or less of. “It’s a profound experience to me, because I can’t look up at the night sky and get other sensory experiences of the cosmos,” she says. “When I listen to a sonification and really try to understand what I’m hearing by reading the explanations, it’s engaging in a visceral way that just reading about things isn’t.”
Malec thinks that these works could be used for educational purposes as well. For example, she says, there’s a lot to learn from the sonification of data from TRAPPIST-1, a solar system with seven known planets that move in resonant orbits, meaning that their orbital periods form ratios of whole numbers. (For every two orbits of the outer planet, the next one inward orbits thrice.) That sonification isn’t actually a translation of a pixelated image. Instead, it turns the planets’ orbits into sound, with a piano note representing each one. The composition starts with the outermost planet and adds one planet at a time. It also uses different drums to signal when each planet passes its outer neighbor, showing the rhythm of their gravitational influences, so that it ends with seven piano notes and six drums.
Alicia Aarnio, cofounder of the American Astronomical Society’s working group on accessibility and disability, says that the astronomical community should accept sonifications as legitimate scientific tools. The human eye is valuable for making classifications of objects in space photos, picking out sets of features that computer algorithms can’t do well yet. But using multiple senses can be helpful; ears are sensitive to changes in pitch, just as eyes sense changes in brightness, says Aarnio, an astronomer at University of North Carolina Greensboro. Sonifications have, in fact, already been used for research, including by the astrophysicist Wanda Díaz-Merced, who has been blind since her 20s and now works at the European Gravitational Observatory in Cascina, Italy.
SYSTEM Sounds isn’t the only group trying to make the cosmos audible. A group of researchers at UCLA and NASA have translated space weather signals into sound. In a recent study, other astronomers described sonification software they are developing called Astronify, though it’s designed for 1-dimensional data like light-curve data and spectra, not the 2D images typically used by SYSTEM Sounds.
The SYSTEM Sounds team has surveyed thousands of sighted, low vision, and blind people who listened to their sonifications of space images, and is about to submit a study for peer review showing that the overall response was positive, with people saying the audio pieces made them feel relaxed—but also curious and interested in space science. “When you have deep-space esoteric scientific data of things that sound super abstract—like exploding stars, colliding galaxies, and galaxy clusters—sonifications can bring them down to Earth in a very practical and emotion-driven way,” Arcand says.
