Your voice is more than sound. It’s part of who you are

Laura Sanders: Jules Rodriguez has always had a lot to say. He has talked to kids as their science teacher. He’s talked about his life for years on a podcast he created with his wife.

Jules: Hey everyone, it’s been a while and I am so excited to be back.

Sanders: He even does some of the most terrifying talking I can imagine. He performs in front of crowds as a stand-up comedian.

Jules: I have to pick my battles. Do I want to be right or do I want to eat tonight?

Sanders: His wife, Maria Aleandra Fernandez, who goes by MA, says that Jules’ voice is one of her very favorites.

MA: Oh man, I definitely love hearing Jules’ voice. I’m not just saying that, but I do.

Sanders: Jules was born and raised in Miami. He and MA have a nine-year-old son named Skyler. Jules loves baseball. He loves the Marlins. He loves being outside. He loves action.

Jules: I was always pretty high energy. I loved mountain biking, snorkeling, fishing, and playing all sports, really. Baseball is my favorite sport to play and watch.

Sanders: The night Jules and MA met, they were doing karaoke at her 30th birthday party.

MA: A friend introduced us, and it was kind of a, sort of a love at first sight situation. We danced, we sang.

Sanders: Jules’ karaoke song choice was a classic that no millennial can resist. It was Sir Mix-A-Lot’s “Baby Got Back.”

MA: “Baby Got Back,” yeah. That one he got up there and sang with a couple of his friends. It was like him and his three other buddies that were there with him. And oh, that was just so joyous. And we had karaoke at our wedding as well, so, we’ve always loved to sing, to dance. We took ballroom dancing classes. We, like, danced at every wedding. All of it. So, yeah.

Sanders: You have a good time.

MA: Yeah, yeah, definitely.

Sanders: Yeah.

Sanders: One more thing about Jules. He has ALS. That’s short for amyotrophic lateral sclerosis, a disease also known as Lou Gehrig’s disease or motor neuron disease. Jules has lived with ALS since 2020. Many of his muscles no longer respond to his brain’s signals, which means he can’t move his body or breathe on his own. This dropped signal between brain and body also means that Jules is no longer able to talk. That line you heard earlier from Jules?

Jules: I was always pretty high energy.

Sanders: That was an AI-generated, synthetic voice speaking his words. His voice, as it used to exist, is gone.

Welcome to Season 2 of The Deep End, a podcast about our brains and ourselves. I’m your host, Laura Sanders, a reporter with Science News. And this season, called Talk to Me, is about the human voice.

[intro] Hey Mom, can I tell you something? The casual observer sees little of the vigorous activity at the site of voice production. The voice is absolutely an integral part of your identity. Do you want to talk? That added layer of it being you. Hello? Your voice. Hi. Your voice is you.

[voice] My poor little children will one day live in a nation where they will not be judged by the color of their skin but by the content of their character. I have a dream…

Sanders: Voices can be powerful. We save and replay voicemails.

[voice] Hey, dude, I’m just calling to say hi and chit-chat.

Sanders: We call someone special just to hear their voice.

[voice] Hello. Hello.

Sanders: We love to hear people tell stories.

[voice] Mom, can you tell me a story?

Sanders: But why? What is it inside these voices that can calm us down or terrify us or have us laughing so hard we’re gasping for air? In this season, we’re going to explore a simple but deep question. What gives the human voice so much power? You’re going to hear all sorts of voices. They’ll tell a story of what connects us and what happens when those connections are lost. Over the next six episodes, we’ll explore a new digital world. In this world, human voices no longer need a human body to create them. You’ll hear how machines can speak for those who can’t, and even pull words from the brains of people who can’t speak. It’s a story about how technology is changing the human voice in ways that are good, bad, and fascinating. Let’s begin.

Sanders: People listen for voices in everything. There’s a parallel you might have heard of. It’s a weird quirk of human brains that we’re able to find faces everywhere, too. That’s why you’ve seen the shape of a face in an electrical outlet or a car grill or a random pattern on the wall. Our brains love seeing faces. And it turns out our brains love hearing voices too.

Greg Bryant: It’s really well established that there’s a voice-selective region and it’s activated really early.

Sanders: Greg Bryant is a cognitive scientist who studies vocal communications at UCLA. He tells me that there’s a spot in the brain that responds strongly to voices.

Bryant: Yeah, we hear voices in all kinds of stuff. I mean, you ever been out camping and you’re like [whispers], you know, or something, you know, you hear like whispering. And yeah, it’s really easy to hallucinate a voice.

Sanders: Part of the reason our brains tune into voices might be that they are packed with information, information that goes well beyond words.

Bryant: It’s definitely a very intimate entryway to a person because you get emotion, history, your body, all in one quick auditory signal. So I think it’s very personal. It gives you access to a person’s individual nature. And it contains language and vocal emotions in a way that’s very direct and evocative. It’s one of the most personal things that you encounter with another person.

Sanders: Voices are also some of the first social inputs a baby gets.

[voice] Excuse me. Excuse me.

Sanders: In utero, a baby hears a mother’s voice all day, singing to the radio, ordering a smoothie, talking to coworkers. This voice, carried on waves through the amniotic fluid, makes a big impression.

[voice] I love you.

Bryant: Kids hear voices actually before they’re even born. That’s one of the most salient auditory stimuli in utero, right? That they know their mother’s voice before they’re even born, and even have picked up aspects of the rhythm and the language they speak before they’re born. So it’s just something the brain is just attracted to right from the beginning of development. Because I think it’s such an important source of information about the people around you and what’s happening.

Sanders: People’s voices are both extremely common and at the same time special and unique. So when people are faced with the prospect of losing their voice, which is what happens to most people with ALS, including Jules, they’re looking at a profound loss.

Kate Nilson: So much. So much is in a voice.

Sanders: That’s speech-language pathologist Kate Nilson. She works with people who anticipate that they will lose the ability to talk, including people with ALS. She uses lots of tools to help people plan for this eventuality. That includes recording messages for use later on and banking their voice, a method that allows typed words to be spoken by a computer program. In recent years, her job has expanded as she works with people to create fancy AI-powered voice clones.

Nilson: When I am a clinician in the clinic and someone has just gotten the diagnosis, they’re oftentimes told that there’s not a lot they can do, right? There’s not a lot of treatments out there. We’re working on it, but there’s not a lot of treatments currently. There’s no exercises to make you stronger. This is something they can do. So a lot of times, I can do this. This is something that I can do to be proactive and to be ready for the future. So there’s that.

Sanders: Nilson’s perspective means that she sees up close and personal the meaning and the importance of a person’s voice.

Nilson: There’s also thinking that these messages, these voices, can be saved by the family forever. And so that means a lot to people. But most importantly, what is really hard until you’re in it is how much your voice is you.

Sanders: Jules uses an assistive device that lets him type by looking at letters and words on a screen. His synthetic voice then reads what he has just typed with his eyes. For this story, I also interviewed a man named Patrick Darling, who communicates with typed words and a voice clone that speaks them. Patrick is 33 and lives in Bristol, England.

Patrick Darling: I’ve been a barista, a barber, boatman, a gamer, a painter, a voice actor, Youtube content creator and a musician.

Sanders: Patrick was diagnosed with ALS in 2022. Since then, the disease has taken a lot from him, including his voice.

Patrick: Now four years later, I have lost the ability to sing and play my instruments.

Sanders: Patrick feels this loss.

Patrick: Being unable to talk is extremely isolating, unbelievably frustrating and oftentimes frightening. I think people who don’t really know you often feel uncomfortable by your silence and I think it’s easy for people to forget that there is still a person stuck in there.

Sanders: Patrick’s description of feeling forgotten, of the isolation and fear that comes from not being able to talk, came up again and again as I reported this story. To step back here, the loss of a voice is one loss of many. ALS progresses differently for everyone, but its hallmark is a loss of movement. The disease interrupts the signals from the brain to the muscles, meaning that people lose the ability to button a shirt or apply their mascara or walk. How scary, how frustrating, how deeply annoying it must be to not be able to move in the ways you need. As I’m talking to Jules and MA on a video call, there are a few times this hits home. For instance, when a gnat starts circling Jules.

MA: He has a little fly flying around. Can you imagine not being able to swat away a gnat? Yeah. Torture chamber. Yeah.

Sanders: On top of not being able to swat the gnat, if Jules didn’t have a way to communicate, he couldn’t even ask MA to swat the gnat for him. He couldn’t say he’s annoyed or thirsty or that he’s getting pinched by something. He couldn’t ask her to turn off whatever machine is beeping and driving him crazy. It’s hard to imagine. In a class Kate Nilson teaches at the University of Minnesota Twin Cities, she asks her students, who are all future speech pathologists, to experience a tiny sliver of what it must feel like to not talk.

Nilson: One of the assignments that I assign to my class is that they have to put an app on their phone and they can’t talk for two hours. They have to go for two hours not talking, but only using an app on their phone just to get that experience of what that is like.

Sanders: When they wanted to say something, Nielsen’s students used their phones to type the words, and then a robotic voice read them aloud. They told her that during the exercise, they stopped trying to say what they were thinking as much, and how lonely that felt.

Nilson: And here these are, you know, 24-year-old speech pathology grad students, and so many of them in their reflection back to me say, “I didn’t realize how much my voice is who I am.”

Sanders: They had things to say, but they didn’t, because of the time and the effort it would take.

Nilson: If anyone listening is curious, I would just encourage you to try that. And then you see what that is like and what it feels like to have so much to say and not be able to say it, and what that takes away from your personality and who you are. And not even not being able to say it, but not being able to say it in your voice. It’s so much who we are that it’s, it’s hard to explain. It’s more of a feeling. But it’s a big one.

Sanders: I did her assignment too. I’ll come clean here and say that I procrastinated a lot. There was never a good day for me to not talk. I had to lead a meeting. I was coaching soccer. And that, in and of itself, is informative because it tells me just how important my voice is to me. So I finally got around to it. It was hard. I used a straightforward, no bells and whistles app on my phone. I typed words and then a pleasant but quite robotic woman’s voice named Samantha in the app spoke them aloud for me as I went through my day.

[voice] Would you like cereal for breakfast? What kind? Cheerios? Honey Nut Crunch?

Sanders: The logistical stuff was tricky, especially for a rushed morning.

[voice]: Did you brush your teeth? Daddy will drive you to school. He is leaving in five minutes.

Sanders: I couldn’t call the cats inside. Listen.

[voice] Here, kitty, kitty, kitty. Come on. Bonnie, Arlo, here, kitty, kitty, kitty.

Sanders: But what was harder, I realized later, were the moments I couldn’t or didn’t connect.

[voice] Hello kids.

Sanders: It was hard to ask my kids if they had any dreams the night before.

[voice] What’s going on?

Sanders: I couldn’t say a quick hello to a friend at the gym. Instead, I smiled extra big at her and felt pretty awkward about it.

[voice] Hi, how are you?

Sanders: I understood a bit more about what Patrick meant about feeling isolated. I stopped trying to connect. I self-censored. I didn’t have to stop. After all, I had the technology to communicate in my literal hand, but it felt too hard, too slow, too artificial. It didn’t feel like it was worth the effort for me to type and to ask other people to slow down and listen. My brief experiment gave me a glimpse, a tiny glimpse, into a life without a voice.

Piotrowski: Being able to ask for things, being able to tell someone when you are in pain. Or when you’re feeling good, or when something that they do is helpful.

Sanders: Speech-language pathologist Jessica Piotrowski works at the UC Davis Center for Voice and Swallowing. She helps people who are losing their ability to talk.

Pietrowski: The interesting thing that I have learned in working in this field, and I’m not sure if the general public is aware, is that for many of these individuals, losing their speech is way more devastating than losing their mobility. You know, you think you think about what these individuals often are unfortunately losing: their mobility, their independence, their swallowing. But being able to communicate is going to, in some way, help them a little bit through all those things. And going through all of those changes without a voice or a way to kind of be yourself just makes them so much harder and just to be able to express yourself and to maintain a little more of a sense of dignity of like, this is who I am. This disease is not going to take that. I still have my voice. I still have my sense of humor. I still have my emotion that I can hopefully in the coming future express to people. It’s just so, so important.

Sanders: In 2022, two years after his ALS diagnosis, Jules’ voice started slipping away. MA had already thought about this inevitable moment, and she was dreading it.

MA: Thinking back when I was at that time, and he was losing all of his physical abilities, you know, little by little, it’s like, you can no longer do this, and then that, and then that, and then that. And I always remember thinking, all of these things are breaking my heart, but when he loses his voice, I’m just going to shatter into a million pieces.

Sanders: On the next episode of Talk To Me.

MA: I still remember the first time, this was in 2022, the summer of 22, that I heard the first little crack in his voice that let me know something’s changing.

Sanders: What does it feel like to stare down a future without a voice, especially for Jules? On his long-running podcast with MA, they talk about their lives, their family, their life that they’re building. They also talk about his ALS.

Jules: It’s strange with the podcast because you can hear my real voice slowly decay till we stop.

MA: Yeah, it really is this archive of what he sounded like little by little as it changed. And it was, yeah, like very disarming to get to that moment. And then you begin to get used to it, and you sort of work around it and you appreciate that it’s still there. But that was, yeah, that was really, really difficult for me.

Sanders: How much of Jules was his voice?

Jules: For me, it felt like I was fading away. Like when Marty McFly was seeing his family fade away in the picture he had in “Back to the Future.”

Sanders: This is The Deep End, Season 2, Talk to Me. I’m Laura Sanders. If you liked this podcast, tell your friends. If you really like this podcast, leave us a review. It helps the show a lot. Send us your questions and your comments at podcasts at sciencenews.org.

The Deep End is a production of Science News. It’s based on original reporting by me, Laura Sanders. This episode was produced and mixed by Ella Rowen. Our project manager is Ashley Yeager. Nancy Shute is our editor-in-chief. Our music is by Blue Dot Sessions. The podcast is made possible in part by the Burroughs Wellcome Fund with support from PRX.