Twitter Sensory-Sensitive Earcons – Design

A person on the left whispers something to a person on the right, who holds their hand up to their ear to listen. In the middle is the Twitter bird logo and an illustration of a sound wave.

Twitter Sensory-Sensitive Earcons


Context

Problem

In 2021, Twitter’s CEO requested an entire new set of earcons for the platform and I was the team member responsible for designing them to be accessible.

Earcons can trigger sensory sensitivities with neurodivergent individuals, yet must be accessible for those who prefer to use them. Accessible earcons are a unique design challenge in that they have needs beyond WCAG’s sound requirements.

Summary

I created a novel type of earcon, a sensory-sensitive earcon. I designed and evaluated a set of them through qualitative interviews, design guideline creation, detailed feedback to our sound vendor, and concept testing with neurodivergent Twitter users.

By including neurodivergent representation in the design process, I designed guardrails that invited more people in. The earcons shipped in H2 2022 and the work was published at ASSETS 2022. I presented it in-person in Athens, Greece in October 2022.

Platform: iOS

Timeframe: Q2 2021 – Q2 2022

Constraints: We had no precedent for an accessibility sound design project. So, upon realizing WCAG’s limitations for this project, I had to design an entirely new way to think about sound, centering the lived experience of people with disabilities.

Role: Responsible and accountable for all accessibility design. Wrote design guidelines for and gave accessibility feedback to our sound vendor. Leveraging my accessibility research experience at the NYU Ability Project, I stepped in to conduct all generative and iterative research, and partnered with quant for evaluative research.

What happens when WCAG doesn’t apply to a design deliverable?

WCAG Criteria for Sound

  1. 1.4.2 Audio Control: Require the ability to turn off sound longer than three seconds
  2. 1.3.3 Sensory Characteristics: Make sure sound is not the only output modality
  3. 1.4.7 Low or No Background Audio: Make sure that users can separate speech from background sounds

Earcons Are a Unique Use Case

  1. They’re often shorter than three seconds
  2. They have to work for those who require sound to perceive information and have sensory sensitivities, such as individuals with multiple disabilities
  3. And they’re primary sounds, not background sounds

WCAG is the minimum; earcons still need to be thoughtfully designed.

A white wave form on a blue background, starting thin—with wave form peaks an eighth of the way in and in the middle—before the line flattens.
A sensory-sensitive earcon for Send Tweet. The sound wave of a sensory-sensitive earcon is one second or less, minimizes wavelength space between repetitive notes, has an ~ 40-50 dB intensity, ~ 1k Hz frequency, and uses a slow attack and decay with no sustain and release for a smooth sound envelope
Sensory-sensitive earcon for Send Tweet on Twitter’s consumer product

Process

Magnifying glass icon

Generative Research

With a neurodiverse team, I conducted a generative research study on Twitter using semi-structured interviews with 9 neurodivergent Twitter users to understand how to design sensory-sensitive earcons.

Clipboard icon

Thematic Analysis

I then created codes, supported codes with data, grouped codes into 6 key themes. Each theme represented a criterion for a sensory-sensitive earcon.

Itemized list icon

Design Guidelines

I used the themes to create mapped guidelines to avoid sensory sensitivities with each theme. I then provided our sound vendor, Listen, with the guidelines generated by our study to produce sound concepts.

Design icon

Designing Concepts

Working with the sound vendor, I ensured the sound design guidelines were applied correctly, while meeting Twitter brand standards. I provided detailed feedback over 4 iterations before concept testing.

Hand holding pencil icon

Crafting Final Earcons

To further craft the concepts, I synthesized feedback from concept testing with 7 neurodivergent customers to ensure their preferences were included: 1) remove repetitive notes; 2) take intensity and frequency down to avoid high-pitched notes; 3) smooth out the shape of the sound envelope.

Hand putting vote in a slot icon

Customer Voting

Through partnership with a quantitative researcher, I invited customers to vote for the final sounds used on Twitter’s consumer product, since Twitter belongs to the people who use it.


Generative Research

Recruiting

I recruited participants by writing a screener survey that we sent to 22,439 individuals to find Twitter users between the ages of 18-65, asking them their age, gender, whether they self-identified as neurodivergent, and if they had used Twitter in the past month. We received 201 qualified responses before I identified nine Twitter users (1 nonbinary, 1 trans male, 3 female, 4 male; ages 18-35) who self-identified as neurodivergent and had experience using audio notifications on their mobile devices.

Procedure

With informed consent, I conducted qualitative, semi-structured interviews via teleconferencing software. I recorded video, transcribed the audio, and stored the data on a secure cloud server. I reviewed the data and performed a thematic analysis.


Design Guidelines

Duration

Keep duration to one second or less

Repetition

Minimize wavelength space between repetitive notes to avoid percussive sounds

Sound Envelope

Use a smooth sound envelope, with a slow attack and decay, and no sustain or release

Intensity & Frequency

Keep intensity mid-range (~ 40-50 dB) and the frequency close to the human voice (~ 1k Hz)

Familiarity

Use recognizable, skeuomorphic sounds

Purpose

Only use earcons for confirming something has happened or indicating an error


Designing Concepts

Version 1

Pull-to-Refresh
The first iteration of pull-to-refresh generated by the guidelines.

My Design Feedback for Vendor: Remove the series of repetitive notes during the “refresh” and to make it sound more familiar and skeuomorphic.

Notification
The first iteration of notification generated by the guidelines.

My Design Feedback for Vendor: Reduce the intensity and frequency to avoid high-pitched notes and to smooth out the sound envelope so it was less tactile and jarring sounding.

Send Tweet
The first iteration of send tweet generated by the guidelines.

My Design Feedback for Vendor: Reduce the intensity and frequency and to soften the shape of the sharp sound envelope to avoid discomfort.

A sound wave of an earcon for send tweet that has sharp peaks and valleys and repetitive wave peaks
The 1st iteration of send tweet was high-pitched and slightly jarring.

Crafting Final Earcons

Concept Testing with Neurodivergent Customers

I wanted to make sure we incorporated feedback from neurodivergent customers in our design iterations, so I concept tested with 7 Twitter users who self-identified as neurodivergent (1 trans male, 1 prefer not to answer, 2 male, 3 female; ages 18-65). I asked participants key questions such as:

  1. On a scale of 1-10, 1 being no pain at all, 10 being extremely painful, how did this earcon make you feel?
  2. What could we change, if anything, about the sound to make it more enjoyable to listen to (duration, pitch, different instruments, sound wave shape, etc.)?

With participant feedback I narrowed down the earcons that caused the least amount of pain or discomfort with participants. I collected itemized feedback and incorporated it into guidance for our sound vendor to finalize the sounds.

Version 4

Pull-to-Refresh
The 4th iteration of pull-to-refresh with concept testing and guideline tweaks.
Notification
The 4th iteration of notification with concept testing and guideline tweaks.
Send Tweet
The 4th iteration of send tweet with concept testing and guideline tweaks.

Analyzing the Sound Envelope Shape

A sound wave of an earcon for send tweet that has sharp peaks and valleys and repetitive wave peaks
Not Sensory-Sensitive: the 1st iteration of send tweet has a sharp sound envelope with repetitive notes.
A sound wave of an earcon for send tweet that has smooth peaks and valleys and no repetitive wave peaks
Sensory-Sensitive: the 4th iteration of send tweet has a smooth sound envelope and no repetitive notes.

Customer Voting

Survey

We surveyed over 2,700 Twitter customers, who were both neurodivergent and neurotypical. I asked respondents to vote between two options for pull-to-refresh, notification, and send tweet.

Curb Cut Effect

Neurodivergent and neurotypical customers all voted for the exact same sounds in every category, producing the Curb Cut Effect. The Curb Cut Effect is rooted in the disability justice movement, when advocates requested ramps on the corners of sidewalks for wheelchair users. But then everyone started using them: people with strollers, skateboards, and rolling suitcases.

Bar charts for pull-to-refresh, notification, and send tweet that show both neurodivergent and neurotypical customers voting for the same sounds in every category.

Outcomes

Invited More People In

Designing guidelines through interviews with neurodivergent customers created guardrails that invited more people in to Twitter’s consumer product.

Created a Novel Type of Earcon

By building a neurodiverse team and including neurodivergent voices throughout the design process, I was able to navigate inapplicable WCAG sound guidelines and innovate in the accessibility space.

Published at an Academic Conference

I led and first-authored a publication for ASSETS 2022, the premier conference on accessibility and technology. I spearheaded a partnership with New York University’s Ability Project, inviting Dr. Amy Hurst, the director to co-author. I presented the work in-person in Athens, Greece in fall 2022.

Me in a mask, smiling and standing before a poster that reads: "Understanding Design Preferences for Sensory-Sensitive Earcons with Neurodivergent Individuals. I'm wearing my favorite black and white saddle oxfords.
Me, very excited to be at ASSETS 2022 in Athens, Greece

gm to those who just discovered the new pull to refresh sound on twitter and can’t stop because it’s giving you just enough serotonin to finally start your day

@arialxjade

What if it becomes normal for tech companies to design things with sensory sensitivities in mind? Wouldn’t that be lovely? ^_^

@LadyAnnalyn

Hey, Twitter has replaced the old pull-to-refresh sound AND the Tweet sound with what sounds like a bird chirp. 🐤Excellent touch! And right on brand. I love it!🐤

@SaraSoueidan
Scroll to top