Reflection On Process
In this short documentary interview, Olivia Ting reflects on the creative works produced through the CripTech Incubator.
Above: Debrief Interview
Artist Statement
Olivia Ting is a hard of hearing visual artist, designer, photographer, video projectionist, and pianist. She explores audial perception without hearing and the intersections of sound perception interpreted as speech, noise, and music (organized sound). Without her hearing aid and cochlear implant, she perceives nearly no sounds, so visuals stand in for the audio that she is familiar with, but hears and not hears. Formerly a pre-med major at Pomona College, she went on to a second degree in graphic design at Art Center College of Design in Los Angeles. She worked for design and branding agencies in New York City for several years before returning to San Francisco, where her work expanded to collaborative video projections with dance choreographers and museum exhibits. She received her MFA Art Practice from U.C. Berkeley and is currently freelancing and developing new work reconnecting with her music background.
Project Description
An immersive multi-channel projection, Song Without Words renders the gestural continuities between piano, musical conducting and sign language to redefine the phenomenological experience of listening. Inspired by Beethoven’s ‘Choral’ Symphony No. 9, which he composed when he was deaf, Ting’s work comprises three movements (allegro, scherzo, rondo) that layer visual and audio compositions like notes in a chord. Projected on translucent panels arrayed as hanging fragments, the videos signal an environment of splintered audial and visual cues that Deaf and hard of hearing individuals inhabit daily.
Each movement contemplates the technical and ability scripts of Ting’s performances as a pianist, conductor, signer, and listener. Referencing the vibroacoustic method that Helen Keller employed to listen to music, vintage radios introduce a haptic translation of the fourth and final movement of “Ode to Joy” from Symphony No. 9. The installation animates sound as movement, inviting gallery visitors to participate in and reflect on these choreographies and soundscapes. The instruments we listen to and with—from our fingertips to our assistive listening devices—share a special kinship.
Process
Audiovisual Installation - Using spatial projection
These initial conceptual sketches capture my idea of a space that would allow people to walk through; the element of self-navigation is important. The proportion of this space goes back to my test in my garage—long and narrow, more tube-like. The side view shows the scale and the height of these panels, and that people will be walking through it. The overhead view is to show that the angle of the panels were not all the same; some of them were tilted to allow for a certain amount of distortion of the video that's projected on them. It’s also important that these screens were intended not to be large enough to encompass the entirety of the video. They’re narrow to allow a kind of a fracturing of the video, because this fracturing is integral to the experience of hard of hearing and deaf in that we only have access to certain amount of sound at a given time, and it's a multisensory puzzling to put all these pieces back together. So this initial sketch is a navigation through how to create a sense of disorientation as a substrate on which you project video and audio information.
Haptic Development: Initial Concepts
Initially, the haptic elements of this project were envisioned as a wearable that would allow someone to feel sound on the body. This direction didn’t get pursued but was picked up in work after the CripTech incubator as part of Ting’s Touch Aesthetics Fellowship.
Audio Development: My Story
The first movement of Song Without Words plays around with captioning and a couple levels of sound perception. For example, there’s a voiceover of me reading my story, but that voiceover weaves in and out with a recording of this Piano Scribe app that transforms that same voiceover into pitches on a piano. So you're hearing the pitches, but you cannot understand because there's just not enough information to pull words out of it. I used to get asked so many times when I put on a hearing aid, “Well, you put on your hearing aid—why couldn't you understand me?” It is not that simple. Maybe people have this idea that you just flip on a hearing aid and it's like a caption; you turn on the caption and I get it. So I thought of captions. I took the captions of my voiceover, but I fuzzified and degraded the text to correlate that idea of you are seeing it, but that doesn't mean that you can always understand it. So hearing is one thing, but comprehension is a different thing. This is just two recordings: me reading my story, and that reading transformed into piano pitches by Piano Scribe.
Sorting it Out and Lip Reading
Sorting It Out and Lip Reading are choose-your-own-adventure flowcharts that visualize the complex strategies involved in understanding spoken language as a deaf person. Mapping the circular, obstructed and sometimes comical paths of listening, they reveal the active process of decoding, guesswork, and meaning-making in understanding speakers. This set includes the flowcharts as exhibited, along with an accessibility experiment that translated this experience into a screen-reader-friendly format for the web.
Example of projection and fabric setup
These images from Tip of my Tongue, an earlier work of mine, show what a fractured projected visual could look and feel like. It has the narrow panels that can’t contain the whole projection, like I used in Song Without Words. Tip of my Tongue was a dance performance; that means the dancers had choreography so they couldn’t get lost in the panels, and the audience was just watching from their seats. I wanted to take it one step further and have the panels set up in a way that the audience themselves could be navigating through them.
CripTech Incubator Exhibition, Beall Center for Art and Technology, UC Irvine
An early explorations developed in relation to the Beall Center’s architecture, this installation explores how the work could be shaped by the physical site of the exhibition. Responding to the gallery’s spatial conditions, it considers how the arrangement of projection, light, and bodies in space might create an immersive encounter for the viewer.
Video development: Visual translation of signing and conducting
These are examples of raw footage that I started with, and then the heavily digitized and manipulated video that you see later. I just wanted to make a point that they come from actual people signing and moving; these are from a human source as opposed to CGI or anything. The way the video is manipulated, I was really intrigued by the idea of echoes, or the connecting of these echos like a trail of energy. Like if you move your hand across, you disturb all these molecules—if you could see those molecules and the trails, what would that calligraphy look like? I recruited my friend, a conductor, and a dancer who is hard of hearing, so she is fluent in American Sign Language. These are both representatives of the kind of movements and languages that I wanted to explore: music conducting and American Sign Language. The source of these two different types of movements is Beethoven’s Ode to Joy Symphony. I reference Beethoven because this symphony was the last large-scale piece he composed, and he was completely deaf. This particular movement has instrumental music and vocals set to a poem by Schiller. The conductor is conducting the instrumental and the signer is signing the words, so there are two aspects of sound that make up this work.
Video development: Notations of volume and discernment
The ones like “ppp” and “mf” are dynamics notations in music. It's how loud and soft the sound is intended to be. To me, it's more about how much I can hear. The phonemic chart is a different kind of writing and notation. And the “dz June” in the yellow square, for example, made me think of how the “f” musical notation is swoopy. So I tried redesigning some of these phonemes notation in the style of music notation, according to whether the dynamics of the phoneme are loud or soft. Some of these phonemic sounds are a lot harder to hear because of their high frequency. So phonemes that I perceive as louder, I did them in the style of a forte (“f”) or a double forte (“ff”) or a triple forte (“fff”), which indicates “loud” or “very loud.” Then, in the Song Without Words video, there are all these floating symbols. I compare them to how I hear all these fragments that I have to put together, and as soon as I do, they fly off again.
Video Development: Piano as a Body
Piano as a Body is a previous work that I built on for Song without Words. For Piano as a Body, I took my piano apart and I put my camera inside, and it's like, wow. I always imagine, if you're inside the factory of sound, what would that feel like? You know, suppose that factory of sound is a musical instrument or it could have been a cochlea. I guess the idea of being in a cochlea is a tiny, tiny space, so the inside of a piano is an inversion of that. I took some of the visuals that I had made of the interior of this piano and repurposed them for the video section of Song Without Words. It makes the feeling more grounded in music, and piano. The piano felt like a friend to me; it's just a special space.
Video Development: Audiovisual Installation-Panels and Space
The video for Song Without Words was developed to be projected on a series of fabric panels as part of an experiential installation; visitors would walk through it, as opposed to seeing it on a flat surface like a film. Early on, I tested the idea in my garage using different fabrics. I tried white and black to get a feeling what the experience would be like. I liked the black ones because of how they kind of disappeared into the space. With the white, even if you dim the light, it's still quite visible, and sort of reminds you that there is a substrate you're projecting on. The black allows the illusion that the visuals are just kind of floating. I also experimented with different locations, to get a feel for different spatial experiences. My garage is long and linear, so it feels a little tunnel-like, whereas the other space I tried, UC Berkeley's Center for New Music Audio Technology, is a big square space and I could feel more spread out. I liked the more tunnel-like space because it made me think of being in a cochlea, which is a spiral shape. You can imagine it is sort of leading you and you just have to follow that tube-like space.
Video Development: Movements
Song without Words is composed of three movements: The first movement: explores the relationship between body and piano through touch, movement, and vibration The second movement: investigates disruptive hearing and missed meaning, emphasizing the gap between access and understanding The third movement: pushes beyond the sound barrier through haptic and visual translations of Beethoven’s Ninth Symphony
Haptic Development: Radios
One of the original ideas for this project was a wearable haptic object, but that did not get made due to logistic challenges and time limitations. The sonic hack and the spatially designated layers of sound were part of the wishful thinking that didn't really come to realization with the wearable haptic. In the final iteration of the installation, there were wooden radio sets that visitors could touch and "feel" vibrations that were "transcribed" from audio files via transducers attached inside the radio. Helen Keller's story, and the photo of her with her arms on a radio, has a direct link to the use of radios and the third movement video. She wrote a letter to (I believe) the New York Philharmonic saying, oh my God, this broadcast on the radio is my first "hearing" of the Beethoven's "Ode to Joy." She described how she was able to distinguish different timbres of instruments and how she could imagine that these different levels of instruments and voices would come together in this grand "Ode to Joy" performance. This radio technology allowed her to feel something that she wouldn't have before, because if she went to the Philharmonic, she wouldn't be able to have that experience.
Process Archive
image
Artist: Olivia Ting.
image
1stmvmt_projection test
Artist: Olivia Ting.
video
IMG_0176
Artist: Olivia Ting.
video
Extension of movements into gesture calligraphy
Artist: Olivia Ting.
image
Overhead view of proposed installation
Artist: Olivia Ting.
image
Justin_walkthru
Artist: Olivia Ting.
video
interior of beall
Artist: Olivia Ting.
image
Montaged with other images to form a visual conversation
Artist: Olivia Ting.
image
Extension of movements into gesture calligraphy
Artist: Olivia Ting.
image
Piano as a body
Artist: Olivia Ting.
animated_image
Cover image: Song without Words
Artist: Olivia Ting.
image
Montaged with other images to form a visual conversation
Artist: Olivia Ting.
image
Lip Reading - Interactive Flowchart
Artist: Olivia Ting.
embed (2023-12-01T00:00:00.000-05:00 -> 2023-12-31T00:00:00.000-05:00).
OliviaTing_space schematic2B
Artist: Olivia Ting.
image
Helen Keller’s connection through radio
Artist: Olivia Ting.
image
Extension of movements into gesture calligraphy
Artist: Olivia Ting.
image
Side view of installation
Artist: Olivia Ting.
image
Transducer_1
Artist: Olivia Ting.
image
IMG_0175
Artist: Olivia Ting.
video
Montaged with other images to form a visual conversation
Artist: Olivia Ting.
image
Ting McLain Reflection 03mar23
Artist: Olivia Ting.
audio
Montaged with other images to form a visual conversation
Artist: Olivia Ting.
image
Montaged with other images to form a visual conversation
Artist: Olivia Ting.
image
Set/video projection design: The Tip of My Tongue with David Herrera Performance Group at Z Space
Artist: Olivia Ting.
image
floor plan of beall
Artist: Olivia Ting.
image
Set/video projection design: The Tip of My Tongue with David Herrera Performance Group at Z Space
Artist: Olivia Ting.
image
“sonic hack” Spatially designated layers of sound
Artist: Olivia Ting.
image
3rd mvmt_Ode_to_Joy projection test
Artist: Olivia Ting.
video
OliviaTing_space schematic2C
Artist: Olivia Ting.
image
Extension of movements into gesture calligraphy
Artist: Olivia Ting.
image
3rd mvmt_3.07.23_sign+conductB
Artist: Olivia Ting.
video
Grove Haptic Motor
Artist: Olivia Ting.
image
Piano as a body
Artist: Olivia Ting.
image
Dynamic Notation Chart
Artist: Olivia Ting.
image
1st movement B_5.10.23
Artist: Olivia Ting.
video
Extension of movements into gesture calligraphy
Artist: Olivia Ting.
image
Piano as a body
Artist: Olivia Ting.
animated_image
Debrief Interview
Artist: Olivia Ting.
video
Extension of movements into gesture calligraphy
Artist: Olivia Ting.
image
My Story Piano Scribe (audio)
Artist: Olivia Ting.
My Story (audio)
Artist: Olivia Ting.
Voice comprehension flow charts (horizontal version)
Artist: Olivia Ting.
document
Montaged with other images to form a visual conversation
Artist: Olivia Ting.
image
panels_walking through projection
Artist: Olivia Ting.
video
Phonemes and dynamics . ai
Artist: Olivia Ting.
image
Extension of movements into gesture calligraphy
Artist: Olivia Ting.
image
English-Phonemic-Chart
Artist: Olivia Ting.
image
Montaged with other images to form a visual conversation
Artist: Olivia Ting.
image
Voice comprehension flow charts (Vertical version)
Artist: Olivia Ting.
document
Montaged with other images to form a visual conversation
Artist: Olivia Ting.
image
Song without Words 2nd Movement (captioned)
Artist: Olivia Ting.
Song without Words Captioned
Artist: Olivia Ting.
Credits
-
Audio technical support by Luke Dzwonczyk
-
Haptics designed by Lloyd May
-
Music conducting by Wei Cheng
-
ASL signing by Zahna Simon
-
Audio description by Social Audio Description Collective, written & narrated by Oliver Baker