Our 2nd annual Storyscapes program, created in collaboration with Bombay Sapphire®, celebrates new trends in digital media and the filmmakers and content creators who employ interactive, web-based or cross-platform approaches to story creation. Matthew Matthew spoke to us about the interactivity of human nature, finding that perfect note, and taking to the streets—literally—to create his project, On A Human Scale.

Tribeca: Tell us a little bit about On A Human Scale. What was the impetus behind this project?

Matthew Matthew: On a Human Scale is an attempt to reimagine the people of New York City as an immersive video instrument that anyone and everyone is invited to play. Each key of a keyboard triggers a different video of a different person, from a different walk of life, singing a different note. Playing the instrument brings to life an audiovisual installation that fuses music, film, and people into a living, singing tapestry of humanity.

The idea was born as I was walking through Shanghai listening to Philip Glass’ "Einstein on the Beach." At some point the crowds of faces and the relentless repetition of the choir in his music coalesced into a really clear idea: Humanity is a vast interworking instrument and each of us is a note.  I knew immediately that I had to bring that idea to life and make an instrument of us.

Tribeca: Do you believe in the transformative power of music?

MM: No, but I believe in the transformative power of neurochemistry. The juices and electrical pulses in our brains can do amazing things, and music really gets those juices flowing and pulses pulsing. Science continues to prove that listening to and making music powerfully affects our brains and, in turn, our emotions, psyches, moods, relationships, communities, civilizations and species. There is a reason music has played a role in every known culture, past and present, around the globe since basically forever. It moves us.

As code continues to become more baked into the creative process, any story or experience that can be imagined will be able to be crafted.

Tribeca: How did you approach New Yorkers about appearing in your project?

MM: I smile, explain the project and ask them to take part. People fall into three camps. Those who just say no and walk away. Those who are instantly game the moment I mention “singing,” and those who like the idea but are terrified by singing. This third group usually requires some form of begging, peer pressure, sweet-talking and a lot of convincing that they don’t have to be a good singer. This is the truth--you don’t have to be a good singer. Since each person is only one key on the final instrument, we really only need one good note from everyone. The beauty is, even the totally tone deaf can sing at least one good note.

Tribeca: What are some of the most memorable stories of individuals you interviewed and had sing for you?

MM: We’ve sung with all sorts of people, from trained opera singers to people sleeping on the street. They’re all memorable in their own way, but my favorites are the people who are terribly reluctant and self-proclaimed tone deaf but, when they open their mouths, have beautiful voices and are right on pitch. It’s such a thrill to watch them light up.

Tribeca: Describe the technology that went in to crafting this “human instrument’?

MM: Our goal is to make the technology as invisible as possible so that people can focus on the experience, on the humanity.

But if you insist—we use Canon cameras, Apple computers, Optoma projectors, a Matrox Triple Head Pro, a 50’s Zuckerman harpsichord mashed-up with a hacked Akai midi controller and a splendid graphical programming environment called Isadora.

Tribeca: How long did it take you to perfect this process?

MM: The process is far from perfect. We definitely fumbled hard the first few times we hit the streets. Now, we really know how to talk to people, find their ranges, build their confidence, and get them singing. 

The building of the scale itself took a long time to get right, mostly because it’s hard to get every note to sit in the right place musically and visually. But after listening and watching a few thousand notes, this has gotten easier.

Tribeca: Beyond visual elements, what impacts do you see interactive instrumentation having on filmmaking?

MM: Instruments are interactive by nature. My hope is to co-opt this familiar form of interaction to lure people into an unexpected experience of wonder, harmony and fantastic humanity caught on film.

Tribeca: How do you approach creating an immersive installation? Was this approach what you always planned for On A Human Scale?

MM: Yeah, it kind of was. The idea emerged almost fully formed. There have been evolutions along the way—the way we project, the realism of the instrument itself—but they are all iterations on the central idea.

Tribeca: What is the future of On A Human Scale?  Would you like to take the project global?

MM: I daydream (and night dream) of an ever-growing and interactive musical cross section of humanity that anyone can add their voice to and anyone can play. I would love for On a Human Scale to continue to grow and to go around the planet singing and creating instruments of people from all different backgrounds and walks of life. On a Human Scale: New York and the installation at Storyscapes is the first big leap towards that dream.

Instruments are interactive by nature.

Tribeca: What are you most looking forward to at Tribeca?

MM: I’m dying to see the other Storyscapes projects come to life. The programmer, Ingrid Kopp, has done an incredible job to curate a really diverse program of innovative and daring work. I really appreciate that the installations are free and open to the public and that you can actually physically experience and play with the works.  This will be a magically groundbreaking place.

Tribeca: Where do you think the future of interactive storytelling is headed? 

MM: What makes me thrilled on a daily basis is that there are as many answers to this question as there are creators. As code continues to become more baked into the creative process, any story or experience that can be imagined will be able to be crafted. New media are being born faster than ever, and they are going to evolve in ways we can’t even picture. I think our future selves will consider the medium as a crucial part of the telling of any story.

Experience On A Human Scale in person at the Bombay Sapphire House of Imagination running April 23-26. Registration is now open!