Help Create the “Koko App”

Intro      Design Features     The Gorilla Foundation      The Ideal Partner     Contact Us

The Gorilla Foundation is looking for the right tech partner to help us develop the Koko App — an educational tool designed to apply interspecies communication (via sign language) to save gorillas from extinction, improve their lives in captivity, continue our ground-breaking dialogue with great apes,  and inspire children of all ages to become scientists and voices for Nature.  Please watch the following video to learn more, read the details below the video, and contact us if you’re interested in being a partner, by emailing [email protected].

 

 

Who is Koko and What is the Koko App?

 

Koko is perhaps the world’s most famous gorilla.   Known mainly for her ability to communicate in sign language (she learned over 1000 signs, and understood even more spoken words), and her gentleness with (and empathy for) other species (such as cats, birds, butterflies and humans)  she almost single-handedly changed the paradigm of gorillas from “King Kong” to “Koko’s Kitten.” 

 

Though Koko passed away in 2018, our 5-decade interspecies communication study with her produced an immense multimedia research archive.  The Koko App is designed to apply this rich archive to benefit future generations of humans and other great apes.  In particular, it will enable humans and gorillas to learn sign language from Koko herself (an inspiring teacher) and to apply this language (which is valuable in its own right) to interpret Koko’s videos and books in ways that will expand interspecies empathy, promote conservation, and continue our dialogue with other animals.

The Koko App is designed to transmit Koko’s legacy of “conservation through communication” by teaching humans to communicate with gorillas like Koko (and each other) in American Sign Language (ASL) — with Koko as their teacher — as well as using gorilla natural gestures.  It also opens a window into the gorilla mind that, with the help of compelling video content collected over 5 decades, will increase interspecies empathy and enable us all to continue our dialogue with great apes.

Interspecies Communication is a vital pathway to appreciating and protecting our vanishing species as well as Nature itself.  The KokoApp will make this powerful tool more accessible with respect to great apes, and leverage one of the greatest lessons of Project Koko — all gorillas are essentially Kokos!  Thus, they all can benefit from the gifts of 2-way communication with their human caregivers, and act as articulate ambassadors for their critically endangered species.

We have a preliminary design for the app, as described below, but need help with the final design and implementation. 

The 5 principal capabilities of the Koko App are:

          1) Learn to Sign with Koko (and Penny)

          2) Learn about the Gorilla Mind

          3) Interpret the Project Koko Video Archive

          4) Translate Zoo Gorilla Natural Gestures

          5) Help Save Gorillas from Extinction and Preserve Nature

 

as described below in greater detail (and illustrated dynamically in the video above).

 

Top

Koko App Design Features

The following design mockup images indicate the kind of functions and features we’d like to offer in the KokoApp.   Each of these images represents a short video clip — from about 5 seconds each for individual sign demo clips, to about 30 seconds for Koko-and-Penny dialogue videos.

 

1) Learn Signs Directly from Koko (and Penny)

Look up hundreds of Koko’s favorite signs by name (eg, EAT, DRINK, MORE, LOVE, SORRY, THANK YOU …) and watch her demonstrate them in short, video clips. Alternatively, click on “Penny”‘s icon and watch her and a native signer, Darlene, demonstrate how the sign is done in strict ASL (American Sign Language) compared with how Koko signs it (GSL).   

Currently, we have over 1000 of Koko’s signs demonstrated in video by Penny and Darlene, but only 100 or so demonstrated by Koko.  As the Koko App evolves, we will be adding more and more Koko demo videos.  

Note that sign language is an important language to learn for other reasons besides interspecies communication.  It can facilitate communication with the deaf community, is useful in situations where speaking is prohibitive (eg, underwater or during presentations) and teachers tell us that it accelerates the acquisition of other spoken languages (eg, Spanish or French), as associating physical gestures with words reinforces the memory process.

 

 

2) Explore the Mind of a Gorilla (via Koko and Michael)

We have hundreds of videos of gorillas Koko and Michael signing with their caregivers (especially Penny) and occasionally with celebrity visitors (like Robin Williams).   These videos demonstrate the many facets of Koko’s (and Michael’s) personality, and how they use sign language to communicate their thoughts and feelings.  Sign language thus offers us a window into the mind of a gorilla, and this treasure of research videos offers not only a way to get to know Koko (and Michael) as a “person,” but also a great tool to help you learn sign language — by watching the videos with and without subtitles.

 

Michael’s videos are more rare than Koko’s (for now, though we are in the process of digitizing many hours of early footage that should reveal much more of Michael) but when he does communicate, it’s usually about something fairly intense, and often in the form of a story.  For example, in the clip where he’s asked if what he remembers about his mother, he tells a shocking story about witnessing his mother killed by bushmeat hunters, when he was just an infant living in Africa (before he was rescued).

 

Both Koko’s story about the kitten she loved and lost, and Michael’s story about witnessing his mother’s death, show that gorillas are much more like us than we previously thought possible.  And this naturally leads to expanded empathy for gorillas as a species, which can, in turn, engender more support for both gorilla conservation and more humane captive care in zoos.

So these videos have the power to change hearts and minds about gorillas, and thus we are extremely excited about packaging them within the framework of the Koko App, as this will make it possible for citizens of gorilla habitat countries (in Africa) to see them for the first time — due to the exponential growth of cell phone use in AFrica.

In addition to videos, the Koko App will also include all of The Gorilla Foundation’s books about Koko and Michael, in the form of eBooks, with links to relevant signing videos, which which will also reinforce sign language acquisition.

 

eBooks Accessible within the Koko App

In addition to learning about the gorilla mind from videos, all of our Koko/Michael books will be accessible as interactive eBooks within the Koko App.   This will allow people to get much more detail on the subject matter without leaving the app, and will feature 2-way links between the books and the signing videos, to reinforce sign language acquisition.

 

3) Interpret Koko’s Videos (Citizen Science)


The Gorilla Foundation began working with Koko in 1972 and Michael in 1974, and has accumulated nearly 5 decades of interspecies communication research videos that amount to over a thousand hours of footage — most of it un-reviewed.

This is an excellent opportunity for crowd-sourced translation of Koko’s sign language videos — which could take us years to complete otherwise.   

Sign language translation of Koko’s (and Michael’s) videos does not require any prior knowledge of sign language, and in fact, it’s a good way to learn sign language.

The Koko App features 3 ways to look up signs:  by a) Sign Name, b) Category, and c) Description. 

When you don’t know the name (or category) of a sign that you’ve just watched, you can look it up by Description.  When you search for a sign by Description, you can describe the sign via up to 3 standard parameters:  Hand Shape, Movement and Location.   So, for example, suppose you saw Koko signing something like what’s in the image on the left (within a video).  You could then simply search for all Koko signs “Located” at her Mouth, as on the right.  This would bring up a small subset of signs (which would be even smaller if you added a description of the Hand Shape and/or Movement) in your search results. 

You can then quickly identify which of the found signs looks most like what Koko is signing on the left.  And in the process, you would have “learned” the sign for (say) “Frown” and wouldn’t have to look it up by Description the next time, you could just search for “Frown” by “Name” to confirm you remembered it correctly (and revert to Description if you’ve forgotten the sign).

Note that while this process can be very accurate and fast in principle,  it may require some refereeing by Gorilla Foundation staff (and/or AI).

By opening up our newly digitized Koko Video Archive (over 1000 hours) to crowd-sourcing through the Koko App, we can continually update the app with both new sign demos, and new gorilla videos (with and without sign subtitles) to enrich all of the other features of the app (learning to sign, exploring the mind of a gorilla, translating signs, and conservation)!

The KokoArc Video Archive Spans5 Decades of Interspecies Communication

4) Help Translate Zoo Gorilla Gestures (improve care)

Gorillas have a natural capacity for gestural communication.  Zoo gorillas have been observed using over 100 natural (untaught) gestures consistently in both the US and the UK.  However, we haven’t yet been able to translate many of these gestures, and learning them could create a huge advantage for gorilla care — as their caregiver would be better able to understand their needs and wants without having to teach them a “new ” language  such as ASL.

For example, the gorilla at left (JJ at the Miami Zoo) was having a rough day.  It looks like he was trying to say:  “Help, no more help” in response to a bunch of visitors attempting to throw food at him.   By adding more context, experience and data, it may be possible to translate a large number of such gestures.

Here’s how the Koko App can help.  Allow users to upload videos that they either take themselves at zoos, or capture from the internet, and ask others to submit their “guesses” (from context) as to what the gorillas are saying.  Then combine expert (relevant zookeeper) inputs with a deep learning algorithm to find the best match for each word or phrase.  As we develop a large enough database of natural gesturals, it will become easier to translate new footage with existing gestures, or to identify new gestures to translate.

We can implement this process in 2 different ways:  automatic uploads of videos and translation guesses, and/or refereed submissions, selectively uploaded for the public to see.  In either case, we could add regularly scheduled Zoom video panel discussions to discuss status and give tutorials on how to participate in the process of learning gorilla natural gestures.  We can also provide updates from specific Zoos on how expanding our shared language with gorillas is improving their care and enrichment in captivity.

Zoom conference between The Gorilla Foundation and a Kids4Koko member to discuss what a zoo gorilla might be trying to say with his natural gestures.

 

 

5) Help Save Gorillas from Extinction (Become a Voice for Nature)

The “Gorillas” section of the app will help people “apply” the above tools to collectively achieve real-world results for gorilla conservation and captive care.  It will feature video tutorials and panel discussion on how to use the Koko App to: 

5.1) learn sign language

5.2) understand the gorilla mind

5.3) help interpret/transcribe Koko’s video archive

5.4) help translate zoo gorilla natural gestures

5.5) facilitate Interspecies Communication with gorillas in zoos

5.6) increase worldwide empathy for gorillas, esp. in their African homeland 

plus suggestions, plans and status updates on new features that can enhance the functionality of the Koko App in each of these areas

Each video tutorial/discussion will be hosted by a Gorilla Foundation staff member and/or an expert in a related field,  and optionally include selected Koko App users as guests (both kids and adults).  Koko App users who want to participate in these tutorials/discussions can submit video proposals (via the app) to be selected in advance.

This will make the Koko App a portal for “conservation through communication” where results will be measured via pre/post app usage surveys, and used to optimize the app’s functionality.

Mock-up tutorial featuring world-renowned
gorilla and elephant conservationist Ian Redmond.

 

Mock-up tutorial featuring Dr Penny Patterson discussing interspecies communication at zoos
with Chaz, a member of The Gorilla Foundation’s KokoKids online network.

 

Top

The Gorilla Foundation

The Gorilla Foundation (aka Koko.org) was established as a nonprofit foundation (501c3) in 1976 to explore interspecies communication with gorillas for the benefit of all great apes, through science, education and direct conservation efforts.    It has tens of thousands of supporters (subscribers and donors) who resonate with our mission of Conservation through Communication.

Our supporters want us to bring Kokos Legacy to fruition; but we can’t do it alone; we need partners whose expertise is complementary to ours.  Now that Koko has passed, the mission of the Gorilla Foundation is fueled by its data, educational resources and resident expertise (cofounders and senior staff).  Our current strategic plan is built around 3 cornerstone projects:  Koko Arc, Koko Kids and the Koko App (see figure below).

 

The Koko Arc (digital archive) project provides the fuel for achieving the mission — with 5 decades of multimedia data, most of which has not been seen by the public, or even analyzed internally.  Once the digitization is completed and catalogued, it will be used to generate new content for the other 2 cornerstone projects.

Top

The Ideal “Koko App Partner”

The ideal partner to help The Gorilla Foundation develop the Koko App is a company or an individual with the following characteristics:

— Large-scale success with at least one successful app used on both IOS and Android devices 
— Significant experience embedding video content within apps 
— Experience enabling users to upload video content (for review) via an app
— Experience with crowd-sourcing visual data analysis via an app
— A passionate interest in helping the Gorilla Foundation accomplish its mission of bringing interspecies communication to the public to accelerate great ape conservation, improve captive care of gorillas, and enable our children to continue “talking with the animals”

Ideally, this would be a pro bono effort, at least initially, with the possibility of sharing revenue if you can propose the right business model.  The Gorilla Foundation’s goal is to maximize the number of people who engage with The Koko App, in order to create millions of advocates for the use of interspecies communication to propel great ape conservation and care.

Contact Us

Call or email Dr. Gary Stanley, The Gorilla Foundation’s COO,  at 650.722.6810 or [email protected] to schedule an introductory Zoom call and explore the possibilities.  Thank you!

Top