TELESTORY, SIFTABLE APPS
David Merrill, the primary designer of Siftables , was my officemate at the Media Lab for two years. Siftables are compact devices with sensing, graphical display, and wireless communication. They can be physically manipulated as a group to interact with digital information and media. During that time, I was the first application developer to work with the siftables API. Over a period of 6 months, I designed and implimented several childrens applications.



Fiddle, Diddle, Make a Riddle - shown above - was made after I consulted with a first grade teacher in the Boston Public School system to design and author a language-learning application using Siftables. It is targeted for children aged 4-7, and is based loosely on the children's book Hop on Pop, by Dr. Seuss. The focus of the application is to teach children basic sentence-construction skills through creative play. The application allows children to explore creative analogies by introducing humorous relationships between words and allowing them to structure those relationships by arranging Siftables in multiple sequences.

The application aims to teach phonemic awareness (rhyme endings with different spellings), semantic sentence structures (ex. what image does the sentence imply?), creativity through control of language (experimenting with different arrangements), and vocabularies and fluency through image/word association, grapheme awareness through word families (the -ice of mice and lice), and basic punctuation (when blocks are switched the capital letters and periods are switched).



Katya Popova (artist/animator), and I (director/programmer/concept) worked closely together on Telestory, a language-learning application sponsored by SamSung, designed to teach vocabulary to children that are not yet old enough to read. This application is comprised of interactive characters on a large display each character represented both on a siftable and on an HD television display. Children build the story themselves by picking up siftables, thereby triggering actions on screen. Episodic narratives between the main characters, a dog and a cat, are triggered by placing objects in relation to each other.

We experimented with an interactive tilt-based menu for Telestory to allow a child to select props. The menu divides the screen of the Siftable into four quadrants, and shows a thumbnail image of each prop in each quadrant. One of the four quadrants starts out in full color, while the other three are faded. Tilting the Siftable toward a particular corner causes the thumbnail in that corner to become the selected item, and to be shown in full color, while the others become faded. If a given prop remains selected for two seconds the selected item enters the cartoon on the large screen.

We observed two children aged 4 and 5 interacting with Telestory in separate sessions and found that both children seemed to easily understand the connection between the Siftables and the corresponding on-screen characters. Direct correspondences, such as raising the Siftable displaying the sun to make the sun rise in the cartoon, or shaking the Siftable showing the dog to make the on-screen dog shake, were the most popular.

In April of 2009, the Discovery Channel did a spot on the siftables project, and tried the application with six children at once. Although some elements of this type of setting (a film shoot) for evaluating an application are contrived, It helped us think about how applications might potentially be designed for use between children, in the form of 2-4 person games. Here the children learned to use the Telestory application with the show host.


Making the Moon Rise by lifting the siftable up.

  
Tilting menu system, choosing a flower here.

  
Triggering episodic narratives on screen by placing siftables adjacent to each other.


Screenshot of the Television, Artwork by Katya Pupova