Overview Today, the way children learn to read is very different from the way they learn from playing with toys. Books present static images and text on the page whereas toys allow for manipulation and interactive exploration of cause-effect relations. What if books were "tinkerable"? What if children could actively explore and modify story, through voice and touch, to dynamically explore meaning as conveyed by the relationship of text to illustrated concept? How might this impact how books are experienced, explored, and shared between parent and child? And in turn, how might this impact how children learn to read? The main hypothesis explored in this work is that by allowing children to actively explore and test the relationship between words and concepts within a story, children may understand that words represent concepts more easily and learn these concepts more quickly.
We present the design of Tink-R-Books, an interactive storybook for parents and children to explore and read together. Importantly, the Illustrated Primer is designed to allow emergent readers to actively explore the abstract relationship between printed words and their meanings via experiment, even before this relationship is properly understood. There are two proposed contributions of this work. The primary design contribution is to derive a principled set of multisensory interaction techniques, suitable for emergent readers, for altering the text-graphic relationship in storybooks. The primary technical contribution, called illustrative commonsense, is a set of computational techniques that combines commonsense reasoning with procedural animation techniques to dynamically generate the illustrative text + visual representation in response to the readers vocal or haptic interaction with the story. The result is a storybook that invites modification and exploration by the emergent reader while supporting the main narrative. Finally, we propose a set of reading experiments with parent-child pairs to evaluate this work.