✦ Jeon Stack

Overview

Jeon Stack turns Korean folktales into a hands-on storytelling system where the story changes based on how you stack the books.

Instead of reading stories linearly, users physically stack unmodified folktale books on a bookshelf.

The system responds by generating a real-time audiovisual scene — complete with animated backgrounds, character actions, synthesized dialogue, and dynamic world transitions

The Design Question

"How to turn Korean culture into a material for tangible media design?"

As the question suggested, we set two requirements:

• The material has to be really Korean in the form of visual design, language, and cultural values
• It must be engaging and fun for the user to interact with

The Approach

My team decided to use Korean folktales as the material for the project as it satisfies all the reqirements: it has Korean significant visual design, told in Korean language, and reflects Korean cultural values.

And that was how we come to the final question: "If we were able to make a space where all the characters from differents folktale stories come together and talk and interact, how would that look like?". That is the whole ideas behind JeonStack.

Technical Implementation

Each book carries a fiducial marker that the system detects via an overhead webcam.

Webcam detects ArUco markers to identify which books are stacked and in what order, triggering a live audiovisual narrative in real time.

The order books are stacked determines the narrative: the first book sets the world, the second introduces a character, and the third triggers a dialogue exchange between two characters. Swapping books reshapes the story on the fly

Visual and Sound Design

We digitized each physical folktale book and picked an illustration from inside it as the world’s source art. We then used AI image-to-video generation to turn that illustration into a 2 to 5 second seamless loop, so the original artwork stays recognizable while the scene feels alive.

Why AI?: Animating the original book illustrations keeps the visuals authentically Korean. Without AI, we would redraw and hand-animate every world, or settle for static images.

Each character speaks as itself. We fed the LLM every character’s description, personality, behavior, and sayings from the original Korean folktale, and the model generates their conversation from that.

Prototype Video

Acknowledgements

Supervisor

Professor Woohun Lee

Co-author

Intae Hwang

Co-author

Seunghwa Pyo

This project would not have been possible without the contributions of my KAIST professor and beloved classmates, whose collaborative effort brought JeonStack to life