# Puffy: A Step-by-step Guide to Craft Bio-inspired Artifacts with Interactive Materiality




Puffy: A Step-by-step Guide to Craft Bio-inspired Artifacts with Interactive Materiality

Sark Pangrui Xing (1, 2), Bart van Dijk (2), Pengcheng An (3), Miguel Bruns (2), Yaliang Chuang (2), Stephen Jia Wang (1)

sark.xing@connect.polyu.hk; contact@b-vandijk.nl; anpc@sustech.edu.cn; mbruns@tue.nl; y.chuang@tue.nl; stephen.j.wang@polyu.edu.hk

1 School of Design, The Hong Kong Polytechnic University, Hong Kong SAR, China
2 Industrial Design, Eindhoven University of Technology, Eindhoven, The Netherlands
3 School of Design, Southern University of Science and Technology, Shenzhen, China

### Abstract

A rising number of HCI scholars have begun to use materiality as a starting point for exploring the design's potential and restrictions. Despite the theoretical flourishing, the practical design process and instruction for beginner practitioners are still in scarcity. We leveraged the pictorial format to illustrate our crafting process of Puffy, a bio-inspired artifact that features a cilia-mimetic surface expressing anthropomorphic qualities through shape changes. Our approach consists of three key activities (i.e., analysis, synthesis, and detailing) interlaced recursively throughout the journey. Using this approach, we analyzed different input sources, synthesized peers' critiques and self-reflection, and detailed the designed experience with iterative prototypes. Building on a reflective analysis of our approach, we concluded with a set of practical implications and design recommendations to inform other practitioners to initiate their investigations in interactive materiality.

### Introduction

The field of HCI has developed a growing interest (e.g., [4], [11], [15], [27]) in addressing materiality as an entry for design research [10] by entangling miniaturized sensing and actuating technologies with advanced materials [19]. Such entanglement has enabled HCI scholars to create matters with shape-changing [1, 26] quality, interactive materiality [30], expressivity [2], to name but a few, navigating HCI research toward its next wave [7]. In response, explorations in forms of design notions (e.g., materiality of information [5], materiality of interaction [38], form-giving [36]), design methods (e.g., [10, 16, 39]) have sparked among the HCI communities.

While researchers are still developing theories to support this emerging field, many have developed tools to support concept ideation e.g., [45] and prototyping (e.g., [20, 22, 42]). In particular, Morphino [24] has proposed a card-based kit that archives illustrations of how organisms change shape, allowing practitioners to widely explore suitable shape-changing mechanisms from nature. Additionally, design practitioners have explored adopting materiality in practices for behavior change [17, 29, 33], lived experiences [43, 44], emotional expression [13], smart garment [34, 41] to name but few. Despite those prior works, it seems few studies address on the process and rationale of how their shape-changing concepts were ideated and concretized; and how the materiality is mapped and correlated with user interactions. In other words, the overarching question is how are certain qualities of organic shape changes, such as subtlety, ambiguity, and temporality transformed into the materiality of shape-changing?

To address this question, we present a mixed materiality approach by which we yielded Puffy, a shape-changing artifact that spontaneously changes its shape and materiality to express its emotions. Our design process consisting of three key activities (A-S-D, i.e., analysis, synthesis, and detailing, see p. 2) is mainly based on two approaches, namely biomimicry [25] and interactive materiality [30]. We used biomimicry-alike to generate implications serving as a source of inspiration. In the follow-up steps, we analyzed different sources of input gained in various stages, synthesized self-reflections and peers' critiques, and detailed the design through iterative prototyping.

In short, we contribute the HCI community a concrete interactive materiality design case and moreover we offer an illustrative step-by-step guide and prototyping suggestion to assist interaction design practitioners to design interactions that feature in rich temporality, interactive materiality, and computational aesthetics.



![A hand hovers over a white fibrous surface on a dark rectangular base.](/papers/puffy/images/v-p01-surface.webp)

Visual description — v-p01-surface

Source: bottom-right photograph beside the introduction.

Original caption: 

Short description: A hand hovers over a white fibrous surface on a dark rectangular base.

Long description:

The photograph is a close view, not a numbered figure. A light-skinned hand enters from the upper right, palm down, fingers slightly curled, above a dense white fibrous surface. The fibers read as a soft mat rather than a smooth skin. That mat sits on a dark rectangular block whose front face is in shadow. No scale, material specification, or state label is printed on the photograph.


Reading notes (site-compiled): Site-compiled. The introduction on PDF page 1 presents Puffy as a shape-changing artifact. This photograph shows a hand and a white fibrous surface; it does not by itself show a shape change or name a state.




Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

TEI '23, February 26–March 1, 2023, Warsaw, Poland
© 2023 Copyright is held by the owner/author(s).
ACM ISBN 978-1-4503-9977-7/23/02.
https://doi.org/10.1145/3569009.3572800


Step-by-step Guide to Craft Bio-inspired Artifacts with Interactive Materiality

The page is a process overview. Three headers run across the page: ANALYSIS, Analyzing various input; SYNTHESIS, Synthesizing the findings; DETAILING, Detailing the design outcome. Numbered activities sit in the column of their phase. The numbers are not arranged as a left-to-right sequence.

Authors Keywords: The Materiality of Interaction; Shape-changing Artifact; Haptic Interaction Design; Computational Aesthetics; Crafting

CCS Concept: Human-centered computing~Interaction design~Interaction design process and methods



![A process overview groups thirteen numbered activities under Analysis, Synthesis, and Detailing. Their placement preserves phase membership rather than a left-to-right numeric sequence. ](/papers/puffy/images/v-p02-overview.webp)

Visual description — v-p02-overview

Source: whole-page process overview.

Original caption: 

Short description: A process overview groups thirteen numbered activities under Analysis, Synthesis, and Detailing. Their placement preserves phase membership rather than a left-to-right numeric sequence.


Long description:

The page title repeats "Step-by-step Guide to Craft Bio-inspired Artifacts with Interactive Materiality". Under it, three colored headers run across the page. Analysis, labeled "Analyzing various input", is on the left. Synthesis, labeled "Synthesizing the findings", is in the center. Detailing, labeled "Detailing the design outcome", is on the right. Illustrated panels occupy the matching columns. The activity numbers are not a left-to-right sequence: reading the headers from left to right does not yield activities 1, then 2, then 3.

| Phase shown in the figure | Numbered activity labels |
|---|---|
| Analysis | 1 Shape transitions; 2 Analogue materials; 8 Computational mechanism explorations; 9 Affirming the computational whole; 11 Analyzing feedback |
| Synthesis | 3 Materials samples; 4 Transition samples; 7 Materiality experiencing setup; 10 Design critique; 12 Synthesizing feedback |
| Detailing | 5 Quick-and-dirty mimicking; 6 Interaction mapping; 13 Detailing the design artifact |

Panel 1, Shape transitions, is a pair of small drawings: a fish and a plant or clump of vegetation, with arrows toward a round form, and a small screen-like sketch beneath them. Panel 2, Analogue materials, is a two-by-two set of square swatches with different surface patterns. Panel 3, Materials samples, is a three-by-three array of square samples, some with radiating or hourglass-like marks. Panel 4, Transition samples, is three rows marked with roman numerals, each a short sequence of profile curves. Panel 5, Quick-and-dirty mimicking, shows a soft cube-like volume with a round reddish form and a thin protrusion against the surface.

Panel 6, Interaction mapping, places an approaching hand above a horizontal line, a balloon-like form beneath that line, and the printed state labels Approaching and Alerted. Curved arrows sit beside the balloon. The panel does not name a sensor, a board, or a measured result.

Panel 7, Materiality experiencing setup, shows a table-like frame with objects suspended underneath, wires, and round bases. Panel 8, Computational mechanism explorations, shows mechanism sketches: a dome or diaphragm with arrows, a container, and a box with a raised center. Panel 9, Affirming the computational whole, shows a seated figure with a thought bubble beside a box-like volume. Panel 10, Design critique, shows speech bubbles and a cluster of colored notes. Panel 11, Analyzing feedback, is a field of small blue and pink rectangles arranged like notes. Panel 12, Synthesizing feedback, shows pink notes beside three dome-like forms. Panel 13, Detailing the design artifact, shows a board whose visible label reads Arduino, wiring from that board to a small cube-like object, and a person beside a larger cube-like object. The board label does not identify an Arduino model.

A footer on the same page prints Authors Keywords (The Materiality of Interaction; Shape-changing Artifact; Haptic Interaction Design; Computational Aesthetics; Crafting) and one CCS concept line. Those lines are page text, not a separate figure.


Reading notes (site-compiled): Site-compiled. PDF page 1 points to this overview as the A-S-D process. Phase membership is taken from the column under each colored header, not from the numeric order. The overview alone should not be described as proof of a strictly linear, one-pass workflow; any account of iteration has to come from the surrounding paper text. Panel 13's board label is not a hardware model, and the drawings do not establish materials or experimental outcomes.





### The Puffy artifact

Puffy is a cilia-mimetic interactive artifact that responds to human proximity and expresses "emotions" through shape changes. It mimics how a pufferfish responds to predators by straightening spikes and inflation. Depending on external surrounding such as a danger nearby, Puffy reveals three kinds of anthropomorphic characteristics (i.e., Calm, Alerted, and Relaxed) through the changes of haptic shapes and fabric sturdiness.

The conceptualization of Puffy followed the mimicking strategy described in [25] and the nature-inspired card deck [24]. Specifically, we looked at a number of lively inspirations that contain dynamic behavior from nature. However, instead of focusing on how biologically or mechanically a creature changes its shape, we, as interaction designers, focused on how the shape changes affect the aesthetic qualities (e.g., figures shown on p. 3) of the creature, which influences how the creature is perceived and feelings that are evoked by the viewer. In other words, we were not simply mimicking the shape-changing mechanism, e.g., the puffer fish's accordion-like folded stomach, but rather the experiences evoked by the movement and somaesthetics derived from the selected natural element. We chose the most intriguing one – Pufferfish as our inspirational reference because of its spiky texture, rebelling behavior, and the correlations of these two with its surroundings. The subsequent sections will elaborate on how we transformed those pufferfish's characteristics into Puffy's interactive materiality.



![The Puffy artifact](/papers/puffy/images/v-p03-artifact-states.webp)

Visual description — v-p03-artifact-states

Source: left column, three stacked photographs beside the section The Puffy artifact.

Original caption: The Puffy artifact

Short description: Three stacked photographs show a hand over a white fibrous surface on a grey block, with the surface changing height.


Long description:

Three photographs are stacked vertically on a black background. Each shows a hand above a white fibrous rectangle that rests on a grey block. In the top frame the fibrous surface is low and the hand is close to it. In the middle frame the surface is taller and more rounded. In the bottom frame the surface is high and domed while the hand stays above it. The vertical label beside the stack reads "The Puffy artifact". The photographs do not print the words Calm, Alerted, or Relaxed on the images themselves.


Reading notes (site-compiled): Site-compiled. The adjacent prose says Puffy reveals Calm, Alerted, and Relaxed through haptic shape and fabric sturdiness. That mapping is the paper's text. The photographs show three heights of a white fibrous surface and a hand; they do not label which frame is which state.




### Analyzing various input (ANALYSIS)

#### 1 Shape transitions

We first explored aesthetic inspiration from nature such as animals and plants. We found that the shape transitions of pufferfishes had some attractive attributes:

1. the shape change communicates tension as the fish expands or squeezes;
2. the growth and angle change of its spikes emphasizes its repelling emotion;
3. the underlying relationship between these changes and the pufferfish's intention and surroundings (e.g., the predators).

Labels on the stills: inflate; deflate; straight spikes.

Credit: Author, Author, Stelio Puccinelli, Martin Kleppe.
Inspiration sources: Puffer Fish Puffing up when caught, 2011. https://youtu.be/ccsvJMkF5Bs; Blow me, Beautiful, 2013. https://youtu.be/S7y4quhmMW0; Dogface Puffer Fish Puffs Up Like Balloon, 2019. https://youtu.be/-qf5vPq_z7U



![1 Shape transitions](/papers/puffy/images/v-p03-shape-transitions.webp)

Visual description — v-p03-shape-transitions

Source: right column, section 1 Shape transitions, three video stills.

Original caption: 1 Shape transitions

Short description: Three stills of a pufferfish are labeled inflate, deflate, and straight spikes.

Long description:

Under the heading 1 Shape transitions, a black strip holds three stills. The left still shows a pale rounded fish with a curved arrow and the label inflate. The center still shows a smaller or lowered fish with the label deflate. The right still shows a fish covered in projecting spines with the label straight spikes. A credit line names Author, Author, Stelio Puccinelli, and Martin Kleppe, and lists three video URLs as inspiration sources. No timeline scale or measured angle is printed.


Reading notes (site-compiled): Site-compiled. The numbered list in the same section says the shape change communicates tension, the spikes emphasize a repelling emotion, and the changes relate to the fish's intention and surroundings. Those claims are the prose, not values read off the stills.




#### 2 Analogue materials

Consecutively, to eliminate the confounding bias from the colors of the material, we purposely explored and examined extensive analogue materials that were in white and matched the interesting attributes of shape transitions gathered from the prior analysis. We obtained a profound and embodied understanding of the selected behavior through first-hand explorations, emphasizing the visual and sensory feelings.



![2 Analogue materials](/papers/puffy/images/v-p03-analogue-materials.webp)

Visual description — v-p03-analogue-materials

Source: bottom of the right column, section 2 Analogue materials.

Original caption: 2 Analogue materials

Short description: Three white material swatches sit in a row: a hairy sample held in a hand, a tufted square, and a sheet of upright fibers.


Long description:

Beneath the analogue-materials paragraph, three photographs share one black strip. The left image shows fingers holding a small white hairy sample. The middle image is a square swatch with a regular tufted or looped texture. The right image is a dense field of upright white fibers. No dimensions or material names are printed on the photographs.


Reading notes (site-compiled): Site-compiled. The prose says the samples were white so color would not confound the comparison, and that the exploration was first-hand and sensory. The photographs show three white textures. They do not name the stock or the dimensions.





### 3 Material samples (SYNTHESIS)

Among the explorations, some focused on manipulating material qualities, while some consisted of combining multiple materials to allow for expansion. Besides, we also looked at how we could produce our own materials with custom specifications (e.g., 3D printed hairy material, Cilllia [23]). While the exploration provided directions for material adaptation, we found that our transition could be best expressed through an un-adapted, hairy fabric combined with a shape-changing mechanism that worked best as it emphasizes the sturdy visual expression and haptic experience.



![3 Material samples (SYNTHESIS)](/papers/puffy/images/v-p04-material-samples.webp)

Visual description — v-p04-material-samples

Source: left half, section 3 Material samples.

Original caption: 3 Material samples (SYNTHESIS)

Short description: A contact sheet of white textile squares on a black field, plus a circular close-up of a hairy edge.


Long description:

The left half is one contact sheet, not a set of unrelated figures. White squares are arranged in rows on black. Several squares carry handwritten labels. Visible constructions differ: some have loose hairs at the edge, some a grid of tufts, some short upright elements, and some a smoother woven face. A circular inset at the lower right magnifies a white hairy surface. The sheet does not print measurements.


Reading notes (site-compiled): Site-compiled. The paragraph under the sheet says the preferred direction was an un-adapted hairy fabric with a shape-changing mechanism, and it names Cilllia [23] as an example of a custom hairy material. The contact sheet shows the range of samples; it does not mark which sample was chosen.




### 4 Transition samples (SYNTHESIS)

We used pufferfish as an inspiration to explore different transition techniques and aimed to find inherently coupled materials and transitions. We then synthesized our explorations regarding surface texture, shape transitions, and behavior movements respectively. To highlight the subtlety and temporality of the organic shape changes, we provided considerable attention to extracting the underlying information. For example, without a looping replay and multiple rewinds of the pufferfish video clips, we would not have been able to see that the deflation was gradual, sluggish, and non-linear, while the inflation was abrupt and linear.

Diagram labels: (Straight spikes to flat); (Non-linear deflation); (Movement and expansion).



![4 Transition samples (SYNTHESIS)](/papers/puffy/images/v-p04-transition-samples.webp)

Visual description — v-p04-transition-samples

Source: right half, section 4 Transition samples.

Original caption: 4 Transition samples (SYNTHESIS)

Short description: Three rows pair a photograph of a hairy or spiked surface with a schematic of a profile change.


Long description:

Three rows share one section. Each row has photographs on the left and a simple schematic on the right. The top schematic is a black wedge with a curved arrow and the label (Straight spikes to flat). The middle schematic is a wavy line with an arrow and the label (Non-linear deflation). The bottom schematic is a circle beside a dotted circle, with an arrow, and the label (Movement and expansion). The photographs show white hairy or spiked surfaces, including pins standing out of a soft sheet and a hand pressing a furry edge. The schematics have no axes, units, or printed numbers.


Reading notes (site-compiled): Site-compiled. The prose says deflation in the source videos looked gradual, sluggish, and non-linear, while inflation looked abrupt and linear. That comparison is the authors' reading of the videos. The schematics illustrate the three transition ideas; they are not plots of measured data.





How will the materiality react to the user with the synthesized materials and transitions?

### 5 Quick & Dirty mimicking (DETAILING)

To answer the question from an experience perspective, we used a quick-and-dirty setup consisting of a texture of interest and a balloon placed underneath it. We mimicked the previously synthesized materials and transitions e.g., the spikes, the shape changes, and the underlying pufferfish-predator relationship. Through the mimicking, we were intrigued by how those reciprocally incorporated elements could be mapped and manifested in detailing and mimicking the desired materiality, resulting in an intriguing interaction design.

#### Hand-controlled actuation

As we intended to mimic the escaping behavior of pufferfish, we explored techniques that allow for rapid examination of materiality experiences, one of which inspired us the most is ClothSurface [12]. We consecutively replicated it through a quick-and-dirty setup in which a piece of fabric was wrapped around the legs of an upside-down chair. By manipulating a balloon placed underneath (i.e., inflating, deflating, or moving), the shape transitions could be mimicked, allowing us to experience and evaluate the aesthetic qualities promptly and haptically. At last, we were impressed by the hairy quality of the textile and the expansion of movement on the textile.



![5 Quick & Dirty mimicking (DETAILING)](/papers/puffy/images/v-p05-quick-dirty.webp)

Visual description — v-p05-quick-dirty

Source: left side, section 5 Quick & Dirty mimicking, drawing and two photographs.

Original caption: 5 Quick & Dirty mimicking (DETAILING)

Short description: A drawing of a soft volume with a balloon inside, then two photographs of checkered fabric over a yellow balloon on a chair frame.


Long description:

A line drawing shows a pale draped volume with a pink-orange round form inside and small arrows at the surface. Below the paragraph, two photographs sit side by side under the label Hand-controlled actuation. Both show a dark checkered cloth held by clips on a metal frame, with a yellow balloon visible underneath. In one frame a hand is at the balloon; in the other the balloon sits more centrally under the cloth. The cloth texture and the balloon are visible. The photographs do not print a pressure or a distance.


Reading notes (site-compiled): Site-compiled. The prose says the setup mimics spikes, shape change, and a pufferfish-predator relationship, and that it replicates ClothSurface [12] with fabric wrapped on an upside-down chair and a balloon underneath. Those intentions are the text. The images show the cloth, the balloon, the frame, and a hand.




### 6 Interaction mapping (DETAILING)

Column labels: No interaction; Hand approaching; Hand leaving.
State labels: Calm; Alerted; Relaxed.

No interaction / Calm: When no human interaction is involved, no significant changes in materiality will occur. The object stays calmly in back-and-forth looping transitions, behaving like 'breathing'.

Hand approaching / Alerted: When a user attempts to approach, the object becomes alerted and expands significantly. Its surface becomes more sturdy and consecutively moves always to escape from the 'danger'.

Hand leaving / Relaxed: When the 'danger' disappears, the object becomes relaxed and gradually squeezes itself, and returns back to center.



![6 Interaction mapping (DETAILING)](/papers/puffy/images/v-p05-interaction-mapping.webp)

Visual description — v-p05-interaction-mapping

Source: right side, section 6 Interaction mapping, three columns.

Original caption: 6 Interaction mapping (DETAILING)

Short description: Three columns map No interaction, Hand approaching, and Hand leaving onto Calm, Alerted, and Relaxed, with balloon drawings above the table.


Long description:

The right side is one mapping figure. Line drawings of a balloon under a horizontal stroke occupy the top, with hands and curved arrows in the approaching and leaving sketches. A three-column table sits below. The headers are No interaction, Hand approaching, and Hand leaving. The state names in the columns are Calm, Alerted, and Relaxed. Calm's cell says that with no human interaction the object stays in back-and-forth looping transitions, like breathing. Alerted's cell says that when a user approaches, the object expands, the surface becomes more sturdy, and it moves to escape from the danger. Relaxed's cell says that when the danger disappears the object squeezes and returns to center. Dotted leaders connect the drawings to the columns. No sensor trace or numeric threshold is shown.


Reading notes (site-compiled): Site-compiled. This table is the paper's mapping of proximity to three anthropomorphic states. The cell text is printed in the figure. It is not an independently measured result, and the drawings do not identify hardware.





### 7 Materiallity experiencing setup (SYNTHESIS)

As the selected analogue material will primarily provide the texture and materiality for the design artifact, choosing the appropriate one was key for this process. First, we advanced the materiality experiencing setup by integrating a 6DOF Stewart platform [31] with two vacuum pumps to manipulate the attached balloon. Then, we explored and examined different combinations of material and transition techniques. Exploring the defined transition samples synthesis (see p. 4), we noticed the captivating capability of the pufferfish's spikes which would stand out from its skin. At rest, the spikes align with the skin forming a solid surface. However, when inflated, the spikes deviate from the skin and start to stand out, resulting in a thicker and less dense outer shell. We also sought this quality in our material samples, in which the observed density of the outer shell would change in response to the changes in shape and/or form. This quality was found in a fabric that had numerous small 'hairs' attached to it, which would stand out, resembling the movements of the spikes.

#### Pneumatic container

Inspired by PneUI [40], we implemented airtight pneumatic containers resembling [28, 35] that can be inflated by a vacuum pump with manual control by a switch. We iteratively explored different inflatable structures and elastic materials to find the best inflation quality.

#### Actuation integration

Then, we integrated the container with the prior actuating prototype and used it to evaluate the quality of shape transitions and behavior movements with semi-manual control over a laptop.

#### Sensing integration

Next, we introduced a capacitive sensor for human behavior detection and exposed the sensing connectors with four sheets of aluminum foil distributed at the corners. This way, the prototype knows where and how close the user is and takes action spontaneously.

Photo labels include: Turn on!; "Inflate" Send!; Approaching.



![7 Materiallity experiencing setup (SYNTHESIS)](/papers/puffy/images/v-p06-experiencing-setup.webp)

Visual description — v-p06-experiencing-setup

Source: upper band and the three columns under section 7.

Original caption: 7 Materiallity experiencing setup (SYNTHESIS)

Short description: Photographs show a Stewart-style platform, a balloon, a pneumatic container, a laptop, and foil sheets, in three labeled steps.


Long description:

The section heading is printed as "7 Materiallity experiencing setup (SYNTHESIS)". A top row of photographs shows hands working a wooden circular platform with copper-colored coils, a green balloon, and a later frame where a white furry top is mounted over the mechanism. Under that row, three columns are labeled Pneumatic container, Actuation integration, and Sensing integration. The pneumatic column includes a hand at a pump or valve with the overlaid words Turn on! The actuation column shows a laptop beside the prototype and the overlaid words "Inflate" Send! The sensing column shows four pale sheets on a square top and the word Approaching with an arrow toward the center. Wires, wooden plates, and a clear dome-like inflatable are visible. The page does not print a parts list on the photographs.


Reading notes (site-compiled): Site-compiled. The prose says a 6DOF Stewart platform [31] and two vacuum pumps move an attached balloon, and that four sheets of aluminum foil expose a capacitive sensor. Those component names come from the paragraphs and column text, not from unspoken detail in the photographs. The double-l spelling Materiallity is the printed heading.




### 8 Computational mechanism explorations (ANALYSIS)

To have precise and comprehensive control over the materiality, we iteratively explored various computational mechanisms. We introduced pneumatic containers for customizing the shape transitions as well as capacitive sensors for human behavior detection.

The photographs for this section are the same band as section 7. They are recorded once as v-p06-experiencing-setup.


Participants (N=9) with design background at MSc. level

### 10 Design critique (ANALYSIS)

To capture user perceptions of and experiences with Puffy, the first and second authors of this pictorial hosted a design critique session. This session was aimed to help gain different perspectives of feedback for materiality experiences improvements in the consecutive stage. All recruited participants (N=9) were asked to follow the steps indicated in the section below.

#### 9 Affirming the artifact

This step intends to grasp and feel the context. Participants leveraged their modality of touch and bodily movements to interact with the artifact. They experienced both visual effects and appreciated their sensational responses to the behavior of the artifact.

#### Interpreting the emotion

Once their first-person experience with the artifact was gained, participants interpreted the relationship between the behavior and the emotion of the artifact they perceived while interacting with it.

#### Reflecting the symbolic notions

Participants reflected on what messages the artifact and/or the designer intended to convey. These reflections were written on pink sticky notes. After that, they left questions and suggestions regarding the artifact on the green ones.

### Initial computational whole

From top to bottom, the initial computational whole Puffy consists of a white hairy fabric that covers four sheets of aluminum foil wired with a capacitive sensor. At the center of Puffy's inside, a Stewart platform is positioned to actuate a custom inflatable that manipulates the fabric through inflation, deflation, and circular actuation.



![Initial computational whole](/papers/puffy/images/v-p07-computational-whole.webp)

Visual description — v-p07-computational-whole

Source: left column, two interior photographs and one exterior cube, with the caption Initial computational whole
.

Original caption: Initial computational whole

Short description: Interior photographs show a wooden box, a white disk, motors, and wiring; a separate view shows the closed cube with a round opening.


Long description:

Two stacked photographs show the open build. The upper one is a top view of a square wooden frame with a white circular pad and a hand at the edge. The lower one shows motors, wires, and a clear rounded inflatable inside a dark surround. Below those, a third photograph shows a closed light-wood cube with a circular opening on the top and a dark circular opening on the side. The caption Initial computational whole is printed under that cube. The paragraph says the stack is a white hairy fabric over four sheets of aluminum foil wired to a capacitive sensor, with a Stewart platform actuating a custom inflatable. The closed-cube photograph does not show the hairy fabric; the interior photographs show the mechanism before that cover.


Reading notes (site-compiled): Site-compiled. Component names in the paragraph are the paper's description of this prototype. The photographs support the wooden box, the round pad, the wiring, and the inflatable. They do not print a sensor part number.






![10 Design critique (ANALYSIS)](/papers/puffy/images/v-p07-design-critique.webp)

Visual description — v-p07-design-critique

Source: right side, critique session photograph and three steps.

Original caption: 10 Design critique (ANALYSIS)

Short description: A workshop photograph and three steps show people with a wooden cube, notes, and the labels Affirming the artifact, Interpreting the emotion, and Reflecting the symbolic notions.


Long description:

A wide photograph is captioned "Participants (N=9) with design background at MSc. level". People stand and sit around tables in a workshop; faces are blurred; laptops, a wooden cube, and notes are on the tables. Three columns follow, joined by a grey arrow bar. Column one is labeled 9 Affirming the artifact and shows hands on a wooden cube that has a circular top opening. Column two is Interpreting the emotion and shows two people looking down at a cube with notes nearby. Column three is Reflecting the symbolic notions and shows the cube on a table covered with pink, green, and orange notes. The section heading is printed as 10 Design critique (ANALYSIS). The prose says reflections were written on pink notes and questions on green notes.


Reading notes (site-compiled): Site-compiled. PDF page 2 places activity 10 in the Synthesis column of the overview, while this page's heading says ANALYSIS. Both labels are printed. This description does not collapse them into one phase. N=9 is printed on the photograph's caption and repeated in the prose.





### 11 Analyzing feedback (ANALYSIS)

The resulting remarks and questions were analyzed through Affinity Diagramming [18] and clustered into five main categories. We learned that the fabric's visual expression and the touching of the pneumatic object underneath contradicted each other. The furry and soft property gives viewers a sense of inviting and touching. However, the sturdy pneumatic material underneath the top surface gives an opposite feeling when petting it. Most of the participants indicated that the initial breathing behavior was calm and humble, but it later became aggressive when they approached the fabric. Moreover, we learned that participants were confused about the delay of Puffy's reaction. In some cases, the fabric and actuators did not synchronize well while reacting to a user's interaction.

### 12 Synthesizing feedback (SYNTHESIS)

We acknowledged that the invitingness of the surface material and the aggressive behavior conflicted too much. During the final step of our process we aimed to bring these two conflicting aspects more towards each other, to further align the physical form, temporal form, and interaction gestalt [3].



![11 Analyzing feedback (ANALYSIS)](/papers/puffy/images/v-p08-affinity.webp)

Visual description — v-p08-affinity

Source: upper two-thirds, affinity board for sections 11 and 12.

Original caption: 11 Analyzing feedback (ANALYSIS)

Short description: An affinity board groups notes under Other, Sound, Material, Behavior, and Interaction, with two large callout bubbles.


Long description:

One board fills the top of the page. A blue header reads DESIGN CRITIQUE OF PUFFY. Five column labels read OTHER, SOUND, MATERIAL, BEHAVIOR, and INTERACTION. Under them, smaller blue labels include Soft & Inviting, Material vs. Balloon, Aggressive, Soft, Slow & Fragile, Animal like & Playful, Inviting, and Centered Interaction. Pink and green notes fill the columns. Two large outlined circles carry short slogans: a green circle asks whether to keep the contradiction between fur and the solid balloon, and pink circles include "it may bite me!" and "I WANNA TOUCH IT!". A small miro mark sits at the lower right of the board. The board is the figure for both section 11 and section 12; the two paragraphs underneath are the analysis and the synthesis, not a second picture.


Reading notes (site-compiled): Site-compiled. The prose says Affinity Diagramming [18] produced five main categories, and that the furry surface invited touch while the pneumatic body felt opposing and later aggressive. Those conclusions are the paragraphs. The board shows the categories and a sample of note texts; most note bodies are too small to transcribe in full. The five header words are the visible top-level groups.





### 13 Detailing the design artifact (DETAILING)

Callout labels on the exploded view:

- Nested casing: outer box to create seamless finish
- Cilia-mimetic surface
- Aluminum foil connected with 12-pin MPR121 capacitive sensing touch sensor
- Nested casing: inner box to provide constructional support
- The inflatable inspired by [28, 35, 40], shaped elastic material made by silicon
- DC 12V Vacuum pumps enable the inflatables to inhale and exhale
- Stewart 6-axis actuating system, modifications based on [31]
- Arduino UNO MCU
- MOSFET amplifiers to control the vacuum pumps respectively
- Maintenance portal to refine and replace the hardware e.g., the inflatable

Photo labels:

- a Nested casing for seamless and aesthetic appearance
- b Shaving unwanted hairy fabric for seamless assembly
- c Attempting to assemble
- d The final assembly
- e Adjusting the actuators with slower and more fluent movements
- f Vertically placing the artifact high off the ground to ensure hands approaching from the edges



![13 Detailing the design artifact (DETAILING)](/papers/puffy/images/v-p09-assembly.webp)

Visual description — v-p09-assembly

Source: whole page, exploded technical view with photos a–f.

Original caption: 13 Detailing the design artifact (DETAILING)

Short description: An exploded cube labels the casing, cilia-mimetic surface, foil, inflatable, pumps, Stewart platform, Arduino UNO, MOSFET amplifiers, and a maintenance door, beside six assembly photos.


Long description:

This is one technical figure. On the left, six photographs are labeled a through f. a shows two wooden boxes, one with a white hairy top. b shows a tool trimming white fibers at an edge. c shows the hairy top being lowered onto a wooden box. d shows a closed grey cube with a white hairy top. e shows hands over a white disk inside an open round frame. f shows the cube raised, with a hand at the side. The printed captions are: a Nested casing for seamless and aesthetic appearance; b Shaving unwanted hairy fabric for seamless assembly; c Attempting to assemble; d The final assembly; e Adjusting the actuators with slower and more fluent movements; f Vertically placing the artifact high off the ground to ensure hands approaching from the edges. On the right, an exploded drawing labels, from top to bottom: Nested casing: outer box to create seamless finish; Cilia-mimetic surface; Aluminum foil connected with 12-pin MPR121 capacitive sensing touch sensor; Nested casing: inner box to provide constructional support; The inflatable inspired by [28, 35, 40], shaped elastic material made by silicon; DC 12V Vacuum pumps enable the inflatables to inhale and exhale; Stewart 6-axis actuating system, modifications based on [31]; Arduino UNO MCU; MOSFET amplifiers to control the vacuum pumps respectively; Maintenance portal to refine and replace the hardware e.g., the inflatable. The drawing shows a ring-shaped platform, rods, a clear dome, a blue board, red and black wiring, and a side panel with a round hole. No schematic voltages beyond the printed DC 12V label, and no measured sensor values, are shown.


Reading notes (site-compiled): Site-compiled. Every part name above is a printed callout on PDF page 9. Arduino UNO is printed here; it is not inferred from the unlabeled board sketch on PDF page 2. The silicon and MPR121 claims are labels in the figure, cited by the paper to [28, 35, 40] and [31] for the inflatable and the platform.





### Reflections

In this pictorial, we described the process of transforming lively inspirations into a cilia shape-changing artifact through an interactive materiality lens. Although our main approach is based on the three steps, i.e., Analysis, Synthesis, and Detailing (A-S-D) proposed in [30], from our first-person practice and instructions from Materiality of interaction [37], we found that it was not adequate to apply the approach for one round. Alternatively, the design activities (A-S-D) interlaced recursively several times, or in Wiberg's words, "work back and forth between wholeness to details" [37], to deepen our practical comprehension of materiality, as well as to evaluate and finetune the material experience and haptic qualities. The interlacing process is not to simply redo analysis or redesign concepts. Instead, we iteratively tackled the complexity of digital and analog materials and continually extracted insights for better material experiences.

Throughout the study, we found this design process resembles artists' creation process. Firstly, many artists have been cultivating to gain inspiration from nature (e.g., sketches of humans, animals, or plants). This consequently allows them to build a great repertoire of inspirational ideas. At the beginning of our design process, we looked for a nature analogy and were inspired by the pufferfish's form-changing and repelling behaviors. Secondly, they both do intense analysis and synthesis in the creation process. For instance, when Pablo Picasso created the famous painting The Bull in 1945, he went through several iterations to analyze the shape of a bull and portray it from hooved, horned, and muscled life-like to an abstract representation without losing identifiability. Such a process mirrors the 'Synthesis' activity of our approach that a designer synthesizes the most valuable elements from tons of inspirations. Lastly, they both require careful attention to the subtle expressivity of the materiality and created forms. A sculptor constantly negotiates the texture and its expressivity with his hands and eyes. Similarly, in our design process, we continuously explored the subtle touch feelings with alternative materials [14] and temporal forms [36] to investigate their expressivity through the creator's first-person experiences. In addition to the mentioned similar parts, there were salient differences in terms of design considerations and evaluation. Before entering the fine-tuning step of detailing the design artifact, we recruited participants for a design critique to collect, analyze and synthesize feedback from a third-person point of view. However, this might not be necessary for artists during their creation of art pieces.

Based on our practice, we thought our mixed materiality approach could instigate designers to focus on the emotional or experiential aspects in addition to the pragmatic or functional features of shape-changing interfaces (SCIs) [6, 21]. It may involve various creators and researchers interested in SCIs and TUIs to explore new design forms that deliver meaningful and affective experiences. By reflecting on our journey of design, we now summarize a set of design implications that are intended to inform interaction practitioners taking a similar approach: meaningful and affective experiences.

#### Leverage the open-endedness and unfinishedness in the early stage.

As described previously, in the early stage of this design approach, we started with extensive freedom for design explorations. While such a large amount of freedom also created uncertainty, in the end, we recognized the benefits of having multiple open-ended directions to explore. Such open-endedness might be a key to success in interactive materiality design. It enables us to avoid falling into the "trap" of being self-constrained by the availability, feasibility, or functionality of the materials we need in the coming phases. Also, just like other designerly or artistic crafting processes (e.g., with clay or wood), interactive materiality design heavily relies on the designer's embodied comprehension or the tacit feel for the designed materiality. And the freedom for exploration in the early stage ensures that the designers can conduct broad experimentations along various open-ended routes, which enables them to develop a sufficient feel for the crafting materials at hand. And this will extensively benefit the later design stages in which they need to decide which materials to use or how to polish the chosen materials further. And in such early open-ended experimentations, we also recommend that practitioners should not pursue the 'perfect' design samples but feel comfortable with the unfinishedness of the samples (e.g., taking a quick-and-dirty technique) so that the experimentations could yield richer design possibilities or options.

#### Enable multi-modal appreciation and documentation throughout the process.

Another implication we gained from reviewing this project is to have rich documentation that captures the subtlety, and temporality of shape-changing, for example, recording videos or sketching transition graphs as shown on p. 4. The purpose of having such documentation is not only for post-hoc analysis or inspiration for future practice, but also very much needed in supporting sensemaking and decision-making throughout the design process. In retrospect, the process started with selecting inspirations through abstracting features and fabricating and evaluating the materiality. Throughout these steps, we centralized our designer's embodied perception and comprehension of the material composites or materiality to enable our best multi-modal appreciation. As mentioned earlier, our design process has been constituted by multiple rounds of Analysis, Synthesis, and Detailing. In each round, we also heavily built upon the design rationales generated in previous rounds. However, much of our design experimentations, evaluations, and decisions cannot be fully communicated by texts, but we need to rely on visual communication. On the other hand, visual documentation can also help designers review how their decisions were made since design decisions can sometimes go intuitive and unconscious. Rich visual documentation using video clips, photos, or sketching could therefore benefit communication and deliberation throughout the process.


### Emphasize the hedonic and experiential aspects in the exploration.

As demonstrated in our process, our exploration has focused heavily on the nuances of the hedonic and experiential aspects of the materiality and the designed artifact. And we recognize this as an advantage of such interactive materiality approach, which could complement the design approaches that purely focus on the pragmatic aspects of design (e.g., utility, usability, or efficiency). With our addressed approach, much of the designer's attention could be effectively directed to the subtle differences of the sensory, experiential, and aesthetic aspects of the interactive artifacts, with the very depth that is often not likely to achieve in pragmatic approaches of design. From our own experiences, such an approach could shift designers from a problem-solving mindset to a curiosity-driven mindset, and help them get immersed in the playful, embodied, and purposeless experimentations with computational and analog materials at hand. Therefore, we recommend that such an interactive materiality approach could be more widely adopted as a complementary or additional method to traditional interaction design processes so that the designers could be equally facilitated in both exploring the pragmatic qualities and the hedonic qualities.

In short, we provide the following takeaways for those intending to leverage bio-inspiration for interactive materiality:

1. during the early stage, try freely exploring inspiration sources; try appreciating them and documenting the shape transitions, material textures, and underlying intentions in a multi-modal manner; and avoid concerning the technical and engineering feasibility;
2. while mimicking, try reasoning the relationship between the selected natural element and its intentions/surroundings; try mimicking that with rough setups to experience effects addressed on potential viewers;
3. while fabricating, try hacking, mixing, and iterating prototypes with parts from daily life or the HCI community;
4. lastly, try centralizing sensations and experiences throughout all steps, aiming to reveal nuanced qualities rather than exactly replicating the natural elements.

### Future work

When considering potential application areas, stimulating multi-modal and haptic-rich user-product relationships seem prominent. By exploiting the physically rich characteristics of interactive materiality instead of using vocal or graphical UIs, we might be able to achieve the goal of creating physically rich interactions [8, 9]. Specifically, as Puffy reveals different degrees of shape changes, it might benefit some fabric-wrapped yet screenless devices such as internet-of-things virtual assistants (e.g., Apple HomePod, Google Home mini) or smart home appliances such as OSKARRR [32] to exhibit rich haptic experiences as well as anthropomorphic attributes with dynamic forms. In these ways, Puffy's cilia-mimetic surface could change its shape in accordance with the mood or intention of the interactive systems.

### Limitations

Although we used the puffer fish which is more object-like as inspiration, the end artifact turned out to be more surface-like. This might be self-limiting in terms of interaction possibilities. However, as we aimed to focus solely on the material's quality and the designed interactions, for this project, we argued for experiencing and evaluating materiality in an exploratory and first-hand manner over biologically or mechanically mimicking a puffer fish. With these considerations in mind, we leaned towards a quick and flexible setup that allowed us to swap material samples handily and promptly. Otherwise, we might fall into the 'trap' of concerning technical and engineering matters.

### Suggestions for prototyping

#### High degrees of durability with high degrees of hot-swap functionality

Shape-changing prototypes need to be highly durable as they are expected to interact physically. Yet, the materiality experience only works the best and most authentic when all materials and elements are entangled as an assembled whole. That means, the materiality shall be experienced and evaluated through the computational whole instead of any division part of it. However, as modifications might be applied during the fine-tuning process, it might be tricky if a maintenance portal or so-called the 'backdoor' is not configurated beforehand. Hereby, we highlight the hot swap functionality design in designing interactive materiality.

#### Hot-swap 1: maintenance portal - the "backdoor" design

Experiencing and evaluating interactive materiality requires all materials, both digital and analogue ones, to be integrated as a whole. And yet, during our materiality experience examination with the initial computational whole, the inflatables blew out serval times due to pump overinflation caused by false sensing detection. The disassembling and reassembling of the MDF box made us to implement a "backdoor" for replacing the inflatables handily. The portal also benefited us when fine-tuning Puffy with different thicknesses of rubber for the inflation. Thus, while crafting, question yourself:

1. Will your prototype involve consumables that are fragile?
2. Would it be convenient to replace them when they are assembled inside a computational whole, and how?

#### Hot-swap 2: nested casing

To ensure both durability and hot-swap functionality, we suggest building the computational whole in a nested structure. For example, Puffy is configured within a double-nested box: the inside one (assembled as a tooth-and-slot box) provides construction, and the outside one (painted in grey) creates a seamless appearance of Puffy.


### Hacking, mixing, and iterating prototypes as an approach

Before originating your prototype for creating materiality, we recommend hacking, mixing the existing prototypes from everyday life or the HCI communities, and trying to iterate those prototypes into a satisfying form. In reviewing the process of crafting Puffy, we attempted to hack, e.g., Ink printer or Joystick Stewart considering their actuating systems; we tried to integrate the inflatables with the Stewart 6DOF platform, which both have been extensively investigated in the field of morphing material and robotics respectively and incorporated sensing technologies and fabric materials iteratively to create the desired pufferfish-alike materiality experiences. While iterating, we started with the most accessible materials (e.g., balloons, latex gloves, etc.) to familiarize ourselves with the physicality and gradually shifted to the exploration and fabrication of custom-shaped materials.

Diagram title: Hacking and mixing materials coming from daily life or the HCI community. The center is labeled iterations, process inspired by [28, 35, 40], leading to New compositions or customized materials.

Labels in the mixing sequence: Mixing liquid silicon part A & part B; A; B; Pouring liquid silicone mixture into 3D printed mould; Shaped elastic rubber.

Assembly labels: M3x6 bolts; Top frame; Airtight piece; Supporting plate; Stewart connector; M3x6 nuts; Assembly.



![Hacking and mixing materials coming from daily life or the HCI community](/papers/puffy/images/v-p12-mixing.webp)

Visual description — v-p12-mixing

Source: top half, hacking and mixing diagram.

Original caption: Hacking and mixing materials coming from daily life or the HCI community

Short description: A left-to-right process shows found objects, silicone mixing, an exploded inflatable stack, and finished disks.


Long description:

The diagram is one process, not four separate figures. A dotted arrow runs from the left title to the right title "New compositions or customized materials". The center is labeled iterations, process inspired by [28, 35, 40]. On the left, circular photographs show a green balloon on a mechanism and two yellow balloon-like forms. The mixing sequence shows beakers marked A and B, the words Mixing liquid silicon part A & part B, and Pouring liquid silicone mixture into 3D printed mould, ending at a disk labeled Shaped elastic rubber. The assembly stack is labeled M3x6 bolts, Top frame, Airtight piece, Supporting plate, Stewart connector, M3x6 nuts, and Assembly. The right side shows a round furry photograph and a fan of clear and white disks. Bolt size M3x6 is printed. No other dimensions are printed.


Reading notes (site-compiled): Site-compiled. The prose below recommends hacking everyday or HCI prototypes, names an ink printer and a joystick Stewart as examples the authors tried, and says early trials used balloons and latex gloves before custom-shaped materials. The diagram shows the silicone pour and the labeled stack. It does not show the ink printer or the joystick.




### Conclusion

As material, sensors, and actuators are becoming more entangled in forming new interactions, HCI scholars have started to take materiality as an entry point for conducting design research. Nonetheless, design cases and instructions that show designerly ways of attending materiality in design practices are still needed. This pictorial presents a concrete case of designing a shape-changing artifact using the materiality approach. The approach consists of three key activities (analysis, synthesis, and detailing) interlaced recursively along the whole design process. The 'analysis' activity gained nature-inspired analogy and iteratively explored from shape transitions through analog materials and computational mechanisms to gain an understanding of the design context; The 'synthesis' activity synthesized findings regarding digital and analog materials, both self-reflection and peers' critiques, to navigate the consecutive activity; The 'detailing' activity encoded the designers' symbolic notions of the interactive materiality as well as the synthesized critique from the audience into a set of iterative prototypes. Our reflection surfaced the value of having such an interlaced iterative process. As a result, by offering a reflective analysis of our approach, we contribute an illustrative step-by-step guide of our highly embodied design process and a set of practical implications and suggestions, to inspire future creators to design interactions with interactive materiality.

### Acknowledgment

The authors want to appreciate the technical support from Jasper and Chet at the Rapid Prototyping lab (TU Eindhoven), Material Workshop arrangement by Simone G. de Waart, substantial suggestions from the anonymous reviewers, kind advice from Prof Stephan Wensveen and Jeffrey Ho, supplementary video clip by Wei Lai. This project is supported by the UGC Funding Scheme (RHCE & G.73.xx.R006) from The Hong Kong Polytechnic University.


### References

[1] Jason Alexander, Anne Roudaut, Jürgen Steimle, Kasper Hornbæk, Miguel Bruns Alonso, Sean Follmer, and Timothy Merritt. 2018. Grand Challenges in Shape-Changing Interface Research. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18, 1–14. https://doi.org/10.1145/3173574.3173873

[2] Miguel Bruns, Stijn Ossevoort, and Marianne Graves Petersen. 2021. Expressivity in Interaction: a Framework for Design. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1–13. https://doi.org/10.1145/3411764.3445231

[3] Peter Dalsgaard, Kim Halskov, and Ole Sejer Iversen. 2016. Participation Gestalt: Analysing Participatory Qualities of Interaction in Public Space. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 4435–4446. https://doi.org/10.1145/2858036.2858147

[4] Tanja Döring, Axel Sylvester, and Albrecht Schmidt. 2012. Exploring material-centered design concepts for tangible interaction. In CHI '12 Extended Abstracts on Human Factors in Computing Systems (CHI EA '12), 1523–1528. https://doi.org/10.1145/2212776.2223666

[5] Paul Dourish. 2017. The Stuff of Bits: An Essay on the Materialities of Information. MIT Press, Cambridge, MA. https://doi.org/10.7551/mitpress/10999.001.0001

[6] Sean Follmer, Daniel Leithinger, Alex Olwal, Akimitsu Hogge, and Hiroshi Ishii. 2013. inFORM: dynamic physical affordances and constraints through shape and object actuation. In Proceedings of the 26th annual ACM symposium on User interface software and technology - UIST '13, 417–426. https://doi.org/10.1145/2501988.2502032

[7] Christopher Frauenberger. 2019. Entanglement HCI The Next Wave? ACM Transactions on Computer-Human Interaction 27, 1: 2:1-2:27. https://doi.org/10.1145/3364998

[8] J.J. Frens. 2006. Designing for Rich Interaction: Integrating Form, Interaction, and Function. Technische Universiteit Eindhoven, Eindhoven, The Netherlands.

[9] Joep Frens, Mathias Funk, Bastiaan van Hout, and Joep Le Blanc. 2018. Designing the IoT Sandbox. In Proceedings of the 2018 Designing Interactive Systems Conference (DIS '18), 341–354. https://doi.org/10.1145/3196709.3196815

[10] Elisa Giaccardi and Elvin Karana. 2015. Foundations of Materials Experience: An Approach for HCI. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 2447–2456. https://doi.org/10.1145/2702123.2702337

[11] S. Gross, Jeffrey Bardzell, and Shaowen Bardzell. 2013. Structures, forms, and stuff: the materiality and medium of interaction. Personal and Ubiquitous Computing 18: 637–649. https://doi.org/10.1007/s00779-013-0689-4

[12] Jeffrey C. F. Ho. 2019. ClothSurface: Exploring a Low-Cost Prototyping Tool to Support Ideation for Shape Displays. Multimodal Technologies and Interaction 3, 3: 64. https://doi.org/10.3390/mti3030064

[13] Yuhan Hu, Zhengnan Zhao, Abheek Vimal, and Guy Hoffman. 2018. Soft skin texture modulation for social robotics. In 2018 IEEE International Conference on Soft Robotics (RoboSoft), 182–187. https://doi.org/10.1109/ROBOSOFT.2018.8404917

[14] Heekyoung Jung and Erik Stolterman. 2010. Material probe: exploring materiality of digital artifacts. In Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction (TEI '11), 153–156. https://doi.org/10.1145/1935701.1935731

[15] Heekyoung Jung and Erik Stolterman. 2011. Material probe: exploring materiality of digital artifacts. Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction. https://doi.org/10.1145/1935701.1935731

[16] E. Karana, B. Barati, V. Rognoli, and A. Z. V. D. Laan. 2015. Material Driven Design (MDD): A Method to Design for Material Experiences. International Journal of Design 9: 35–54.

[17] Sofie Kinch, Erik Grönvall, Marianne Graves Petersen, and Majken Kirkegaard Rasmussen. 2013. Encounters on a shape-changing bench: exploring atmospheres and social behaviour in situ. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction - TEI '14, 233–240. https://doi.org/10.1145/2540930.2540947

[18] Andrés Lucero. 2015. Using Affinity Diagrams to Evaluate Interactive Prototypes. In Human-Computer Interaction – INTERACT 2015, Julio Abascal, Simone Barbosa, Mirko Fetter, Tom Gross, Philippe Palanque and Marco Winckler (eds.). Springer International Publishing, Cham, 231–248. https://doi.org/10.1007/978-3-319-22668-2_19

[19] M. A. McEvoy and N. Correll. 2015. Materials that couple sensing, actuation, computation, and communication. Science 347, 6228: 1261689. https://doi.org/10.1126/science.1261689

[20] Hila Mor, Ken Nakagaki, Yu Tianyu, Benjamin Harvey Miller, Yichen Jia, and Hiroshi Ishii. 2020. Prototyping Interactive Fluidic Mechanisms. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction, 881–884. https://doi.org/10.1145/3374920.3374967

[21] Ken Nakagaki, Luke Vink, Jared Counts, Daniel Windham, Daniel Leithinger, Sean Follmer, and Hiroshi Ishii. 2016. Materiable: Rendering Dynamic Material Properties in Response to Direct Physical Touch with Shape Changing Interfaces. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI '16, 2764–2772. https://doi.org/10.1145/2858036.2858104

[22] Ryosuke Nakayama, Ryo Suzuki, Satoshi Nakamaru, Ryuma Niiyama, Yoshihiro Kawahara, and Yasuaki Kakehi. 2019. MorphIO: Entirely Soft Sensing and Actuation Modules for Programming Shape Changes through Tangible Interaction. In Proceedings of the 2019 on Designing Interactive Systems Conference (DIS '19), 975–986. https://doi.org/10.1145/3322276.3322337

[23] Jifei Ou, Gershon Dublon, Chin-Yi Cheng, Felix Heibeck, Karl Willis, and Hiroshi Ishii. 2016. Cilllia: 3D Printed Micro-Pillar Structures for Surface Texture, Actuation and Sensing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 5753–5764. https://doi.org/10.1145/2858036.2858257

[24] Isabel P. S. Qamar, Katarzyna Stawarz, Simon Robinson, Alix Goguey, Céline Coutrix, and Anne Roudaut. 2020. Morphino: A Nature-Inspired Tool for the Design of Shape-Changing Interfaces. In Proceedings of the 2020 ACM Designing Interactive Systems Conference, 1943–1958. https://doi.org/10.1145/3357236.3395453

[25] A. Rapp. 2020. In Search for Design Elements: A New Perspective for Employing Ethnography in Human-Computer Interaction Design Research. International Journal of Human-Computer Interaction. https://doi.org/10.1080/10447318.2020.1843296

[26] Majken K. Rasmussen, Esben W. Pedersen, Marianne G. Petersen, and Kasper Hornbæk. 2012. Shape-changing interfaces: a review of the design space and open research questions. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems - CHI '12, 735. https://doi.org/10.1145/2207676.2207781


[27] Erica Robles and Mikael Wiberg. 2010. Texturing the material turn in interaction design. In Proceedings of the fourth international conference on Tangible, embedded, and embodied interaction, 137–144. https://doi.org/10.1145/1709886.1709911

[28] Marie Louise Juul Søndergaard, Ozgun Kilic Afsar, Marianela Ciolfi Felice, Nadia Campo Woytuk, and Madeline Balaam. 2020. Designing with Intimate Materials and Movements: Making "Menarche Bits." In Proceedings of the 2020 ACM Designing Interactive Systems Conference (DIS '20), 587–600. https://doi.org/10.1145/3357236.3395592

[29] Jelle Stienstra, Miguel Bruns Alonso, Stephan Wensveen, and Stoffel Kuenen. 2012. How to design for transformation of behavior through interactive materiality. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction Making Sense Through Design - NordiCHI '12, 21. https://doi.org/10.1145/2399016.2399020

[30] Jelle Stienstra, Miguel Bruns Alonso, Stephan Wensveen, and Stoffel Kuenen. 2012. How to design for transformation of behavior through interactive materiality. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design (NordiCHI '12), 21–30. https://doi.org/10.1145/2399016.2399020

[31] ThomasKNR. Arduino Controlled Rotary Stewart Platform. Instructables. Retrieved August 6, 2022 from https://www.instructables.com/Arduino-controlled-Rotary-Stewart-Platform/

[32] James Thorp, Daniel Richards, Nick Dunn, Michael Stead, and Katerina Gorkovenko. 2021. OSKARRR: Data-driven Design Speculations For The Future of Domestic Waste. Springer, Hong Kong. https://eprints.lancs.ac.uk/id/eprint/162624/

[33] Jonas Togler, Fabian Hemmert, and Reto Wettach. 2009. Living interfaces: the thrifty faucet. In Proceedings of the 3rd International Conference on Tangible and Embedded Interaction - TEI '09, 43. https://doi.org/10.1145/1517664.1517680

[34] Lianne Toussaint and Marina Toeters. 2020. Keeping the Data within the Garment: Balancing Sensing and Actuating in Fashion Technology. In Proceedings of the 2020 ACM Designing Interactive Systems Conference, 2229–2238. https://doi.org/10.1145/3357236.3395577

[35] Daniela Ghanbari Vahid, Lee Jones, Audrey Girouard, and Lois Frankel. 2021. Shape Changing Fabric Samples for Interactive Fashion Design. In Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction, 1–7. https://doi.org/10.1145/3430524.3440633

[36] Anna Vallgårda. 2014. Giving form to computational things: developing a practice of interaction design. Personal and Ubiquitous Computing 18, 3: 577–592. https://doi.org/10.1007/s00779-013-0685-8

[37] Mikael Wiberg. 2014. Methodology for materiality: interaction design research through a material lens. Personal and Ubiquitous Computing 18, 3: 625–636. https://doi.org/10.1007/s00779-013-0686-7

[38] Mikael Wiberg. 2018. The materiality of interaction: notes on the materials of interaction design. The MIT Press, Cambridge, Massachusetts.

[39] Amy K.M. Winters, Simone G. de Waart, and Miguel Bruns. 2022. Tension-and-Release: A Design Principle for Dynamic Materials. International Journal of Design 16, 2: 1–14. https://doi.org/10.57698/v16i2.01

[40] Lining Yao, Ryuma Niiyama, Jifei Ou, Sean Follmer, Clark Della Silva, and Hiroshi Ishii. 2013. PneUI: Pneumatically Actuated Soft Composite Materials for Shape Changing Interfaces. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13), 13–22. https://doi.org/10.1145/2501988.2502037

[41] Lining Yao, Helene Steiner, Wen Wang, Guanyun Wang, Chin-Yi Cheng, Jifei Ou, and Hiroshi Ishii. 2016. Second Skin: Biological Garment Powered by and Adapting to Body in Motion. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16), 13. https://doi.org/10.1145/2851581.2889437

[42] Iddo Yehoshua Wald and Oren Zuckerman. 2021. Magnetform: a Shape-change Display Toolkit for Material-oriented Designers. In Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction, 1–14. https://doi.org/10.1145/3430524.3446066

[43] Ce Zhong, Ron Wakkary, Amy Yo Sue Chen, and Doenja Oogjes. 2021. deformTable: Crafting a Shape-changing Device for Creative Appropriations Over Time. In Designing Interactive Systems Conference 2021 (DIS '21), 1253–1265. https://doi.org/10.1145/3461778.3462112

[44] Ce Zhong, Ron Wakkary, Xiao Zhang, and Amy Yo Sue Chen. 2020. transTexture Lamp: Understanding Lived Experiences with Deformation Through a Materiality Lens. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20), 1–13. https://doi.org/10.1145/3313831.3376721

[45] AskNature. AskNature. Retrieved February 11, 2021 from https://asknature.org/