Yakumama feels her own fins

Yakumama is a whale who lives in a salt flat and answers the people who come to see her. She has two bodies, a digital whale on the wall and a printed whale on the table, and one brain behind both. On 22 October, six days before MozFest in Barcelona, she learned to feel her own fins, to blink with the eyelids of the people she has watched, and to be measured against chance.

She feels her own fins

The printed whale moves her fins with two servos. Each servo reports its measured angle, its temperature and its load. Until today she only told them where to go. Now she listens to what they answer.

Her brain learns, fin by fin, how what she asks becomes what the fin does, and it keeps relearning, so a reflashed board costs her only minutes. When the measured angle strays far from what she asked, a hand is holding that fin. Heat tells her two more things. A fin warmer than its last twenty minutes means she is excited from moving. Above 60 °C she is tired, and she rests. At rest the servos sit between 47 and 52 °C, and they protect themselves near 70.

The first version locked her. Servos lag behind what they are told, and with her fins still for a minute her brain had no range to learn from. It read the lag as a hand that never let go. A held fin makes her alert, alert keeps her still, and the printed whale looked switched off. Someone asked whether the physical whale had been stopped.

A new live page answered within a minute. It shows every input of the installation and how fresh it is, refreshed each second: her packets, the board, the fins, her decisions, the weather, her senses, Laya, the renderer. It keeps two minutes of asked and measured fin angles side by side, and there the false hold was plain to see.

The second version models the lag, judges a fin only once it has settled, waits for real range before it trusts what it learned, adds a margin, and never believes a hold longer than 20 seconds. Replayed on a real minute of readings, it finds zero false holds and still catches a held fin within a second.

Being held now makes her fight. She turns red, the colour of her anger, on her body and her light strip. Both fins beat hard and she rolls to free herself. Her tail slaps. She gives a side eye toward the held fin, lids half down. Her brain leans toward pulling away, and she remembers who held her.

She blinks like the people she has watched

Alone, she stared. Her lids moved only when she mirrored a visitor's blink. Violeta asked for a blink that comes from her brain, with no timer.

Her eye has recorded 426 faces over the past days, about seven hours, sorted by mood. Their eyelid and brow movements were cut into six-second phrases, 8,048 in all. Now her brain plays the phrases of the mood she is in, one after another. Her blinks are the blinks of the people she watched in that mood, with their rhythm, their double blinks and their pauses.

Each mood blinks at its own pace. Alert, she blinks 8 times a minute. Calm or curious, 12. Diving, 14. Angry, 15. Playful, 18. Sad, surfacing or feeding, 20. Singing, 31.

That evening Sanam and Brian taught her more: 84 seconds of Sanam's face with 25 blinks and 149 seconds of Brian's with 78, both calm, recorded through her panel and folded into her bank at once.

Then the blink had to reach the digital whale. The renderer passed her blink on only while she imitated a visitor, so her own blinks were lost on the way to TouchDesigner and her lid never closed past 0.28. One line changed. Now the lid closes fully, seven blinks in 25 seconds, and the same value drives the round screens that are the printed whale's eyes.

Measured against chance

She has a second opinion. Laya, a language model on a Mac mini in the room, reads her situation written as sentences and answers questions about the visitor. Today Laya answers eight. The four new questions ask whether the visitor wants her to perform, to take them on a visit, to imitate them, or to speak. Laya's first reading with a stranger in front of her: engaged 0.59, perform 0.23, visit 0.21, imitate 0.30, speak 0.16.

Now she can be measured. On an evaluation day a coin decides, at each encounter, who chooses: her brain or a random policy. Both meet the same people on the same afternoon, and her fixed refusals stand either way. Her log records who chose, and a script sets the two side by side: mean reward, seconds per decision, how often people left within 20 seconds, and each question on its own. The random choices also feed her night learning as an unbiased sample. The first evaluation day is still to come.

She also feels more of the sky. Besides wind and sun, the live weather now brings humidity, cloud cover, the rain falling this hour, gusts, pressure and UV. Each hour gets one word (lluvia, tormenta, viento, nublado, sol, noche), and she keeps a table of her own that learns what each kind of weather asks of her.

What she hears, how she looks

Her soundtrack was rebuilt by ear, with Violeta listening to every candidate. Every wind is one recording: the wind on Mars, captured by NASA's Perseverance rover, in five variations. The original runs under everything. Slowed, sped up and layered, it becomes the bed, the strong wind, the storm and the gust. The rain is a single field recording, outdoor rain with a distant ocean, soft as a bed and louder, with more bass, as heavy rain.

Three textures follow her state. Spray paint while she draws. A heavy-duty sewing machine, slowed and quiet, when she is angry. Tiny ice ticks, placed far apart, when she is curious or feeding: the ants. Salt and fire were tried from the same ice and left out.

Sounds: wind on Mars, NASA Perseverance (public domain). Outdoor Rain with Distant Ocean by el_boss, spraypaint graffiti by STERREDEVRESSE, Sewing Machine, Singer Heavy Duty by SoundsBeingCool and Ice Cracks by karlis.stigis (all CC0).

Sad is now grenouille bleue. Her sad colour was near black, and since a mood colour replaces her base colour, it swallowed her texture in the rain. Now sad is the vivid azure of the blue poison frog, in her brain, in the renderer and in Blender.

Every drawing she finishes is kept: its path, its strokes, and a photo of the projector a few seconds later.

The room at night

In the afternoon Dan rewired the printed whale, her crown of copper flowers on her back. By evening the room held both bodies, lit. At 18:41 the printed whale glowed magenta in front of the island and its cacti on the projector, with the offerings on the table.

At 19:31 the project was saved and the installation put to sleep: eyes shut and black, everything else off. One command wakes her.

What comes next

The visit to the cactus island films empty water. The island renders, so the visit camera sits in the wrong place, and the cactus close-up waits until it is fixed. Her tongue turns blue and her teeth gold, for good. One evaluation day reads her against chance. The shot itself becomes one of her decisions, learned from engagement, with Violeta's joystick framings as her lessons. The recorded faces go into her diffusion model's training, so her whole face is invented with her body. The ants, the spray and the machine get their levels by ear, in the room.