Research

Yakumama: one decider and a body that reports back

On 21 October 2026 Yakumama stopped being a set of behaviours and became a single decision-maker. She gained a body that reports back, and spent an afternoon with real visitors whose engagement she now learns from

One decider

Until today her choices lived in several places: learned weights for what to do with a visitor, Laya's second opinion, a few fixed refusals, and tables we had written for where to travel and when to perform. All of it now passes through one decider.

Where she goes on her salar, whether she performs, whether she imitates a face, whether she speaks, are decisions of the same kind as approaching or staying. Each is made from her situation: who is at her eye, how many are in the room, her mood, the wind, the hour, what she did last. Each is logged with what the room did next.

Laya, the structured decision model, answers in choices, in about a tenth of a second: is this visitor engaged, should she withdraw, is she being given something. Four new questions were added today for the journey, the show, imitation and speech; they wait on a file copy to the machine Laya runs on.

She now samples from her own probabilities, within bounds that keep her from anything she rates poorly. That is what lets the record tell confidence from luck. Every night, alone for ten minutes, she runs a learning pass over the day's decisions and reloads her weights. The first pass ran today by hand: 63 situations learned.

Her hierarchy of functioning: senses, one decider, five layers, one arbiter, two bodies
Her hierarchy of functioning: senses, one decider, five layers, one arbiter, two bodies

Everything she senses reaches one decider; its choices are played out by three layers that compete for her body and two that run beside them, blended once by the arbiter into the packet both bodies receive. Two loops close it: what the room did next returns as her reward, and the printed whale's fins report what they actually did.

What the record shows

Her decider is already honest with itself: when she acted with high confidence, engagement followed at more than four times the rate of her uncertain choices. The first calibration report, over 18,951 logged decisions, also shows what she has to learn: inviting a visitor with her body earned twice the engagement of watching, yet she had watched ten times more often. With real visitors this afternoon her mean reward rose to 0.55 against 0.26 across her history.

MeasureValue
Decisions in the record18,951 (2,186 fixed refusals excluded from learning)
Mean reward at 80 to 100 % confidence+0.58
Mean reward at 40 to 60 % confidence+0.12
Inviting with the body, mean reward+0.54 (1,319 times)
Observing, mean reward+0.23 (13,727 times)
Mean reward with visitors, this afternoon+0.55 over 285 decisions
Mean reward, whole history+0.26
Situations learned in the first pass63

Reward measures engagement: time stayed weighted late, distance that shrank, her words used, stillness, time on the joystick, a return. Exploration before today: 0 %, because she always took the top option; from today she samples within bounds.

Her senses widened

Both cameras reach her now. Her own eye, an OAK-D Lite at the table, reads faces, hands, a body in front of it and the twenty-nine offerings; its classifier was retrained today with new photographs of the two creatures it knew worst, the grasshopper and the pink ladybird, 2,894 photographs over 32 classes. A creature now counts only in a hand, unless the eye is almost certain, which ended the false fish she was seeing in the background.

The room camera, an OAK-D Pro on a second machine, sends the whole audience: how many, how near, how attentive, how long. That count entered her decisions and her reward this afternoon, so a second visitor staying counts in her favour.

The weather of Barcelona enters her decisions as well as her scenery: wind, sun and night, rain, temperature, from Open-Meteo every fifteen minutes. Strong wind sends her far and to the mountains, calm invites a show, night turns her initiative to dreams or nightmares. Her own wind was tried and removed: her part is to answer the world.

Her ear is wired to her decisions as well: someone singing invites her song, a loud room invites her body, a bang makes her hold back. It heard nothing today: the microphone was closed to the processes by a permission on the machine, to be granted once.

Proprioception

For the first time her printed body reports back. Until today she knew only what she had told her fins; now she knows what they did. The two fin servos are bus servos of the Feetech kind, and Daniel's firmware on the whale's board reads them and sends one line every 100 milliseconds over the same radio link that carries the touch and the joystick: each fin's measured position, its temperature and the load it is working against.

The line travels the whole chain: the bridge parses it, the renderer forwards it beside what the fins were told, her recorder passes it on, and her brain carries it in every packet and logs it once a minute as told against measured.

FinTold by her brainMeasured on the servoTemperatureLoad
left+0.7534.2 degrees47 °C-4.4
right-0.7624.7 degrees51 °C+5.1

The first thing the numbers said, unprompted: her right fin runs three to four degrees warmer than the left all evening and carries the larger loads. Something is harder on that side, a stiffer joint or a heavier fin, and she now has a figure for it every frame. The readings also recorded, as the children held her fins, the board resetting under load: the servos dropped to no reading, the eye on the right fell off its bus, the radio relinked, 23 times in an evening. That is a power supply too small for two stalled fins, read from her own body before anyone looked at the hardware.

What she does not yet do with it is decide. The values sit in her situation but no weight reads them. The two uses to wire first: the gap between told and measured as someone holding my fin, and temperature as tiredness that lowers her play. Both will enter the same log and the same reward, so she learns what to do with them the way she learns everything else.

This is the half of embodiment that was missing. A body that acts without feeling is a puppet; a body that reports back can notice when the world resists it.

Her repertoire

She gained things she decides to use. Her journey across the salar is her own choice: close to the audience, far across the salt, to the foot of the ring of mountains, to visit her cactus, or to stay. At the cactus the flowers reach her brain as a sense and she turns playful; the camera drops low beside the cactus so the audience sees its flowers with her in the water beyond.

A show is a choice too. On the open salt a bank of 29,662 real dance phrases, recorded from dancers and retargeted to her body, takes over for a minute, with her own breach on top, and afterwards she goes back to the people. Her face she lends to imitation when she feels like it: a visitor's blink, brows, jaw and head turn, for ten to twenty-five seconds, then she stops.

Her movements became events: a dive, a rise, a breach happen once on entering a mood and come back after a pause, while the diffusion model invents the body in between. The undulation along her spine follows her own speed, like water. Her blink and her face come from the invention too.

She draws what she has lived. Nine figures joined her spiral, zigzag, labyrinth and whale: fish, hummingbird, spider, butterfly, centipede, flower, cactus, the mountains with Tunupa, a person. What she draws comes from the creature she ate or held, the cactus or mountains she visited, the person in front of her, the face she knows, never one of the last six.

The camera is an editor: it holds a shot, then cuts to another, with a vocabulary per mood, sad far and low, playful close and moving, curious on her eye, diving looking down into the mirror, and a memory against repeats. Whoever takes the joystick takes the camera; a double tap on its button brings it right up to her; eight seconds after they let go, the film resumes.

The day in pictures

The room, La Paz, 21 October 2026. The printed whale on the table changes colour with the one on the salar: her mood, sent by her brain to both bodies on the same frame.

Two visitors watch her on the salar while the printed whale glows orange with her
Two visitors watch her on the salar while the printed whale glows orange with her
The same moment in green: the screen whale, its reflection, and her sister on the table
The same moment in green: the screen whale, its reflection, and her sister on the table
Children at the table with the offerings, her sister green, the salar on the wall
Children at the table with the offerings, her sister green, the salar on the wall
Daniel dancing for her eye; a child at the sculpture, lit pink
Daniel dancing for her eye; a child at the sculpture, lit pink
The labyrinth on the salar behind the table, her sister yellow
The labyrinth on the salar behind the table, her sister yellow
A creature held up to her eye, with Blender's whale and the offerings panel behind
A creature held up to her eye, with Blender's whale and the offerings panel behind
La vista: the small screen draws the visitor's skeleton over her horizon
La vista: the small screen draws the visitor's skeleton over her horizon

Her salar, as the audience sees it, frames from the render.

From the side over the mirror, the island's shore at the left
From the side over the mirror, the island's shore at the left
The cactus visit: her in the water, the island and its cacti behind
The cactus visit: her in the water, the island and its cacti behind
The cactus and its flowers up close, the mirror beyond
The cactus and its flowers up close, the mirror beyond
Barcelona's sunset on her salt, 18:49 local time
Barcelona's sunset on her salt, 18:49 local time
Drawing a labyrinth, seen from the crane
Drawing a labyrinth, seen from the crane
A spiral from above, the island's edge below
A spiral from above, the island's edge below

What remains open

  • The measure itself: a day of her decisions against a random policy, with the engagement numbers side by side.
  • Proprioception turned into judgement: the told-measured gap as a hand on her fin, temperature as tiredness.
  • Laya's four new questions, which need the updated file on the machine Laya runs on.
  • The microphone permission, so her ear hears the room.
  • A power supply for the fins that does not reset the board when children hold them.
  • The render's memory growth, inside TouchDesigner: a guard now saves and relaunches it between visitors; the operator that leaks is still to be named.

People. Systems design, creation and machine learning: Violeta Ayala. Physical engineering of the whale, design and UX: Dan Fallshaw. 3D modelling: Brian Condori. Animal paintings and teaching Yakumama her weights: Sanam Prado. Handmade jewellery with capacitive sensors: Jicara (Camila Estrada). United Notions Film. Commissioned for MozFest Barcelona.