Academic
AI
Experimental

Human–AI collaboration

Human–AI collaboration exploring how communication pathways shape collaboration between 2 perspectives and AI. Combining multi-agent simulation, A/B experiments, and participant evaluation, it turns a keyword-based control into a toolkit-guided agent that makes need interpretable and design decisions inspectable. Participants rate, reconstruct, and annotate generated outputs, connecting human intent to actionable feedback.

My role: Design engineer

Coding and prototyping

Years

2026 Sept - Nov

Patents

Utility & design

Problem

When two people with different needs brief an AI through each other, details drop out at every hand-off. The AI ends up building for the loudest keyword instead of for either person.

Toolkit

Helps you

Captures everything people say as separate cards, so no need gets merged away or reduced to a keyword.

How it works
  1. Pin each thing a person says as its own card, in their words.
  2. Note the need, why it matters, any concern, and when it comes up.
  3. Keep who said it and how, so every card can be traced back.
In the study

106 cards. A keyword builder kept only 0.57 keys per sentence from the same words.

Helps you

Clears up ambiguous words before one guess quietly becomes the design.

How it works
  1. When a phrase could mean more than one thing, write two or three readings.
  2. Turn them into one short clarifying question.
  3. Let the person tap the reading they meant, or none of them.
In the study

Asked 10 times, answered twice.

Helps you

Moves from the part someone asked for to the purpose behind it, so a different part can do the job.

How it works
  1. Write down the part they named.
  2. Ask what it is for and note it as a hypothesis.
  3. Build for the purpose, with that part or a better one.
In the study

Used 3 times. It is what lets another part meet the need.

Helps you

Makes sure every need gets an explicit decision instead of silently dropping out.

How it works
  1. Go through the wall card by card.
  2. Answer each with a part, a new part, a rule, or “no part” and why.
  3. Keep every decision in one shared record.
In the study

31 answers: 20 new parts and 11 rules. 72 cards still unverified.

Helps you

Lets people confirm the AI understood them before anything gets built.

How it works
  1. Play each decision back in plain words.
  2. Ask: did I get that right?
  3. Let the person accept or decline, and revise what they decline.
In the study

0 of 31 accepted. This is the biggest gap.

Helps you

Checks from the viewer’s side whether the drawing shows what was intended.

How it works
  1. Show the drawing without explaining it.
  2. Ask the person to describe what they see, in their own words.
  3. Compare their description with what was meant to be there.
Hover to preview · Tap to understand

Study setup

Two participants play roles with conflicting needs and brief the AI about one bridge. Each pair tried 8 communication routes, from discussing together to telling the AI separately, once with a keyword-based control and once with the toolkit-guided agent.

Discuss togetherDiscuss privately, then tell AITell AI separatelyRole 1 → Role 2 → AIRole 1 → AI → Role 2Role 2 → AI → Role 1Role 1 onlyRole 2 only
Before: read by keywords

The AI pulls keywords out of what each person says and builds from them, so needs that are never named directly get dropped.

After: multi-agent negotiation simulation

AI agents play both roles and negotiate first, so the toolkit is tested on conflicting needs before real people use it.

How participants gave feedback

For every bridge the AI built, participants rated each matchstick, rebuilt the bridge the way they wanted, and marked up the AI’s sketch.

1  Rate every match
1 Rate every match
Red means it doesn’t work for her, blue means it does.
2  Rebuild it
2 Rebuild it
Blue matches are the ones she added.
3  Mark up the sketch
3 Mark up the sketch
Numbered notes on what the AI got right and wrong.

Eillen · Discuss together

Results

Two participants rated what the AI built from the keyword control and from the toolkit-guided agent, on every communication route. Scores run from −3 (very unsatisfied) to +3 (very satisfied).

Keyword controlToolkit, higherToolkit, lower

Two participants, so read these as direction, not statistics.

Eillen · Discuss together
Keyword control bridge
Keyword control −2
Toolkit bridge
Toolkit +2
Vicky · Discuss privately, then tell AI
Keyword control bridge
Keyword control −3
Toolkit bridge
Toolkit +3

Human feedback

Participants rated each AI sketch, numbered what worked and what didn’t, and explained why in their own words.

WorkedDidn’t workNeutral
Sketch rated by Eillen
Eillen
Wants decks that keep people and cars apart, shade, night lights and grass for pets
  • 1. triple deck for different purposes
  • 2. shades for sunny & rainy days
  • 3. high rails for protection
  • 4. fake grass for pets
Sketch rated by Vicky
Vicky
Wants a bridge that opens for boats, with a sidewalk and room for two cars
  • 1. splitting request is met
  • 2. pathway created, barrier between driveway and pathway added
  • 3. nice wide driveway for two cars to pass at the same time
  • 4. no stop sign and guard to stop people from entering the bridge when it’s raised
Sketch rated by Guanjie
Guanjie
Walks a dog and wants a sidewalk with guardrails on the same bridge
  • 1. Wide and smooth
  • 2. Pavement
  • 3. Guardrail
Sketch rated by Yoyo
Yoyo
Wants a quiet footbridge for dog walks, with cars in a tunnel below
  • 1. Love the cat deco with a lot of expressions.
  • 2. No environment.
Sketch rated by Han
Han
Wants dogs and people apart, big and small dogs apart, and no trucks
  • 1. The distinction between big dogs and small dogs, as well as the seating areas for people to rest, make this park a pleasant place for relaxation.
  • 2. The guardrail also prevented vehicles from entering, creating a friendly space for dogs and pedestrians.
Sketch rated by Vicky
Vicky
Wants a bridge that opens for boats, with a sidewalk and room for two cars
  • 1. The split doesn’t let ships pass