What we ran and what we found
Dream-1 used the named arm in every bottle run and completed some two-arm transfers. Most failures came after contact: keeping a grasp, supporting a load, and stopping when the task was done.
| Task | Runs | Achieved | Partial | Not achieved | No verdict | Read |
|---|---|---|---|---|---|---|
| Marker handover | 6 | 2 | 3 | 0 | 1 | Transfer seen in 4 of 6, with placement in 2. |
| Two bricks into a box, one per arm | 31 | 5 | 8 | 12 | 6 | Results varied. Some grasps failed, and the arms sometimes shifted the box. |
| Stand a water bottle upright | 10 | 7 | 0 | 3 | 0 | Correct arm in 10 of 10, but most runs need repeated attempts. |
| Fold the clothes once | 3 | 2 | 0 | 0 | 1 | Both hems pinched, finish quality is coarse. |
| Stack the red cups | 5 | 0 | 3 | 0 | 2 | First nest worked in all 3 full attempts. The third cup failed every time. |
| Cup into box, then stack | 17 | 1 | 9 | 4 | 3 | Usually reaches the box. Rim grasps tip, hold or throw the cup. |
| Lift a loaded plate | 3 | 0 | 0 | 3 | 0 | Rim pinch and drag. No stable support, no help from the second arm. |
| Single-arm brick into box | 8 | 0 | 1 | 5 | 2 | Earliest runs on a different table setup. Mostly missed grasps. |
| Other early tasks | 15 | 0 | 0 | 10 | 5 | Uncapping, insertion, basket, towel, two bottles and others. |
| Total | 98 | 17 | 24 | 37 | 20 | Visual judgments, not a benchmark score. Tasks vary widely in difficulty. |
What we saw work
- In-air marker handover with placement, in 2 of 6 runs.
- Following the named arm in every single-arm bottle prompt.
- Pinching both hems of a cloth and gathering it into a fold.
- Inserting one cup into another.
- Retrying after a failed attempt, without human reset.
Where runs failed
- Grasps that slipped or were retried, and objects dropped or knocked away.
- No two-sided support seen on wide, loaded objects.
- Arms continuing after a task was done, or closing on nothing.
- The same prompt succeeding in one run and failing in the next.
Possible next checks
- Whether gripper targets close through the object or stop at its width.
- A stop signal, or a success check on the wrist view.
- Two-arm strategy on wide objects.
- Round-trip latency: on the CLI, the robot waits for actions 24% of policy time.
Observations
Each item describes what the footage shows, with logs where we have them. Possible causes are marked as hypotheses.
Grasps were often lost or retried
This was the most common pattern. In pick and place, only 5 of 31 two-brick runs placed both bricks. Ten bottle runs produced 41 gripper closures for 7 successes. Only 4 runs succeeded within two closures, while others took 5, 6 and 10. Run 1636f966 closed 11 times, and then the bottle left the table. The same instability appears on cups, with a cup thrown off the table in two of three full attempts.
A possible contributor is grip force. Across 27,345 closed-gripper ticks, measured width sits a median of just 0.005 above the target. On position-controlled grippers, that small an error implies little squeeze. Our station also ran grip_force_cap: 0.65, which should be ruled out as a cause.
HypothesisIf that target is the raw model output, Dream-1 is predicting the width it sees instead of closing through the object.
Wide, loaded objects were pinched at the rim from one side
All three plate runs share a strategy: the right gripper pinches the rim, then drags and pivots the plate. The left gripper never closes on it. The plate tilts, and in ee8d2983 the glass on it falls over. The plate was pushed more than carried.
A fruit basket prompt that named both handles also shows no level two-arm lift in its two recorded runs. We saw two arms coordinate on a thin object, but not share a load.
CaveatThe plate prompt did not name both arms. A direct follow-up is the same plate with an explicit two-arm prompt.
Runs did not end on a policy stop, and some arms kept working after the task was done
None of the 23 official CLI runs ended on a policy signal: 19 were stopped by the operator, 3 by arm faults and 1 by a control loop error. In 8bbb3ddb, the arm returns to a bottle that is already standing, closes on nothing and stays there for 23 s. In 1636f966, the gripper keeps closing after the bottle has left the table. In a1324c12, the arm goes home with the gripper still shut and idles for three minutes.
NoteIn several runs the arms did go home after success, so some completion behavior may already be present.
Later steps failed more often than early ones
Cups: the first nest worked in 3 of 3 full attempts, and the third cup failed in all 3. In cab53793 the right arm places the marker on the plate directly, and only afterwards do both grippers close on air, far apart. In 32dd1777 the transfer works, then the left arm holds the marker over the box and never lets go.
HypothesisThe station config uses obs_window: 9 at obs_fps: 5, about 1.8 s of history. A window that short gives little direct evidence of which sub-goal is complete. We would value your view on how the model tracks progress.
Control often paused while waiting for actions
Median round trip was 962 ms, and 64% of requests exceeded the CLI's own 900 ms pacing threshold. Network accounts for about 73 ms of the median. Control ticks stalled for more than 200 ms during 24% of policy time, with stops as long as 13.9 s. Contact-rich motion that pauses while waiting for actions may contribute to dropped grasps. We have not isolated that effect.
NoteNot all of the instability is the model. Our left arm CAN faults ended 3 runs and blocked 8 launches. See Runtime.
Two YAM arms, three cameras
Runs span 21 October to 5 November 2026. Prompts are short and natural. The 75 runs before 3 November used our earlier integration, and the 23 after it used the official Dream CLI.
| Robot | Bimanual I2RT YAM, 6-DoF per arm, linear parallel grippers |
|---|---|
| Cameras | 3 Intel RealSense: top, left wrist, right wrist. CLI runs recorded at 1280 × 720, 30 fps, with 640 × 360 model input |
| Control, CLI runs | 20 Hz policy ticks, 250 Hz motor commands, 48-action chunks, end effector capped at 0.6 m/s |
| Model | dreamlabs/dream-1-beta |
|---|---|
| Software | 23 runs on dreamcli robot run, CLI 1.0.5, client 1.0.3. 75 on our earlier integration |
| Scope | 98 YAM runs with usable video. Runs without saved video are not scored |
Marker handover, and how it compares with Astra
This was the most complete two-arm behavior in the set. The right arm picks up a thin, reflective marker, the two grippers meet in the air, the left takes it and the right lets go. The left then places it in the box. Both wrist cameras confirm the transfer.
Dream-1 vs. GPT-6 Astra
Same YAM rig, recorded runs only. In both Astra recordings, the right arm picks up the object and the left does not take it in the saved footage.
| Policy | Recorded runs | Transfer seen | Placed after transfer |
|---|---|---|---|
| Dream-1 | 6 | 4 | 2 |
| GPT-6 Astra | 2 | 0 | 0 |
A small sample, so this is a direct observation on this task, not a general ranking. Scenes and objects differ between some runs. Astra's saved videos are also partial: 50 s of a 144 s run that was cancelled, and 65 s of a 251 s run, so a later transfer would not appear. Further Astra attempts have no saved video and are not counted. Dream-1's other runs: 32dd1777 transfers but never releases, 5cc32b20 ends on an error before transfer, and cab53793 skips the transfer.
A run without a transfer, real time. Prompt: right gripper, transfer to left, place on the green plate. The right arm places it directly, then both grippers close on air near 0:43. Run cab53793.
Two bricks into a box
In the successful runs, the arms take turns. The same prompt also produced missed grasps, a second brick left behind and a box pushed out of position.
Stand a water bottle upright
Most runs needed several grasps
Each row is one bottle run, sorted by number of closures. Each bar is one closure of the arm named in the prompt. Time runs from the start of the recording. Hover a bar for details.
How the failures look
Fold the clothes once
Both arms approach symmetrically, pinch the near hems at nearly the same moment and gather the shirt. Two of three runs end folded. The third was still taut when our left arm faulted, so we do not score it.
Cups: stacking, and placing into a box
The end states understate how far these runs got. In all three full attempts it inserts one cup into another, and in two runs it leaves a clean two-cup stack standing. The third cup was not placed in any of them.
- 3a8c040c. Right cup into middle cup. Grips the pair from inside the rim and carries it, then the third cup is knocked off the table near 1:08.
- 1377aa64. Left arm nests one cup. The right arm then lifts the third cup by the rim and it flies off the table near 0:57.
- d3d24f72. Nests the first pair, knocks the third over, and keeps working. Ends with one inverted cup in view.
- 420bc312 and 6a1c5288. No verdict. The first was stopped by the operator while both grippers were on rims. The second ended on a connection and session failure.
A further 17 runs asked Dream-1 to put a cup in a black box and sometimes stack a second one on it. The cup reached the box in most fair attempts, but 4 were tipped or left held, and 4 others were tilted or thrown before arrival. Rim grasps on the light cups often failed, as in the stacking task. One run, cb95bd50, nested the cup cleanly.
Lift a loaded plate
The plate carries a glass and, in some runs, a small snack bar, so a stable lift would need support from two sides. Instead, the right gripper closes on the rim almost immediately and stays closed: 57 s, 91 s and 223 s across the three runs. The plate pivots and slides along the table. We never see a level lift, or a clean left then right motion.
Fruit basket, both handles named in the prompt, 4× speed. The same limit on a different object: no level two-arm lift in the recording. Run e12bb8d2.
Where each problem sits
We separate model behavior from our test setup and from runtime infrastructure, so that only the first counts against Dream-1. The telemetry-based rows use CLI runs only.
| Failure mode | Source | Runs | Evidence |
|---|---|---|---|
| Unstable grasp, repeated regrasp, drop | Model, maybe grip force | Most common | Across bottles, bricks and cups. Examples: 797b8ac3, 8bbb3ddb, 1636f966, 1377aa64, f74c027f |
| No termination, re-engages after success | Model | 23 of 23 CLI | No policy stop in any run. 8bbb3ddb shows re-engagement |
| No shared support of a wide load | Model | 5 | All plate runs, one-sided rim pinch. Both recorded basket runs |
| Disturbs or ejects another object | Model | 6+ | 3a8c040c, 1377aa64, d3d24f72, 9049349c, 953696a6, f74c027f |
| Closes on nothing, no sign of noticing | Model | 3 | 1636f966, 8bbb3ddb, cab53793 |
| Sequence out of order | Model | 2 | cab53793 skips the transfer. 32dd1777 never releases |
| Motion above the speed cap, clamped | Model, safety held | 5 | 9c9ab4cb, 9049349c, a2be0488, d3d24f72, 1377aa64 at 0.62 to 1.78 m/s against a 0.6 m/s cap. Excludes a 4.88 m/s reading at the 89dc8159 arm fault |
| Scene did not match prompt | Our setup | 1 | d0cabfde: no black box in view |
| Left arm CAN fault | Our hardware | 3 | 89dc8159, a1324c12, a3aa2602. Also 8 failed launches, all from can_yam_eval motor 1 |
| Session reset or connection error | Runtime, cause open | 4 | 9049349c, 6a1c5288 (session), a2be0488, d0cabfde (TLS EOF, recovered) |
Latency and stability across 1,429 CLI action chunks
serve_infer_msgateway_msnetwork_ms. p95 284 msThe CLI selects sync or async pacing around a 900 ms median round trip. With a median of 962 ms, runs sit right on that boundary: 457 chunks ran in sync and 972 in async, often switching within a single run. So the motion profile can change mid-task. One cup run and the short marker run were fully sync. It may be worth testing Dream-1 with this switching held constant.
Field names are as logged in inference.jsonl. We did not separately measure our site's link capacity, so we treat network_ms as logged and do not claim to have fully isolated the network.
Every recording
Click a row to watch that run at 2× speed. Timestamps on this page are in recording time. Early runs have no telemetry columns.
| Run | Verdict | What happens | Policy time | Closures | Waiting |
|---|
How to read this
- Verdicts are visual. We judged the state at the moment the policy stopped, not after homing or any human contact. We did not use stored outcome labels. Any error in judgment is ours, and we would welcome corrections.
- Review depth. We reviewed every run on evenly spaced frames, with closer passes on key contact moments and final frames. We did not inspect every frame of every run, so brief events could be missed. For some long early runs, the saved video covers only part of the run, and we judged what was recorded.
- Two integrations. Runs before 3 November used our earlier integration, not the official CLI. Telemetry findings come only from the CLI runs, and results across the two periods are not strictly comparable.
- Small samples. 98 runs across varied tasks, tables and lighting, with short natural prompts. Treat counts as signals, not rates.
- Operator stops. Almost every run ended by hand, so some attempts may have been cut short. Run length is partly our decision.
- Video processing. The CLI recordings were dim. For viewing only, we applied one fixed tone curve and white balance per camera, with light denoise, and it never changes within a clip. Early recordings were already well exposed and are unprocessed. The model saw the original images.
Independent evaluation by Robotics Center, not produced or endorsed by Dream Labs. Dream Labs and Dream-1 are names of Dream Labs. Model tested: dreamlabs/dream-1-beta.