Jim Fan推文
GEN-1.5 热潮背后的关键：数据中的自然重复与 UMI 数据采集
作者认为 GEN-1.5 的成功源于人类数据中的自然重复动作，包括对称模式和恢复动作，并强调保留失败片段对模型学习的重要性。同时，作者指出 UMI 直接采集人类数据优于遥操作，能保留物理直觉，使模型具备零样本能力。
译文
GEN-1.5 的 hype 浪潮来了，而且名正言顺。非常佩服 Pete 和 Andy 的执行力。秘密在于人类采集的数据中天然存在的重复动作。这类重复有两个主要来源：（1）对称模式。整理、归置、组装几乎从来不会一步完成。翻开任何一本 IKEA 的组装说明书，你会发现大多数物体都是对称的。你拧一颗螺栓，然后是它的孪生兄弟，再是下一对。每一对 {螺栓 A，螺栓 B} 在上下文里都是自然的延续，而第二个实例就是模仿第一个（“提示”）的免费训练信号。（2）恢复。人类随时都会掉东西，但我们捡起来的速度太快，以至于根本注意不到。这种修复反射占据了身体能力的一半。关键洞察在于保留失败的前半段，而不是把它剪掉。如果模型消费完整的弧线——失手、接住、继续——那么恢复就会在测试时自然地涌现。有趣的是，上下文中的改进恰恰来自 *不* 过度净化你的数据。另一个关键要素是 UMI。我一直在说遥操作不会长久，而 GEN-1.5 正在钉下最后一颗棺材钉。UMI 本质上是人直接戴着机器人夹爪采集数据（人 → 数据）。遥操作则插入了一层隔阂：人 → VR/骨骼设备 → 机器人 → 数据，这会把人类所有的“物理直觉”都放干。我们与物体之间不断进行的微妙手法、微调、零件卡入到位的手感，在你无法直接感知环境时几乎不可能捕捉。一旦你有了足够的数据，很多行为其实可以零样本完成。例如，你甚至不需要微调就能拿起一个新物体。面对训练分布中相似的场景，模型“就是知道”该怎么做。上下文学习是否真正有效，也取决于测试与训练之间的距离。目前，演示还太简单，不足以定论。我持谨慎乐观态度。不过，这依然是机器人领域的好日子。
Seeing a hype wave around GEN-1.5, and rightfully so. Lots of respect to Pete & Andy for executing so well. The secret is in the naturally repetitive motions in human-collected data. There're 2 main sources for such repetitions: (1) Symmetric patterns. Sorting, tidying, and assembling almost never finish in one motion. Open any assembly manual from IKEA, and you find most objects symmetrical. You drive one bolt, then its twin, then the next pair. Every {bolt A, bolt B} pair is a natural continuation in context, and the second instance is a free training signal that imitates the first ("prompt"). (2) Recovery. Humans drop things all the time, but we pick them up so fast, we don’t even notice. That reflex to fix is half of our physical competence. The key insight is to keep the failed first half instead of trimming it away. If the model consumes the full arc, fumble, catch, continue, then recovery shows up organically at test time. It's funny that in-context improvement results from *NOT* over-sanitizing your data. The other critical ingredient is UMI. I've been saying for a while that teleop will not last, and GEN-1.5 is driving the final nail in the coffin. UMI is essentially a human wearing the robot gripper to collect data directly (human → data). Teleop inserts a layer of separation: human → VR/skeletal device → robot → data, which bleeds out all the human "physical intuition". The subtle sleight of hand we perform constantly with objects, the micro-adjustments, the feel of a part snapping into place, is nearly impossible to capture when you can't feel the environment directly. Once you have enough data, many behaviors can actually be zero-shot. For example, you don't even need finetuning to pick up a novel object. The model "just knows" what to do given a similar scene in the training distribution. Whether in-context learning truly works or not also depends on how far away the test is from training. Currently, the demos are still a bit too simple to conclude. I'm cautiously optimistic. Still, it's a great day in robotics.