Results for Real2Sim
Each action below, shows the video, and corresponding segmentation masks and neural renderer outputs for the three methods.
- A. Ground-truth boxes, ground-truth action phase
- B. Ground-truth boxes, predicted action phase
- C. Predicted boxes, predicted action phase
Action 1: Pull left to right
| ID | Video | Method A | Method B | Method C |
|---|---|---|---|---|
| Vid 1: 8675 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 2: 13956 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 3: 6848 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 4: 20193 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 5: 10960 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 6: 15606 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
Action 2: Pull right to left
| ID | Video | Method A | Method B | Method C |
|---|---|---|---|---|
| Vid 1: 13309 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 2: 3107 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 3: 11240 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 4: 2458 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 5: 10116 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 6: 11732 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
Action 3: Push left to right
| ID | Video | Method A | Method B | Method C |
|---|---|---|---|---|
| Vid 1: 3694 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 2: 601 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 3: 9889 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 4: 4018 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 5: 6955 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 6: 6132 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
Action 4: Push right to left
| ID | Video | Method A | Method B | Method C |
|---|---|---|---|---|
| Vid 1: 3949 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 2: 1987 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 3: 4218 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 4: 4378 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 5: 1040 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 6: 8844 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
Action 5: Pick up
| ID | Video | Method A | Method B | Method C |
|---|---|---|---|---|
| Vid 1: 7194 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 2: 38559 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 3: 12359 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 4: 1838 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 5: 2875 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 6: 24925 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
Action 6: Put behind
| ID | Video | Method A | Method B | Method C |
|---|---|---|---|---|
| Vid 1: 1044 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 2: 6538 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 3: 3074 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 4: 12986 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 5: 779 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 6: 4388 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
Action 7: Put in front of
| ID | Video | Method A | Method B | Method C |
|---|---|---|---|---|
| Vid 1: 1663 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 2: 41177 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 3: 28603 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 4: 5967 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 5: 14841 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 6: 2114 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
Action 8: Put next to
| ID | Video | Method A | Method B | Method C |
|---|---|---|---|---|
| Vid 1: 4053 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 2: 19 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 3: 874 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 4: 3340 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 5: 1890 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 6: 1642 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
Action 9: Put onto
| ID | Video | Method A | Method B | Method C |
|---|---|---|---|---|
| Vid 1: 8801 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 2: 7655 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 3: 13390 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 4: 757 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 5: 7504 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
|
| Vid 6: 16310 |  Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |
 Segmentation mask  Renderer output |