# NOAI 2026: Embodied Intelligence Sim2Real State Prediction Based on NVIDIA Isaac Sim

**Note: This problem will be evaluated in an environment based on the `noai:2026v1.1` image. Participants should select `noai:2026v1.1` as the training image.**



## 1. Task Description

In embodied intelligence tasks, simulation environments can generate large amounts of robot trajectories at low cost, but simulated trajectories and real robot trajectories do not perfectly align. Participants must use the complete simulation trajectory, timestamp information, and a visible prefix of real robot states provided at the beginning of each trajectory, to predict the real robot states corresponding to the subsequent hidden frames.

Each frame contains a 6-dimensional joint position `simulation_positions` from the simulation environment. Participants must predict the corresponding real robot's 6-dimensional joint state `observation_state` for that frame.

Participants may choose any appropriate modeling approach, including but not limited to regression modeling, feature engineering, similar trajectory retrieval, and nearest-neighbor methods. Suggested focus areas:

- The relationship between simulated trajectories and real trajectories;
- How the visible prefix within the same task provides hints for the subsequent hidden segment;
- ...



## 2. Dataset

### 2.1 Data Format

The dataset is in CSV format. Each row corresponds to one time frame within a task trajectory.

| Field                  | Meaning                                                      |
| ---------------------- | ------------------------------------------------------------ |
| `index`                | Global sample index                                          |
| `task_index`           | Task index; rows with the same `task_index` form one complete task trajectory |
| `frame_index`          | Frame index within the task                                  |
| `timestamp`            | Timestamp, in seconds                                        |
| `simulation_positions` | 6-dimensional joint position in the simulation environment   |
| `observation_state`    | 6-dimensional joint state of the real robot                  |

In the training set, all frames provide the complete `observation_state`. In the validation set and test set, the `observation_state` for a visible prefix at the beginning of each task trajectory is provided, while the state values for subsequent frames are set to null; participants must predict the real states for these missing positions.

### 2.2 Data Scale

| Dataset                     | # Tasks | Total Rows | Rows to Predict |
| --------------------------- | ------- | ---------- | --------------- |
| Training Set (Train)        | 336     | 176,675    | 0               |
| Validation Set (Validation) | 72      | 35,136     | 24,562          |
| Test Set (Test)             | 72      | 37,632     | 26,310          |

### 2.3 Data Example

```csv
index,task_index,frame_index,timestamp,observation_state,simulation_positions
0,0,0,0.0,"[-0.153,-1.707,1.601,1.664,0.179,0.008]","[-0.142,-1.627,1.524,1.598,0.173,0.005]"
1,0,1,0.033,"[-0.153,-1.707,1.601,1.664,0.179,0.008]","[-0.142,-1.639,1.535,1.609,0.175,0.005]"
```

### 2.4 Training Data Access

During the development phase, participants can only directly access the training set. The validation set and test set are only available in the official evaluation environment; their file paths must be obtained via specified environment variables. Please refer to the [baseline](https://www.bohrium.com/notebooks/88428539178) Notebook for the specific access method.

## 3. Task

Participants must predict the `observation_state` for all hidden frames in the validation set and test set:

- **Input information**: Complete `simulation_positions`, `timestamp`, `frame_index`, and the visible `observation_state` prefix from the same trajectory;

  **Prediction target**: The `observation_state` numerical array of length 6 corresponding to each hidden frame.



## 4. Submission

Participants must submit a Notebook named `submission.ipynb`. This Notebook should include the complete workflow for data reading, data processing, model training, prediction, and submission file generation, and must be runnable from scratch in the evaluation environment.

### 4.1 Input and Output

- **Input**: Training set CSV file; validation set and test set data are obtained at submission time via environment variables (see [baseline](https://www.bohrium.com/notebooks/88428539178) code for details).
- **Output**: A compressed file named `submission.zip`, containing:
  - `submission_val.csv` — validation set prediction results
  - `submission_test.csv` — test set prediction results

### 4.2 File Structure

Please refer to the complete file structure in the [baseline](https://www.bohrium.com/notebooks/88428539178) Notebook.

### 4.3 Submission CSV Format

The prediction file should only include the hidden frames to be predicted in the validation set or test set, saved in the following field order:

```csv
index,task_index,frame_index,timestamp,observation_state
4045,4,268,8.933,"[-0.207,-0.699,0.709,1.212,0.186,0.363]"
```

Requirements:

| Rule                                                 | Description                                                  |
| ---------------------------------------------------- | ------------------------------------------------------------ |
| `observation_state`                                  | Must be saved as a numeric array string of length 6, and the format must be consistent with the `observation_state` in the original data. |
| `index` / `task_index` / `frame_index` / `timestamp` | Must match the values in the target rows                     |
| Row count                                            | The submission file must contain exactly all hidden frames to be predicted in the corresponding dataset — no omissions, duplicates, or additional rows. |
| Data usage                                           | Participant programs must not access, recover, or use the true labels of hidden frames in the validation set or test set in any way. |

**Note: Failure to comply with these requirements will result in a direct error during evaluation.**

### 4.4 Only one Notebook file may be submitted for this problem; additional datasets or other files are not permitted.



## 5. Scoring

### 5.1 Metric

For each hidden frame to be predicted, let the true state be $y$ and the predicted state be $\hat y$. The Root Mean Square Error (RMSE) across the six dimensions is first calculated:

$$\text{RMSE} = \sqrt{\frac{1}{6} \sum_{i=1}^{6}(\hat{y}_i - y_i)^2}$$

The base score for that row is:

$$\text{Score}_{\text{row}} = \exp(-10 \cdot \text{RMSE})$$

### 5.2 Score Calculation

For the validation set and test set, the average row score across all hidden frames to be predicted is calculated separately:

$$\text{Score} = \frac{1}{n}\sum_{i=1}^{n} \text{Score}_{\text{row},i} = \frac{1}{n}\sum_{i=1}^{n} \exp(-10 \cdot \text{RMSE}_i)$$

The final score range is $(0,1]$. A smaller prediction error yields a higher score; when the predicted values perfectly match the true values, the score for that frame is 1.

### 5.3 Public Leaderboard and Private Leaderboard

- **Public Leaderboard (A)**: Calculated based on the Validation Set;
- **Private Leaderboard (B)**: Calculated based on the Test Set, and published after the competition ends.



## 6. Constraints

- Downloading or using any external data beyond the dataset provided for this problem is not allowed; however, participants may construct features, transform data, or perform secondary annotation based on the provided data;

- Using external large language model APIs (such as GPT or Claude) for prediction, feature generation, data annotation, or model ensembling is not allowed;

- The evaluation environment does not provide internet access; participant programs must not perform any network operations, nor install additional dependencies via `pip install`. Participants may only use packages pre-installed in the specified image;

- This problem uses a **CPU** for training and evaluation; the total time for training + inference must not exceed 25 minutes.

  

## 7. Baseline Score and Reference Score

- **Leaderboard B Baseline Score ([baseline](https://www.bohrium.com/notebooks/88428539178))**: 0.5647
- **Scientific Committee Reference Solution B Leaderboard Score (Reference Result)**: 0.7861



## 8. Acknowledgements

Thanks to the technical experts at NVIDIA for providing this problem.