## NOAI2025 Compound Word Segmentation Task ### Reference time to solve: within 1 hour ### I. Overview Compound words (Compounds) are new words formed by combining multiple shorter words, which is especially common in German. For example, “Fußball” is formed by combining “Fuß” and “Ball”, meaning “foot” and “ball” respectively; “Autobahnanschlussstelle” (highway interchange/connection point) is formed by “Autobahn”, “Anschluss”, and “Stelle”, meaning “highway”, “connection”, and “place” respectively. In this task, we need to split compound words in a German sentence into shorter words separated by spaces. For example, “Fußballspieler” should be segmented into “Fuß”, “ball”, and “spieler”. ### II. Dataset `data/train.json` contains more than 90,000 German compound words, each already segmented into shorter words. Each record includes two fields: the compound word and its segmentation labels. The validation set (`val.json`) and test set (`test.json`) each contain more than 10,000 German compound words. The dataset sizes are: - **Training set**: 94,306 samples, stored in `train.json`; - **Validation set**: 11,788 samples, stored in `val.json`; - **Test set**: 11,789 samples, stored in `test.json`. The training data can be accessed and downloaded directly: [Training dataset](https://www.bohrium.com/en/competitions/53892361357?tab=datasets). The validation and test sets cannot be downloaded directly; they must be accessed via encrypted environment variables. See [baseline.ipynb](https://www.bohrium.com/en/notebooks/59712856135) for details. Example training data: ```json { "Sprachbereich": [ 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1 ], "Autobahnanschlussstelle": [ 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1 ], ... } ``` The data uses JSON format, where the key is the compound word, and the value is a 0–1 array. Each position in the array corresponds to the corresponding letter of the compound word, where 1 indicates the end of a word, and 0 indicates the beginning or the middle of a word. For example, the first piece of data indicates that "Sprachbereich" is split into "Sprach" and "bereich", so the value at the 5th position (the position index is counted starting from 0) and the last position is 1, and the other positions are 0. The 0–1 arrays of the value in the validation set and the test set are empty. ### III. Task Please implement a compound word segmenter, and fill in the value of the validation set and the test set. The specific requirements are as follows: 1. The training time + testing time using GPU must not exceed 10 minutes; connection time and queueing time are not included in the total time. Reference value: in offline testing, if the correct model is chosen, and the number of epochs is controlled between 8 and 32, Tesla T4 can train a fairly good result within 10 minutes. 2. Hint: it is recommended to use Embedding + deep learning model. ### IV. Submission Contestants need to submit the **model training and inference code**, named "submission.ipynb", in which it **must include the training process of training the model and the testing process of predicting the validation set and the test set**, and it cannot submit only the trained model. The output of "submission,ipynb" is a packaged zip, and the zip contains two files, "submissionval.json" and "submissiontest.json". The format is the same as the training set, and the content is the prediction for val.json and test.json. A submission example is given in [baseline.ipynb](https://www.bohrium.com/en/notebooks/59712856135). ### V. Scoring 1. The final score is the average F1-score of word segmentation for each compound word. The scoring result of val.json can be queried in the A leaderboard during the competition; the scoring result of test.json cannot be queried during the competition, and it will be displayed after the competition ends, and will be used as the final scoring result. 2. The specific calculation process of F1-score is as follows. **Hint: you do not need to read it carefully; you can intuitively understand that the more accurate the predicted positions are, the higher the score.** For the word segmentation result of each compound word, we first restore it to a “segmentation boundary set”. Let: - $G$ be the true segmentation set - $P$ be the predicted segmentation set Among them, each segmentation is represented by an interval $(start, end)$, indicating that the characters within $(start, end)$ belong to the same segmentation. Note: according to the consistent expression of computer languages, it includes $start$ but does not include $end$. **(1) True Positive (TP)** Only when a certain predicted segmentation and the true segmentation are completely consistent in the start and end positions, it is considered correct. $$ TP = \left| G \cap P \right| $$ **(2) Precision and Recall** The formula of precision is $$ Precision = \frac{TP}{|P|} $$ where $|P|$ is the total number of predicted segmentations. If $|P| = 0$, define precision as 0. The formula of recall is $$ Recall = \frac{TP}{|G|} $$ where $|G|$ is the total number of true segmentations. If $|G| = 0$, define recall as 0. **(2) F1-score formula** Using precision and recall, the F1 score formula is: $$ F1 = \frac{2 \times Precision \times Recall}{Precision + Recall} $$ When $Precision + Recall = 0$, define $F1 = 0$. **(3) Overall average F1-score** Assume there are a total of $N$ samples. For the $i$-th sample, its F1 score is recorded as $F1_i$. Then the overall F1 mean is: $$ F1_{avg} = \frac{1}{N} \sum_{i=1}^{N} F1_i $$ #### Example: calculating for two compound words Below are two examples, respectively the words **"Sprachbereich"** and **"Autobahnanschlussstelle"**. **Example 1: "Sprachbereich"** **True labels and segmentation boundaries** Assume the true label sequence is: ``` [0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1] ``` Traverse the sequence: - When reaching index 5, the label is 1, corresponding segmentation boundary is $(0, 6)$ - Starting from index 6, until index 12 the label is 1, obtain segmentation boundary is $(6, 13)$ So the true segmentation set is: $$ G = \{ (0,6), (6,13) \} $$ **Predicted labels and segmentation boundaries** Assume the predicted label sequence is completely consistent with the true one, then: $$ P = \{ (0,6), (6,13) \} $$ **Metric calculation** - Intersection: $$ G \cap P = \{ (0,6), (6,13) \} $$ so $TP = 2$. - Precision: $$ Precision = \frac{TP}{|P|} = \frac{2}{2} = 1 $$ - Recall: $$ Recall = \frac{TP}{|G|} = \frac{2}{2} = 1 $$ - F1 score: $$ F1 = \frac{2 \times 1 \times 1}{1 + 1} = 1 $$ **Example 2: "Autobahnanschlussstelle"**: **True labels and segmentation boundaries** Assume the true label sequence is: ``` [0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1] ``` Traverse this sequence (character index counts from 0): - Index 3 label is 1: segmentation boundary is $(0,4)$ - Index 7 label is 1: segmentation boundary is $(4,8)$ - Index 22 label is 1: segmentation boundary is $(8,23)$ So the true segmentation set is: $$ G = \{ (0,4), (4,8), (8,23) \} $$ **Predicted labels and segmentation boundaries** Assume the predicted label sequence makes the restored predicted segmentation set be: $$ P = \{ (0,4), (4,10), (10,23) \} $$ **Metric calculation** - Intersection: $$ G \cap P = \{ (0,4) \} $$ therefore $TP = 1$. - Precision: $$ Precision = \frac{1}{|P|} = \frac{1}{3} \approx 0.3333 $$ - Recall: $$ Recall = \frac{1}{|G|} = \frac{1}{3} \approx 0.3333 $$ - F1 score: $$ F1 = \frac{2 \times 0.3333 \times 0.3333}{0.3333 + 0.3333} \approx 0.3333 $$ **Overall average F1 score calculation** Assume the F1 scores of the two samples are: - For "Sprachbereich": $F1_1 = 1$ - For "Autobahnanschlussstelle": $F1_2 \approx 0.3333$ Then the overall average F1 score is: $$ F1_{avg} = \frac{F1_1 + F1_2}{2} = \frac{1 + 0.3333}{2} \approx 0.6667 $$