How to Participate

Participation Steps

1. Register: Sign up on the CodaBench competition platform, and join the competition.
2. Subtasks: Choose the subtask(s) you want to participate in (Subtask 1, 2, 3, or all).
3. Download: Go to the Files tab to download the dataset, and a starter kit. CLICK HERE FOR A GUIDE.
4. Develop: Build your models using the provided data.
5. Submit: Upload your predictions on the CodaBench competition platform

Evaluation Metrics

• Subtask 1: Macro F1
• Subtask 2: Macro F1
• Subtask 3: Macro F1

Competition Rules and Terms

1. Consent to Public Release of Scores

• By submitting results, you consent to the public release of your scores on:
  • the competition website
  • at the designated workshop
  • in associated proceedings
• Task organizers have discretion over the release and choice of metrics.
• Scores may include:
  • automatic and manual quantitative judgments
  • qualitative judgments
  • other metrics as deemed appropriate

2. Score Release and Validity

• Task organizers reserve the right to withhold scores for:
  • incomplete submissions
  • erroneous submissions
  • deceptive submissions
  • rule-violating submissions
• Inclusion of a submission's scores does not constitute endorsement.

3. Team Participation Rules

• Participants may be involved in only one team.
• Exceptions may be granted with prior approval from organizers.

4. Account Management

• Each team must create and use exactly one account on the designated platform.

5. Team Constitution

• Team membership cannot be changed after the evaluation period begins.

6. Development Period Rules

• Teams can submit up to 999 submissions.
• Results are visible only to the submitting team.
• Leaderboard is disabled.
• Warnings and errors are visible for each submission.

7. Evaluation Period Rules

• The teams are constrained to make 3 submissions.
• Only the final submission will be considered official.
• Warnings and errors are visible for each submission.

8. Post-Competition

• The gold labels will be released after the competition.
• The teams are encouraged to report results on all their system variants in their description paper.
• The official submission results must be clearly indicated.

9. Public Release of Submissions

• Final team submissions may be made public after the evaluation period.

10. Disclaimer about the Datasets

• Organizers and affiliated institutions provide no warranties on dataset correctness or completeness.
• They are not liable for dataset access or usage.

11. Peer Review Process

• Each participant will review another team's system description paper.

12. Dataset Usage Restrictions

• Datasets should only be used for scientific or research purposes.
• Any other use is explicitly prohibited.
• Datasets must not be redistributed or shared with third parties.
• Interested parties should be directed to the official website.

13. Final Ranking

• To be included in the official task ranking, you MUST submit a system description paper.