Frequently Asked Questions

Participation Questions

Do I have to participate in all 3 subtasks?

No. You may choose one or more subtasks.

Do I have to participate in all languages for a given subtask?

No. You may choose one or more languages.

When will all datasets for the 21 languages be released?

We will announce the release of each dataset as it becomes available. Currently, datasets for 9 languages have been released, and the remaining ones will follow soon.

How will you verify my submitted model?

To be included in the final team rankings, participants must submit a system description paper detailing their approach and methodology. This ensures transparency and scientific integrity.

Will I be included in the final ranking if I do not write a system description paper?

No. A system description paper is mandatory for inclusion in the final ranking.

I have never written a system description paper. How can I write one?

We will provide an online tutorial and resources to help guide you.

Data and Methodology

When will you release the gold labels?

• Dev set: Released at the start of the evaluation phase.
• Test set(s): Released only after the competition ends.

Can I use LLMs in the different subtasks?

Yes.

Can I use additional datasets (e.g., publicly available ones)?

Yes, but you must cite them properly in your system description paper.

How was the data collected?

• Data sources include news websites, Reddit, blogs, Bluesky, and regional forums, covering events such as elections, conflicts, gender rights, migration, and more.
• Each language contains 3,000–5,000 annotated instances. Annotation platforms include Label Studio, Prolific, Potato, and Mechanical Turk.

How was the data annotated, and did you use LLMs?

No LLMs were used. All data was annotated by at least three native speakers per instance.

Publication and Conference

Do I need to pay conference registration fees or attend SemEval for my paper to be published?

No. Publication does not require attendance. If you do not attend the workshop, you do not need to pay any fees. If you do wish to attend, standard registration fees apply.

Our system did not perform well. Should I still write a system description paper?

Yes! We strongly encourage all teams to submit a paper. Negative results are valuable to the community and will help advance research on polarization.