SemEval 2026 Task 9 / ACL 2026

Multilingual Polarization Detection Using Ensemble Gemma Models with Synthetic Data Augmentation

Srikar Kashyap Pulipaka

Independent Researcher

2nd overall across 22 languages / 0.811 mean macro-F1 / 3 first-place languages

System Overview

Per-language modeling beats one global recipe.

The system keeps validation data real, adds filtered synthetic training examples, fine-tunes Gemma 3 models per language, tunes thresholds, and selects the strongest 12B/27B strategy for each language.

1 Data Real split

80/20 train-validation split.

2 Augment Synthetic data

Direct generation, paraphrases, contrastive pairs.

3 Filter Quality checks

Leakage checks, dedupe, translation consistency.

4 Train Gemma LoRA

12B and 27B language-specific models.

5 Calibrate Threshold tuning

Search 0.30 to 0.70 for macro-F1.

6 Select Final strategy

Choose 12B, 27B, average, or weighted ensemble.

0.811

Mean macro-F1

2nd

Overall rank

18/22

Languages improved by final strategy

14/22

Languages where ensembles won

Abstract

We present Team PSK's system for SemEval-2026 Task 9: Multilingual Polarization Detection, a binary classification task across 22 languages. The system fine-tunes separate Gemma 3 models with LoRA, augments the training data with GPT-4o-mini synthetic examples, applies multi-stage quality filtering, tunes classification thresholds per language, and selects a language-specific 12B/27B ensemble strategy. The final submission achieves 0.811 mean macro-F1 and ranks 2nd overall.

Method

Synthetic Data

The augmentation mix uses direct generation, label-preserving paraphrasing, and contrastive pair creation. Synthetic examples are added only to the training split.

Quality Filtering

The pipeline removes obvious artifacts with basic cleaning, label-leakage checks, embedding-based deduplication, and consistency checks.

Modeling

Gemma 3 12B is the primary model. Gemma 3 27B provides additional predictions for per-language ensemble selection.

Calibration

Thresholds are tuned per language over 0.30 to 0.70, because the default 0.50 threshold is often not optimal for macro-F1.

Results

Official test submissions

Sub 1: Gemma 12B

0.797

Sub 2: XLM-R/Qwen3

0.665

Sub 4: Ensemble

0.811

Sub 5: Post-hoc best

0.812

Leaderboard coverage

1st place

Top 3

Top 10

Outside top 10

The system was broadly competitive across languages, not carried by a small subset.

Analysis

Gemma transferred better

XLM-R and Qwen3 looked competitive on development data but dropped sharply on test. Gemma generalized more reliably.

Ensembling helped selectively

Ensemble methods won for 14 of 22 languages, but single-model strategies remained best for some languages.

Thresholds mattered

Per-language threshold tuning gave meaningful F1 gains without retraining and exposed calibration differences.

Coverage gaps remained

Italian was the weakest case, largely because major test topics were absent from the train and development splits.

BibTeX

@misc{pulipaka2026semeval,
  title={PSK at SemEval-2026 Task 9: Multilingual Polarization Detection Using Ensemble Gemma Models with Synthetic Data Augmentation},
  author={Pulipaka, Srikar Kashyap},
  year={2026},
  eprint={2605.05159},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2605.05159}
}