EEUCA 2026 Shared Task / ACL 2026

Fine-Tuning Large Language Models with Synthetic Data Augmentation for Multi-Class Toxicity Detection in Gaming Chat

Srikar Kashyap Pulipaka

Independent Researcher

4th of 35 teams / 0.6234 macro-F1 / 5% synthetic augmentation

System Overview

A small synthetic boost beat larger, higher-validation systems.

The final system keeps validation fully real, adds a narrow 5% minority-class paraphrase set, fine-tunes Llama 3.1 8B with LoRA, and avoids systems that overfit the validation distribution.

1 Data GameTox chat

World of Tanks messages with severe class imbalance.

2 Split Real validation

Validation remains 100% real data.

3 Generate Paraphrases

Short gaming-style rewrites for minority classes.

4 Filter Quality checks

Cleaning, leakage removal, and synthetic dedupe.

5 Train Llama LoRA

4-bit NF4, LoRA rank 16, class-weighted loss.

6 Submit 5% synthetic

Final run with small Class 2 boost.

0.6234

Official macro-F1

4th

Rank out of 35 teams

Best synthetic ratio

Toxicity classes

Abstract

This paper describes Team PSK's system for the EEUCA 2026 Shared Task on Understanding Toxic Behavior in Gaming Communities. The task classifies World of Tanks chat messages into six toxicity categories: Non-toxic, Insults/Flaming, Other Offensive, Hate/Harassment, Threats, and Extremism. The final system combines Llama 3.1 8B, LoRA fine-tuning, class-weighted training, and carefully calibrated 5% synthetic paraphrase augmentation, achieving 0.6234 macro-F1 and ranking 4th of 35 teams.

Method

Minority-Class Paraphrases

Synthetic data is generated as short, slang-aware paraphrases for minority toxicity classes, rather than broad direct generation.

Strict Ratio Control

The final system samples 1,921 synthetic examples, keeping synthetic data at approximately 5% of the training set.

Llama 3.1 8B

The best model uses 4-bit NF4 quantization, LoRA rank 16, alpha 64, and class-weighted cross-entropy.

Validation Trap Avoidance

Several systems with stronger validation F1 transferred poorly to test, so final selection prioritized test transfer behavior.

Results

Validation trap

Two-stage

0.47 test

Gemma 12B

0.52 test

Llama 8B

0.597 test

Llama + 5%

0.623 test

Light bars show validation F1; dark bars show test F1.

Synthetic ratio sensitivity

The 5% setting was the only synthetic ratio that improved test transfer.

System comparison

Two-stage

0.47

Gemma 12B

0.52

Llama 8B

0.597

Llama + 5%

0.623

Training-set class imbalance

81% Non-toxic 13.8% Insults <1% Hate, threats, extremism

Analysis

Validation F1 was misleading

Two-stage and larger-model systems scored well on validation but were too conservative on the shifted test distribution.

5% synthetic was the sweet spot

Small paraphrase augmentation increased minority-class predictions without overwhelming the real data pattern.

Direct generation was too generic

Paraphrases better preserved short gaming slang than broad synthetic toxic-message generation.

Ensembles did not help

Averaging and routing methods hurt because the best Llama 8B configuration dominated the alternatives.

BibTeX

@misc{pulipaka2026eeuca,
  title={PSK@EEUCA 2026: Fine-Tuning Large Language Models with Synthetic Data Augmentation for Multi-Class Toxicity Detection in Gaming Chat},
  author={Pulipaka, Srikar Kashyap},
  year={2026},
  eprint={2605.07201},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2605.07201}
}