bluesky-collector/FINDINGS.md

Bluesky Toxicity Analysis - Main Findings

Study Overview

Period: January 1 – March 30, 2026 (89 days) Monitored Accounts: 159 Dutch political accounts Total Posts Collected: 15,190 posts

---

1. Data Collection Summary

Content Distribution

• Primary Content (by tracked accounts):

  • Original Posts: 3,032
  • Replies: 3,652
  • Total Primary: 6,684 posts

• Secondary Content (mentions of tracked accounts):

  • Unique Mention Posts: 8,506
  • Note: Posts mentioning multiple tracked accounts counted once

Total Dataset

• Combined Content: 15,190 posts
• Collection Method: Automated via Bluesky Public API (every 4 hours)
• Infrastructure: Docker containers with PostgreSQL database

---

2. Toxicity Detection Results

AI Model Performance

• Model Used: OpenAI GPT-4.1-nano
• Classification Categories: 12 toxicity dimensions
• Flagging Threshold: Overall toxicity score ≥ 0.5 (50%)

Flagged Content

• Primary Content (Posts/Replies): 97 posts flagged
• Secondary Content (Mentions): 413 unique posts flagged
• Total Flagged: 510 unique posts

Distribution Insight

• 81% of flagged content came from mentions (external users → politicians)
• 19% of flagged content came from politicians themselves
• External users directed significantly more toxic content toward politicians than politicians produced

---

3. Human Review Results

Review Completion

• Total Items Reviewed: 510 posts (100% of flagged content)
• Review Period: January 1 – March 30, 2026
• Review Interface: Custom web application with ✓/✗/? buttons

Validation Results

Primary Content (Posts/Replies by Politicians)

Status	Count	Percentage
✓ Correctly Flagged	32	33.0%
✗ Incorrectly Flagged	65	67.0%
? Unsure	0	0.0%
Total	97	100%

Secondary Content (Mentions of Politicians)

Status	Count	Percentage
✓ Correctly Flagged	174	42.1%
✗ Incorrectly Flagged	239	57.9%
? Unsure	0	0.0%
Total	413	100%

Combined Results

Status	Count	Percentage
✓ Correctly Flagged	206	40.4%
✗ Incorrectly Flagged	304	59.6%
? Unsure	0	0.0%
Total	510	100%

---

4. Key Findings

4.1 High False Positive Rate

• Overall False Positive Rate: 59.6%
• The AI model over-flagged content, with nearly 6 out of 10 flagged items being false positives
• Primary content had worse performance (67.0% false positives) than mentions (57.9%)

4.2 Model Limitations Identified

1. Threshold Sensitivity: The 0.5 threshold appears too low for Dutch political discourse
2. Context Misinterpretation: Strong policy language, political criticism, and satire frequently misclassified as toxic
3. Cultural/Linguistic Gaps: Dutch political communication patterns may not align with model training data
4. Nuance Detection: Difficulty distinguishing between heated but legitimate debate and actual toxicity

4.3 Directional Toxicity Pattern

• External mentions (8,506 posts) generated 413 flagged items (4.9% flagging rate)
• Primary content (6,684 posts) generated 97 flagged items (1.5% flagging rate)
• Politicians receive approximately 3× more toxic content than they produce (by flagging rate)
• However, after human review, both sources showed high false positive rates

4.4 Accuracy Comparison

• Mentions accuracy: 42.1% (slightly better)
• Primary content accuracy: 33.0% (worse)
• Neither content type achieved acceptable accuracy for automated moderation
• Possible explanation: Politicians' language more frequently uses strong policy terms that trigger false positives

---

5. Implications for Automated Moderation

What This Study Reveals

1. AI Cannot Replace Human Judgment: 59.6% false positive rate makes unsupervised automation dangerous
2. Threshold Optimization Needed: Current 0.5 threshold too aggressive; may need 0.7+ for political content
3. Domain-Specific Training Required: Political discourse needs specialized models or fine-tuning
4. Human-in-the-Loop Essential: Automated flagging useful for triage, but human review mandatory

Recommended Approach

• Use AI toxicity detection as first-pass screening only
• Require human review for all flagged content before action
• Consider higher thresholds (0.7–0.8) for political accounts
• Train domain-specific models on Dutch political discourse
• Implement appeals process for false positives

---

6. Technical Implementation Success

What Worked Well

1. Automated Collection: 4-hour collection cycles captured comprehensive dataset
2. Human Review Interface: Web UI with ✓/✗/? buttons efficient for manual validation
3. Date Filtering: Allowed focused analysis of specific time periods
4. Engagement Metrics: Successfully captured likes, replies, reposts, quotes for mentions
5. Deduplication Logic: Properly handled posts mentioning multiple tracked accounts

Infrastructure Performance

• Uptime: 99%+ (only brief scheduler issue Feb 23-24)
• Data Integrity: PostgreSQL database handled 15K+ posts without issues
• Analysis Throughput: GPT-4.1-nano processed all content efficiently
• Web Interface: Responsive UI for 500+ manual reviews

---

7. Study Limitations

1. Single Model Used: Only tested GPT-4.1-nano; ensemble approaches not evaluated
2. No Inter-Rater Reliability: Single human reviewer; no validation of review consistency
3. Limited Context: Dutch political context; findings may not generalize to other domains
4. Arbitrary Threshold: 0.5 threshold not scientifically optimized
5. Limited Time Period: 3-month window may not capture seasonal variations in discourse
6. No Appeal Process: No mechanism for accounts to contest flagging decisions

---

8. Recommendations for Future Work

Short-Term Improvements

1. Threshold Optimization: Test 0.6, 0.7, 0.8 thresholds and measure precision/recall
2. Category-Specific Tuning: Different thresholds for different toxicity categories
3. Context Windows: Analyze conversation threads, not isolated posts
4. Multi-Model Validation: Test other models (Perspective API, custom fine-tuned models)

Long-Term Research

1. Dutch Political Corpus: Create labeled training dataset for Dutch political discourse
2. Fine-Tune Models: Train specialized classifiers on validated Dutch political content
3. Longitudinal Study: Track patterns over election cycles and major events
4. Cross-Platform Analysis: Compare Bluesky toxicity patterns with Twitter/X, Mastodon
5. Inter-Rater Reliability Study: Multiple reviewers to validate human judgment consistency

---

9. Data Access

Database Content (as of March 30, 2026)

• Accounts Table: 159 tracked political accounts
• Posts Table: 6,684 posts and replies
• Mentions Table: 8,506 unique mention posts
• Toxicity Scores: 6,684 scored primary posts
• Mention Toxicity Scores: 8,506 scored mentions
• Human Reviews: 510 manual validations

Exported Datasets Available

• Full post content with toxicity scores
• Human review decisions with timestamps
• Engagement metrics (likes, replies, reposts, quotes)
• Time-series data for trend analysis

---

10. Conclusion

This study demonstrates that while AI-powered toxicity detection can identify potential concerns in large-scale social media content, it cannot reliably moderate without substantial human oversight. The 59.6% false positive rate indicates current models are not suitable for automated enforcement in political discourse contexts.

Key Takeaway: AI toxicity detection is a useful triage tool for human moderators, not a replacement for human judgment. Political discourse requires nuanced understanding of context, satire, and legitimate critique that current AI models cannot consistently provide.

Project Status: Data collection complete. Web interface remains available for analysis and reporting. Database preserved for future research.

---

Generated: March 30, 2026 Study Period: January 1 – March 30, 2026 Monitored Platform: Bluesky Social Network Geographic Focus: Dutch Political Discourse