Commit graph

10 commits

Author SHA1 Message Date
Pieter
bddbcd74ce Complete toxicity analysis implementation with manual review
- Fixed review submission bug (item_id now uses internal database ID)
- Added comprehensive logging to review API endpoint
- Updated analysis report for Jan 1 - Mar 30, 2026 period
- Report includes all 44 manually reviewed posts
- 4 confirmed toxic, 40 false positives (90.9% FP rate)
- Improved table layout: reduced column widths, smaller text
- Fixed horizontal scrolling with max-width override
- All flagged posts now successfully reviewed and stored

Key findings:
- 7,506 posts collected, 3,938 analyzed
- Only 0.10% confirmed toxic (4 of 3,938)
- High false positive rate shows challenge of automated detection
- Most FPs were legitimate political discourse about extremism

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-31 17:50:23 +02:00
Pieter
8989c5728d Fix double @ symbols and make status text clickable
- Fixed author display to show single @ (was showing @@account)
- Made status text clickable linking to full Mastodon post
- Fixed author links to point to Mastodon instances instead of Bluesky

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-31 10:03:05 +02:00
Pieter
ac2b50751b Fix flagged page styling by adding CSS/JS blocks to base template
- Add {% block extra_css %} to base.html head section
- Add {% block extra_js %} to base.html before closing body tag
- Add encode_uri template filter for URL encoding

Flagged page CSS and JavaScript now load correctly, fixing:
- Filter bar styling
- Table formatting
- Review button styles and functionality

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-31 09:25:18 +02:00
Pieter
c177725ebc Fix analysis dashboard trend data field mapping
- Update web.py to use flagged_posts/flagged_mentions fields
- Matches data structure returned by analysis_helpers

Analysis dashboard and trend graph now working correctly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-31 09:03:33 +02:00
Pieter
c7844ec502 Fix analysis data loading and template compatibility
- Add template compatibility fields to flagged items (item_id, item_type, source_type, text)
- Fix trend data field names (flagged_posts, flagged_mentions instead of flagged_statuses)
- Change trend sort order to ASC for chronological display
- Ensure API endpoint field mapping matches template expectations

Fixes:
- Internal server error on /analysis/flagged
- Empty trend graph (data now loads correctly)
- Hover tooltips now show correct flagged counts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-31 09:02:43 +02:00
Pieter
95a56ee833 Improve analysis dashboard UI
- Fix chart y-axis to use suggestedMax (0.2) instead of max (1.0) for better visibility of low toxicity values
- Improve decimal precision in chart tooltips (0.1% instead of 0%)
- Style quick-link buttons with solid background, shadow, and hover effects like Bluesky collector

Charts now properly display low toxicity values (0.02-0.05 range)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-31 08:59:32 +02:00
Pieter
2172efa701 Fix toxicity analysis web interface
- Fix analysis_helpers stats to match template expectations (posts/replies/mentions breakdown)
- Fix SQL interval syntax in trend query
- Fix URL routing in templates (analysis_flagged, accounts_list)
- Add .claude/ to .gitignore

Analysis dashboard now accessible at http://localhost:8585/analysis

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-30 17:07:12 +02:00
Pieter
a8b28f63a0 Complete toxicity analysis system setup and testing
- Copy and integrate analysis templates from Bluesky collector
- Add template filters (format_number, time_ago, truncate_text)
- Add Analysis link to navigation
- Fix analyzer database schema compatibility (account_db_id, status_type)
- Add OPENAI_API_KEY to docker-compose environment
- Successfully tested analyzer on 100 statuses ($0.0116, 75.4 seconds)

Web interface available at /analysis and /analysis/flagged

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-30 15:39:36 +02:00
Pieter
0aa4a16fab Add toxicity analysis system for Mastodon statuses
Implements comprehensive toxicity analysis following the Bluesky collector architecture:

- Analyzer module with async batch processing using GPT-4o-mini
- Database schema for toxicity scores and analysis run tracking
- 12 toxicity categories (toxic, threat, hate_speech, racism, antisemitism, islamophobia, sexism, homophobia, insult, dehumanization, extremism, ableism)
- Web interface routes for analysis dashboard and flagged content review
- Manual review API endpoint for human validation
- Analysis helper functions for database queries
- Dutch language support with coded political term recognition

Usage:
  docker exec mastodon-collector-collector-1 python -m app.analyzer

See TOXICITY_ANALYSIS.md for full documentation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-30 14:43:35 +02:00
Pieter
72dbf0d2b6 Initial commit: Mastodon collector application
Add Flask-based application for collecting and archiving Mastodon posts from configured accounts.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-09 08:05:54 +01:00