58 lines
3.5 KiB
Markdown
58 lines
3.5 KiB
Markdown
# Discord Data Collector - Agent Guide
|
|
|
|
## Commands
|
|
- **Setup**: `python setup.py` - Install dependencies and create config files
|
|
- **Run Main**: `python main.py` - Start Discord data collector
|
|
- **CLI Tools**: `python cli.py [command]` - Available: export, stats, search, servers, user-servers, server-users, backup, cleanup, test
|
|
- **Test Imports**: `python test_imports.py` - Verify all dependencies work
|
|
- **Install Dependencies**: `pip install -r requirements.txt`
|
|
|
|
## Architecture
|
|
- **Entry Points**: `main.py` (main app), `cli.py` (CLI tools), `setup.py` (installation)
|
|
- **Core Modules**:
|
|
- `src/client.py` - Discord client implementation with data collection
|
|
- `src/database.py` - Database manager with MariaDB->JSON fallback, UserData dataclass
|
|
- `src/config.py` - TOML/env configuration management
|
|
- `src/rate_limiter.py` - API rate limiting
|
|
- `src/logger.py` - Logging setup
|
|
- **Data Storage**: MariaDB (primary) or JSON files in `data/` directory, backups in `data/backups/`
|
|
- **Configuration**: `config.toml` for app settings, `.env` for Discord token
|
|
|
|
## Code Style
|
|
- **Python**: 3.8+ required, async/await patterns throughout
|
|
- **Imports**: Standard library first, third-party, then local (`from .module import Class`)
|
|
- **Error Handling**: Comprehensive try/catch with logging, graceful degradation
|
|
- **Naming**: snake_case for variables/functions, PascalCase for classes
|
|
- **Types**: Type hints required (`typing` module), dataclasses for data structures
|
|
- **Async**: All I/O operations use async/await, proper cleanup in finally blocks
|
|
|
|
## Recent Changes & Fixes
|
|
|
|
### Database Integration (2025-07-14)
|
|
- **✅ Remote MariaDB Support**: Added full support for remote MariaDB connections via .env credentials
|
|
- **✅ MariaDB->JSON Fallback**: Smart database factory that tries MariaDB first, falls back to JSON
|
|
- **✅ Dictionary Cursors**: Fixed asyncmy integration to return proper dictionary results instead of tuples
|
|
- **✅ Transaction Handling**: Proper async transaction management with START TRANSACTION/COMMIT/ROLLBACK
|
|
- **✅ Duplicate Key Handling**: Added INSERT IGNORE to prevent duplicate user-server entries
|
|
|
|
### Discord Data Collection Improvements
|
|
- **✅ Enhanced Bio Collection**: Multiple fallback methods for user bio extraction (direct, profile fetch, activity-based)
|
|
- **✅ Better Status Tracking**: Comprehensive status collection including platform-specific status (desktop/mobile/web)
|
|
- **✅ Activity Tracking**: Multi-activity support with proper type detection and formatting
|
|
- **✅ Discord API Fixes**: Fixed Guild.fetch_members() coroutine handling for discord.py-self
|
|
|
|
### CLI Enhancements
|
|
- **✅ Server Management Commands**: Added `servers`, `user-servers`, `server-users` commands for detailed server tracking
|
|
- **✅ Enhanced Search**: Search results now include bio, status, activity, and server membership info
|
|
- **✅ Better Formatting**: Improved output formatting with truncation and better readability
|
|
|
|
### Network & Connectivity
|
|
- **✅ Connection Testing**: Added early Discord connectivity testing with clear error messages
|
|
- **✅ Graceful Error Handling**: Better error messages for network issues and missing dependencies
|
|
|
|
## Production Status
|
|
- **Database**: ✅ Working (11 users collected from 1 server)
|
|
- **Discord Integration**: ✅ Working (logged in as _pixelparadox#0, 5 servers connected)
|
|
- **Data Collection**: ✅ Working (users, status, server membership being tracked)
|
|
- **Remote Storage**: ✅ Working (MariaDB at xargana.tr:3306, ~52KB database)
|