Problem: Bulk fix operations were hitting rate limits (100 backups per scan).
Solution:
- Implemented 10x higher rate limits for API key authenticated requests
- These are trusted requests from the platform for legitimate bulk operations
- Regular file token auth keeps original limits for security
Rate Limit Changes:
- backup-file: 100 → 1000 for API key auth
- get-file: 500 → 5000 for API key auth
- replace-file: 100 → 1000 for API key auth
- rename-file: 50 → 500 for API key auth
- delete-file: 50 → 500 for API key auth
This allows bulk fix operations to process up to 1000 files without hitting rate limits,
while maintaining security for regular scan operations.
Problem: VPS API keys were generated dynamically but never saved to the database,
causing file fix operations to fail with "Invalid token" error.
Root Cause:
- When CyberPanel runs on VPS, it calls platform's /api/vps/generate-api-key/
- The returned API key was used for scan submission but not persisted
- Later file fix operations couldn't validate this API key
Solution:
- Save VPS API key to ai_scanner_settings table whenever obtained
- Modified 4 locations where VPS API key is retrieved:
1. startScan() - Save when initiating a scan
2. getPlatformMonitorUrl() - Save when checking scan status
3. addPaymentMethod() - Save when configuring payment (2 locations)
Implementation:
- Use get_or_create() to ensure scanner settings exist
- Update existing settings if API key is empty or different
- Set is_payment_configured=True for VPS accounts (implicit payment)
- Continue operation even if saving fails (non-blocking)
Result:
- VPS API keys are automatically persisted on first use
- File fixes work without manual database intervention
- All AI Scanner features work seamlessly on VPS instances
Problem: "Failed to backup file before replacement" error when using API key authentication.
Root Cause:
- The externalApp field (which contains the system user for file operations) was sometimes None
- When using API key authentication, the code couldn't determine which user to run commands as
Solution:
- Added fallback logic to ensure we always have a valid user for file operations:
1. First try wp_site.owner.externalApp
2. If not set, try Websites.objects.get(domain).externalApp
3. If still not set, fall back to admin username
- Added detailed error messages and logging throughout backup operations
- Enhanced error reporting to include user context and operation details
- Added validation for backup directory creation with proper error handling
Changes:
- Modified validate_access_token() OPTION 2 and OPTION 3 to ensure external_app is always set
- Enhanced backup and replace operations with better error messages
- Added detailed logging for debugging file operation failures
- Include user context in error messages for easier troubleshooting
This ensures file operations work correctly even when externalApp field is not properly configured.
Problem: File fixes were failing with "Invalid token" even though the platform was sending the correct API key.
Solution:
- Updated validate_access_token() to accept CyberPanel's own API keys for file operations
- Added three validation options:
1. FileAccessToken (temporary tokens for active scans)
2. API key validation with less restrictive admin check
3. Simple validation for platform callbacks (any valid API key + valid scan)
- Added extract_auth_token() helper to support both Bearer and X-API-Key headers
- Created debug endpoints for testing authentication (/api/ai-scanner/test-auth)
- Added test script for validation testing (supports remote servers via env vars)
This fix allows the platform to use CyberPanel's API key to fix files for any scan,
solving the "File access token has expired" issue for older scans.
Usage for remote testing:
CYBERPANEL_SERVER=http://your-server:8001 \
CYBERPANEL_API_KEY=cp_your_key \
CYBERPANEL_SCAN_ID=your-scan-id \
./test_api_key_fix.sh
- Added extract_auth_token() function to handle both Bearer and X-API-Key authentication
- Updated all file operation endpoints to support X-API-Key headers:
- list_files()
- get_file_content()
- scanner_backup_file()
- scanner_get_file()
- scanner_replace_file()
- scanner_rename_file()
- scanner_delete_file()
- Maintains backward compatibility with existing Bearer token authentication
- Added test script to verify both authentication methods work correctly
- Enables permanent API key authentication for file fixes (no more expired token issues)
This change allows the platform to fix files using the permanent CyberPanel API key
instead of temporary file access tokens that expire after ~1 hour.
- Write temp files to /tmp (accessible by all users)
- Website user can read from /tmp for cat command
- No permission issues with cross-directory operations
- Simplifies file operations (no intermediate copy needed)
- Clean up temp file with os.remove() as root
- Fixes: Files becoming empty due to permission issues
- Shell redirection requires shell=True parameter in executioner
- Without shell=True, the > is treated as literal argument
- Fixes: File contents not being replaced
- Change from cp to 'cat temp > target' for replacing file contents
- cat redirection is more reliable for overwriting existing files
- Ensures file contents are actually replaced
- Fixes: Files not being replaced even though cp returns success
- Write to /home/cyberpanel temp first (no user permission issues)
- Copy from /home/cyberpanel to user directory using executioner
- Use cp instead of mv for final file replacement (more reliable)
- Clean up temp files after successful operations
- Fixes: File corruption due to failed mv command via lscpd
- Write content to /home/cyberpanel/scanner_temp_{scan_id}_{timestamp}.tmp first
- Use Python open() instead of shell here-document to avoid:
- lscpd connection reset for large files (>2KB)
- EOF_MARKER conflicts in file content
- Shell command size limits
- Copy temp file to user directory using executioner with user context
- Clean up CyberPanel temp file after copy
- Fixes: [Errno 104] Connection reset by peer when writing large files
- executioner() returns 1 for success, 0 for failure (inverted)
- Fix mkdir check: if mkdir_result != 1 (was != 0)
- Fix cp check: if cp_result[0] != 1 (was != 0)
- Fix double slash in backup path by stripping trailing slash from wp_path
- Add logging to show mkdir_result value for debugging
- Add external_app field to AuthWrapper class
- Get externalApp from website object during authentication
- For FileAccessToken: Use Websites.externalApp
- For API Key: Use WPSites.owner.externalApp
- Update all 5 file operation endpoints to use file_token.external_app
- scanner_backup_file
- scanner_get_file
- scanner_replace_file
- scanner_rename_file
- scanner_delete_file
- Ensures externalApp matches wp_path since they come from same source
- Fixes backup API failing due to incorrect user context
