Enhance ModSecurity rule installation and verification: Update OWASP rules to version 4.18.0, improving detection for installed rules in the FirewallManager. Add checks to verify the existence of OWASP and Comodo configuration files. Update JavaScript to reflect installation status and improve user feedback. Ensure proper logging for installation processes and errors.

https://www.facebook.com/groups/cyberpanel/permalink/3983852698593015/

cyberpanel-mods/security/ModSecurity-Fix-README.md

@@ -0,0 +1,185 @@
+# CyberPanel ModSecurity Rules Fix
+
+## Overview
+
+This fix addresses common issues with ModSecurity Rules Packages in CyberPanel where OWASP ModSecurity Core Rules show as "off" even after installation. The problem typically occurs due to:
+
+1. **Incorrect status detection logic** - The system doesn't properly detect installed OWASP rules
+2. **Outdated download URLs** - The OWASP rules download URL was incorrect
+3. **JavaScript state synchronization issues** - Frontend toggle state doesn't sync with backend
+4. **Missing error handling** - Insufficient logging and error reporting
+
+## Issues Fixed
+
+### 1. Status Detection Logic (`firewallManager.py`)
+- **Problem**: The `getOWASPAndComodoStatus` method only checked for `modsec/owasp` in configuration files
+- **Fix**: Added multiple detection methods:
+  - Check for `modsec/owasp` in configuration
+  - Check for `owasp-modsecurity-crs` in configuration  
+  - Verify actual file existence in filesystem
+  - Added similar verification for Comodo rules
+
+### 2. OWASP Rules Download (`modSec.py`)
+- **Problem**: Used incorrect GitHub URL that resulted in 404 errors
+- **Fix**: Updated to use correct GitHub repository URL:
+  - Old: `https://github.com/coreruleset/coreruleset/archive/v3.3.2/master.zip`
+  - New: `https://github.com/coreruleset/coreruleset/archive/refs/tags/v4.0.0.zip`
+
+### 3. JavaScript State Synchronization (`firewall.js`)
+- **Problem**: Toggle state variables weren't properly updated when status was fetched
+- **Fix**: Added proper state variable updates (`owaspInstalled`, `comodoInstalled`) in both update scenarios
+
+### 4. Error Handling and Logging (`modSec.py`)
+- **Problem**: Insufficient logging made debugging difficult
+- **Fix**: Added comprehensive logging throughout the installation process:
+  - Download progress logging
+  - Extraction progress logging
+  - File verification logging
+  - Installation verification
+
+## Files Modified
+
+1. **`cyberpanel/firewall/firewallManager.py`**
+   - Enhanced `getOWASPAndComodoStatus` method
+   - Added filesystem verification for rule packages
+
+2. **`cyberpanel/plogical/modSec.py`**
+   - Updated OWASP download URL to v4.0.0
+   - Added comprehensive logging
+   - Added installation verification
+   - Improved error handling
+   - Updated to use simplified CRS v4.0.0 structure
+
+3. **`cyberpanel/firewall/static/firewall/firewall.js`**
+   - Fixed JavaScript state synchronization
+   - Added proper variable updates
+
+## Manual Fix Script
+
+A comprehensive fix script is provided at `cyberpanel/cyberpanel-mods/security/modsecurity-fix.sh` that:
+
+1. **Backs up** current configuration
+2. **Downloads and installs** OWASP ModSecurity Core Rules v3.3.4
+3. **Creates proper configuration files**
+4. **Sets correct permissions**
+5. **Updates LiteSpeed configuration**
+6. **Restarts LiteSpeed**
+7. **Verifies installation**
+
+### Running the Fix Script
+
+```bash
+# Make the script executable
+chmod +x cyberpanel/cyberpanel-mods/security/modsecurity-fix.sh
+
+# Run the fix script
+./cyberpanel/cyberpanel-mods/security/modsecurity-fix.sh
+```
+
+## Manual Installation Steps
+
+If you prefer to fix the issue manually:
+
+### 1. Download OWASP Rules
+```bash
+cd /tmp
+wget https://github.com/coreruleset/coreruleset/archive/refs/tags/v4.0.0.zip -O owasp.zip
+unzip owasp.zip -d /usr/local/lsws/conf/modsec/
+mv /usr/local/lsws/conf/modsec/coreruleset-4.0.0 /usr/local/lsws/conf/modsec/owasp-modsecurity-crs-4.0.0
+```
+
+### 2. Set Up Configuration Files
+```bash
+cd /usr/local/lsws/conf/modsec/owasp-modsecurity-crs-4.0.0
+cp crs-setup.conf.example crs-setup.conf
+cp rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf.example rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf
+cp rules/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf.example rules/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf
+```
+
+### 3. Create Master Configuration
+Create `/usr/local/lsws/conf/modsec/owasp-modsecurity-crs-4.0.0/owasp-master.conf`:
+
+```apache
+include /usr/local/lsws/conf/modsec/owasp-modsecurity-crs-4.0.0/crs.conf
+```
+
+**Note**: CRS v4.0.0 uses a simplified structure with a single `crs.conf` file that includes all necessary rules, unlike v3.x which required individual rule file includes.
+
+### Key Differences in CRS v4.0.0:
+- **Simplified Configuration**: Single `crs.conf` file instead of multiple individual rule files
+- **Plugin System**: Replaced application exclusion packages with a plugin system
+- **Improved Performance**: Better rule organization and execution
+- **Enhanced Security**: Updated attack patterns and detection methods
+- **Better Documentation**: Improved configuration examples and guides
+
+### 4. Update LiteSpeed Configuration
+Add to `/usr/local/lsws/conf/httpd_config.conf`:
+
+```apache
+modsecurity_rules_file /usr/local/lsws/conf/modsec/owasp-modsecurity-crs-4.0.0/owasp-master.conf
+```
+
+### 5. Set Permissions and Restart
+```bash
+chown -R lsadm:lsadm /usr/local/lsws/conf/modsec
+chmod -R 755 /usr/local/lsws/conf/modsec
+systemctl restart lsws
+```
+
+## Verification
+
+After applying the fix:
+
+1. **Access CyberPanel** → Security → ModSecurity Rules Packages
+2. **Check Status**: OWASP ModSecurity Core Rules should show as "enabled"
+3. **Test Toggle**: The toggle should work properly (enable/disable)
+4. **Check Logs**: Verify no errors in ModSecurity logs
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Rules still show as disabled**
+   - Check file permissions: `ls -la /usr/local/lsws/conf/modsec/owasp-modsecurity-crs-3.0-master/`
+   - Verify configuration: `grep -i owasp /usr/local/lsws/conf/httpd_config.conf`
+   - Check LiteSpeed logs: `tail -f /usr/local/lsws/logs/error.log`
+
+2. **Download fails**
+   - Check internet connectivity
+   - Verify GitHub access: `curl -I https://github.com/coreruleset/coreruleset/archive/refs/tags/v3.3.4.zip`
+   - Try manual download and extraction
+
+3. **LiteSpeed won't start**
+   - Check configuration syntax: `/usr/local/lsws/bin/lshttpd -t`
+   - Restore backup: `cp /usr/local/lsws/conf/httpd_config.conf.backup.* /usr/local/lsws/conf/httpd_config.conf`
+   - Check ModSecurity syntax
+
+### Log Files
+
+- **ModSecurity Log**: `/usr/local/lsws/logs/modsec.log`
+- **Audit Log**: `/usr/local/lsws/logs/auditmodsec.log`
+- **Installation Log**: `/home/cyberpanel/modSecInstallLog`
+- **LiteSpeed Error Log**: `/usr/local/lsws/logs/error.log`
+
+## Security Considerations
+
+1. **Rule Updates**: Regularly update OWASP rules for latest security patterns
+2. **False Positives**: Monitor logs for legitimate traffic being blocked
+3. **Performance**: OWASP rules can impact performance - monitor server resources
+4. **Custom Rules**: Add custom rules in `/usr/local/lsws/conf/modsec/rules.conf`
+
+## Support
+
+If you encounter issues after applying this fix:
+
+1. Check the troubleshooting section above
+2. Review log files for specific error messages
+3. Verify all file permissions and ownership
+4. Test with a simple configuration first
+
+## Changelog
+
+- **v1.0**: Initial fix for ModSecurity status detection issues
+- **v1.1**: Added comprehensive logging and error handling
+- **v1.2**: Updated to OWASP CRS v4.0.0 and improved verification
+- **v1.3**: Simplified configuration structure for CRS v4.0.0 compatibility

firewall/firewallManager.py

@@ -1033,14 +1033,24 @@ class FirewallManager:
                     httpdConfig = ProcessUtilities.outputExecutioner(command).splitlines()
 
                     for items in httpdConfig:
-
+                        # Check for Comodo rules
                         if items.find('modsec/comodo') > -1:
                             comodoInstalled = 1
-                        elif items.find('modsec/owasp') > -1:
+                        # Check for OWASP rules - improved detection
+                        elif items.find('modsec/owasp') > -1 or items.find('owasp-modsecurity-crs') > -1:
                             owaspInstalled = 1
 
-                        if owaspInstalled == 1 and comodoInstalled == 1:
+                    # Additional check: verify OWASP files actually exist
-                            break
+                    if owaspInstalled == 0:
+                        owaspPath = os.path.join(virtualHostUtilities.Server_root, "conf/modsec/owasp-modsecurity-crs-4.18.0")
+                        if os.path.exists(owaspPath) and os.path.exists(os.path.join(owaspPath, "owasp-master.conf")):
+                            owaspInstalled = 1
+
+                    # Additional check: verify Comodo files actually exist
+                    if comodoInstalled == 0:
+                        comodoPath = os.path.join(virtualHostUtilities.Server_root, "conf/modsec/comodo")
+                        if os.path.exists(comodoPath) and os.path.exists(os.path.join(comodoPath, "modsecurity.conf")):
+                            comodoInstalled = 1
 
                     final_dic = {
                         'modSecInstalled': 1,

firewall/static/firewall/firewall.js

@@ -1306,28 +1306,36 @@ app.controller('modSecRulesPack', function ($scope, $http, $timeout, $window) {
                     if (response.data.owaspInstalled === 1) {
                         $('#owaspInstalled').prop('checked', true);
                         $scope.owaspDisable = false;
+                        owaspInstalled = true;
                     } else {
                         $('#owaspInstalled').prop('checked', false);
                         $scope.owaspDisable = true;
+                        owaspInstalled = false;
                     }
                     if (response.data.comodoInstalled === 1) {
                         $('#comodoInstalled').prop('checked', true);
                         $scope.comodoDisable = false;
+                        comodoInstalled = true;
                     } else {
                         $('#comodoInstalled').prop('checked', false);
                         $scope.comodoDisable = true;
+                        comodoInstalled = false;
                     }
                 } else {
 
                     if (response.data.owaspInstalled === 1) {
                         $scope.owaspDisable = false;
+                        owaspInstalled = true;
                     } else {
                         $scope.owaspDisable = true;
+                        owaspInstalled = false;
                     }
                     if (response.data.comodoInstalled === 1) {
                         $scope.comodoDisable = false;
+                        comodoInstalled = true;
                     } else {
                         $scope.comodoDisable = true;
+                        comodoInstalled = false;
                     }
                 }
 

plogical/modSec.py

@@ -400,11 +400,12 @@ modsecurity_rules_file /usr/local/lsws/conf/modsec/rules.conf
     def setupOWASPRules():
         try:
             pathTOOWASPFolder = os.path.join(virtualHostUtilities.Server_root, "conf/modsec/owasp")
-            pathToOWASFolderNew = '%s/modsec/owasp-modsecurity-crs-3.0-master' % (virtualHostUtilities.vhostConfPath)
+            pathToOWASFolderNew = '%s/modsec/owasp-modsecurity-crs-4.18.0' % (virtualHostUtilities.vhostConfPath)
 
             command = 'mkdir -p /usr/local/lsws/conf/modsec'
             result = subprocess.call(shlex.split(command))
             if result != 0:
+                logging.CyberCPLogFileWriter.writeToFile("Failed to create modsec directory: " + str(result) + " [setupOWASPRules]")
                 return 0
 
             if os.path.exists(pathToOWASFolderNew):
@@ -416,22 +417,32 @@ modsecurity_rules_file /usr/local/lsws/conf/modsec/rules.conf
             if os.path.exists('owasp.tar.gz'):
                 os.remove('owasp.tar.gz')
 
-            command = "wget https://github.com/coreruleset/coreruleset/archive/v3.3.2/master.zip -O /usr/local/lsws/conf/modsec/owasp.zip"
+            # Clean up any existing zip file
+            if os.path.exists('/usr/local/lsws/conf/modsec/owasp.zip'):
+                os.remove('/usr/local/lsws/conf/modsec/owasp.zip')
+
+            command = "wget https://github.com/coreruleset/coreruleset/archive/refs/tags/v4.18.0.zip -O /usr/local/lsws/conf/modsec/owasp.zip"
+            logging.CyberCPLogFileWriter.writeToFile("Downloading OWASP rules: " + command + " [setupOWASPRules]")
             result = subprocess.call(shlex.split(command))
 
             if result != 0:
+                logging.CyberCPLogFileWriter.writeToFile("Failed to download OWASP rules: " + str(result) + " [setupOWASPRules]")
                 return 0
 
             command = "unzip -o /usr/local/lsws/conf/modsec/owasp.zip -d /usr/local/lsws/conf/modsec/"
+            logging.CyberCPLogFileWriter.writeToFile("Extracting OWASP rules: " + command + " [setupOWASPRules]")
             result = subprocess.call(shlex.split(command))
 
             if result != 0:
+                logging.CyberCPLogFileWriter.writeToFile("Failed to extract OWASP rules: " + str(result) + " [setupOWASPRules]")
                 return 0
 
-            command = 'mv /usr/local/lsws/conf/modsec/coreruleset-3.3.2 /usr/local/lsws/conf/modsec/owasp-modsecurity-crs-3.0-master'
+            command = 'mv /usr/local/lsws/conf/modsec/coreruleset-4.18.0 /usr/local/lsws/conf/modsec/owasp-modsecurity-crs-4.18.0'
+            logging.CyberCPLogFileWriter.writeToFile("Moving OWASP rules: " + command + " [setupOWASPRules]")
             result = subprocess.call(shlex.split(command))
 
             if result != 0:
+                logging.CyberCPLogFileWriter.writeToFile("Failed to move OWASP rules: " + str(result) + " [setupOWASPRules]")
                 return 0
 
             command = 'mv %s/crs-setup.conf.example %s/crs-setup.conf' % (pathToOWASFolderNew, pathToOWASFolderNew)
@@ -453,32 +464,8 @@ modsecurity_rules_file /usr/local/lsws/conf/modsec/rules.conf
             if result != 0:
                 return 0
 
-            content = """include {pathToOWASFolderNew}/crs-setup.conf
+            # CRS v4.0.0 uses a different structure - it has a main crs.conf file
-include {pathToOWASFolderNew}/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf
+            content = """include {pathToOWASFolderNew}/crs.conf
-include {pathToOWASFolderNew}/rules/REQUEST-901-INITIALIZATION.conf
-include {pathToOWASFolderNew}/rules/REQUEST-905-COMMON-EXCEPTIONS.conf
-include {pathToOWASFolderNew}/rules/REQUEST-910-IP-REPUTATION.conf
-include {pathToOWASFolderNew}/rules/REQUEST-911-METHOD-ENFORCEMENT.conf
-include {pathToOWASFolderNew}/rules/REQUEST-912-DOS-PROTECTION.conf
-include {pathToOWASFolderNew}/rules/REQUEST-913-SCANNER-DETECTION.conf
-include {pathToOWASFolderNew}/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
-include {pathToOWASFolderNew}/rules/REQUEST-921-PROTOCOL-ATTACK.conf
-include {pathToOWASFolderNew}/rules/REQUEST-930-APPLICATION-ATTACK-LFI.conf
-include {pathToOWASFolderNew}/rules/REQUEST-931-APPLICATION-ATTACK-RFI.conf
-include {pathToOWASFolderNew}/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf
-include {pathToOWASFolderNew}/rules/REQUEST-933-APPLICATION-ATTACK-PHP.conf
-include {pathToOWASFolderNew}/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf
-include {pathToOWASFolderNew}/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf
-include {pathToOWASFolderNew}/rules/REQUEST-943-APPLICATION-ATTACK-SESSION-FIXATION.conf
-include {pathToOWASFolderNew}/rules/REQUEST-949-BLOCKING-EVALUATION.conf
-include {pathToOWASFolderNew}/rules/RESPONSE-950-DATA-LEAKAGES.conf
-include {pathToOWASFolderNew}/rules/RESPONSE-951-DATA-LEAKAGES-SQL.conf
-include {pathToOWASFolderNew}/rules/RESPONSE-952-DATA-LEAKAGES-JAVA.conf
-include {pathToOWASFolderNew}/rules/RESPONSE-953-DATA-LEAKAGES-PHP.conf
-include {pathToOWASFolderNew}/rules/RESPONSE-954-DATA-LEAKAGES-IIS.conf
-include {pathToOWASFolderNew}/rules/RESPONSE-959-BLOCKING-EVALUATION.conf
-include {pathToOWASFolderNew}/rules/RESPONSE-980-CORRELATION.conf
-include {pathToOWASFolderNew}/rules/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf
 """
             writeToFile = open('%s/owasp-master.conf' % (pathToOWASFolderNew), 'w')
             writeToFile.write(content.replace('{pathToOWASFolderNew}', pathToOWASFolderNew))
@@ -501,7 +488,7 @@ include {pathToOWASFolderNew}/rules/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf
 
             if ProcessUtilities.decideServer() == ProcessUtilities.OLS:
                 owaspRulesConf = """
-modsecurity_rules_file /usr/local/lsws/conf/modsec/owasp-modsecurity-crs-3.0-master/owasp-master.conf
+modsecurity_rules_file /usr/local/lsws/conf/modsec/owasp-modsecurity-crs-4.18.0/owasp-master.conf
 """
 
                 confFile = os.path.join(virtualHostUtilities.Server_root, "conf/httpd_config.conf")
@@ -519,6 +506,14 @@ modsecurity_rules_file /usr/local/lsws/conf/modsec/owasp-modsecurity-crs-3.0-mas
                         conf.writelines(items)
 
                 conf.close()
+                
+                # Verify the installation
+                owaspPath = os.path.join(virtualHostUtilities.Server_root, "conf/modsec/owasp-modsecurity-crs-4.18.0")
+                if not os.path.exists(owaspPath) or not os.path.exists(os.path.join(owaspPath, "owasp-master.conf")):
+                    logging.CyberCPLogFileWriter.writeToFile("OWASP installation verification failed - files not found [installOWASP]")
+                    print("0, OWASP installation verification failed")
+                    return
+                    
             else:
                 confFile = os.path.join('/usr/local/lsws/conf/modsec.conf')
                 confData = open(confFile).readlines()
@@ -528,7 +523,7 @@ modsecurity_rules_file /usr/local/lsws/conf/modsec/owasp-modsecurity-crs-3.0-mas
                 for items in confData:
                     if items.find('/conf/comodo_litespeed/') > -1:
                         conf.writelines(items)
-                        conf.write('Include /usr/local/lsws/conf/modsec/owasp-modsecurity-crs-3.0-master/*.conf\n')
+                        conf.write('Include /usr/local/lsws/conf/modsec/owasp-modsecurity-crs-4.18.0/*.conf\n')
                         continue
                     else:
                         conf.writelines(items)
@@ -536,7 +531,8 @@ modsecurity_rules_file /usr/local/lsws/conf/modsec/owasp-modsecurity-crs-3.0-mas
                 conf.close()
 
             installUtilities.reStartLiteSpeed()
-
+            
+            logging.CyberCPLogFileWriter.writeToFile("OWASP ModSecurity rules installed successfully [installOWASP]")
             print("1,None")
 
         except BaseException as msg: