I have not seen any comprehensive how-to for installing mod_security on Linux server running Plesk, so I decided to make this one here.
The system being used in this example is RHEL3 and Plesk 7.5.4
 
1) First install httpd-devel. If you are using RedHat you can use
up2date -uf httpd-develOtherwise you can use YUM or something.
2) Now download mod_security
wget http://www.modsecurity.org/download/modsecurity-apache-1.9.2.tar.gz You can check here to see what is the most current version
http://www.modsecurity.org/download/
3) Untar it
tar -zvxf modsecurity-apache-1.9.2.tar.gz4) Now descend into \'apache2\' since that is the version Plesk comes with
cd modsecurity-apache-1.9.2/apache25) Compile
/usr/sbin/apxs -cia mod_security.cor
/[APACHEHOME]/bin/apxs -cia mod_security.c([APACHEHOME] = the installation directory, ex /etc/httpd/)
6) Configure apache
The installer should have added the appropriate LoadModule line to your Apache\'s httpd.conf, however you should verify that it did.
nano -w /etc/httpd/conf/httpd.confLook for the line

LoadModule security_module /usr/lib/httpd/modules/mod_security.so

Under that add this

Include "/etc/httpd/conf/modsec.user.conf"

Save the file
7) Make the config file
nano -w /etc/httpd/conf/modsec.user.confAdd the following to the new file

SecFilterEngine On
SecFilterCheckURLEncoding Off
SecAuditEngine RelevantOnly
SecAuditLog logs/audit_log
SecFilterDebugLog logs/modsec_debug_log
SecFilterDebugLevel 1
SecFilterDefaultAction "deny,log,status:406"
SecFilterSelective REMOTE_ADDR "^127.0.0.1$" nolog,allow
# These rules work with mod_security 1.9.x and above only
# This is a rule template, with limited application specific matches
# To prevent functionality loss
# Updated 2/20/2006
# Tested to work with apache1 and apache2
#
# BEGIN RULES
#
# Basic rules with arbitrary command detection
SecFilterSelective THE_REQUEST "\\.htgroup"
SecFilterSelective THE_REQUEST "\\.htaccess"
SecFilterSelective THE_REQUEST "cd\\.\\."
SecFilterSelective THE_REQUEST "///cgi-bin"
SecFilterSelective THE_REQUEST "/cgi-bin///"
SecFilterSelective THE_REQUEST "/~root"
SecFilterSelective THE_REQUEST "/~ftp"
SecFilterSelective THE_REQUEST "/htgrep" chain
SecFilterSelective THE_REQUEST "/htgrep" log,pass
SecFilterSelective THE_REQUEST "/\\.history"
SecFilterSelective THE_REQUEST "/\\.bash_history"
SecFilterSelective THE_REQUEST "/~nobody"
SecFilterSelective THE_REQUEST "SecFilterSelective THE_REQUEST "psybnc"
SecFilterSelective THE_REQUEST "cmd=cd\\x20/var"
SecFilterSelective THE_REQUEST "\\?STRENGUR"
SecFilterSelective THE_REQUEST "/etc/motd"
SecFilterSelective THE_REQUEST "/etc/passwd"
SecFilterSelective THE_REQUEST "conf/httpd\\.conf"
SecFilterSelective THE_REQUEST "/bin/ps"
SecFilterSelective THE_REQUEST "bin/tclsh"
SecFilterSelective THE_REQUEST "tclsh8\\x20"
SecFilterSelective THE_REQUEST "chsh"
SecFilterSelective THE_REQUEST "psybnc"
SecFilterSelective THE_REQUEST "udp\\.pl"
SecFilterSelective THE_REQUEST "wget\\x20"
SecFilterSelective THE_REQUEST "bin/nasm"
SecFilterSelective THE_REQUEST "nasm\\x20"
SecFilterSelective THE_REQUEST "/usr/bin/perl"
SecFilterSelective THE_REQUEST "links -dump "
SecFilterSelective THE_REQUEST "links -dump-(charset|width) "
SecFilterSelective THE_REQUEST "links (http|https|ftp)\\:/"
SecFilterSelective THE_REQUEST "links -source "
SecFilterSelective THE_REQUEST "cd\\x20/(tmp|var/tmp|etc/httpd/proxy|dev/shm)"
SecFilterSelective THE_REQUEST "cd\\.\\."
SecFilterSelective THE_REQUEST "///cgi-bin"
SecFilterSelective THE_REQUEST "/cgi-bin///"
SecFilterSelective THE_REQUEST "/~named(/| HTTP\\/(0\\.9|1\\.0|1\\.1)$)"
SecFilterSelective THE_REQUEST "/~guest(/| HTTP\\/(0\\.9|1\\.0|1\\.1)$)"
SecFilterSelective THE_REQUEST "/~logs(/| HTTP\\/(0\\.9|1\\.0|1\\.1)$)"
SecFilterSelective THE_REQUEST "/~sshd(/| HTTP\\/(0\\.9|1\\.0|1\\.1)$)"
SecFilterSelective THE_REQUEST "/~ftp(/| HTTP\\/(0\\.9|1\\.0|1\\.1)$)"
SecFilterSelective THE_REQUEST "/~bin(/| HTTP\\/(0\\.9|1\\.0|1\\.1)$)"
SecFilterSelective THE_REQUEST "/~nobody(/| HTTP\\/(0\\.9|1\\.0|1\\.1)$)"
SecFilterSelective THE_REQUEST "/\\.history HTTP\\/(0\\.9|1\\.0|1\\.1)$"
SecFilterSelective THE_REQUEST "/\\.bash_history HTTP\\/(0\\.9|1\\.0|1\\.1)$"
SecFilter "(cmd|command)=(cd|\\;|perl|python|lynx|links|mkdir|elinks|cmd|wget|uname|(s|r)(cp|sh)|net(stat|cat)|rexec|smbclient|curl)"
SecFilterSelective REQUEST_URI "/nessus_is_probing_you_"
SecFilterSelective REQUEST_URI "/NessusTest"
SecFilter "javascript\\://"
SecFilter "img src=javascript"
SecFilter "_PHPLIB\\[libdir\\]"
SecFilter "hdr=/"
#Require Content-Length to be provided with every POST request
SecFilterSelective REQUEST_METHOD "^POST$" chain
SecFilterSelective HTTP_Content-Length "^$"
SecFilterSelective HTTP_Transfer-Encoding "!^$"
#Specific XML-RPC attacks on xmlrpc.php
SecFilterSelective THE_REQUEST "(/xmlrpc|.*xmlrpc_services)\\.php" chain
SecFilter "(\\SecFilter "(echo( |\\(|\\\').*\\;|chr|fwrite|fopen|system)\\(.*\\)\\;"
#XML-RPC SQL injection generic signature
SecFilterSelective THE_REQUEST "(/xmlrpc|.*xmlrpc_services)\\.php" chain
SecFilter ".*.*.*(delete|insert|drop|replace|update|create)[[:space:]]+[A-Z|a-z|0-9|\\*| |,]+[[:space:]](from|into|table).*methodName\\>"
#Exploit phpBB Highlighting Code Execution/SQL Injection - Santy.A Worm
SecFilter "&highlight=\\\'\\.fwrite\\(fopen\\("
SecFilter "&highlight=\\x2527\\x252Esystem\\("
SecFilter "&highlight=\\\'\\.mysql_query\\("
SecFilterSelective THE_REQUEST "/quick-reply\\.php" chain
SecFilterSelective THE_REQUEST "(\\;|\\&)highlight=\\\'\\.system\\("
SecFilterSelective THE_REQUEST "&highlight=\\\'\\.mysql_query\\("
SecFilterSelective THE_REQUEST "&highlight=\\\'\\.fwrite\\(fopen\\("
SecFilterSelective THE_REQUEST "&highlight=%2527%252E"
SecFilterSelective THE_REQUEST "&highlight=\\x2527\\x252Esystem\\("
SecFilterSelective THE_REQUEST "/viewtopic\\.php\\?.*(highlight.*(\\\'\\.|\\x2527|\\x27)|include\\(.*GET\\[.*\\]\\)|=(http|https|ftp)\\:/|(printf|system)\\()"
#phpBB remote command execution exploit
SecFilterSelective REQUEST_URI "profile\\.php\\?GLOBALS\\[signature_bbcode_uid\\]=\\(\\.\\x2B\\)/e\\x00"
SecFilterSelective REQUEST_URI|POST_PAYLOAD "r57phpBB2017xpl"
SecFilterSelective POST_PAYLOAD "[EMAIL=_bill_gates@microsoft\\.com]_bill_gates@microsoft\\.com[/EMAIL]"
SecFilterSelective THE_REQUEST "/admin/admin_forums\\.php\\?sid=.*" chain
SecFilter "(forumname|forumdesc)=*\\<[[:space:]]*(script|about|applet|activex|chrome)"
SecFilterSelective REQUEST_URI "/posting\\.php\\?mode=reply\\&t=.*userid.*phpbb2mysql_t=(<[[:space:]]*script|(http|https|ftp)\\:/)"
SecFilterSelective REQUEST_URI "/posting\\.php\\\\?.*(<[[:space:]]*script|(http|https|ftp)\\:/)"
SecFilterSelective THE_REQUEST "changedir=%2Ftmp%2F.php"
SecFilter "^/viewtopic\\.php\\?" chain
SecFilter "chr\\(([0-9]{1,3})\\)"
SecFilterSelective THE_REQUEST "viewtopic\\.php" chain
SecFilterSelective "THE_REQUEST|ARG_VALUES" "(passthru|cmd|fopen|exit|fwrite)"
SecFilter "phpbb_root_path="
SecFilterSelective THE_REQUEST "/calendar_scheduler\\.php\\?start=(<[[:space:]]*script|(http|https|ftp)\\:/)"
SecFilterSelective REQUEST_URI "/groupcp\\.php\\?g=.*sid=\\\'"
SecFilterSelective REQUEST_URI "/index\\.php\\?(c|mark)=*\\\'"
SecFilterSelective REQUEST_URI "/portal\\.php\\?article=*\\\'"
SecFilterSelective REQUEST_URI "/viewforum.php?f=.*sid=\\\'"
SecFilterSelective REQUEST_URI "/viewtopic.php?p=.*sid=\\\'"
SecFilterSelective REQUEST_URI "/album_search\\.php\\?mode=\\\'"
SecFilterSelective REQUEST_URI "/album_cat\\.php\\?cat_id=.*sid=\\\'"
SecFilterSelective REQUEST_URI "/album_comment\\.php\\?pic_id=.*sid=\\\'"
SecFilterSelective REQUEST_URI "calendar_scheduler\\.php\\?d=.*&mode=&start=\\\'\\">"
SecFilterSelective REQUEST_URI "/profile\\.php\\?mode=viewprofile&u=.*((script|script|about|applet|activex|chrome)\\>|html|(http|https|ftp)\\:/)"
SecFilterSelective REQUEST_URI "/viewtopic\\.php\\?p=.*&highlight=.*((script|script|about|applet|activex|chrome)\\>|html|(http|https|ftp)\\:/)"
#awstats XSS vulnerabilities
SecFilterSelective THE_REQUEST "awstats" chain
SecFilterSelective ARGS "(pluginmode|loadplugin|debug|configdir|perl|cgi|chmod|exec|print)"
SecFilterSelective REQUEST_URI "/awstats\\.pl\\?(configdir|update|pluginmode|cgi)=(\\||echo|\\:system\\()"
SecFilterSelective REQUEST_URI "/awstats\\.pl\\?(debug=1|pluginmode=rawlog\\&loadplugin=rawlog|update=1\\&logfile=\\|)"
SecFilterSelective REQUEST_URI "/awstats\\.pl\\?[^\\r\\n]*logfile=\\|"
SecFilterSelective REQUEST_URI "/awstats\\.pl\\?configdir="
SecFilterSelective REQUEST_URI "awstats\\.pl\\?" chain
SecFilterSelective ARGS "(debug|configdir|perl|chmod|exec|print|cgi)"
Save and restart Apache.
 
Kiss a lot of your security woes goodbye Most people should notice a dramatic drop in server load and number of creepy processes running.
TIP: Setting the mod_security rules:
The rules are what drives the mod. You can customize them to match and deny access to any string found in inbound http requests.
If the rule set is too strong, functionality to legitimate web-applications can be lost.
The mod_security rules I included above are very effective in protecting against some of the web\'s most common exploits including phpbb2 awstats just to mention a couple.
It can be inserted directly into httpd.conf, but I suggest this is placed in its own .conf file and included as I did here. This keeps your httpd.conf file from getting too messy.
All matches will be logged to logs/audit_log for your due vigilence.