NGINX reverse proxy in front of Apache: hardened, high performance dedicated server setup

From DP

Revision as of 10:02, 14 January 2012 by Alexlwa (Talk | contribs)
Jump to: navigation , search

Contents

Overview

NGINX reverse proxy in front of Apache-HTOP screenshot

Apache is a reliable HTTP server that still holds more than 66%, according to W3Techs, of the web server market, but Apache was not designed with performance or scalability in mind.

When the traffic is too high, it generate a lot of Apache processes and consume too much memory and generate many high CPU processes.This situation cause server to crash.

You can speed up your current HTTP server by installing a reverse proxy server in front of it. A reverse proxy fetches resources from one or more servers and returns them to the client as if they originated from the proxy server itself.

We will use web server Nginx, Apache serve all dynamic content and Nginx handle all static files without consuming lots of system resources, combining the benefits of both servers.

Hardware

Dedicated Server EX 4 http://www.hetzner.de/en/hosting/produkte_rootserver/ex4

  • Intel® Core™ i7-2600 Quadcore
  • RAM 16 GB DDR3 RAM
  • Hard disks 2 x 3 TB SATA 6 Gb/s HDD 7200 rpm
  • NIC1 GBit OnBoard connected at 100 MBit
  • Traffic 10TB/monthly

Operating System:

  • Debian-60-squeeze-64-minimal (Hetzner image)


Basic server setup - LAMP

Debian OS - upgrade to latest packages

    # apt-get update 
    # apt-get upgrade


Packages installation

Apache
    # apt-get install apache2
    # a2enmod rewrite
    # /etc/init.d/apache2 restart


configuration: 

    # nano /etc/apache2/sites-enabled/000-default
      (default webroot directory: /var/www/)


check configuration: 

    # apachectl -t


After enabling, disabling, or modifying any part of your Apache configuration, you will need to reload or restart the Apache configuration again with command: 

    # /etc/init.d/apache2 reload or 
    # /etc/init.d/apache2 restart



PHP
    # apt-get install php5 php-pear php5-suhosin php5-mysql


configuration: edit /etc/php5/apache2/php.ini Make sure that the following values are set, and relevant lines are uncommented (comments are lines beginning with a semi-colon (;)): 

max_execution_time = 60
memory_limit = 128M
error_reporting = E_COMPILE_ERROR|E_RECOVERABLE_ERROR|E_ERROR|E_CORE_ERROR
display_errors = Off
log_errors = On
error_log = /var/log/php5.log
register_globals = Off


To apply PHP configuration changes Apache need to be restarted: 

    # /etc/init.d/apache2 restart



MySQL
    # apt-get install mysql-server

During the installation, you will be prompted for a password. Choose something secure and record it for future reference. At this point, MySQL should be ready to configure and run. While you shouldn't need to change the configuration file, note that it is located at /etc/mysql/my.cnf for future reference.



ProFTPD
    # apt-get install proftpd

select: standalone mode


configuration: 

    # nano /etc/proftpd/proftpd.conf


check configuration: 

    # proftpd -t


After modifying any part of your ProFTPD configuration, you will need to restart the ProFTPD service: 

    # /etc/init.d/proftpd restart



Postfix
    # apt-get install postfix

select: internet site


configuration: 

    # nano /etc/postfix/main.cf  
    # nano /etc/postfix/master.cf


After modifying any part of your Postfix configuration, you will need to restart service: 

    # /etc/init.d/postfix restart



List of all installed packages for new installation

    # dpkg --get-selections > all-installed-software.log
    # dpkg --set-selections < all-installed-software.log 
    # dselect (select i -install)


References:

Advanced server setup - NGINX

Installation

Using http://www.dotdeb.org/instructions/ repo for Nginx 1.0.11 last stable For the main Dotdeb repository add these two lines to: /etc/apt/sources.list file 

    # deb http://packages.dotdeb.org stable all
    # deb-src http://packages.dotdeb.org stable all


Then fetch the appropriate GnuPG key 

    # wget http://www.dotdeb.org/dotdeb.gpg
    # cat dotdeb.gpg | sudo apt-key add -



    # apt-get update    
    # apt-get install nginx

Nginx configuration

Stop the Nginx server if it was started automatically by the package manager and create a new nginx.conf configuration file – installed in /etc/nginx/ by default – by pasting the following and adjusting the paths to those of your installation: 

user www-data; #change to the same user apache runs as
worker_processes 8; #change to the number of your CPUs/Cores
worker_rlimit_nofile 8192;

error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;

events {
  worker_connections 1024;
  use epoll;
  accept_mutex off;
}

http {
  server_names_hash_bucket_size 64;
  include /etc/nginx/mime.types;
  default_type application/octet-stream;
  access_log /var/log/nginx/access.log;
  sendfile on;
  tcp_nopush on;
  keepalive_timeout 65;

  # reverse proxy options
  proxy_redirect off;
  proxy_set_header Host $host;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

  # gzip compression options
  gzip on;
  gzip_http_version 1.0;
  gzip_comp_level 6;
  gzip_min_length 0;
  gzip_buffers 16 8k;
  gzip_proxied any;
  gzip_types text/plain text/css text/xml text/javascript application/xml application/xml+rss application/javascript application/json;
  gzip_disable "MSIE [1-6]\.";
  gzip_vary on;

  # include virtual hosts configuration
  include /etc/nginx/virtual.d/*.conf;
}


Nginx should run as the same user Apache runs, to avoid file permission problems. Replace apache with the Apache user of your setup, for www-data in default Debian and Ubuntu installations.

Besides the proxy setup this configuration file includes some generic performance tuning, such as use epoll as the event model method, which works effectively on Linux 2.6+ kernels. This works in tandem with the next line, accept_mutex off, to improve performance a bit more. Enabling sendfile allows nginx to use the kernel’s sendfile support to send files to the client regardless of their contents. This can help with large static files, such as images, that have no need for a multiple request/confirmation system to be served. Enabling gzip compression for static files can make a big performance difference. The lines starting with gzip enable compression for common web files, such as .css and .js files, on supported browsers. You can find more information about these options, as well as the complete documentation for nginx, on the project’s wiki - http://wiki.nginx.org/

Apache reverse proxy forward module(mod_rpaf)

If you check the Apache access log files you should see that all incoming requests are coming from 127.0.0.1. To fix this you need to install mod_rpaf, the reverse proxy add forward module for Apache. 

    # apt-get install libapache2-mod-rpaf


check content of /etc/apache2/mods-enabled/rpaf.conf : 

<IfModule mod_rpaf.c>
RPAFenable On
RPAFsethostname On
RPAFproxy_ips 127.0.0.1
</IfModule>


restart apache: 

    # /etc/init.d/apache2 restart

Apache configuration

Nginx now acts as the front-end web server – waiting for requests on port 80 – you need to configure Apache to listen on a different port (8080 in this case) and preferably only on localhost, open the file /etc/apache2/ports.conf and change the line Listen 80 to Listen 127.0.0.1:8080, if you use name-based virtual hosts you should have a line NameVirtualHost *:80 in the same file. Change that to NameVirtualHost *:8080.

If you have configured Keep-Alive support in Apache you should disable it since it is already enabled in Nginx. Change KeepAlive On to KeepAlive Off in /etc/apache2/apache2.conf . You can also disable the mod_deflate module since Nginx already provides gzip compression.

WordPress, MediaWiki, phpBB configurations

WordPress

http://wordpress.org/extend/plugins/nginx-proxy-cache-integrator/

MediaWiki

phpBB

Hardening

sysctl.conf security hardening

    # 
    #

/tmp, /var/tmp directory hardening

RootKit hunter

CSF firewall & LFD

Anti-DOS configuration
Brute force detection and prevention
Port scan detection and prevention
Securing SSH server
Root logger

Log analysis

FTP hardening

Apache mod_security

Monitoring

AWS EC2

    # 
    #
Nagios
Munin

Backups!!!

Second HDD synchronization

Database dump

Offsite encrypted backups - Jungle Disk

Offsite encrypted backups on AWS EC3 or Rackspace Cloud Files

Troubleshooting

References

Retrieved from "http://dangerousprototypes.com/docs/NGINX_reverse_proxy_in_front_of_Apache:_hardened,_high_performance_dedicated_server_setup"