These instructions absolutely assume that you are working with a Linux distribution that has yum support, which includes the popular CentOS and Red Hat Enterprise Linux.
These instructions also assume that you'll be deploying on a large server. The reference configuration files we provide assume a server with the equivalent of 32 CPU cores and 64 GB of RAM. If this seems outrageous you probably do not need to be installing Nginx! We only recommend this configuration for clients that will be serving billions of impressions per month.
Tip
Downloading and installing Nginx is as easy as running the following command:
yum install nginx
You may, however, need to download and install the Extra Packages for Enterprise Linux repository configuration (EPEL) if you get an error saying that Nginx is not available.
rpm -Uvh http://mirrors.kernel.org/fedora-epel/6/x86_64/epel-release-6-8.noarch.rpm
Of course this only provides a basic Nginx installation which is not quite what we need. Integrating the ey_balancer module with Nginx requires compiling a new Nginx binary from source. The source of the ey_balancer module also needs a minor patch to work with Nginx 1.0.x versions. To reduce errors in these steps, we strongly recommend that you download this already patched Nginx 1.0.14 bundle that we've prepared:
To be able to compile Nginx from source you are also going to need GCC, make and a couple of other required libraries for regex and SSL support, which can also be installed with yum:
yum install gcc
yum install make
yum install openssl-devel
yum install openssh-clients
yum install pcre-devel
Note
Downloading and installing MySQL is as easy as running the following command:
yum install mysql mysql-server
MariaDB is also supported provided that you use the MyISAM storage engine. In the future we may support the newer Aria storage engine. However, at the time of writing we do not recommend using Aria in a production environment as it has not been tested sufficiently.
Another option is TokuDB, which we highly recommend for customers with high-traffic deployments. In fact, you can use TokuDB with either MySQL or MariaDB. Please make sure that you are using AdvertPro 3.0 or newer versions, however, as previous versions are not fully optimized for the TokuDB storage engine.
Note
The following steps are necessary to compile and install Nginx with the ey_balancer module. We also include the http_ssl module for SSL support. All other modules that are not necessary for the operation of AdvertPro have been disabled. This results in a faster binary that is better hardened for security as well.
tar -xzvf nginx-1.0.14-eybalancer.tar.gz
cd nginx-1.0.14
./configure --user=nginx --group=nginx --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --with-http_ssl_module --without-http_auth_basic_module --without-http_autoindex_module --without-http_charset_module --without-http_empty_gif_module --without-http_fastcgi_module --without-http_geo_module --without-http_limit_req_module --without-http_limit_zone_module --without-http_map_module --without-http_memcached_module --without-http_referer_module --without-http_scgi_module --without-http_split_clients_module --without-http_ssi_module --without-http_userid_module --without-http_uwsgi_module --add-module=../ngx_max_connections-0.0.5 --with-file-aio --with-ipv6 --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' --with-ld-opt=-Wl,-E
make
cp objs/nginx /usr/sbin/nginx_eybalancer
rm -f /etc/nginx/conf.d/*.conf
rm -f /etc/nginx/fastcgi*
rm -f /etc/nginx/koi-*
rm -f /etc/nginx/mime.types.default
rm -f /etc/nginx/nginx.conf.default
rm -f /etc/nginx/scgi*
rm -f /etc/nginx/uwsgi*
rm -f /etc/nginx/win-utf
mkdir -p /var/lib/nginx/cache
After the compilation is done you will need to edit the /etc/rc.d/init.d/nginx startup file and replace the definitions of the nginx and prog variables with the following:
nginx="/usr/sbin/nginx_eybalancer"
prog="nginx"
This facilitates starting the custom nginx_eybalancer binary rather than the nginx binary that was installed from EPEL, which lacks the ey_balancer module.
To be safe, we also suggest that you disable new versions of Nginx from being installed automatically via yum update by editing the /etc/yum.repos.d/epel.repo file and appending the following line to it:
exclude=nginx,nginx*
This prevents newer versions of Nginx from being installed automatically and possibly changing the startup script back to the EPEL provided binary without your knowledge!
rpm -i jdk-{version}-linux-i586.rpm
rpm -i jdk-{version}-linux-x64.rpm
Note
mv apache-tomcat-{version}.tar.gz /usr/local
cd /usr/local
tar -xzvf apache-tomcat-{version}.tar.gz
mv apache-tomcat-{version} tomcat
rm -rf apache-tomcat-{version}.tar.gz
cd tomcat
chmod 755 bin/*.sh
cd webapps
rm -rf *
mkdir ROOT
yum install apr-devel
yum install zlib-devel
cd /usr/local/tomcat/bin
tar -xzf tomcat-native.tar.gz
cd tomcat-native*
cd jni/native
./configure --with-java-home=/usr/java/latest --with-apr=/usr/bin/apr-1-config --with-ssl=no
make
make install
unzip javamail-{version}.zip
cp javamail-{version}/mail.jar /usr/local/tomcat/lib/mail.jar
unzip jaf-{version}.zip
cp jaf-{version}/activation.jar /usr/local/tomcat/lib/activation.jar
tar -xzvf mysql-connector-java-{version}.tar.gz
cd mysql-connector-java-{version}
cp mysql-connector-java-{version}-bin.jar /usr/local/tomcat/lib/mysql.jar
cp advertpro-{version}.war /usr/local/tomcat/webapps/ROOT
cd /usr/local/tomcat/webapps/ROOT
/usr/java/latest/bin/jar -xf advertpro-{version}.war
rm -f advertpro-{version}.war
Edit the /etc/profile file and append the following to it to set the JAVA_HOME environment and add the various JDK utilities to your path:
export JAVA_HOME=/usr/java/latest
export PATH=$PATH:$JAVA_HOME/bin
Replace the entire contents of the /etc/my.cnf file with the following recommended configuration:
[client]
port = 3306
socket = /var/lib/mysql/mysql.sock
[mysqld]
skip-name-resolve
port = 3306
socket = /var/lib/mysql/mysql.sock
datadir = /var/lib/mysql
default-character-set=utf8
skip-external-locking
max_connections = 4096
key_buffer_size = 6144M
table_open_cache = 8192
thread_cache_size = 128
bulk_insert_buffer_size = 16M
join_buffer_size = 2M
read_buffer_size = 4M
read_rnd_buffer_size = 4M
sort_buffer_size = 16M
max_allowed_packet = 16M
max_heap_table_size = 768M
tmp_table_size = 768M
query_alloc_block_size = 16384
query_cache_limit = 2M
query_cache_size = 768M
query_cache_type = 1
query_prealloc_size = 16384
myisam_sort_buffer_size = 256M
open-files-limit=65536
server-id = 1
symbolic-links = 0
[mysql.server]
basedir = /var/lib
user = mysql
[mysqld_safe]
err-log = /var/log/mysqld.log
pid-file = /var/run/mysqld/mysqld.pid
[mysqldump]
max_allowed_packet = 16M
quick
[mysql]
default-character-set=utf8
no-auto-rehash
[isamchk]
key_buffer = 256M
read_buffer = 2M
write_buffer = 2M
sort_buffer = 256M
[myisamchk]
key_buffer = 256M
read_buffer = 2M
write_buffer = 2M
sort_buffer = 256M
[mysqlhotcopy]
interactive-timeout
Note
Replace the entire contents of the /etc/nginx/nginx.conf file with the following recommended configuration:
#---------------------------------------------------------------------
# Runtime Settings
#---------------------------------------------------------------------
user nginx;
pid /var/run/nginx.pid;
error_log /var/log/nginx/error.log crit;
#---------------------------------------------------------------------
# Performance Settings
#---------------------------------------------------------------------
worker_processes 32; # change this to match number of CPU cores
worker_rlimit_nofile 32768;
events {
worker_connections 8192; # (32*(8192/4)) = 65,536 max clients
}
#---------------------------------------------------------------------
# HTTP Settings
#---------------------------------------------------------------------
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
access_log off;
gzip on;
gzip_comp_level 6;
gzip_http_version 1.1;
gzip_min_length 0;
gzip_types application/json application/x-javascript text/plain text/css text/javascript text/xml;
gzip_vary on;
sendfile on;
tcp_nopush on;
keepalive_timeout 60;
proxy_cache_path /var/lib/nginx/cache levels=1:1:2 keys_zone=shared:100m inactive=60m max_size=2500m;
proxy_cache_use_stale updating;
server_tokens off;
upstream tomcat {
server 127.0.0.1:8009;
max_connections 256; # set to (tomcat_max_threads / nginx_worker_processes)
max_connections_queue_timeout 30000;
}
server {
listen 80;
server_name _;
location / {
proxy_pass http://tomcat;
proxy_connect_timeout 30;
proxy_set_header Host $http_host;
proxy_set_header Accept-Encoding "";
proxy_set_header User-Agent $http_user_agent;
proxy_set_header Referer $http_referer;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $http_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_redirect http://$http_host $scheme://$http_host;
}
location /servlet/files {
proxy_pass http://tomcat;
proxy_set_header Host $http_host;
proxy_cache shared;
proxy_cache_key "$scheme$http_host$request_uri";
}
location ~* \.(css|html|js|txt|xml)$ {
proxy_pass http://tomcat;
proxy_set_header Host $http_host;
proxy_set_header Accept-Encoding "";
proxy_cache shared;
proxy_cache_key "$scheme$http_accept_encoding$http_host$request_uri";
}
location ~* \.(gif|jpg|png|jar)$ {
proxy_pass http://tomcat;
proxy_cache shared;
proxy_cache_key "$scheme$http_host$request_uri";
}
}
}
Note
Edit /usr/local/tomcat/bin/catalina.sh and add the following after the comments/instructions at the top.
# Tomcat Home.
CATALINA_HOME="/usr/local/tomcat"
# Java Home.
JAVA_HOME="/usr/java/latest"
#
# Change -Xms and -Xms if necessary to allocate more or less memory.
# It is important that they have the same value for optimal performance.
# Note that on some operating systems you may need to increase the value of
# -Xss to 512k or higher (do not exceed 1024k) if you experience stability
# problems or start seeing stack overflow exceptions in your application
# server logs.
#
# If you've changed -Xms and -Xms, you should change -XX:NewSize and
# -XX:MaxNewSize to have values equal to half of the -Xms and -Xmx
# values.
#
# Suggested values based on available physical server memory (RAM):
#
# 2 GB RAM -Xms1024m -Xmx1024m -XX:NewSize=512m -XX:MaxNewSize=512m
# 4 GB RAM -Xms2048m -Xmx2048m -XX:NewSize=1024m -XX:MaxNewSize=1024m
# 8 GB RAM -Xms4096m -Xmx4096m -XX:NewSize=2048m -XX:MaxNewSize=2048m
# 16 GB RAM -Xms8192m -Xmx8192m -XX:NewSize=4096m -XX:MaxNewSize=4096m
# 32 GB RAM -Xms16384m -Xmx16384m -XX:NewSize=8192m -XX:MaxNewSize=8192m
# 64 GB RAM -Xms32768m -Xmx32768m -XX:NewSize=16384m -XX:MaxNewSize=16384m
#
CATALINA_OPTS="$CATALINA_OPTS -server -Xms32768m -Xmx32768m -Xss256k -XX:+DisableExplicitGC"
CATALINA_OPTS="$CATALINA_OPTS -XX:NewSize=16384m -XX:MaxNewSize=16384m"
#
# If you're using a multi processor system, uncomment the following
# options and set the value of -XX:ParallelGCThreads equal to the
# number of processors that are in your system.
#
#CATALINA_OPTS="$CATALINA_OPTS -XX:+UseParNewGC -XX:ParallelGCThreads=32"
#CATALINA_OPTS="$CATALINA_OPTS -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=80"
#CATALINA_OPTS="$CATALINA_OPTS -XX:SurvivorRatio=128 -XX:MaxTenuringThreshold=0"
#
# APR Connector JNI Library
#
CATALINA_OPTS="$CATALINA_OPTS -Djava.library.path=/usr/local/apr/lib"
#
# Uncomment the following options to help debug garbage collection
# related performance problems.
#
#CATALINA_OPTS="$CATALINA_OPTS -verbose:gc -Xloggc:/usr/local/tomcat/logs/gc.txt -XX:+PrintGCDetails"
#
# Java Options.
#
JAVA_OPTS="$JAVA_OPTS -Djava.awt.headless=true -Dsun.net.inetaddr.ttl=28800 -Dsun.net.inetaddr.negative.ttl=60"
#
# Raise open file limit and stack size.
#
ulimit -n 524288
ulimit -s 2048
Edit the /usr/local/tomcat/conf/logging.properties file and change the following log handlers from INFO to SEVERE level:
1catalina.org.apache.juli.FileHandler.level = SEVERE
2localhost.org.apache.juli.FileHandler.level = SEVERE
3manager.org.apache.juli.FileHandler.level = SEVERE
4host-manager.org.apache.juli.FileHandler.level = SEVERE
java.util.logging.ConsoleHandler.level = SEVERE
Replace the entire contents of the /usr/local/tomcat/conf/server.xml with the following recommended configuration:
<?xml version="1.0" encoding="utf-8"?>
<Server port="8005" shutdown="SHUTDOWN">
<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="off" />
<Listener className="org.apache.catalina.core.JasperListener" />
<Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
<GlobalNamingResources>
<Resource name="UserDatabase" auth="Container"
type="org.apache.catalina.UserDatabase"
description="User database that can be updated and saved"
factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
pathname="conf/tomcat-users.xml" />
</GlobalNamingResources>
<Service name="Catalina">
<Connector protocol="org.apache.coyote.http11.Http11AprProtocol"
acceptCount="128"
connectionTimeout="15000"
disableUploadTimeout="false"
enableLookups="false"
maxKeepAliveRequests="100"
keepAliveTimeout="30000"
maxThreads="8192"
port="8009"
pollTime="2000"
pollerSize="32768"
useSendfile="true"
sendfileSize="1024"
URIEncoding="UTF-8"/>
<Engine defaultHost="ads.yoursite.com" name="Catalina">
<Host name="ads.yoursite.com" appBase="webapps" unpackWARs="true">
<Context docBase="ROOT" path="" privileged="false" swallowOutput="true" />
</Host>
<Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase" />
</Engine>
</Service>
</Server>
Note
Create /etc/rc.d/init.d/tomcat using the following recommended settings.
#!/bin/bash
#
# Startup script for the Tomcat Web Server
#
# chkconfig: 345 92 16
# description: Tomcat is a World Wide Web server. It is used to serve \
# HTML, JSP, and servlets, and CGI if needed.
# processname: java
CATALINA_HOME=/usr/local/tomcat
case "$1" in
start)
rm -rf $CATALINA_HOME/work
mkdir $CATALINA_HOME/work
$CATALINA_HOME/bin/startup.sh
;;
restart)
$CATALINA_HOME/bin/shutdown.sh
sleep 45
killall -9 java
sleep 2
rm -rf $CATALINA_HOME/work
mkdir $CATALINA_HOME/work
$CATALINA_HOME/bin/startup.sh
;;
stop)
$CATALINA_HOME/bin/shutdown.sh
sleep 45
killall -9 java
sleep 2
;;
*)
echo $"Usage: tomcat {start|restart|stop}"
exit 1
esac
exit 0
chmod 755 /etc/rc.d/init.d/mysqld
/sbin/chkconfig --add mysqld
/sbin/chkconfig --level 2345 mysqld on
/sbin/chkconfig --list mysqld
chmod 755 /etc/rc.d/init.d/nginx
/sbin/chkconfig --add nginx
/sbin/chkconfig --level 2345 nginx on
/sbin/chkconfig --list nginx
chmod 755 /etc/rc.d/init.d/tomcat
/sbin/chkconfig --add tomcat
/sbin/chkconfig --level 345 tomcat on
/sbin/chkconfig --list tomcat
AdvertPRO needs the following ports to be open in your firewall in order to function properly:
| Port Number | Protocol | Allow Input | Allow Output | Client/Source Address | Destination Address | Used By |
|---|---|---|---|---|---|---|
| 25 | TCP | Yes | Yes | Any | Any | SMTP |
| 80 | TCP | Yes | Yes | Any | Any | HTTP |
| 443 | TCP | Yes | Yes | Any | Any | HTTPS |
| 3306 | TCP | Yes | Yes | localhost | localhost | MySQL |
| 9000 | TCP | No | Yes | localhost | websvc1.advertpro.com websvc2.advertpro.com | AdvertPRO |
| 9001 | TCP | No | Yes | localhost | websvc1.advertpro.com websvc2.advertpro.com | AdvertPRO |
| 9002 | TCP | No | Yes | localhost | websvc1.advertpro.com websvc2.advertpro.com | AdvertPRO |
| 9003 | TCP | No | Yes | localhost | websvc1.advertpro.com websvc2.advertpro.com | AdvertPRO |
| 9004 | TCP | No | Yes | localhost | websvc1.advertpro.com websvc2.advertpro.com | AdvertPRO |
| 9005 | TCP | No | Yes | localhost | websvc1.advertpro.com websvc2.advertpro.com | AdvertPRO |
Failure to open the aforementioned ports in your firewall will cause the AdvertPRO setup utility to abort.
/etc/rc.d/init.d/mysql start
/etc/rc.d/init.d/tomcat start
/etc/rc.d/init.d/nginx start
Log in to the MySQL client of your choosing and execute the following SQL queries to create the AdvertPRO database and user to access it.
CREATE DATABASE advertpro;
GRANT ALL PRIVILEGES ON advertpro.* TO advertpro@"localhost" IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON advertpro.* TO advertpro@"localhost.localdomain" IDENTIFIED BY 'password';
FLUSH PRIVILEGES;
Warning
Keep in mind that the database will be created using the default MySQL character set, which is specified by the character_set_system option in the my.cnf configuration file. For older versions of MySQL the option is named character_set instead. By default MySQL uses the Latin1 character set, so if you wish to use UTF-8 to support non-Latin1 characters create the MySQL database using this command unless you already changed the default MySQL character set to UTF-8:
CREATE DATABASE advertpro CHARACTER SET utf8 COLLATE utf8_general_ci;
When the AdvertPRO setup utility asks you for the JDBC URL you should changed it to specify that you want to use UTF-8 unless again you already changed the default MySQL character set to UTF-8 as the driver will always use the default MySQL character set regardless of the character set of your database:
jdbc:mysql://localhost:3306/advertpro?useUnicode=true&characterEncoding=UTF-8
If you have purchased the GeoIP Country or GeoIP City database, you should install that prior to running the AdvertPRO setup utility.
There are also free versions of GeoIP Country and City that you can download for evaluation purposes, but you should bear in mind that they are not as accurate as the paid versions!
Note