Getting real IP addresses using Nginx and CloudFlare

[Update]: Thanks to digitaltoast for informing me about the missing ~~real_ip_header CF-Connecting-IP;~~ from the script and providing a patch for it.

OK, I suppose you know what CloudFlare is, and are familiar with Nginx configuration process, before we proceed any further. Just in case you don’t know, CloudFlare offers free and commercial, cloud-based services to help secure and accelerate websites. The thing is, I’m really satisfied with the services they offer except a repellent issue about logging the real IP address of your website’s visitors. Since CloudFlare acts as a reverse proxy, all connections come from CloudFlare’s IP addresses, not the real visitors anymore. Anyway, using Nginx there’s a simple workaround for this issue, which I’ll describe in the rest of this post.

HttpRealipModule

Well, to achieve over goal we have to first build Nginx with HttpRealipModule support. I’ll describe three methods to build Nginx with RealIP module support on FreeBSD, Funtoo or Gentoo GNU/Linux variants and any other distros.

Building from Ports on FreeBSD

In fact, building and installing Nginx from Ports is very easy and straightforward. Just issue the following commands:

$ cd /usr/ports/www/nginx
$ make config

You have to find and choose HTTP_REALIP by hitting Spacebar on your keyboard in the opening configuration window:

[*] HTTP_REALIP           Enable http_realip module

By issuing the following command Nginx will be built and installed by Ports.

$ make install clean

Building from Portage on Funtoo / Gentoo

To build and install Nginx with RealIP support from Portage on Gentoo or Funtoo we have to first touch the make.conf file with the following content. Just add the realip keyword to NGINX_MODULES_HTTP:

/etc/make.conf

1
2


# nginx
NGINX_MODULES_HTTP="some other modules ... realip"

Start the building and installation process by using the emerge command:

$ emerge -avt www-servers/nginx

You should see the realip keyword without “-” sign in the NGINX_MODULES_HTTP on the emerge command output:

Calculating dependencies... done!
[ebuild   R    ] www-servers/nginx-1.x.x  USE="some use flags -some -disbaled -use -flags" NGIN
X_MODULES_HTTP="some other modules ... realip ... -some -disbaled -modules" NGINX_MODULES_MAIL=
"some modules -some -disbaled -modules"

Building from Source

By following the instructions given below, it is easy to build Nginx from source with RealIP module support on any other distro. Because the RealIP module won’t build by default, you need to enable it with the configuration option with-http_realip_module.

$ wget http://nginx.org/download/nginx-1.x.x.tar.gz
$ tar xvzf nginx-1.x.x.tar.gz
$ cd nginx-1.x.x
$ ./configure --with-http_realip_module
$ make
$ make install

Nginx Configuration

After building Nginx with RealIP support enabled, now it’s time to configure our vhosts in order to enable real visitors IP logging. I assume you’ve configured your vhosts and Nginx installation already.

You need to first fetch CloudFlare’s IPv4 and IPv6 IP Ranges. You should consider, this list changes from time to time. So, it’s vital to fetch and synchronize them with your own list on regular basis. However, it’s possible to write an automated update script to accomplish this task. Therefore, I’ll provide one at the end of the article.

Let’s say we’ve chosen /usr/local/www/nginx as Ngix default host root. So, we put each vhost in its own directory at /usr/local/www. And, also put the vhost configurations inside the /usr/local/www/_vhosts directory. In addition to that, we create an _include directory at /usr/local/www for sharing CloudFlare settings among our vhosts.

$ mkdir -p /usr/local/www/_include/

Anyway, our CloudFlare configuration file at /usr/local/www/_include/cloudflare looks like this:

/usr/local/www/_include/cloudflare

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


# CloudFlare
# https://www.cloudflare.com/ips-v4
# https://www.cloudflare.com/ips-v6
set_real_ip_from 204.93.240.0/24;
set_real_ip_from 204.93.177.0/24;
set_real_ip_from 199.27.128.0/21;
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 103.21.244.0/22;
set_real_ip_from 103.22.200.0/22;
set_real_ip_from 103.31.4.0/22;
set_real_ip_from 141.101.64.0/18;
set_real_ip_from 108.162.192.0/18;
set_real_ip_from 190.93.240.0/20;
set_real_ip_from 188.114.96.0/20;
set_real_ip_from 197.234.240.0/22;
set_real_ip_from 198.41.128.0/17;
set_real_ip_from 2400:cb00::/32;
set_real_ip_from 2606:4700::/32;
set_real_ip_from 2803:f800::/32;
set_real_ip_from 2405:b500::/32;
set_real_ip_from 2405:8100::/32;
real_ip_header CF-Connecting-IP;

Finally, you can enable RealIP for each vhost by including the /usr/local/www/_include/cloudflare file:

/usr/local/www/_vhosts/example.com

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


server {
    server_name  example.com;
    rewrite ^(.*) http://www.example.com$1 permanent;

    # CloudFlare
    include /usr/local/www/_include/cloudflare;
}

server {
    listen  80;
    server_name   www.example.com;

    root           /usr/local/www/example.com/www/;

    # CloudFlare
    include /usr/local/www/_include/cloudflare;
}

server {
    listen         80;
    server_name    sub.example.com;
 
    root           /usr/local/www/example.com/www-sub/;

    # CloudFlare
    include /usr/local/www/_include/cloudflare;
}

Automated CloudFlare IP Ranges Update

You have to first create the /usr/local/www/_cron directory:

$ mkdir -p /usr/local/www/_cron

Then, put the cloudflare-ip-ranges-updater.sh file with the provided contents inside the /usr/local/www/_cron directory. The major prerequisites for this script to run correctly are AWK and FreeBSD fetch or GNU Wget. Also, you may need to change the shebang line or script variables such as CLOUDFLARE_IP_RANGES_FILE_PATH, WWW_GROUP and WWW_USER according to your OS requirements.

/usr/local/www/_cron/cloudflare-ip-ranges-updater.sh

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55


#!/bin/sh

#  (The MIT License)
#
#  Copyright (c) 2013 Mamadou Babaei
#
#  Permission is hereby granted, free of charge, to any person obtaining a copy
#  of this software and associated documentation files (the "Software"), to deal
#  in the Software without restriction, including without limitation the rights
#  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
#  copies of the Software, and to permit persons to whom the Software is
#  furnished to do so, subject to the following conditions:
#
#  The above copyright notice and this permission notice shall be included in
#  all copies or substantial portions of the Software.
#
#  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
#  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
#  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
#  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
#  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
#  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
#  THE SOFTWARE.


CLOUDFLARE_IP_RANGES_FILE_PATH="/usr/local/www/_include/cloudflare"
WWW_GROUP="www"
WWW_USER="www"

CLOUDFLARE_IPSV4_REMOTE_FILE="https://www.cloudflare.com/ips-v4"
CLOUDFLARE_IPSV6_REMOTE_FILE="https://www.cloudflare.com/ips-v6"
CLOUDFLARE_IPSV4_LOCAL_FILE="/var/tmp/cloudflare-ips-v4"
CLOUDFLARE_IPSV6_LOCAL_FILE="/var/tmp/cloudflare-ips-v6"

if [ -f /usr/bin/fetch ];
then
    fetch $CLOUDFLARE_IPSV4_REMOTE_FILE --no-verify-hostname --no-verify-peer -o $CLOUDFLARE_IPSV4_LOCAL_FILE --quiet
    fetch $CLOUDFLARE_IPSV6_REMOTE_FILE --no-verify-hostname --no-verify-peer -o $CLOUDFLARE_IPSV6_LOCAL_FILE --quiet
else
    wget -q $CLOUDFLARE_IPSV4_REMOTE_FILE -O $CLOUDFLARE_IPSV4_LOCAL_FILE --no-check-certificate
    wget -q $CLOUDFLARE_IPSV6_REMOTE_FILE -O $CLOUDFLARE_IPSV6_LOCAL_FILE --no-check-certificate
fi

echo "# CloudFlare IP Ranges" > $CLOUDFLARE_IP_RANGES_FILE_PATH
echo "# Generated at $(date) by $0" >> $CLOUDFLARE_IP_RANGES_FILE_PATH
echo "" >> $CLOUDFLARE_IP_RANGES_FILE_PATH
awk '{ print "set_real_ip_from " $0 ";" }' $CLOUDFLARE_IPSV4_LOCAL_FILE >> $CLOUDFLARE_IP_RANGES_FILE_PATH
awk '{ print "set_real_ip_from " $0 ";" }' $CLOUDFLARE_IPSV6_LOCAL_FILE >> $CLOUDFLARE_IP_RANGES_FILE_PATH
echo "real_ip_header CF-Connecting-IP;" >> $CLOUDFLARE_IP_RANGES_FILE_PATH
echo "" >> $CLOUDFLARE_IP_RANGES_FILE_PATH

chown $WWW_USER:$WWW_GROUP $CLOUDFLARE_IP_RANGES_FILE_PATH

rm -rf $CLOUDFLARE_IPSV4_LOCAL_FILE
rm -rf $CLOUDFLARE_IPSV6_LOCAL_FILE

After that, set the script file’s permissions to executable by all users:

$ chmod a+x /usr/local/www/_cron/cloudflare-ip-ranges-updater.sh

Now, you may want to test the script by issuing the following set of commands:

$ /usr/local/www/_cron/cloudflare-ip-ranges-updater.sh
$ cat /usr/local/www/_include/cloudflare

The output must be something like this:

/usr/local/www/_include/cloudflare

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


# CloudFlare IP Ranges
# Generated at Sat Mar  9 00:15:58 UTC 2013 by /usr/local/www/_cron/cloudflare-ip-ranges-updater.sh

set_real_ip_from 204.93.240.0/24;
set_real_ip_from 204.93.177.0/24;
set_real_ip_from 199.27.128.0/21;
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 103.21.244.0/22;
set_real_ip_from 103.22.200.0/22;
set_real_ip_from 103.31.4.0/22;
set_real_ip_from 141.101.64.0/18;
set_real_ip_from 108.162.192.0/18;
set_real_ip_from 190.93.240.0/20;
set_real_ip_from 188.114.96.0/20;
set_real_ip_from 197.234.240.0/22;
set_real_ip_from 198.41.128.0/17;
set_real_ip_from 2400:cb00::/32;
set_real_ip_from 2606:4700::/32;
set_real_ip_from 2803:f800::/32;
set_real_ip_from 2405:b500::/32;
set_real_ip_from 2405:8100::/32;
real_ip_header CF-Connecting-IP;

If you want to update the CloudFlare IP Ranges every 24:00 hours as root user you should add the following at the end of your system’s crontab file:

$ sudo crontab -e -u root

sudo crontab -e -u root

1
2
3


# CloudFlare IP Ranges Automatic Update
# Every 24:00 hours at 04:00am UTC
00      04      *       *       *       root    /usr/local/www/_cron/cloudflare-ip-ranges-updater.sh >/dev/null 2>&1

It’s also possible to set time intervals weekly or several times a month, a day or even hours:

'sudo crontab -e -u root' other time interval samples

1
2
3
4
5


# Every 30 minutes
*/30     *       *       *       *       root    /usr/local/www/_cron/cloudflare-ip-ranges-updater.sh >/dev/null 2>&1
# or
# Every 7 days at 02:30am UTC
30      02      1,8,15,22,28    *       root    /usr/local/www/_cron/cloudflare-ip-ranges-updater.sh >/dev/null 2>&1

In order to verify your crontab entry:

$ sudo crontab -l -u root

Finally, if necessary restart the cron service:

$ service cron restart

Source Code

Check out the source code on GitLab

Check out the source code on GitHub

AWK Bash CloudFlare Cron csh FLOSS FOSS FreeBSD Funtoo Gentoo GNU Linux Nginx sh Shell Scripting Unix