Difference between revisions of "Reverse Proxy with Squid"

From RSWiki
Jump to navigation Jump to search
Line 17: Line 17:
 
By default Squid is configured to listen on TCP port 3128. As we wish to use Squid as a web server we need to tell it to listen on port 80 instead. So the first line of our new squid.conf is as follows:
 
By default Squid is configured to listen on TCP port 3128. As we wish to use Squid as a web server we need to tell it to listen on port 80 instead. So the first line of our new squid.conf is as follows:
  
http_port 80 accel defaultsite=www.sweetnam.eu vhost
+
<syntaxhighlight lang="squid.conf">
forwarded_for on
+
http_port 80 accel defaultsite=www.sweetnam.eu vhost
 +
forwarded_for on
 +
</syntaxhighlight>
  
 
The default site to be served is www.sweetnam.eu and we will use vhost directives to configure the other servers. In addition, the forwarded_for on is required for logging the original client address to the back end server. See below.
 
The default site to be served is www.sweetnam.eu and we will use vhost directives to configure the other servers. In addition, the forwarded_for on is required for logging the original client address to the back end server. See below.
Line 24: Line 26:
 
The next lines in the configuration are merely Squids default:
 
The next lines in the configuration are merely Squids default:
  
refresh_pattern ^ftp:          1440    20%    10080
+
<syntaxhighlight lang="squid.conf">
refresh_pattern ^gopher:        1440    0%      1440
+
refresh_pattern ^ftp:          1440    20%    10080
refresh_pattern .              0      20%    4320
+
refresh_pattern ^gopher:        1440    0%      1440
 +
refresh_pattern .              0      20%    4320
 +
</syntaxhighlight>
  
 
The next lines are where we begin to configure the two seperate servers.
 
The next lines are where we begin to configure the two seperate servers.
Line 32: Line 36:
 
First we add a directive to tell Squid the IP address of the first server:
 
First we add a directive to tell Squid the IP address of the first server:
  
cache_peer 172.20.1.10 parent 80 0 no-query no-digest originserver name=apache login=PASS
+
<syntaxhighlight lang="squid.conf">
+
cache_peer 172.20.1.10 parent 80 0 no-query no-digest originserver name=apache login=PASS
 +
</syntaxhighlight>
 +
 
 
You can see from above that I have given a name of apache. All sites that will be hosted on this server will be under this group. Appending login=PASS at the end tells Squid to pass authentication on to the back end server. For example if you have directories password protected using .htaccess.
 
You can see from above that I have given a name of apache. All sites that will be hosted on this server will be under this group. Appending login=PASS at the end tells Squid to pass authentication on to the back end server. For example if you have directories password protected using .htaccess.
  
 
Next we must tell Squid the domains that will be served under apache:
 
Next we must tell Squid the domains that will be served under apache:
  
acl sites_apache dstdomain www.sample.com sample.com
+
<syntaxhighlight lang="squid.conf">
acl our_sites dstdomain www.sample.com sample.com
+
acl sites_apache dstdomain www.sample.com sample.com
cache_peer_access apache allow sites_apache
+
acl our_sites dstdomain www.sample.com sample.com
 +
cache_peer_access apache allow sites_apache
 +
</syntaxhighlight>
  
 
To add more domain names simply add them after the sample ones above.
 
To add more domain names simply add them after the sample ones above.
Line 46: Line 54:
 
Now we will configure the second server:
 
Now we will configure the second server:
  
cache_peer 172.20.1.4 parent 80 0 no-query no-digest originserver name=iis login=PASS
+
<syntaxhighlight lang="squid.conf">
acl sites_iis dstdomain www.sample2.com sample2.com
+
cache_peer 172.20.1.4 parent 80 0 no-query no-digest originserver name=iis login=PASS
acl our_sites2 dstdomain www.sample2.com sample2.com
+
acl sites_iis dstdomain www.sample2.com sample2.com
cache_peer_access iis allow sites_iis
+
acl our_sites2 dstdomain www.sample2.com sample2.com
 +
cache_peer_access iis allow sites_iis
 +
</syntaxhighlight>
  
 
Next we must ensure that Squids default acl rules are in place. For this I simply copied the defaults:
 
Next we must ensure that Squids default acl rules are in place. For this I simply copied the defaults:
  
acl all src 0.0.0.0/0.0.0.0
+
<syntaxhighlight lang="squid.conf">
acl manager proto cache_object
+
acl all src 0.0.0.0/0.0.0.0
acl localhost src 127.0.0.1/255.255.255.255
+
acl manager proto cache_object
acl to_localhost dst 127.0.0.0/8
+
acl localhost src 127.0.0.1/255.255.255.255
acl SSL_ports port 443
+
acl to_localhost dst 127.0.0.0/8
acl Safe_ports port 80          # http
+
acl SSL_ports port 443
acl Safe_ports port 21          # ftp
+
acl Safe_ports port 80          # http
acl Safe_ports port 443        # https
+
acl Safe_ports port 21          # ftp
acl Safe_ports port 70          # gopher
+
acl Safe_ports port 443        # https
acl Safe_ports port 210        # wais
+
acl Safe_ports port 70          # gopher
acl Safe_ports port 1025-65535  # unregistered ports
+
acl Safe_ports port 210        # wais
acl Safe_ports port 280        # http-mgmt
+
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 488        # gss-http
+
acl Safe_ports port 280        # http-mgmt
acl Safe_ports port 591        # filemaker
+
acl Safe_ports port 488        # gss-http
acl Safe_ports port 777        # multiling http
+
acl Safe_ports port 591        # filemaker
acl CONNECT method CONNECT
+
acl Safe_ports port 777        # multiling http
 +
acl CONNECT method CONNECT
 +
</syntaxhighlight>
  
 
Next we add two entries to allow the acls that we created for the backend servers:
 
Next we add two entries to allow the acls that we created for the backend servers:
  
http_access allow our_sites2
+
<syntaxhighlight lang="squid.conf">
http_access allow our_sites
+
http_access allow our_sites2
 +
http_access allow our_sites
 +
</syntaxhighlight>
  
 
And finally some more default configuration information:
 
And finally some more default configuration information:
  
http_access allow manager all
+
<syntaxhighlight lang="squid.conf">
http_access allow manager
+
http_access allow manager all
http_access deny !Safe_ports
+
http_access allow manager
http_access deny CONNECT !SSL_ports
+
http_access deny !Safe_ports
http_access deny all
+
http_access deny CONNECT !SSL_ports
+
http_access deny all
access_log /var/log/squid/access.log
 
cache_mgr someone@somewhere.com
 
  
 +
access_log /var/log/squid/access.log
 +
cache_mgr someone@somewhere.com
 +
</syntaxhighlight>
  
 
And that's it your reverse proxy should now be configured and is ready to start serving requests. So you may now start squid by entering the following at a console:
 
And that's it your reverse proxy should now be configured and is ready to start serving requests. So you may now start squid by entering the following at a console:
Line 102: Line 117:
 
To do this you you would use the following:
 
To do this you you would use the following:
  
acl sites_apache dstdomain "/etc/squid/apachesites"
+
<syntaxhighlight lang="squid.conf">
acl our_sites dstdomain "/etc/squid/apachesites"
+
acl sites_apache dstdomain "/etc/squid/apachesites"
cache_peer_access apache allow sites_apache
+
acl our_sites dstdomain "/etc/squid/apachesites"
 +
cache_peer_access apache allow sites_apache
 +
</syntaxhighlight>
  
 
You simply add eash domain to be directed to the back end server to the file specified.
 
You simply add eash domain to be directed to the back end server to the file specified.
  
 
== Complete squid.conf ==
 
== Complete squid.conf ==
 +
<syntaxhighlight lang="squid.conf">
 +
http_port 80 accel defaultsite=www.sweetnam.eu vhost
 +
forwarded_for on
 +
 +
refresh_pattern ^ftp:          1440    20%    10080
 +
refresh_pattern ^gopher:        1440    0%      1440
 +
refresh_pattern .              0      20%    4320
 +
 +
cache_peer 172.20.1.10 parent 80 0 no-query no-digest originserver name=apache login=PASS
 +
acl sites_apache dstdomain www.sample.com sample.com
 +
acl our_sites dstdomain www.sample.com sample.com
 +
cache_peer_access apache allow sites_apache
 +
 +
cache_peer 172.20.1.4 parent 80 0 no-query no-digest originserver name=iis login=PASS
 +
acl sites_iis dstdomain www.sample2.com sample2.com
 +
acl our_sites2 dstdomain www.sample2.com sample2.com
 +
cache_peer_access iis allow sites_iis
 +
 +
acl all src 0.0.0.0/0.0.0.0
 +
acl manager proto cache_object
 +
acl localhost src 127.0.0.1/255.255.255.255
 +
acl to_localhost dst 127.0.0.0/8
 +
acl SSL_ports port 443
 +
acl Safe_ports port 80          # http
 +
acl Safe_ports port 21          # ftp
 +
acl Safe_ports port 443        # https
 +
acl Safe_ports port 70          # gopher
 +
acl Safe_ports port 210        # wais
 +
acl Safe_ports port 1025-65535  # unregistered ports
 +
acl Safe_ports port 280        # http-mgmt
 +
acl Safe_ports port 488        # gss-http
 +
acl Safe_ports port 591        # filemaker
 +
acl Safe_ports port 777        # multiling http
 +
acl CONNECT method CONNECT
 +
 +
http_access allow our_sites2
 +
http_access allow our_sites
 +
 +
http_access allow manager all
 +
http_access allow manager
 +
http_access deny !Safe_ports
 +
http_access deny CONNECT !SSL_ports
 +
http_access deny all
  
http_port 80 accel defaultsite=www.sweetnam.eu vhost
+
access_log /var/log/squid/access.log
forwarded_for on
+
cache_mgr someone@somewhere.com
+
</syntaxhighlight>
refresh_pattern ^ftp:          1440    20%    10080
 
refresh_pattern ^gopher:        1440    0%      1440
 
refresh_pattern .              0      20%    4320
 
 
cache_peer 172.20.1.10 parent 80 0 no-query no-digest originserver name=apache login=PASS
 
acl sites_apache dstdomain www.sample.com sample.com
 
acl our_sites dstdomain www.sample.com sample.com
 
cache_peer_access apache allow sites_apache
 
 
cache_peer 172.20.1.4 parent 80 0 no-query no-digest originserver name=iis login=PASS
 
acl sites_iis dstdomain www.sample2.com sample2.com
 
acl our_sites2 dstdomain www.sample2.com sample2.com
 
cache_peer_access iis allow sites_iis
 
 
acl all src 0.0.0.0/0.0.0.0
 
acl manager proto cache_object
 
acl localhost src 127.0.0.1/255.255.255.255
 
acl to_localhost dst 127.0.0.0/8
 
acl SSL_ports port 443
 
acl Safe_ports port 80          # http
 
acl Safe_ports port 21          # ftp
 
acl Safe_ports port 443        # https
 
acl Safe_ports port 70          # gopher
 
acl Safe_ports port 210        # wais
 
acl Safe_ports port 1025-65535  # unregistered ports
 
acl Safe_ports port 280        # http-mgmt
 
acl Safe_ports port 488        # gss-http
 
acl Safe_ports port 591        # filemaker
 
acl Safe_ports port 777        # multiling http
 
acl CONNECT method CONNECT
 
 
http_access allow our_sites2
 
http_access allow our_sites
 
 
http_access allow manager all
 
http_access allow manager
 
http_access deny !Safe_ports
 
http_access deny CONNECT !SSL_ports
 
http_access deny all
 
 
access_log /var/log/squid/access.log
 
cache_mgr someone@somewhere.com
 
  
 
 
 
== Configuring Logging to the back end servers ==
 
== Configuring Logging to the back end servers ==
  

Revision as of 18:56, 5 April 2016

Template:AdWords

Caveat

While this HowTo was originally written based upon Ubuntu 7.10, the information below should still be valid and accurate. Certainly for the 2.7 releases of Squid - 12/04/11

Introduction

If like me your ISP provides you with a single static IP address you may think that you are limited to running one web server. Or at the very best using NAT to ports other than port 80 on other servers. There are many reasons why you would wish to use more than one webserver. For example you may wish to have Apache serving one site and Microsoft IIS 6.0 serving another or even JBoss, Tomcat or some other application server.

Background

For my scenario I wished to run Apache and PHP on one server and IIS on another. For this HowTo I will be configuring squid on Ubuntu 7.10. The location of your configuration files may depend on your installation. However for this howto I will be assuming that your configuration files are under /etc/squid and the cache itself is under /var/squid.

Editing squid.conf

The supplied squid.conf is over 4000 lines long. Most of this is documentation added in the comments. For my purposes I created a new squid.conf from scratch.

By default Squid is configured to listen on TCP port 3128. As we wish to use Squid as a web server we need to tell it to listen on port 80 instead. So the first line of our new squid.conf is as follows:

http_port 80 accel defaultsite=www.sweetnam.eu vhost
forwarded_for on

The default site to be served is www.sweetnam.eu and we will use vhost directives to configure the other servers. In addition, the forwarded_for on is required for logging the original client address to the back end server. See below.

The next lines in the configuration are merely Squids default:

refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern .               0       20%     4320

The next lines are where we begin to configure the two seperate servers.

First we add a directive to tell Squid the IP address of the first server:

cache_peer 172.20.1.10 parent 80 0 no-query no-digest originserver name=apache login=PASS

You can see from above that I have given a name of apache. All sites that will be hosted on this server will be under this group. Appending login=PASS at the end tells Squid to pass authentication on to the back end server. For example if you have directories password protected using .htaccess.

Next we must tell Squid the domains that will be served under apache:

acl sites_apache dstdomain www.sample.com sample.com
acl our_sites dstdomain www.sample.com sample.com
cache_peer_access apache allow sites_apache

To add more domain names simply add them after the sample ones above.

Now we will configure the second server:

cache_peer 172.20.1.4 parent 80 0 no-query no-digest originserver name=iis login=PASS
acl sites_iis dstdomain www.sample2.com sample2.com
acl our_sites2 dstdomain www.sample2.com sample2.com
cache_peer_access iis allow sites_iis

Next we must ensure that Squids default acl rules are in place. For this I simply copied the defaults:

acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

Next we add two entries to allow the acls that we created for the backend servers:

http_access allow our_sites2
http_access allow our_sites

And finally some more default configuration information:

http_access allow manager all
http_access allow manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny all

access_log /var/log/squid/access.log
cache_mgr someone@somewhere.com

And that's it your reverse proxy should now be configured and is ready to start serving requests. So you may now start squid by entering the following at a console:

/etc/init.d/squid start

Notes

All the above assumes that you have DNS configured so that each domain name that you have is pointing to the IP address of the reverse proxy.

The reverse proxy must be facing the internet. Either directly or by port forwarding.

If you intend on serving many domains such as I do you may find your squid.conf is getting rather ugly and difficult to manage. If this is the case then you can configure squid to lookup the domains to be served in a file instead.

To do this you you would use the following:

acl sites_apache dstdomain "/etc/squid/apachesites"
acl our_sites dstdomain "/etc/squid/apachesites"
cache_peer_access apache allow sites_apache

You simply add eash domain to be directed to the back end server to the file specified.

Complete squid.conf

http_port 80 accel defaultsite=www.sweetnam.eu vhost
forwarded_for on

refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern .               0       20%     4320

cache_peer 172.20.1.10 parent 80 0 no-query no-digest originserver name=apache login=PASS
acl sites_apache dstdomain www.sample.com sample.com
acl our_sites dstdomain www.sample.com sample.com
cache_peer_access apache allow sites_apache

cache_peer 172.20.1.4 parent 80 0 no-query no-digest originserver name=iis login=PASS
acl sites_iis dstdomain www.sample2.com sample2.com
acl our_sites2 dstdomain www.sample2.com sample2.com
cache_peer_access iis allow sites_iis

acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

http_access allow our_sites2
http_access allow our_sites

http_access allow manager all
http_access allow manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny all

access_log /var/log/squid/access.log
cache_mgr someone@somewhere.com

Configuring Logging to the back end servers

When using a reverse proxy you may notice that the logs on the back end servers will show hits as coming from the proxy rather then the client. There is a feature in Squid as noted earlier called forwarded_for on. This will pass the original clients IP address to the back end server. However you need to configure a custom log format on the back end server for logging to occur.

For Apache, I edited the httpd.conf and added a new line in the logging section called cached. This line as it appears in my configuration file looks like this:

LogFormat "%{X-Forwarded-For}i %v %u %t \"%r\" %>s %b \"%{Referer}i\"\"%{User-Agent}i\"" cached

Now you need to configure your default sites or virtual hosts so that they will use the new log format. Simply locate the entry for your host and change the CustomLog entry so that it looks like this:

CustomLog /home/blog/logs/access_log cached

Once you have configured Apache you need to restart the service.

Problems?

If you have any problems then leave a comment over at my blog entry on my reverse proxy


Extras

Squid contains some powerful features that can be used to further secure and limit access to the back end servers. For example limiting who can access your global sites or blocking access by certain useragents. I have another tutorial about Blocking Bots with Squid which you may find interesting.

Thanks and Credits

Many thanks are owed to Paolo Marchiori and John Moylan of the Irish Linux Users Group who helped me figure out how to pass configure the custom log format for Apache.

Template:AdWords2