Personal tools
How to cache Windows Updates using Untangle
From UntangleWiki
WARNING: this is a user contributed how-to and has not been tested or verified by Untangle. These modifications may cause immediate or future problems on your Untangle installation.
There are quite a few guides in the forums describing how to install squid, a fully featured web proxy and caching tool. Squid suffers from being fairly complex to configure and being overkill for a simple problem: how to stop every single computer in your small business or home from separately downloading a full set of updates.
This is particularly important if you, like many techie type people, are regularly called upon to maintain the computers of other people. Usually these tend to be virus ridden and severely lacking in patches and updates and having to wait many hours just to get it up to date with Microsoft's updates alone is painful. What you really want is to connect to your internal network and have all the updates delivered from a local cache at LAN speed.
If like most people you installed Untangle on a reasonable PC, you have a large hard drive sitting there doing nothing. This guide shows you how to install a simple personal web cache and to configure it so it solely caches updates.
If you have questions or problems, you may find this forum thread useful.
Contents |
Before Anything Else
If you have not done so before, before doing absolutely anything else you need to enable the Debian repositories. Open /etc/apt/sources.list using nano or whatever your favourite editor is and uncomment the four items in there. Run 'apt-get clean' and then 'apt-get update'.
The Guide
We shall be using a personal web caching proxy called Polipo which requires very little configuration. To install:
apt-get install polipo nano /etc/polipo/config
In here add the following (I assume a standard Untangle configuration):
proxyAddress = "192.168.2.1" allowedClients = 127.0.0.1, 192.168.2.0/24 disableVia = false relaxTransparency = maybe dontCacheRedirects = false diskCacheUnlinkTime = 10000d diskCacheTruncateTime = 10000d diskCacheTruncateSize = 1073741824
This allows anyone on the internal network to access Polipo but excludes external access - which you definitely want unless you want to run an open proxy. It also turns on the "Via:" HTTP header so you can see when Polipo is working or not, and it relaxes when it will serve stale content e.g. with this setting it will still serve if your internet connection is down.
The disk cache settings basically disable flushing or truncating cache data. This we want because we're caching multi-megabyte update files.
Next thing we must fix a bug in the Debian distro (http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=519907):
nano /usr/lib/polipo/polipo-control
Find the following and modify as below:
polipo_start() {
# DAEMON_OPTS="$DAEMON_OPTS proxyOffline=`offlineMode`"
DAEMON_OPTS="$DAEMON_OPTS "
After these changes:
/etc/init.d/polipo restart
Configuring Untangle
Next we must tell Untangle to allow internal access to Polipo. Go to Untangle => Configuration => Networking => Advanced => Packet Filters and add this User Packet Filter Rule:
Description: Polipo Action: Pass Conditions: Destined Local AND Destination Port =8123 AND Protocol = TCP
Next, go to Untangle => Configuration => Networking => DNS Server and add these Static DNS Entries:
192.168.2.1 wpad.<domain name suffix> 192.168.2.1 wpad
Now check it's working by visiting http://wpad:8123/ in your web browser: you should see a page with two options giving Polipo's manual and its current config (which is modifyable). Inside its config is a section detailing memory and cache usage and other useful things.
Configuring Web Proxy Auto Discovery (WPAD)
We must now add some config files to have web browsers and windows updates in all your computers use the new caching proxy:
nano /var/www/.htaccess
Into this type:
AddType application/x-ns-proxy-autoconfig .dat
This tells the Apache providing the Untangle web server that from now on .dat files are proxy auto-config files. Next:
nano /var/www/wpad.dat
And into this:
function FindProxyForURL(url, host) {
var direct="DIRECT";
var proxy="PROXY wpad:8123";
if(isPlainHostName(host)) return direct;
if(isResolvable(host)) {
var resolvedip=dnsResolve(host);
if(isInNet(resolvedip, "192.168.2.0", "255.255.255.0"))
return direct;
}
// Always use the proxy for patches & updates, but not for TV listings
if(shExpMatch(host, "*tvdownload.microsoft.com")) return direct;
if(shExpMatch(host, "*windowsupdate.microsoft.com")) return proxy;
if(shExpMatch(host, "*download.microsoft.com")) return proxy;
if(shExpMatch(host, "*update.microsoft.com")) return proxy;
if(shExpMatch(host, "*windowsupdate.com")) return proxy;
if(shExpMatch(host, "*adobe.com")) return proxy;
if(shExpMatch(host, "*java.com")) return proxy;
if(shExpMatch(host, "*java.sun.com")) return proxy;
if(shExpMatch(host, "*firefox.com")) return proxy;
if(shExpMatch(host, "*mozilla.com")) return proxy;
if(shExpMatch(host, "*opera.com")) return proxy;
if(shExpMatch(host, "*ubuntu.com")) return proxy;
if(shExpMatch(host, "*debian.org")) return proxy;
if(shExpMatch(host, "*centos.org")) return proxy;
if(shExpMatch(host, "*redhat.com")) return proxy;
if(shExpMatch(host, "*avg.com")) return proxy;
if(shExpMatch(host, "*clamav.net")) return proxy;
if(shExpMatch(host, "*lavasoft.de")) return proxy;
if(shExpMatch(host, "*ca.com")) return proxy;
if(shExpMatch(host, "*grisoft.com")) return proxy;
if(shExpMatch(host, "*symantecliveupdate.com")) return proxy;
if(shExpMatch(host, "*mcafee.com")) return proxy;
if(url.substring(0, 6)=="https:") return direct;
// Choose here which you want the default to be
return direct;
}
This is some Javascript which the web browser runs in order to determine what proxy it should use. The above chooses the proxy for updates but for everything else chooses a direct connection - if you'd like it to always use the proxy, simply change the bottom line.
After a few days check /var/cache/polipo to ensure it's working. You should see a list of domains which the proxy has cached - naturally, these should just be the ones above. Debian and Ubuntu don't auto-discover the web proxy ordinarily speaking so you may need to do the following when you have multiple Ubuntu or Debian installations:
nano /etc/apt/apt.conf
And into this:
Acquire::http::Proxy "http://wpad:8123/";
Note that Polipo never purges except when manually initiated: http://www.pps.jussieu.fr/~jch/software/polipo/manual/Purging.html. In our configuration it would be extremely unlikely to run out of drive space, however if you use it as a general cache then it could fill up. To check how much space it uses do this:
du /var/cache/polipo
Troubleshooting
There is the situation of course when you've done all of the above and it doesn't appear to be working - /var/cache/polipo just doesn't fill no matter what you do.
Note that you need to REBOOT each Windows machine for it to pick up the WPAD. If Windows doesn't find WPAD on first boot then it silently disables it. Similarly, if the wpad.dat Javascript contains an error, OR if you make ANY changes to wpad.dat then you'll need to reboot Windows again for the changes to become seen.
Testing the Polipo proxy
For this bit I recommend Firefox and the "Live HTTP Headers" addon. In Firefox, go Tools->Options->Advanced->Network->Settings and choose "Manual Proxy Configuration". Enter 192.168.2.1 port 8123 and then tick "Use this proxy server for all other protocols". Hit okay.
Now open the Live HTTP Headers window. This will show you what your web browser and the internet say to one another at the implementation level.
Try browsing a site guaranteed to not have been visited before (or clear Firefox's cache before doing something). Check the reply headers for the "Via:" header we inserted in Polipo's configuration above, making sure that one of the Via's is definitely 192.168.2.1 (or the name of the proxy you specified in its config file which usually defaults to the Untangle box's hostname) as your ISP may also have implemented a HTTP cache which adds its own Via. Also check that /var/cache/polipo is now gaining items.
If this isn't working then something is wrong with Polipo, or maybe the User Packet filter rule you added above isn't configured correctly.
Testing the WPAD mechanism
Firstly open a command box on your local computer and type 'ping wpad'. If that doesn't work then you haven't configured the DNS correctly.
Next try http://wpad/wpad.dat in your web browser (make sure that you turned off the manual proxy settings above first!). If this loads in the javascript you entered above then great, if not then there's a config issue in there. Note that most web browsers will ask you to download a .dat file - that's fine, just download it and check its contents are what you expect.
Lastly there may be an issue in the wpad.dat Javascript file - in particular ANY syntax errors in there will render the file silently useless. I know of just one way of testing wpad.dat files: Google's PAC tester which is a pain to configure and install, but there is a useful web front end to it here.
The cache works fine almost all the time, but very rarely Windows will hang at 99% downloading one particular update
Unfortunately this is a known bug in Polipo - its author has moved onto other projects, and until someone else fixes Polipo or writes another small web proxy like Polipo we don't have much option.
When this happens I let Windows timeout and do its other updates, then I do a '/etc/init.d/polipo stop', manually tell Windows to update, and once finished I restart Polipo. The alternative is to shutdown Polipo, find the errant file inside /var/cache/polipo (it's usually the most recently modified), delete it and restart Polipo. When Polipo goes to redownload the update you'll find it gets all of it this time rather than 99.9% of it.
Sadly this means that Polipo is not a fire & forget solution - you couldn't deploy it on a non-techie's Untangle box even though this bug only presents maybe twice a year :(
I hope that you found my guide useful!
--Ned14 04:58, 14 February 2010 (PST)
