From UntangleWiki

Untangle Server User's Guide

Web Filter

Other Links: Web Filter Description Page Web Filter Video Demo Web Filter Screenshots Web Filter Forums Web Filter FAQs

---

Contents 1 About Web Filter 2 Settings 2.1 Block Lists 2.1.1 Categories 2.1.2 Sites 2.1.3 File Types 2.1.4 MIME Types 2.1.5 Enforce safe search on popular search engines 2.1.6 Block pages from IP only hosts 2.1.7 Youtube for Schools 2.1.8 Unblock 2.2 Pass Lists 2.2.1 Passed Sites 2.2.2 Passed Clients 3 Event Log 4 Related Topics 5 Web Filter FAQs 5.1 Can I use both Web Filter and Web Filter Lite? 5.2 Is Web Filter really better than Web Filter Lite? 5.3 Does Web Filter use a lot of memory and CPU? 5.4 How do real-time updates work? 5.5 How long does Web Filter cache category information for sites? 5.6 Can I add additional categories? 5.7 How should I handle false positives? 5.8 Can I use Web Filter to block HTTPS/SSL sites? 5.9 Why can i access a site using HTTPS when I've added it to the block list? 5.10 Why is Web Filter still blocking an HTTPS site even after I added it to the pass list?

About Web Filter

Web Filter offers everything that Web Filter Lite, another application, does and more. Web Filter appeals to customers who require an added level of protection or are subject to regulations. For example, Web Filter helps libraries comply with Children's Internet Protection Act); equally important, Web Filter helps schools control hate speech. Pornography is still a big workplace productivity problem for companies, and Web Filter's categorization is a great solution for this problem.

The main technical differences between these products is that Web Filter offers:

• Real-time classification and updates. Web Filter uses a community-based approach whereby a large base of Web Filter users and Untangle itself categorize URLs. However, Web Filter combines both human beings—and sophisticated web crawlers. Web Filter runs web crawlers throughout the day. If Web Filter's web crawlers detect a new site or if you visit a site that Web Filter doesn't know about, it immediately analyzes it, then does the following:

• If Web Filter's engine can identify the content with high probability, it will categorize it.
• If Web Filter's engine cannot identify the content with high probability, it will be assigned to be identified by a human.
  

Web Filters techniques result in more categorization (over 100,000 URLs daily), more accuracy, and faster turnaround time. When identifies a malicious website, customers get the update within seconds. Time is essential in web content filters because new websites go live very quickly.

• Categorize HTTPS traffic by IP address. Untangle can block site via a URL or the IP address of the web site. Since secure websites cannot be scanned, this feature tries to use reverse DNS to figure out the category. (If https appears in the URL, then you know it's a secure site.) If it cannot resolve, this option will block inbound and outbound SSL traffic on port 443.

So, for example, when you log on to your online banking account to view a statement, a secure site prevents others on the network from capturing private information such as your username and password. But, there's a dirty little secret that's known to those who want to bypass web filters. Often used in the pornography industry, the web site is set up as a secure site. Web Filter easily combats this tactic because Web Filter also cateogorizes the destination IP address, which isn't unknown. This feature is implemented via the Web Filter Scan HTTPS check box.

• Detailed categorization. Web Filter does a good job categorizing, but Web Filter offers over 53 categories and over 450 million categorized sites. The abundance of categories means that you can narrow your scope. For example, maybe you want to block websites related to Dating, but not Social Networking. You can do so with Web Filter.

Settings

This section reviews the different settings and configuration options available for Web Filter.

Block Lists

This tab controls the different mechanisms to classify and block or flag content.

Categories

This allows you to customize which categories of sites will be allowed, flagged as a violation, or blocked.

As sites are visited they are categorized in real time using a categorization service and cached locally.

If Categorize HTTPS traffic by IP address is enabled, HTTPS (encrypted) traffic will be categorized using the IP address if Web Filter is unable to determine the name of the server due to encryption.

Clear Category URL Cache will clear the local cache of categorized sites and URLS. After clearing the cache all new web visits will be looked up fresh using the categorization service. The cache automatically cleans itself as entries become old or stale so this is just for testing purposes.

Sites

1. From Web Filter, click the Block Lists tab and the Edit Sites button.
   1. In the table, click the add (+) button to the left of the table.
   2. In the new entry, add the URL that you want to block.
   3. Click Done
2. Click the Save button to save the newly added entries or the Apply button to save and continue adding new entries.
3. Click the OK button

Tip: If you want to temporarily unblock this website later, deselect the block check box.

File Types

1. From Web Filter, click the Block Lists tab and the Edit File Types button.
2. In the table, do one of the following:
• If the file type that you want to block already appears in the table, select the block or log check box or both.
• If you want to block a file type that isn't in the list, click the add (+) button to the left of the table, then specify the file type that you want to block.
3. Click the Save button, then OK.

MIME Types

1. From Web Filter, click the Block Lists tab, and click the Edit MIME Types button.
2. In the table, do one of the following:
• If the MIME type that you want to block appears in the table, select the block check box for that MIME type.
• If you need to add a new MIME type, click the add (+) button to the left of the table, and in the new entry, add the MIME type that you want to block.
3. Click the Save button, then OK.

Enforce safe search on popular search engines

When this option is enabled, safe search will be enforced on all searches using supported search engines (Google, Yahoo, etc).

Block pages from IP only hosts

When this option is enabled, users entering the IP address rather than domain name will see a block page.

Youtube for Schools

If enabled, this option will inject your youtube indentifier into all youtube web traffic so youtube will enforce the appropriate policy for computers on your network. You can read more about how to setup youtube for schools here.

After creating an account a unique identifier will be supplied (Example: Jvagw05BzSxAntTLKwUw1w). Take the supplied youtube identifier and save it in your settings and Web Filter will rewrite all youtube URLs with this identifier.

After doing this you will need to configure your desired settings on the Youtube for Schools configuration page under "Account->Settings->Manage School" on youtube.com. This pages allows you to create a list of blessed videos and blessed "teacher accounts" and more to enforce proper youtube usage on your network.

If this is configured it may also be necessary to block all HTTPS youtube traffic using Application Control to prevent access to youtube of encrypted channel where the URL can not be rewritten.

Unblock

Some organizations may wish to allow certain users to bypass the block page displayed by Web Filter. This is known as "unblocking" and it is configurable under "Unblock."

If Unblock is set to None no users will be allowed to bypass the block page. If Unblock is set to Temporary users will be allowed to visit the site for one hour from the time it is unblocked. If Unblock is set to Permanent and Global then users will be allowed to visit the site and unblocked sites will be added to the permanent global pass list so it will always be allowed in the future.

Unblock is best when combined with Policy Manager so that only certain users are allowed to bypass the block page.

1. From Web Filter, click the Block Lists tab.
2. Under Unblock choose None, Temporary, or Permanent and Global
3. Click the Save button.

Pass Lists

Pass Lists are used to pass content that would have otherwise been blocked. This can be useful for "unblocking" sites that you don't want blocked or allowing certain users special privileges.

Passed Sites

If your organization deems a specific website to be useful and that site or URL should not be blocked regardless of its categorization, it can be added to the Passed Sites list.

1. From Web Filter, click the Pass Lists tab and the Edit Passed Sites button.
2. In the table, do one of the following:
• If the URL that you want to pass appears in the table, select the pass check box for that URL.
• If you need to add a new URL, click the add (+) button to the left of the table, and in the new entry, add the URL that you want to pass.
3. Click the Save button, then OK.

1. From Web Filter, click the Block Lists tab and the Edit Sites button.
2. In the table, locate an existing URL that you want to pass, and clear the block check box, or simply delete the row.
3. Click the Save button, then OK.

Passed Clients

If you only have a few users that need to completely bypass Web Filter controls, consider using pass lists. If these users simply need a different Web Filter policy you should set up a separate rack using Policy Manager.

Before You Begin: It may be useful to assign the user a static IP address. If the Untangle Server is your router, go to Assigning Network Computers Static IP Addresses.

1. From Web Filter, click the Pass Lists tab and the Edit Passed Client IPs button.
2. In the table, select the add (+) button. A new row appears.
3. In the IP address/range text box, specify the computer IP address and subnet mask of user that you want to be exempt from the web filter.
4. Click the Update button, then Save.

Event Log

Use the following terms and definitions to understand the Web Filter Event Log:

timestamp	The time the event took place.
action	The action which the Untangle Server took on the web request.
client	IP address of the client who made the request.
request	A description of the request made (e.g. http://someurl/somepath.html).
reason for action	The reason the action was taken.
server	The server IP Address. The server is the computer that receives the request.

Related Topics

• Web Filter Lite

Web Filter FAQs

Can I use both Web Filter and Web Filter Lite?

We do not recommend running both Web Filter and Web Filter Lite at the same time - if you have access to the trial or full version of Web Filter, we recommend using it rather than Web Filter Lite.

Is Web Filter really better than Web Filter Lite?

The Web Filter is the same as Web Filter Lite except it is based on SiteFilter technology. Web Filter is better than Web Filter Lite in many ways:

• Many More categories (141 vs. 15)
• Larger database (450+ million URLs vs ~1 million)
• Dynamic categorization of new sites
• Extra features
  • Youtube for Schools support
  • HTTPS Filtering
  • Search Engine SafeSearch enforcement

Does Web Filter use a lot of memory and CPU?

If your Untangle Server is operating well without Web Filter, then you won't see much of a difference if you run Web Filter. Web Filter doesn't use much memory, and its cloud-based architecture adds very little to CPU utilization.

How do real-time updates work?

When a client first vists a site, Web Filter accesses the zveloDB to get the categories the site is under to make a decision to block or pass based on your configuration. The category information is also written to a local cache so it doesn't have to be checked the next time a user visits that site.

How long does Web Filter cache category information for sites?

Several days. Web Filter flushes non-frequently used cache. The website that you visit daily will not be cleared from cache.

Can I add additional categories?

Custom categories are not available, however we provide over 140 categories for granular control over what your clients can access. If you feel there are categories that we can add to make it even better, just let us know.

How should I handle false positives?

While the fastest way to allow clients to access a site that is currently blocked is to add the site to your pass list, you can request recategorization of sites here - the turnaround time is usually less than two days.

Can I use Web Filter to block HTTPS/SSL sites?

Yes - because Web Filter has access to a separate database of IP addresses, it can categorize HTTPS traffic based on the destination IP address. This is not done by individual domain, but by category - for example, if you simply block 'facebook.com'

Note: This does not mean Web Filter can parse HTTPS as it is encrypted. Categorization is done via IP address. This means other forms of blocking like URL, file-type, mime-type, etc can not be done on HTTPS as the stream is encrypted and these require parsing of the HTTP protocol.

Why can i access a site using HTTPS when I've added it to the block list?

Web Filter scans and categorizes HTTPS traffic by IP address because the session itself is encrypted and cannot be scanned. As a result, if you add "example.com" to the block list and go to "https://example.com" it will not be blocked because Untangle can only see the IP address. However, if you block the category "example.com" is in, then go to "https://example.com" it will not connect and you will see a block event in the Event Log.

Why is Web Filter still blocking an HTTPS site even after I added it to the pass list?

This should only be a problem with older browsers that do not provide SNI information in the HTTPS stream - if your browser provides SNI information, adding the domain to the pass list should allow the site to load.

Older browsers that do not provide SNI information can run into this problem, however. If this is the case, it is because Web Filter does categorization of HTTPS traffic by IP address. HTTPS encrypts the hostname and request, so all we can see is the destination IP. This means if https://example.com/ is getting blocked, adding "example.com" to the passlist will have no effect because HTTPS is categorized by IP address. If you add the IP address of example.com to the passlist then HTTPS traffic to example.com will be allowed.