How To Control Access To Unwanted Websites Using URL Blacklist With SafeSquid Proxy Server

SafeSquid – Content Filtering Internet Proxy, has many content filtering features that can be used to decide who is allowed what, when and how much on the net. In this tutorial I will describe how to control access to unwanted categories of websites, by using URL Blacklist database with SafeSquid Proxy Server.

Note: Also see the following articles :
'Deploying A Content Filtering Proxy Server To Distribute Controlled Internet Access With SafeSquid'
Set Up Gateway Level Virus Security With ClamAV And SafeSquid Proxy
How To Set Up Internet Access Control And Internet Filtering With SafeSquid Proxy Server

SafeSquid allows the administrators to use plain text urlblacklist very easily and with a desired level of sophistication. The sites http://www.shallalist.de/ and  http://www.urlblacklist.com maintain a well categorized list of various web-sites and pages like porn, adult, webmail, jobsearch, entertainment, etc. This is an excellent resource for an administrator seeking to granularly enforce a corporate policy that allows or disallows only certain kinds of web-sites to be accessible by specific users, groups or networks.

Note: cProfiles offers the flexibility of many more actions than URL Blacklist, instead of just allowing / blocking categories. For example, you can add a profile to a specific category, and then use that profile in any of SafeSquid's filtering sections, for actions on the category like blocking cookies, ads and banners, ActiveX, Java Scripts, throttling bandwidth (QoS), or simply analyzing what category is most visited, without blocking access.
For Details, see http://www.safesquid.com/html/portal.php?page=132

While Shalla Secure Services offer free downloads and updates for home users, Urlblacklist requires you to subscribe to receive updates. You can download the URL Blacklist by Shalla from HERE, and the trial database by urlblacklist.com from HERE.
Please note that you will be able to download this trial database only once. You need to subscribe to urlblacklist.com to be able to receive regular updates 

Copy the downloaded trial database to /usr/local/bin directory on the SafeSquid Server, and untar the files

cd /usr/local/src
tar -zxvf bigblacklist.tar.gz

This will create a directory 'blacklist'. Create a directory 'urlbl' in /opt/safesquid and copy the contents of blacklist in this directory.

mkdir /opt/safesquid/urlbl
cd blacklist
cp -rf . /opt/safesquid/urlbl

Next, restart SafeSquid

/etc/init.d/safesquid restart

In SafeSquid GUI Interface, click on URL blacklist in the Top Menu It should display a list of all the categories copied to the urlbl directory. Here, you can query the database to find out if a website is listed under any category. For example, to find out what category hackerstuff.com belongs to, type hackerstuff.com in the Domain field and click on Submit below. You should get a screen similar to this –

SafeSquid Interface – URL Blacklist Database Query

Note: This section only allows you to query the database. Selecting or unselecting a category does not enable or disable it. 

Next, click on Config, and from the drop-down menu, select URL blacklist and click on Submit. This is where you allow or deny access the a category, either to a specific Profile, or globally. 

SafeSquid Interface – URL Blacklist Section

By default, the section is disabled. Enable the section by selecting Enabled – Yes.
The Policy is Allow. So you need to specify what you want to deny under the Deny sub-section.
Now suppose, this is what you want to achieve –

Globally block: Categories porn, adult and dating
Profile HRD: Allowed jobsearch category, but denied to everybody else
Profile Finance: Allowed categories banking and financial, but denied to everybody else

Note: Creating users and user profiles has been described in How To Set Up Internet Access Control And Internet Filtering With SafeSquid Proxy Server.

To achieve the above, click on Add under Deny sub-section, and create the following rules –

In the first rule, since the Profiles field is left blank, it will apply to every user, and block access to websites listed under adult, porn and dating categories.

In the second rule, the Profiles is !HRD. The '!' before HRD means NOT HRD, or everyone EXCEPT HRD. So, requests for websites listed under jobsearch will be allowed only to HRD Profile and denied to all other profiles.

The third rule is similar to the second rule and allows access to websites listed under banking and financial only to Finance Profile..

When a user requests for a website that is blocked by URL Blacklist, a template similar to the below is displayed –

SafeSquid – URL Blacklist Template

TIP: You might find that many times a website gets blocked since it is listed under a denied category, but you would want to allow access to it. You have two options to achieve this:

Option 1:
Edit the list and delete the entry for that Website, and restart SafeSquid (any changes to the database requires a restart of SafeSquid, since it loads the database in the memory, when it starts up).
E.g. to edit the domains list under ads category –

vi /opt/safesquid/urlbl/ads/domains

Option 2:
Add a rule under Allow sub-section, to allow access to the whitelist category for everyone, edit the whitelist category, and add the website that you want to allow in domains file. 

TIP: You can also create your own customized categories easily. Create a directory in /opt/safesquid/urlbl and name it what you want the category to be called, e.g. 'custom', and create a 'domains' file in this directory, listing one website per line.

mkdir /opt/safesquid/urlbl/custom
cd /opt/safesquid/urlbl/custom
touch domains
vi domains

Src: http://www.howtoforge.com/control-access-to-unwanted-websites-using-url-blacklist-with-safesquid-proxy-server