Please log in for full Help Center access

Articles in this section

See more

Using Blacklist and Whitelist

  • Updated
Follow

Websites are composed of content from multiple sources. For monitoring or load testing it may be necessary to block certain requests or to allow requests from a specific domain or sub-domain. This is accomplished with the whitelistRequests() and blacklistRequests() commands. The blacklist command is the more common command and many scripts contain the blacklistCommonUrls() method. When a site is blacklisted the request is not made to the endpoint. This command works at the proxy level and as a result the request will still show in the validation log, monitor logs, or load test data, it cannot be removed. However, the request(s) should have the HTTP status code specified (int responseCode) by the command and show with a size of 0 bytes and a download time of 0 ms. 

The method description for blacklistRequests is below.
 

blacklistRequests
 
void blacklistRequests(java.lang.String pattern, int responseCode)
 

Blacklist any HTTP request that matches the supplied regular expression pattern, automatically registering the supplied response code as the response without actually making the call. The response time for these requests will always be 0 ms. These requests will also not be matched against any automatic "expected response code" validation, even if one was supplied before making the request. For example, to prevent Google Analytics requests from being issued, you could do the following:

 

c.blacklistRequests("http://www\\.google-analytics\\.com/.*", 200);

 

Parameters:
pattern – the regular expression string to match against. You can use regex symbols such as ".", "\d", "\W", "[...]", etc.

 

responseCode - the HTTP response code to immediately "return" instead of making the actual HTTP request.

 

blacklistCommonUrls
 
void blacklistCommonUrls()
 

Blacklist well-known sites that are commonly used for site analytics and advertising. Most website monitoring scripts will want to exclude these so that analytics are not messed up. This is currently a shortcut for the following:


The follow is an example of code that would be contained in the script

var c = test.openHttpClient(); //instantiate the HTTP client
c.blacklistCommonUrls(); // blacklist common ad and analytic sites

The blacklistCommonUrls(); method is the equivalent of having the below individual blacklist requests. These examples are also how to block a specific URL, domain, path, etc.Copy and paste them into a script and then edit them to match the desired pattern. 

c.blacklistRequests("http(s)?://[^/]+\\.google-analytics\\.com/.*", 200);
c.blacklistRequests("http://[^/]+\\.quantserve\\.com/.*", 200);
c.blacklistRequests("http://www\\.quantcast\\.com/.*", 200);
c.blacklistRequests("http://c\\.compete\\.com/.*", 200);
c.blacklistRequests("http(s)?://s?b\\.scorecardresearch\\.com/.*", 200);
c.blacklistRequests("http://s\\.stats\\.wordpress\\.com/.*", 200);
c.blacklistRequests("http://partner\\.googleadservices\\.com/.*", 200);
c.blacklistRequests("http://ad\\.adtegrity\\.net/.*", 200);
c.blacklistRequests("http(s)?://[^/]+\\.doubleclick\\.net/.*", 200);
c.blacklistRequests("http(s)?://connect\\.facebook\\.net/.*", 200);
c.blacklistRequests("http://platform\\.twitter\\.com/.*", 200);
c.blacklistRequests("http://[^/]+\\.addthis\\.com/.*", 200);
c.blacklistRequests("http://widgets\\.digg\\.com/.*", 200);
c.blacklistRequests("http(s)?://plus\\.google\\.com/.*", 200);
c.blacklistRequests("http(s)?://plusone\\.google\\.com/.*", 200);


Whereas blacklist blocks traffic for each specified request, the whitelist request only allows traffic from the specified destination. This command works at the proxy level and as a result requests not contained in  the whitelist may show in the validation log, monitor logs, or load test data, they cannot be removed. However, the request(s) should have the HTTP status code specified (int responseCode) by the command and show with a size of 0 bytes and a download time of 0 ms. The method for the command follows.
 

whitelistRequests

void whitelistRequests(java.lang.String[] patterns, int responseCode)
 

Whitelist any HTTP request that matches the supplied regular expression patterns, automatically registering the supplied response code as the response for any request not in the set of whitelists. The response time for those requests will always be 0 ms. Those requests will also not be matched against any automatic "expected response code" validation, even if one was supplied before making the request. For example, to prevent all 3rd party requests from being made and to only allow requests to www.example.com and images.example.com, you would do:
 
c.whitelistRequests(["http://www\\.example\\.com/.*", "http://images\\.example.com\\.com/.*"], 200); 
 
Parameters:
patterns - the array regular expression string to match against. You can use regex symbols such as ".", "\d", "\W", "[...]", etc.

responseCode - the HTTP response code to immediately "return" instead of making the actual HTTP request.

The following is an example of code that would be contained in the script. Notice that the URL is in an array. 

var c = test.openHttpClient(); // instantiate the HTTP client
c.whitelistRequests(["https?://.*\\.vercara.*\\.com/.*"], 200); // only allow traffic from vercara.com
 

Multiple resources are available on the web to assist with creating Regular Expressions.

Related articles