One of the latest under-the-hood upgrades to our User Journey and Page Performance monitoring system has been to introduce the ability to set custom HTTP request headers. This was actually a requirement from one of our clients who was beginning to have trouble maintaining the whitelist of IP addresses that our requests might come from in their Web Application Firewall (WAF). In certain regions our injector IP list is potentially ever-growing and we need to have the ability to bring on new injectors quickly, without having to worry about our client’s whitelists.
WAFs allow users to write access rules based on various conditions, two of which are the
When we launch our browser monitoring software we go through a setup process that includes creating a local proxy servlet and the browser session itself. It’s in the proxy that we set any custom headers and in the browser session that we overwrite the ‘default’ User-Agent. All subsequent requests made during the test will then include the custom headers and User-Agent.
Once this is setup in our software its over to the client’s WAF, or similar system, to look for the headers and/or User-Agent and handle the requests accordingly. This is a much preferred method for allowing our browsers access to the test target as it reduces the maintenance overhead of managing a list of IP addresses and means we are able to be more flexible with our infrastructure.
