The User Agent Field: Analyzing and Detecting the Abnormal or Malicious in your Organization

Hackers are hiding within the noise of HTTP traffic. They understand that within this noise it is becoming increasingly difficult to detect malicious traffic. They know that overworked analysts have little time to detect malicious/abnormal HTTP traffic hiding amongst a mountain of legitimate HTTP traffic. However hackers may be using unusual, alien to your organization, unique or just plain evil HTTP header request user agents. When they do they become easier to identify. This paper aids intrusion analysts in understanding the user agent field and how it can be used to detect malicious traffic. It will then show the analyst how, using free tools like Wireshark, Tshark, Tcpdump and regex commands, to separate the normal from the abnormal. It will build upon what we know about our own organizations. It will look at how hackers are using the HTTP request header user agent field to attack organizations. Malicious attacks using the user agent field in HTTP request headers will be examined and discussed. Cross site scripting, SQL injection and other forms of attacks will be shown along with mitigation techniques to avoid these attacks.