Manually send HTTP requests using Netcat

In my last post, I talked about the importance of reviewing your web server's log files periodically for any unusual behavior.  This is a good practice and should be implemented in your organization.  In OWASP's Log review and management, they point out that the frequency "depends on the criticality (i.e. payment system, customer information, business secret, etc.) of the system labelled by the organization, logs could be reviewed ranging from minute, every day, weekly, monthly or even 3 months." Assessing your log management needs doesn't end here.  Keep in mind that there are other considerations including but not limited to: the need for centralizing your log files, retention periods, and protection and integrity of log information. The latter can be done via a simple checksum to each log file to determine if the file has been tampered with in any way.  Another consideration is to include log files in automatic backups and disaster recovery plans.  How long you choose to keep these files is very important.  Include all of these things in your organization's security policies.

Let's say you come across more than the usual number of requests coming in on Port 80 that should be coming in on the secured Port 443.  You may become suspicious but more than likely if you forced your site to communicate over HTTPS then it's probably a few users who have bookmarked the HTTP url.  If indeed the site is responding to those requests over HTTPS like it should, there should be no harm in that.  But, it might be a good time to do some manual testing to be doubly sure.

The tool we want to use is Netcat. It is a network utility used for sending and receiving data from networked computers. You can transfer files, serve up a single web page, and send messages from one system to another.  In our case we will be sending simple HTTP requests.

It's history goes back as far as 1996. Although, I have some suspicions that it goes back further than that. The original Netcat and today's Ncat that we can download from here is the same tool in the sense that it performs the same function but it doesn't share the same code base.  The original included a port scanner. It wasn't included in Ncat because Nmap replaced it as the de facto tool.

Let's Visit the download page and install it for your system. If you're on windows like I am, open an command prompt and navigate to the folder.  Type in the commands as shown in the following screen shot.  It should be in the following format:

nc [host] [port]
[httpMethod] [url] [httpversion]

You should get back the raw html from the requested page. How often will you use this? Probably not too often. But let's appreciate its simplicity. If you need to quick test some security settings, want to look at some raw JSON or do some testing and want to look through your log files quickly you can add in a custom User-Agent and CTRL-F for the value you put in there.  Like so:

nc yourdomain.com 80
GET / HTTP/1.1
User-Agent: blah

Search for the term "blah" and you'll quickly find them in the log files. Or, set up a batch job in Log Parser Studio as mentioned in my previous blog post and view them there.  There are many other things you can do with this tool. Let me know if you do anything cool with it.