Manually send HTTP requests using Netcat

In my last post, I talked about the importance of reviewing your web server's log files periodically for any unusual behavior.  This is a good practice and should be implemented in your organization.  In OWASP's Log review and management, they point out that the frequency "depends on the criticality (i.e. payment system, customer information, business secret, etc.) of the system labelled by the organization, logs could be reviewed ranging from minute, every day, weekly, monthly or even 3 months." Assessing your log management needs doesn't end here.  Keep in mind that there are other considerations including but not limited to: the need for centralizing your log files, retention periods, and protection and integrity of log information. The latter can be done via a simple checksum to each log file to determine if the file has been tampered with in any way.  Another consideration is to include log files in automatic backups and disaster recovery plans.  How long you choose to keep these files is very important.  Include all of these things in your organization's security policies.

Let's say you come across more than the usual number of requests coming in on Port 80 that should be coming in on the secured Port 443.  You may become suspicious but more than likely if you forced your site to communicate over HTTPS then it's probably a few users who have bookmarked the HTTP url.  If indeed the site is responding to those requests over HTTPS like it should, there should be no harm in that.  But, it might be a good time to do some manual testing to be doubly sure.

The tool we want to use is Netcat. It is a network utility used for sending and receiving data from networked computers. You can transfer files, serve up a single web page, and send messages from one system to another.  In our case we will be sending simple HTTP requests.

It's history goes back as far as 1996. Although, I have some suspicions that it goes back further than that. The original Netcat and today's Ncat that we can download from here is the same tool in the sense that it performs the same function but it doesn't share the same code base.  The original included a port scanner. It wasn't included in Ncat because Nmap replaced it as the de facto tool.

Let's Visit the download page and install it for your system. If you're on windows like I am, open an command prompt and navigate to the folder.  Type in the commands as shown in the following screen shot.  It should be in the following format:

nc [host] [port]
[httpMethod] [url] [httpversion]

You should get back the raw html from the requested page. How often will you use this? Probably not too often. But let's appreciate its simplicity. If you need to quick test some security settings, want to look at some raw JSON or do some testing and want to look through your log files quickly you can add in a custom User-Agent and CTRL-F for the value you put in there.  Like so:

nc yourdomain.com 80
GET / HTTP/1.1
User-Agent: blah

Search for the term "blah" and you'll quickly find them in the log files. Or, set up a batch job in Log Parser Studio as mentioned in my previous blog post and view them there.  There are many other things you can do with this tool. Let me know if you do anything cool with it.

Which HTTP method, GET or POST, is more secure? How about over a secure connection (HTTPS)?

There are many considerations when deciding to send either a HTTP GET or POST request when submitting form data.  Some of those reasons may include: ease of use, allowing the use of the back and reload buttons on the browser, and etc.  Some may implement solutions that use only one type or another exclusively.  But, the main consideration that we will look into is security.

We all know that when we send a GET request, the URL is visible to you and the person right next to you.  Well of course that's insecure!  In a POST request, the form data is sent as a block.  What about GET and POST requests sent via HTTPS?  Surely that's secure, right?

Submitting data via POST is the more secure way less insecure way.  The reasons are pretty simple.  URLs are saved or transmitted in a least a couple places.  1) In the browser's history, 2) in the HTTP Referer field and 3) in the web server's log files.  Attackers have at least these places to look for to get at the juicy URLs.

How hard would it be to put a piece of malicious software on a USB stick around the office or better yet at various conferences and event halls with the label, "try our demo today?"  Once ran, it can crawl your browser history and upload it periodically.  How about another attack vector via the ad networks that will display an ad and log the referer, aka the last page that was visited by the user.  And this URL can very well be that GET request with all kinds of query string information.  Don't even get me started with CDNs and the danger of leaking your URLs when fetching images and javascript files with the referer info.  Just about all webpages these days do this unless you specify this meta element in every page of your site: meta name="referrer" content="never".  Of course, as of yet, not all browsers support this under HTML5.  What's even worse is that most web servers keep logs of all URLs.  And ever single URL can be potentially logged, whether it comes from a secured TCP connection or not.

As a security minded developer, if you stick to this one rule your users and employers will thank you: Never send sensitive data using the GET method.  Ask yourself this question the next time you are working on a web application: "Am I relying too heavily on passing data via the GET request and the query string?"  If the answer is yes, choose POST.  To help you remember, think of the POST OFFICE as being more secure because they package up your data as opposed to the GET OFFICE.  :[