1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

HTTP Explained

Discussion in 'Code Snippets and Tutorials' started by Zer0, Sep 24, 2009.

  1. Zer0

    Zer0 Level IV

    Joined:
    Mar 2, 2008
    Messages:
    3,037
    Likes Received:
    180
    Location:
    Home sweet home
    HTTP Explained

    Table of Contents
    1. Introduction
    2. GET Requests
    3. Cookies
    4. Referers
    5. Finding and manipulating data

    Introduction
    So what exactly is the HTTP? HTTP stands for Hypertext Transfer Protocol, which is a well-defined standards for computers to communicate with each other. It is essentially a language for computers to understand each other when sending data across the Internet. Without such a standard protocol, every computer would be talking in a different "language" and no one would be able to understand each other!

    After establishing a connection between two computers, both computers can send requests to each other using HTTP. There are two main types of requests in HTTP: GET requests and POST requests. I'll be explaining both of these in detail in the next two sections.

    GET Requests
    A GET request is typically made whenever you simply visit a web page by typing in its URL or clicking on a link. These are very basic requests that tell the web server to send over information about the web page. A basic GET request would look something like this:
    Code (Text):
    1. GET http://www.neopets.com
    which would return the HTML code for the main page of Neopets.

    In addition to simply fetching web pages, you can also pass data to the server using a GET request. When browsing the web, sometimes you'll notice a question mark '?' and/or ampersands '&' in the URL. The question mark means that you will be giving data to the server along with your request. For example, when you watch a video on YouTube (e.g. http://www.youtube.com/watch?v=2WPCLda_erI), you see a "?v=somestuff" in the URL. The "v=" means you are assigning "somestuff" to the variable "v" and giving it to the server. The server can then read that variable, and then return the correct video for your viewing.

    You can also pass multiple variables by separating variables by an ampersand. When you click on a related video on YouTube, an additional variable named "feature" will be sent to the server with the value "related".

    e.g.
    Code (Text):
    1. GET http://www.youtube.com/watch?v=mQPLrJ_fL4I&feature=related
    2.  
    3. URL: http://www.youtube.com/watch
    4. Parameter 1: v=mQPLrJ_fL4I
    5. Parameter 2: feature=related
    Although great for basic web requests, GET has some shortfalls. You'll notice that the variables are passed to the server through the URL. This means that they are not hidden and any bystander can see them. If you were logging into a website using GET, both your username and password would show up in the URL and could easily be stolen.

    e.g. of such a URL
    Code (Text):
    1. GET http://www.neopets.com/login.php?username=Zer0&password=omgbbq
    POST Requests
    POST requests solve this problem with GET requests by hiding all their variables. Almost all login schemes are implemented using POST for these security reasons. In addition, web forms are also use POST requests for simplicity's sake. Everytime an <input> tag is used in a form, the corresponding name and value attribute will be sent to the server through POST.

    Going back to our login example, the POST request would look something like this
    Code (Text):
    1. POST http://www.neofriends.net/login.php
    2. Parameter 1: username=Zer0
    3. Parameter 2: password=omgbbq
    Notice that the variables you pass are not visible to the user. Instead, they are sent invisibly.

    Cookies
    Yum!! Cookies are variables given by the server that are stored on your computer. These variables can store information such as settings, sessions, etc. Every time you make a request to the web server that you the cookie, you will also send the cookie back to them. This is particularly useful for keeping track of session data where the web server must verify who you are.

    You can read more about cookies here: http://en.wikipedia.org/wiki/HTTP_cookie

    Referers
    A referer is simply a URL that your web browser sends to the web server telling the server what was the previous web page that you visited. This is one of the ways web sites keep track of where their traffic comes from. It can also be used for validation, which is why sometimes when you get "Oops! It looks like you came from a wrong web page" messages on websites.

    Finding and manipulating data
    Well, knowing how HTTP works is good, but its not particularly useful if you don't know how to manipulate it. Suppose you are trying to write a program that simulates a restocker for you (aka an autobuyer). Your program must know what variables to pass to the server in order to appear exactly like a legit restocker. Well one way to find out what these variables are is to get a packet sniffer. A packet sniffer intercepts requests between your computer and web servers and displays all the data being sent and received.

    There are some very advanced packet sniffers out there, but I use the Tamper Data add-on for Firefox because it deals exclusively with HTTP and thus has some handy tools. You can download Tamper Data here: https://addons.mozilla.org/en-US/firefox/addon/966

    After installing Tamper Data, open it from Firefox by going to Tools > Tamper Data. Now just visit any web page on your Firefox browser, and all the HTTP requests will be logged in Tamper Data where you can view the data. In addition to just viewing the data, you can also tamper with it (hence the name Tamper Data). You can probably figure out how to do that, its pretty self explanatory on the Tamper Data window.