Hypertext Transfer Protocol (HTTP) is an application layer protocol. On the surface, HTTP can appear quite simple, but there are a lot of quirks considering its use across inconsistent browsers and servers.
Breaking down an HTTP request
For each request, we generally have:
- A request line
- A body (optional)
Taking from the example above, the request line is the first line in the request and tells us the HTTP method (GET), the URI of the request (/api/shop/products) and the protocol (HTTP/1.1).
The headers are listed below the request line. On each line, the header itself is defined first, with the value following after the colon. For example, the Host header has a value of ‘localhost’, and the Content-Type header has a value of ‘application/json’.
For GET requests (the most commonly used HTTP method) there should not be a body. This is because we are simply requesting a resource and any parameters are passed via the URI. However, for other HTTP methods such as POST, the body would be listed after the headers, with a link break in between. In the HTTP response example on the right, you can see an example of this.
The response structure is generally the same. We again have a response line, the headers and a body. One of the main things to point out in the response is the status code. This gives an indication of what happened with the request. ‘200 OK’ is a good sign, it means the request was processed normally. There are other response codes and they can be categorized as follows:
- 100-199 are informational responses
- 200-299 are successful responses
- 300-399 are redirect messages
- 400-499 are client errors
- 500-599 are server errors
If you want to dive deep into status codes, you can check out RFC 9110.
These can also be referred to as HTTP verbs. They indicate the intended actions to be performed on a resource. The most common HTTP methods are GET, POST, PUT, and DELETE. Whilst you probably don’t need to know what every method does off the top of your head, knowing the common ones is important.
- GET – retrieve a resource from the server.
- POST – submit a resource to the server.
- PUT – update a resource on the server.
- DELETE – delete a resource on the server.
Other methods to be aware of are HEAD, PATCH, OPTIONS, CONNECT, and TRACE.
Understanding how an application is meant to behave, and which HTTP methods it accepts can sometimes lead us to the discovery of unintended behaviour. For example, when an application accepts and processes HTTP methods that are not intended by the developer.
Headers are used to pass additional information with both requests and responses. When we talk about request headers, we mean the headers that are sent in the initial request to the web server from the client. Response headers are the headers that come back. We’ll go through some of the headers you will need to understand as an application security engineer or web application penetration tester.
Sometimes referred to as ‘CSP’. This helps browsers detect and mitigate some injection attacks such as XSS. When it’s defined, it also needs a policy to go along with it which describes a number of directives to follow. You can find out more about the directives on the mdn web docs.
This tells a browser if it is allowed to render a page in a <frame>, <iframe>, <embed> or <object>. Using X-Frame-Options prevents pages from being loaded into other sites which can prevent click-jacking attacks.
Often referred to as ‘HSTS’. This tells the browser that the site should only be accessed over HTTPS. If an HTTP request is made, it should be upgraded to HTTPS automatically.
Here we just highlighted some of the key ones that might flag up from vulnerability scans or other testing. There are of course many headers to consider, so we encourage you to take some time to review each one. Try to thoroughly understand what they do and, perhaps more importantly, what their limitations are.
Because HTTP is a stateless protocol (i.e. it doesn’t maintain a session or connection between requests), we need a mechanism for session management. Cookies give us just that, and work in a way that goes something like this:
- User authenticates to a web service
- The web server generates a session ID and sends it back to the client as a cookie
- The client sends the session ID back to the server with each subsequent request.
In this scenario, any request coming in with that session ID will be treated as though it comes from the user that authenticated.
There are a couple of drawbacks here. If a session ID was predictable or stolen, an attacker could use it to masquerade as the user simply by sending requests and including the session ID as a cookie. This is called session hijacking.
Cookies are made up of a name, value, and expiration date. When they expire, the browser will discard them. You’ll often see them encoded in base64 (though this is not a hard rule, there are many available encoding schemes) and they can be set with flags that dictate how they can be handled.
There are several flags we need to know as they may come up in casual conversations with developers or other security professionals.
Secure: This will ensure that the cookie is only sent over secure connections (HTTPS).
SameSite: This flag dictates when the cookie is sent. There are three options, strict, lax, and none. Strict means the cookie is only sent with requests that originate from the same site as the cookie. Lax means the cookie is only sent with requests that originate from the same site as the cookie, as well as with some cross-site requests, such as GET requests. None means the cookie will be sent with all types of requests, regardless of the origin. It’s worth noting that if the SameSite flag is not set, some browsers will apply a default value.
Expires: This is exactly as it sounds, it tells the browser when to discard the cookie.
Where to go next?
Well HTTP might seem quite simple, but we can’t overstate the importance of understanding the underlying technology when working in the field of Cyber Security. If you want to learn more about web application security and hacking, head on over to the TCM Security Academy.