Hypertext Transfer Protocol (HTTP) is an application layer protocol. On the surface, HTTP can appear quite simple, but there are a lot of quirks considering its use across inconsistent browsers and servers.
Breaking down an HTTP request
For each request, we generally have:
-
- A request line
- Headers
- A body (optional)
Taking from the example above, the request line is the first line in the request and tells us the HTTP method (GET), the URI of the request (/api/shop/products) and the protocol (HTTP/1.1).
The headers are listed below the request line. On each line, the header itself is defined first, with the value following after the colon. For example, the Host header has a value of ‘localhost’, and the Content-Type header has a value of ‘application/json’.
For GET requests (the most commonly used HTTP method) there should not be a body. This is because we are simply requesting a resource and any parameters are passed via the URI. However, for other HTTP methods such as POST, the body would be listed after the headers, with a link break in between. In the HTTP response example on the right, you can see an example of this.
The response structure is generally the same. We again have a response line, the headers and a body. One of the main things to point out in the response is the status code. This gives an indication of what happened with the request. ‘200 OK’ is a good sign, it means the request was processed normally. There are other response codes and they can be categorized as follows:
-
- 100-199 are informational responses
- 200-299 are successful responses
- 300-399 are redirect messages
- 400-499 are client errors
- 500-599 are server errors
If you want to dive deep into status codes, you can check out RFC 9110.
HTTP methods
These can also be referred to as HTTP verbs. They indicate the intended actions to be performed on a resource. The most common HTTP methods are GET, POST, PUT, and DELETE. Whilst you probably don’t need to know what every method does off the top of your head, knowing the common ones is important.
-
- GET – retrieve a resource from the server.
- POST – submit a resource to the server.
- PUT – update a resource on the server.
- DELETE – delete a resource on the server.
Other methods to be aware of are HEAD, PATCH, OPTIONS, CONNECT, and TRACE.
Understanding how an application is meant to behave, and which HTTP methods it accepts can sometimes lead us to the discovery of unintended behaviour. For example, when an application accepts and processes HTTP methods that are not intended by the developer.
HTTP Headers
Headers are used to pass additional information with both requests and responses. When we talk about request headers, we mean the headers that are sent in the initial request to the web server from the client. Response headers are the headers that come back. We’ll go through some of the headers you will need to understand as an application security engineer or web application penetration tester.
Content-Security-Policy
Sometimes referred to as ‘CSP’. This helps browsers detect and mitigate some injection attacks such as XSS. When it’s defined, it also needs a policy to go along with it which describes a number of directives to follow. You can find out more about the directives on the mdn web docs.
X-Frame-Options
This tells a browser if it is allowed to render a page in a <frame>, <iframe>, <embed> or <object>. Using X-Frame-Options prevents pages from being loaded into other sites which can prevent click-jacking attacks.
Strict-Transport-Security
Often referred to as ‘HSTS’. This tells the browser that the site should only be accessed over HTTPS. If an HTTP request is made, it should be upgraded to HTTPS automatically.
Here we just highlighted some of the key ones that might flag up from vulnerability scans or other testing. There are of course many headers to consider, so we encourage you to take some time to review each one. Try to thoroughly understand what they do and, perhaps more importantly, what their limitations are.
Cookies
Because HTTP is a stateless protocol (i.e. it doesn’t maintain a session or connection between requests), we need a mechanism for session management. Cookies give us just that, and work in a way that goes something like this:
-
- User authenticates to a web service
- The web server generates a session ID and sends it back to the client as a cookie
- The client sends the session ID back to the server with each subsequent request
In this scenario, any request coming in with that session ID will be treated as though it comes from the user that authenticated.
There are a couple of drawbacks here. If a session ID was predictable or stolen, an attacker could use it to masquerade as the user simply by sending requests and including the session ID as a cookie. This is called session hijacking.
Cookies are made up of a name, value, and expiration date. When they expire, the browser will discard them. You’ll often see them encoded in base64 (though this is not a hard rule, there are many available encoding schemes) and they can be set with flags that dictate how they can be handled.
Cookie flags
There are several flags we need to know as they may come up in casual conversations with developers or other security professionals.
HttpOnly: This prevents JavaScript from accessing the cookie. It’s considered best practice to prevent any client-side scripts from stealing or interacting with the user’s session. Occasionally, you will find applications where it has been forgotten.
Secure: This will ensure that the cookie is only sent over secure connections (HTTPS).
SameSite: This flag dictates when the cookie is sent. There are three options, strict, lax, and none. Strict means the cookie is only sent with requests that originate from the same site as the cookie. Lax means the cookie is only sent with requests that originate from the same site as the cookie, as well as with some cross-site requests, such as GET requests. None means the cookie will be sent with all types of requests, regardless of the origin. It’s worth noting that if the SameSite flag is not set, some browsers will apply a default value.
Expires: This is exactly as it sounds, it tells the browser when to discard the cookie.
Where to go next?
Well HTTP might seem quite simple, but we can’t overstate the importance of understanding the underlying technology when working in the field of Cyber Security. If you want to learn more about web application security and hacking, head on over to the TCM Security Academy.
About the Author: Alex Olsen
Alex is a Web Application Security specialist with experience working across multiple sectors, from single-developer applications all the way up to enterprise web apps with tens of millions of users. He enjoys building applications almost as much as breaking them and has spent many years supporting the shift-left movement by teaching developers, infrastructure engineers, architects, and anyone who would listen about cybersecurity. He created many of the web hacking courses in TCM Security Academy, as well as the PJWT and PWPT certifications.
Alex holds a Master’s Degree in Computing, as well as the PNPT, CEH, and OSCP certifications.
About TCM Security
TCM Security is a veteran-owned, cybersecurity services and education company founded in Charlotte, NC. Our services division has the mission of protecting people, sensitive data, and systems. With decades of combined experience, thousands of hours of practice, and core values from our time in service, we use our skill set to secure your environment. The TCM Security Academy is an educational platform dedicated to providing affordable, top-notch cybersecurity training to our individual students and corporate clients including both self-paced and instructor-led online courses as well as custom training solutions. We also provide several vendor-agnostic, practical hands-on certification exams to ensure proven job-ready skills to prospective employers.
Pentest Services: https://tcmdev.tcmsecurity.com/our-services/
Follow Us: Blog | LinkedIn | YouTube | Twitter | Facebook | Instagram
Contact Us: sales@tcm-sec.com
See How We Can Secure Your Assets
Let’s talk about how TCM Security can solve your cybersecurity needs. Give us a call, send us an e-mail, or fill out the contact form below to get started.