When testing web applications, the understanding and use of various encoding schemes is a fundamental skill. In particular, we often see Base64, URL encoding, and HTML encoding used across many applications both as part of the application’s general functionality and offensively by pentesters or attackers.
Encoding and Decoding Schemes
Encoding is a process that transforms data from one form to another according to a specified scheme, whereas decoding reverts this process, returning the data to its original form. These schemes are designed to ensure that data can be safely transmitted, stored, and interpreted by different systems.
Base64 Encoding
This binary-to-text encoding scheme represents binary data in an ASCII string format by translating it into a radix-64 representation. It’s frequently used to transmit data over media that are designed to handle text.
| cheesecake | Y2hlZXNlY2FrZQ== |
URL Encoding
Also known as Percent-encoding, URL encoding is used to represent unreserved characters and reserved characters in URLs, which do not allow certain characters. It replaces unsafe ASCII characters with a “%” followed by two hexadecimal digits.
| tiramasu | %74%69%72%61%6d%61%73%75 |
HTML Encoding
HTML encoding is used to convert characters that are not allowed in HTML into entities that can be used in HTML. The primary use of HTML encoding is to protect the webpage from certain types of attacks, such as Cross-site Scripting (XSS).
| macaroon | macaroon |
How to Use Encoding During a Pentest
Base64 Encoding
It can be used for obfuscating payloads and bypassing input validation filters. For instance, instead of directly using <script>prompt(‘XSS’)</script> in an XSS attack, you can encode it using Base64, making it more likely to bypass security filters. To execute payloads like this we can use functions such as atob().
<img onload="eval(atob('cHJvbXB0KCdwYW5jYWtlcycp'))">
URL Encoding
This is often employed when testing for injection flaws, including SQL Injection and Command Injection. Special characters in a malicious payload can be URL encoded to evade security measures. For instance, the payload ‘; DROP TABLE users; — could be URL encoded to %27%3B%20DROP%20TABLE%20users%3B%20– to bypass security filters.
HTML Encoding
HTML encoding can be used in scenarios where HTML tags might be filtered out to prevent XSS attacks. For example, if you’re testing for an XSS vulnerability and find that the < and > tags are being filtered, you could use HTML encoding to encode these characters as < and >, respectively. So, an XSS payload like <script>alert(‘XSS’)</script> becomes <script>alert(‘XSS’)</script>.
Unearthing Sensitive Data
Sensitive data might include personal user information, passwords, API keys, or proprietary application details. While encoding does obfuscate this data, it doesn’t provide robust protection because it can be easily reversed.
There are many places we can look for this kind of information:
- HTTP headers
- URL parameters
- Source code
- JS files
- API responses
- Cookies
- Hidden form fields
- Server logs
- Metadata
Wrapping Up
Understanding encoding and decoding schemes and their applications is critical. They can often be the key to unlocking vulnerabilities that might otherwise remain hidden. Aside from this, we should always be on the lookout for the misuse of encoding schemes as a way of obfuscating sensitive information.
About the Author: Alex Olsen
Alex is a Web Application Security specialist with experience working across multiple sectors, from single-developer applications all the way up to enterprise web apps with tens of millions of users. He enjoys building applications almost as much as breaking them and has spent many years supporting the shift-left movement by teaching developers, infrastructure engineers, architects, and anyone who would listen about cybersecurity. He created many of the web hacking courses in TCM Security Academy, as well as the PJWT and PWPT certifications.
Alex holds a Master’s Degree in Computing, as well as the PNPT, CEH, and OSCP certifications.
About TCM Security
TCM Security is a veteran-owned, cybersecurity services and education company founded in Charlotte, NC. Our services division has the mission of protecting people, sensitive data, and systems. With decades of combined experience, thousands of hours of practice, and core values from our time in service, we use our skill set to secure your environment. The TCM Security Academy is an educational platform dedicated to providing affordable, top-notch cybersecurity training to our individual students and corporate clients including both self-paced and instructor-led online courses as well as custom training solutions. We also provide several vendor-agnostic, practical hands-on certification exams to ensure proven job-ready skills to prospective employers.
Pentest Services: https://tcmdev.tcmsecurity.com/our-services/
Follow Us: Blog | LinkedIn | YouTube | Twitter | Facebook | Instagram
Contact Us: sales@tcm-sec.com
See How We Can Secure Your Assets
Let’s talk about how TCM Security can solve your cybersecurity needs. Give us a call, send us an e-mail, or fill out the contact form below to get started.