This is how you can bypass any IP based rate limiting defenses in your red teaming/pentesting engagements: An AWS Free Tier account and an API Gateway HTTP Proxy
Imagine you are performing a penetration testing and need to execute some kind of high traffic action such as, for example, a directory/file discovery on a web server. Sometimes this type of activity is not possible due to rate limiting defenses implemented, that is, you have some kind of protection that detects high traffic coming from a single IP and blocks this source from requesting resources from the web server for a period of time or, worst, puts the IP in a blacklist for good.
Here is a very simple diagram of the situation:
In this post we’ll see a tool that was recently introduced to me and we’ll see exactly what it does. But we’ll not dive into the tool. We’ll instead manually apply everything that it automates for us and why it bypasses the rate limiting protection. This is actually a post about one of the resources of the AWS API Gateway service.
On a recent engagement I decided to use a tool called fireprox, introduced to me by my good friend Raphael Mota in order to bypass this protection on the target server. Of course, all the credits for the tool and the use of the API Gateway Proxy resource for this propouse are in the github page of the tool. I just learned and decided to write a post about it.
What this tool does is to automate the process of creating an pass-through HTTP Proxy using AWS API Gateway service. The magic is that every request coming from this proxy has an unique IP address. You read it right! An unique IP address for every request! Well, that’s the solution to our situation. Let’s see this magic happening, but I’ll not cover how to create an AWS free tier account so the post doesn’t get too long.
The first thing we need to do is create a new user in the “Identity and Access Management (IAM)” AWS service:
Let’s give this user a name and mark the “Programmatic access” checkbox:
Now let’s create a new group for this user. This is mandatory for the tool to work:
Now we have to name the group and add the policy “AmazonAPIGatewayAdministrator” to it. This policy is needed so we can use this user’s credentials in the tool:
Check if everything is correct and create the new user:
Now write down the ACCESS KEY and the SECRET ACCESS KEY because we’ll use them in our tool:
Now that we have a user created, we can use the tool. After cloning the repository with git clone https://github.com/ustayready/fireprox and installing the dependencies with python3 -m pip install -r requirements.txt, we can see what it does:
So, we can issue commands using the tool and we have to pass the user’s acecss keys:
So, we don’t have any API’s yet. Let’s create one. But first, we need a target. This target is a service where the proxy will send all the requests to. I’ll use a VPS of mine hosted by DigitalOcean. It will be a simple python web server:
Now we can create a proxy using fireprox that points to our web server:
So now, we can simply curl our newly created proxy and it will forward the request to our web server:
Looking the web server log we see the request. Note the source IP address:
We can see that this is not my real IP address:
In fact, let’s simulate a directory listing attack and see what happens:
See? One single IP address for every request! That’s awesome!!
However, as a very famous saying in my country goes: not everything in life are flowers. Let’s use another python webserver and issue a request again:
Ah-há! There’s a catch. The real source IP address is put in the header “X-Forwarded-For” and it arrives at the target. So if the web server or web application running on this server looks for this header, we’re done.
To fix this, we can use a very neat resource provided to us by the API Gateway: a header mapping. Thanks to Fred Reimer for this awesome discovery!
Turns out that we can send a custom header with our request, because fireprox creates a mapping from the header “X-My-X-Forwarded-For” to “X-Forwarded-For”, so we can control its content. Let’s see:
Now, all we have to do is send one random IP address using this custom header and we’re done!
Now you must be asking yourself what’s in this “server.py” file that allows me to see the header. Let’s take a quick look:
This server simply uses the custom class “MyHandler” as base for actions based on the methods of the requests it gets. In this example, I only have actions for GET requests, jsut to make thing simple. we see that all it does is get the header i want and print it on the screen. The function “connection” serves the service and waits for incoming connections.
Now that we know how the tool works, let’s see what it does with our AWS credentials. Let’s go to the API Gateway service and see what’s there:
As we can see above, fireprox creates a simple API structure. Let’s see the configs for this GET method:
We can see a very clear structure: on the left, the client, in our case the curl command. On the far right, the target application on port 8585. In between, the proxy.
Now, notice that we have the “Method Request” and the “Integration Request” boxes. The first one is the request that comes from the client. See the “X-My-X-Forwarded-For” header? This box knows that this header will come within the request.
Also, we have the header “X-Forwarded-For” in the “Integration Request” box, that represents the request going from the proxy to the final destination. Let’s see how this header mapping works:
Clicking on the “Integration Request” box, we can see its details, which include HTTP headers. See that we have one entry for that and there’s a mapping going on: The contents of the “X-Forwarded-For” header comes from the contents of the header “X-My-X-Forwarded-For” header from the client request. That’s how we can change the header contents.
Now you must be wondering: but what if we simply delete this header entry? Let’s see:
Issuing another request:
See? Somehow the behavior is still the same. My real IP address continues to leak through the “X-Forwarded-For” header. Somehow we still have the mapping in place even after deleting the header. And if we also delete the custom header from the “Method Request” box, the result is the same. So we must use this custom header if we want to prevent our IP from leaking in the final request to the target application.
To conclude our journey, I’ll run a final attack simulation, but this time I’ll also simulate a server that permanently blocks IP addresses that send requests faster than 10 per second.
First, let’s create a very simple Python script that runs with threads and sends requests to our server:
Python script for simulating sending requests using multi threading
- The function random_ip() just returns a random valid IP address;
- The function send_request() sends as many requests as the number passed in command line to the option “–requests”. It’s called once for every thread;
- All threads are spawned, each one calling the function send_request() with the arguments url, requests and delay.
Now the target web server:
It’s needless to say that this is an amazing tool. And because I’m a little bit lazy, I’ll just paste the print for the credits:
As a final thought, please, don’t use this method in places you don’t have permission to. You can violate the AWS policy and get yourself in big trouble.