How to test proxies using bash and cURL with multi-threaded async requests
Introduction
If you’ve recently found some public proxies or even paid proxies you may want to test to see if they’re working correctly.
The quickest and easiest way to do this is with using the cURL cli binary on OSX (or any linux distro).
With this we can easily test HTTP, HTTPS, SOCKS4, SOCKS4a & SOCKS5 proxy protocols using just a few commands.
In this tutorial I will assume you have access to a linux terminal or OSX terminal.
Step 1 – Set your proxies
Copy all of your proxies into a file, in this case we will paste them all into a file called “usa.txt”
Product | Advantage | Link |
---|---|---|
Residential proxies | Never get blocked, choose your location | |
Datacenter proxies | Super fast and reliable | |
3 Day Trial | Test all products to find the best fit |
Example:
209.205.211.251:2000 209.205.211.251:2001 209.205.211.251:2002 209.205.211.251:2003 209.205.211.251:2004 209.205.211.251:2005 209.205.211.251:2006 209.205.211.251:2007 209.205.211.251:2008 209.205.211.251:2009 209.205.211.251:2010 209.205.211.251:2011 209.205.211.251:2012 209.205.211.251:2013 209.205.211.251:2014 209.205.211.251:2015 209.205.211.251:2016 209.205.211.251:2017 209.205.211.251:2018 209.205.211.251:2019
Step 2 – Write our bash / cURL script
Using your favourite text editor (vim/nano/etc) create your shell script file in this case we will call it test_usa.sh
Here are the seperate scripts for each protocol
HTTP protocol:
for proxy in `cat usa.txt` do (curl -s --max-time 10 -x $proxy -U YOURUSER:YOURPASS ip-api.com/json;echo) & done | grep country | wc -l
HTTPS Protocols
for proxy in `cat usa.txt` do (curl -s --max-time 10 -x https://$proxy -U YOURUSER:YOURPASS ip-api.com/json;echo) & done | grep country | wc -l
SOCKS4 Protocol
for proxy in `cat usa.txt` do (curl -s --max-time 10 --socks4 $proxy ip-api.com/json;echo) & done | grep country | wc -l
SOCKS4a Protocol
for proxy in `cat usa.txt` do (curl -s --max-time 10 --socks4a $proxy -U YOURUSER:YOURPASS ip-api.com/json;echo) & done | grep country | wc -l
SOCKS5 Protocol
for proxy in `cat usa.txt` do (curl -s --max-time 10 --socks5 $proxy -U YOURUSER:YOURPASS ip-api.com/json;echo) & done | grep country | wc -l
There is a max execution time of 10 seconds to prevent the proxy from never timing out.
Once it is completed it will output a number into your console and this is the number of successful requests it was able to complete
Example output:
sbook:~$ bash test_usa.sh 19
Finished!
That was pretty easy, right?
It wouldn’t be hard to create a much more advanced script that takes parameters and all that jazz but this is a simple, easy solution that most people can understand more importantly simply copy + paste to get going.
How it works (if you’re new to bash shell scripting)
Line 1:
This is a for loop and every new line it reads it stores it in the variable $proxy
What is cat
? This command reads a file and outputs it into your terminal
for proxy in `cat usa.txt`
Line 2:
This is the syntax we need to create the for loop (bash is really quite like reading english)
do
Line 3:
This executes the cURL command line with our $proxy (the line we just read)
-s
Parameter is setting silent mode otherwise we get a bunch of ambigious text on our screen from the multi-threaded approach
-U YOURUSER:YOURPASS
is the authentication process
--max-time 10
This sets the max time for our request to load the page ip-api.com/json
The &
symbol sends our command into the background and moves onto the next part of the loop, thus creating our async / multi-threaded approach
(curl -s --max-time 10 -x $proxy -U YOURUSER:YOURPASS ip-api.com/json;echo) &
Line 4:
Now we finish the loop with done
just like we started with do
grep country
then echos/prints only if we find the text string “country” from the page we loaded (only successful page loads would be counted)
wc -l
This command counts how many of these instances we find of the above
This is probably not the most in depth explanation you’ll find on these single commands but hopefully it gives you a better understanding. If you genuinely want a further explanation of these commands or want to ask me a question just write below in the comments and I’ll see how I can assist 🙂
3 Day Trial of All Proxy Products
We want you to find the perfect Proxy Product for your purposes. So we have introduced this new trial product that gives you access to all our products. 3 Days for $13.95, Learn more
Product | Advantage | Link |
---|---|---|
Residential proxies | Never get blocked, choose your location | |
Datacenter proxies | Super fast and reliable | |
3 Day Trial | Test all products to find the best fit |