Releasing Flumberbuckets: S3 Bucket Enumeration Tool for Bug Hunters

Flumberbuckets is a part of suite of scripts that I'll be open-sourcing on GitHub in flumberboozle repository, scripts in this repository are supposed to aid bug hunters in hunting, automating workflows, etc. There is also another script in flumberboozle repository which is a portscanner with scan auto-save, auto-startup function called portboozle.

What is flumberbuckets?

Flumberbuckets is is yet another S3 bucket enumeration tool which you can choose to use while hunting on bug bounty programs or during security assessment. I designed this tool to serve my purposes and now I am open-sourcing it, there are several different tools that exist for people with different tastes. The aim of this tools was to present S3 bucket enumeration results in better format which is visually more appealing than scrolling through output of a bash script that just runs  aws s3 ls in a loop.

How does it work?

Flumberbuckets is a really simple script which combines the best of existing S3 bucket enumeration tools to make S3 bucket enumeration simpler and faster. It achieves this by using DNS resolution for sorting out non existent buckets and runs 10 tests on each bucket, several buckets are checked simultaneously to save time.

But why?

• I actually got my first bounty by finding misconfigured S3 bucket so I decided to write a tool for doing it on larger scale, this script is outcome of that effort.
• Tools I used previously weren't so good at presenting results of enumeration visually, it was a pain to scroll through output of those
• Tools I used previously had smaller word-lists and checked for less functions, I suspect that I missed a few easy bounties because of that.
• Tools I used previously were painfully slow they were bash scripts, they weren't multi-threaded they would sort out nonexistent buckets with HTTP requests, flumberbuckets was an attempt to fix all the problems I encountered while using other scripts.

Is it really worth switching? 

Flumberbuckets in action

You may want to switch over for any of the following reasons

• Faster sorting out nonexistent buckets with DNS(massdns) resolution rather than HTTP
• Multi-threaded enumeration of existent buckets to find misconfiguration
• Cleaner visual output, you can select if you want to just see vulnerable buckets or see all the buckets that exist even though they are not vulnerable. You can save output with -o & --no-colour option 
• Several vulnerability tests available like LIST, ACL , POLICY , CORS , REPLICATION , WEBSITE , LOCATION , LOGGING , UPLOAD , DELETE. 
• You can decide what tests you want to run on a buckets run all if you want or run only a few if you're in hurry 
• I also included DELETE test It's the most overlooked by S3 bucket enumeration tools
• Much more functionality and options compared to other tools
• Enumerating a list of custom buckets, single bucket, enumerating all buckets with a keyword in their name
• Option to accept list of FQDNs or domains for enumeration like assets.example.com can be found by sublister and then you can make a list of subdomains you found and feed it into flumberbuckets, It maybe that company is using S3 bucket with the same name as domain name so it'll check if the bucket "assets.example.com" is vulnerable or not
• Just try it, 200 lines of code delivers a lot of functionality here 

Note: The performance of flumberbuckets is subject to your connection bandwidth speed and hardware

Prerequisites & Installation

Please satisfy dependencies in the order given below

1. Have Linux distribution with Python 3.6+
2. Have massdns installed and configured
3. Have aws-cli installed and configured
4. Python's botocore package which is usually installed as dependency for aws-cli so just check if it's there by running following command if you're Arch Linux user. Mostly you won't have to install this separately, it'll be installed as dependency for aws-cli package 

Getting massdns ready

Compile the massdns binary by referring to instruction on their GitHub try using the massdns version build around the time that this article was published if you run into any issues.

$ massdns/bin/massdns -r <(printf 8.8.8.8) -q -o S <(printf example.com) 

Output should look something like this if you have successfully installed massdns

example.com. A 93.184.216.34

Add mdns_path as variable on last line in your ~/.bashrc

export mdns_path="/ABSOLUTE_PATH_TO_BINARY/massdns/bin/massdns"

Getting aws-cli ready

Running following command for Arch Linux users will install aws-cli.

$ sudo pacman -S aws-cli

 For non Arch Linux users like Ubuntu, Kali Linux you may refer to this awesome article for help which shows serveral ways of installing aws-cli

To check if you installed correctly run

$ aws s3 ls s3://flaws.cloud

2017-03-14 08:30:38       2575 hint1.html
2017-03-03 09:35:17       1707 hint2.html
2017-03-03 09:35:11       1101 hint3.html
2018-07-10 22:17:16       3082 index.html
2018-07-10 22:17:16      15979 logo.png
2017-02-27 07:29:28         46 robots.txt
2017-02-27 07:29:30       1051 secret-dd02c7c.html

If output is something like this then you've done it correctly.

Now that you're done with dependency hell now you can move ahead to actual flumberbuckets installation

Flumberbuckets Installation

1. To start using the script just clone flumberboozle git repository in a directory
2. Move into the directory in which flumberbuckets.py exists by using cd command 
3. Start running commands in the "Detailed Usage" section

Running into Issues?

Facing any issues with installation or running it? 

Raise an issue on GitHub and ping me on Twitter, so that everyone can get benefit from your query.

For resolving common issues, just look at the issues already filed by existing users.

1. FIXED: TypeError: unexpected keyword capture_output (Error faced by Python 3.6 users) 

Do remember to submit 

1. Error screenshot
2. Commands that you've trouble with
3. Error output
4. Name linux distribution
5. Output of following command

$ aws --version

Before filing a bug please make sure you're using the commands mentioned in the blog post & just google the error out before submitting most of the time that's enough ☺

And of course pull requests are welcome ☺

Word of caution

• massdns may cause some temporary blockage of your network if you're using big list
• You should not put your primary DNS resolver's IP in the resolver file as you may get blacklisted
• I'm not responsible for anything you do with this script, hopefully you will not try to find evidence of aliens in area 51 with it, use it for good causes
• Some have said that using aws-cli S3 API cost money, while creating AWS API key you'll be asked for credit card, I just skipped that screen and changed the URL and finally got my AWS API Key without handing over any credit card information, I haven't been charged even though I have run thousand tests by now

Detailed Usage 

Commands that you'll often use

To get detailed help

$ python3 flumberbuckets.py -h

 usage: ./flumberbuckets.py [options] -i [bucket] [-h] (-w WORDLIST | -s) [-i INPUT] [-t THREADS] [-o OUTPUT] [-d DOMAINLIST] [-m MDNS_PATH]
                                                 [--resolve RESOLVER_PATH] [-n] [--no-colour] [-p] [-u] [-r] [--acl] [--policy] [--cors]
                                                 [--replication] [--website] [--location] [--logging] [-e]
optional arguments:
  -h, --help            show this help message and exit
  -w WORDLIST, --wordlist WORDLIST
                        location of wordlist from which permutations of keyword will be generated
  -s, --single          check a single bucket only
  -i INPUT, --input INPUT
                        specify keyword or bucket name, supply - as argument to take input from stdin
  -t THREADS, --threads THREADS
                        specify number of threads to be used for enumeration of existent buckets default is 150
  -o OUTPUT, --output OUTPUT
                        location to save output
  -d DOMAINLIST, --fqdn DOMAINLIST
                        specify list of FQDNs to search for buckets having same name as FQDN
  -m MDNS_PATH          specify path to massdns binary
  --resolve RESOLVER_PATH
                        specify path to resolvers file
  -n                    do not print banner
  --no-colour           output is colourless
  -p, --print-everything
                        print bucket information even if it is not vulnerable
  -u, --upload          perform file upload test. default=False
  -r, --remove          remove file from bucket after uploading it. default=False
  --acl                 perform ACL test
  --policy              perform policy test
  --cors                perform CORS configuration test
  --replication         perform replication configuration test
  --website             perform website configuration test
  --location            perform location test
  --logging             perform logging test
  -e, --everything      view all bucket configuration. default=True

To check a specific bucket -i has to be used to specify input

$ python3 flumberbuckets.py -si flaws.cloud

Pass 2 or 3 buckets as input for enumeration 

$ echo "flaws.cloud\nabcxyz" | python3 flumberbuckets.py -si -

Discard output on stderr

$ python3 flumberbuckets.py -si flaws.cloud 2> /dev/null

It'll will prevent this line from appearing

[+] Testing flaws.cloud bucket, meanwhile you do Google & GitHub dorking for buckets

Find several buckets by specifying keywords, like "flaws", "neo"  -i has to be used to specify input -m is the path to massdns binary you can store it as a bash variable

$ python3 flumberbuckets.py -m $mdns_path -w medium.txt -i flaws

After you press enter it'll say it's sorting buckets, in the background massdns is running and it'll take like 1 or 2 minutes depending on your internet speed to complete the enumeration, be warned big.txt which is included in repository is a comprehensive list and can exhaust your data limit if you try running this on metered network

$ python3 flumberbuckets.py -m /home/fellchase/massdns/bin/massdns -w medium.txt -i flaws

Whenever you supply custom path for mdns_path variable be sure to mention the full absolute path to the binary not the relative path, otherwise it won't work.

Include a list of FQDNs you found in sub domain enumeration for S3 bucket enumeration

Sometimes buckets maybe named like FQDNs like assets.example.com that looks like a sub-domain of example.com but it may also be a name of S3 bucket so when you come across list of sub-domains feed that list into flumberbuckets.py option -d will allow you to do it.
The list should be structured like

www.example.com
assets.example.com

$ python3 flumberbuckets.py -d list_of_sub_domains -w medium.txt -i flaws

To save output for future use

$ python3 flumberbuckets.py -m $mdns_path -w medium.txt -i neo -o ~/Desktop/flumberresult --no-colour

-u               Upload Test
-r               Remove the default uploaded file (BugBounty-flumber.txt)
--acl            perform ACL test
--policy         perform policy test
--cors           perform CORS configuration
--replication    perform replication configuration test
--website        perform website configuration
--location       perform location test
--logging        perform logging test
-e               If you use -e option it will make flumberbuckets to perform all these tests if you specifically request one test then other won't be performed by default.

What do those colours mean?

Grey means bucket is secure with regards to that test

Red means that test was not performed at the request of user

Green means the bucket is vulnerable 

Once you see green colour you should run the same test with aws cli tool and further investigate into the issue if the files found are sensitive does the bucket really belong to company, the task of flumberbuckets is just to make you aware of faults, investigation is bug hunters duty.

Credits

Flumberbuckets is inspired from following scripts, it combines all the good things in the following tools.

• mass3
• s3enum
• S3Scanner

Support the Project

Share your story with me! ☺

If you earned a bounty through use of this script do share the story with me I'd be happy to hear that my script was of use to you. You can contact me over twitter @fellchase

Wanna support monetarily 💰?

If you want to thank me monetarily or want to donate to this project you can do so on paypal.me/fellchase  I'll be happy to hear your bug bounty story if you got any bounty with this script.