Greetings dear readers, how are you?! May the god of hacking be present in your lives!
All kidding aside, through this article, which will actually be a 3-part series, I’ll be covering the topic of reconnaissance and information gathering. As the theme is very extensive and deep and to avoid being dull, I decided to separate it.
In this first article, we will talk about the concept of reconnaissance, information gathering, and open-source intelligence.
«SPOILER ALERT » Stay tuned to our series of articles because we will be publishing a contribution to the notorious tool ReconFTW. «SPOILER ALERT »
Perimeter reconnaissance and open-source intelligence
Why and how is information collected from the perimeter of a company? As the Chinese proverb says “To cut down a tree quickly, spend twice as much time sharpening your axe”. Bringing it to our interpretation: “Get the most information the better will be the result of your attack”.
This article will cover the fundamentals of a perimeter enumeration, moving on to the notorious tool “ReconFTW” that automates the entire process, and will conclude by detailing the author’s contribution to the project, which is open-source and was conceived by “Six2dez”.
The subject of red team operations comes from the military environment which represents a systematic way of making critical and contrary thinking part of the strategic planning process of any organization. It was developed by US military and intelligence agencies to overcome cognitive bias and groupthink, force decision-makers to challenge their assumptions, and prevent the “failures of imagination” that led to the 9/11 terrorist attacks and wars and disasters that followed.
As good operators, we have to be able to identify and construct the chain of vulnerabilities. For that, the collection of information and intelligence is essential to have precision in our actions.
Explaining the above infographic, when an operation is started, the crucial point is the collection of information. The focus is on everything pertinent to the company, that is: leaked files/information, public data on the internet, third-party services, technologies, hosts, applications, equipment, and last but not least people (employees).
Entering the part where things start to get interesting, we will address the topics related to the themes presented, explaining what it is and how we do it to achieve the objective.
The subjects that will be addressed are:
- Open-source intelligence (information published on the internet)
- Collection of subdomains
- Host identification
- Verification of WEB applications
- Checking for vulnerabilities automatically (bonus)
Open-source intelligence (OSINT):
OSINT (open-source intelligence or open-source information) is the term used to describe information obtained from sources available to the general public, such as newspapers, scientific journals, and the internet.
These are the sources that have the most valuable information which we can leverage to crawl our robots to identify and collect data to assist in our invasion.
We will talk about:
- Domain information (WHOIS and Amass): “Domain Names” are a fundamental part of the Internet’s infrastructure. They provide a readable address for any web server available on the Internet. Any computer connected to the Internet can be reached via a public IP address, consisting of 32 bits IPv4 (they are usually written as four groups of three numbers between 0 and 255, separated by dots (e.g., 173,194,121,32) or consisting of 128 bits IPv6 (they are usually written as eight groups of 4 hexadecimal numbers, separated by colons, eg. 2027:0da8:8b73:0000:0000:8a2e:0370:1337). Computers can handle these addresses easily, but people have a hard time figuring out who is running the server or what service the site offers. IP addresses are hard to remember and can change over time and a site can even have multiple of them. To solve all these problems we use human-readable addresses called domain names.
Whois is a query and response protocol that provides information about existing domain names and their pertinent data.
Amass performs network mapping of attack surfaces and external asset discovery using open-source information gathering and active reconnaissance techniques. Already does this and other features.
- Information gathering through Search Engines/Social Media (theHarvester and EmailFinder): It consists of searching public information like email addresses, usernames, hosts, links, and other things that are public on the internet. This search takes place through services such as anubis, baidu, bevigil, binaryedge, bing, bingapi, bufferoverun, censys, certspotter, crtsh, dnsdumpster, duckduckgo, fullhunt, github-code, hackertarget, hunter, intelx, otx, pentesttools, projectdiscovery, qwant, rapiddns, rocketreach, securityTrails, sublist3r, threatcrowd, threatminer, urlscan, virustotal, yahoo, and zoomeye. The important thing is to use these search engines as much as possible to find these values.
theHarvester is a powerful tool designed to be used during the reconnaissance stage of a red team assessment or penetration test. It performs open source intelligence (OSINT) gathering to help determine a domain’s external threat landscape. The tool gathers names, emails, IPs, subdomains, and URLs by using multiple public resources.
EmailFinder is designed to find company emails that are filtered by search engines. This is done by searching for @company.com. The goal has to be to have the minimum amount of emails in the search engines.
- Password Leak (H8mail): Password leaks can help us understand the pattern of passwords used by a user or even reuse them. This search is carried out through database services that verify whether the e-mail used is part of a leak.
H8mail is an email OSINT and breach hunting tool using different breach and reconnaissance services, or local breaches such as Troy Hunt’s “Collection1” and the infamous “Breach Compilation” torrent.
- Metadata finder (MetaFinder): Metadata is data that describes other data. The prefix meta means “an underlying definition or description”. Therefore, metadata describes whatever piece of data it is connected to. Since metadata summarizes basic information about the data, such as author type, creation date, usage, file size, and more, metadata is crucial for the efficiency of information systems to classify and categorize data. Additionally, metadata information helps IT systems discover what users are looking for.
MetaFinder searches for documents in a domain through Search Engines. The objective is to extract metadata.
- Google Dorks (dorks_hunter): Google Dorks is a technique used to optimize Google searches using its resources to dig deeper in indexing or even raise sensitive information about companies and people. Also, it’s a great tool to find more specific things within websites and URLs like pdfs, documents, etc.
- Github Dorks (Gitdorks_go): It works in a similar way as Google Dorks, however, operating on the GitHub platform.
Gitdorks_go is a tool that utilizes the GitHub Search API and an extensive list of GitHub dorks that are compiled from various sources to provide an overview of sensitive information stored on GitHub given a search query. The Primary purpose of Git Dork is to provide the user with a clean and tailored attack surface to begin harvesting sensitive information on GitHub. GitDorker can be used with additional tools such as GitRob or Trufflehog on interesting repositories or users discovered from GitDorker to produce the best results.
- GitHub org analysis (Enumerepo) and (Trufflehog): Enumeration of an organization on the GitHub platform. Verification of passwords, keys, and sensitive data that were hosted publicly and erroneously by the developers.
Enumerepo List all public repositories for (valid) GitHub usernames.
Trufflehog Find leaked credentials.
This concludes the first session of our series based on reconnaissance and information gathering. We recommend that the subjects presented be studied in depth and other tools that have the same purpose be tested, with that our mindset will be expanded.