Tutorial #3 - Cyber threat intelligence¶
One of the most important tasks when analyzing the security level of a network is studying any information that can be obtained about this network’s perimeter.
Lampyre comprises a rather big number of tools for performing such kind of tasks: from visualizing the entire network infrastructure to searching and filtering certain information in the obtained data.
In this tutorial, we’ll be using some well-known and publicly accessible tools without scanning for ports or any intervention into the work of the network under analysis.
While working with Lampyre, analysts may work with offline data and they can also download it from different online sources. In this tutorial, we’ll be using the API of such publicly available search engines as "shodan.io", "virustotal.com", "threatcrowd.org", "haveibeenpwned.com" and a tool similar to "TheHarvester" utility from the "Kali Linux" toolbox.
In order to start working in Lampyre you have to create an Investigation or to open a previously created one. Please turn to our previous tutorials if you need any guidance on how to do it. Here we already created an investigation and are good to go.
As an example, in this tutorial we’ll be investigating the "Massachusetts Institute of Technology" - MIT.
As a starting point, we can take either a company’s name or its domain. Let’s begin with the mit.edu domain and see what we can get.
To get the list of subdomains we’ll be using 2 requests at the same time - VirusTotal and ThreatCrowd searches.
We press Ctrl+Y on the keyboard or choose "List of Requests" in the Windows main menu. Then we tick the requests that we need to perform.
These 2 requests require a domain’s name as input parameter. In our case it’s mit.edu. We type it in and click "Execute" to run both requests with one click.
We can monitor the status of our requests in the "Requests" window.
Once they’re done we’ll get 2 results and we’ll have to combine them on a schema. So we select any of our results in the "Requests" window and choose "Schema" in this window’s menu. The System will automatically offer visualization variants available for this request:
Now that we’ve got the results of one of our requests on a schema we have to visualize the results of the other one here as well. To do this we have to click the "Add to active" button first.
Then we select what needs to be visualized. For this request, there are multiple visualization variants available. We’ll choose Sub- and sibling domains.
Thus, we get the mit.edu subdomains on the same schema.
Now we’ll find out what network services are active on the corresponding IP addresses.
"ThreatCrowd" and "VirusTotal" return some IP addresses but not all of them. To get all of the addresses of the sub-domains found let’s resolve the host names to the IP addresses manually.
To do this let’s filter the Host objects in the "Content" window. We double click the Domain object to select these objects on the schema, then we right-click any of the selected vertices and choose the Nslookup request in the context window.
You can also launch this request through the "List of Requests" window.
After our request is complete we’ll get a list of IP addresses which correspond to the hosts. Now let’s obtain data on the active services of the found IP addresses with the help of "Shodan.io".
We select all IP-addresses on our schema (the same way we selected all domains) and then launch the Shodan search: right-click any of the selected objects and then choose Shodan - organisations. This way, apart from the network infrastructure, we'll get the list of organizations which own these IP-addresses.
At this moment you can go and make yourself a cup of tea or a coffee as each IP requires a separate request and Shodan limits the number of these requests to just one per second.
Once our request is complete we get a detailed layout of the network which includes not only domains and their IP addresses (and the companies these IPs belong to), but also all services which were found on these IPs with their names, versions, ports, etc.
The "Content" window provides us with additional functionality. For example, it allows us to search the results of our requests and also to group the objects of the schema by different attributes.
For example, let’s see what companies were found by our requests. In the object attributes section let’s find Organization name and click the "Expand" button to expand the list. We’ll pick the "MIT Laboratory for Nuclear Science" as an example and look into it more closely.
This entity was found in the process of obtaining data by IP addresses. We’ll use it to demonstrate how to make a request to Shodan having only a company name.
We could have begun our investigation at this stage – from a company’s name and not a domain.
So let’s proceed with this company that we chose and double click it in the "Content" window to locate the corresponding object on our schema.
Then we’ll copy it onto a new schema by pressing Ctrl+N on the keyboard or by right-clicking it and choosing "Copy onto new schema" from the context menu.
Once we did that we right-click the object and launch the Shodan - organisations request once again. This request comprises 5 sub-requests. Once it is complete we get the layout of IP-addresses, hosts and services of only this company.
We click in the menu to the left of our schema, to increase the size of the vertices with bigger number of links.
In order to analyze the obtained network infrastructure thoroughly we need to combine the results of our previous request for the mit.edu domain with the results of this Search by company name request.
To avoid being distracted by the company objects and by the central domain mit.edu, we’ll select both our last Shodan request results in the "Requests" window and create a new Infrastructure schema by selecting it in the drop-down list of the corresponding menu in the upper part of the "Requests" window.
On this schema, we’ll see which IPs have more services and see what services are running and where exactly.
All data obtained by our requests is contained in the "Content" window where we can analyze its attribute statistics, like Product, Port, CPE and so on.
We can also search this data to find the services we are interested in using the search field in the upper part of the "Content" window.
We’ll be searching all objects and links’ attributes, including the Data attribute which contains the tcp service banner. If needed we can also search for any specific text, contained on html pages. For example, let’s search by the following key words: ftp, remote, printer, mysql, netbios.
Searching for 230 login will show us all FTPs which can be accessed anonymously.
Searching for Mysql will find databases. We can select some specific database and view it on our schema right away.
We can also establish what software is more popular on the Net. To do this we should create a Software BreakDown schema for the request results that we already have by selecting the appropriate option in the "Schema" drop-down list of the "Requests" window menu.
Then we match the size of the vertices to the number of their links, like we did before, by clicking the button in the left menu of the schema
and change the layout of our graph to hierarchical by clicking and choosing "Hierarchical" in the drop-down list.
This is the result that we get:
What we got here is not only what versions of services are running on what nodes but also which versions are the most popular. In the "Content" window we can also take a look at the CPE statistics for services.
On the Software BreakDown schema by right-clicking a certain service and choosing "Select neighbors" in the context menu, we can select all IP addresses which use the same version of this service.
The same way we can take a look at the services’ ports, by creating a Ports Breakdown schema and changing the size of the vertices and the layout.
Many Services objects contain location data. We can visualize these objects on a map by selecting the appropriate request results and clicking "GIS map" and then - Infrastructure.
After performing these actions, the user will get a map with all the objects, which have geo reference.
To view the captions to our objects we select them on the map, right click any of them and choose "Show captions". The captions contain information on the addresses and ports and also on the names and versions of the software.
If needed we can switch into the heat map mode in the "Explorer" window. For this we open the "Explorer" window in the "Windows" main menu and tick the corresponding option in the Gis-map tree-list.
This is what we get:
As you see we have acquired a lot of information on the services of the mit.edu and MIT LNS subdomains. But in your own research you can go even further and get a broader picture by searching by the company's name. In our case here it would be "Massachusetts Institute of Technology". And if you continued here you'd start with the corresponding Company object on our first schema and find even more addresses this way or another way connected to this company.
If we talk about informational security, getting data only on the network infrastructure is not enough. Information security requires not only technical solutions, as people may also become a vulnerability. In the course of the following research, we'll collect some MIT company e-mail addresses from open sources and we’ll use it as initial data for obtaining additional info on the people working there.
Let’s copy the mit.edu host from any of our schemas and into a new one by selecting it and pressing Ctrl+N on the keyboard or by simply right-clicking it and choosing "Copy onto new schema". On the new schema, we right click our object again and launch the Harvest emails request.
We’ll get these results:
Let’s select all the E-mail objects (by the object type in the "Content" window or by right-clicking our HOST and selecting all its neighbors in the context menu).
Then we right click any of them and choose "To requests". In this list we tick what we need to be performed (see the screenshot below) and click "Execute", thus searching for accounts connected to the e-mail addresses selected.
After our requests are done, we can visualize the results (one by one for now) on a new schema:
In addition to the accounts found, we can check if these e-mails are contained in the "HaveIBeenPwned.com" database of the biggest password leaks. We select all e-mails on our schema, right click any of them and choose haveibeenpwned request in the context menu. When it’s done we change the layout to hierarchical:
What we got to our schema are object leaks and objects, which show on what text exchange websites (for example, "Pastebin" or other) these emails are shared. This information will also show us where these e-mails were registered with.
Each link - between any e-mail address and the data leak - contains information on which data was leaked and when. This can also be analyzed in the "Content" window.
If we match the size of the vertices on our graphs to the number of their links we’ll see whose passwords were leaked more often.
Applying the same scenario when analyzing your own company network will help you locate the weak spots, which require your security assistance.
And Lampyre will certainly be of great help for you here.