Tutorial #3 - Cyber threat intelligence

One of the most important tasks when analyzing the security level of a network is studying any information that can be obtained about this network’s perimeter.

Lampyre comprises a rather big number of tools for performing such kind of tasks: from visualizing the entire network infrastructure to searching and filtering certain information in the obtained data.

In this tutorial, we’ll be using some well-known and publicly accessible tools without scanning for ports or any intervention into the work of the network under analysis.

While working with Lampyre, analysts may work with offline data and they can also download it from different online sources. In this tutorial, we’ll be using the API of such publicly available search engines as "shodan.io", "virustotal.com", "threatcrowd.org", "haveibeenpwned.com" and a tool similar to "TheHarvester" utility from the "Kali Linux" toolbox.

In order to start working in Lampyre you have to create an Investigation or to open a previously created one. Please turn to our previous tutorials if you need any guidance on how to do it. Here we already created an investigation and are good to go.

Figure 1 Figure 2

As an example, in this tutorial we’ll be investigating the "Massachusetts Institute of Technology" - MIT.

As a starting point, we can take either a company’s name or its domain. Let’s begin with the mit.edu domain and see what we can get.

To get the list of subdomains we’ll be using 2 requests at the same time - VirusTotal and ThreatCrowd searches.

We press Ctrl+Y on the keyboard or choose "List of Requests" in the Windows main menu. Then we tick the requests that we need to perform.

Figure 3

These 2 requests require a domain’s name as input parameter. In our case it’s mit.edu. We type it in and click "Execute" to run both requests with one click.

Figure 4

We can monitor the status of our requests in the "Requests" window.

Once they’re done we’ll get 2 results and we’ll have to combine them on a schema. So we select any of our results in the "Requests" window and choose "Schema" in this window’s menu. The System will automatically offer visualization variants available for this request:

Figure 5

Now that we’ve got the results of one of our requests on a schema we have to visualize the results of the other one here as well. To do this we have to click the "Add to active" button first.

Figure 6

Then we select what needs to be visualized. For this request, there are multiple visualization variants available. We’ll choose Sub- and sibling domains.

Figure 7

Thus, we get the mit.edu subdomains on the same schema.

Figure 8

Now we’ll find out what network services are active on the corresponding IP addresses.

"ThreatCrowd" and "VirusTotal" return some IP addresses but not all of them. To get all of the addresses of the sub-domains found let’s resolve the host names to the IP addresses manually.

To do this let’s filter the Host objects in the "Content" window. We double click the Domain object to select these objects on the schema, then we right-click any of the selected vertices and choose the Nslookup request in the context window.

Figure 9 Figure 10

You can also launch this request through the "List of Requests" window.

Figure 11

After our request is complete we’ll get a list of IP addresses which correspond to the hosts. Now let’s obtain data on the active services of the found IP addresses with the help of "Shodan.io".

We select all IP-addresses on our schema (the same way we selected all domains) and then launch the Shodan search: right-click any of the selected objects and then choose Shodan - organisations. This way, apart from the network infrastructure, we'll get the list of organizations which own these IP-addresses.

At this moment you can go and make yourself a cup of tea or a coffee as each IP requires a separate request and Shodan limits the number of these requests to just one per second.

Figure 12

Once our request is complete we get a detailed layout of the network which includes not only domains and their IP addresses (and the companies these IPs belong to), but also all services which were found on these IPs with their names, versions, ports, etc.

Figure 13 Figure 14

The "Content" window provides us with additional functionality. For example, it allows us to search the results of our requests and also to group the objects of the schema by different attributes.

For example, let’s see what companies were found by our requests. In the object attributes section let’s find Organization name and click the "Expand" button to expand the list. We’ll pick the "MIT Laboratory for Nuclear Science" as an example and look into it more closely.

Figure 15 Figure 16

This entity was found in the process of obtaining data by IP addresses. We’ll use it to demonstrate how to make a request to Shodan having only a company name.

We could have begun our investigation at this stage – from a company’s name and not a domain.

So let’s proceed with this company that we chose and double click it in the "Content" window to locate the corresponding object on our schema.

Figure 17

Then we’ll copy it onto a new schema by pressing Ctrl+N on the keyboard or by right-clicking it and choosing "Copy onto new schema" from the context menu.

Once we did that we right-click the object and launch the Shodan - organisations request once again. This request comprises 5 sub-requests. Once it is complete we get the layout of IP-addresses, hosts and services of only this company.

We click Button 1 in the menu to the left of our schema, to increase the size of the vertices with bigger number of links.

Figure 18 Figure 19

In order to analyze the obtained network infrastructure thoroughly we need to combine the results of our previous request for the mit.edu domain with the results of this Search by company name request.

To avoid being distracted by the company objects and by the central domain mit.edu, we’ll select both our last Shodan request results in the "Requests" window and create a new Infrastructure schema by selecting it in the drop-down list of the corresponding menu in the upper part of the "Requests" window.

Figure 20

On this schema, we’ll see which IPs have more services and see what services are running and where exactly.

Figure 21

All data obtained by our requests is contained in the "Content" window where we can analyze its attribute statistics, like Product, Port, CPE and so on.

Figure 22

We can also search this data to find the services we are interested in using the search field in the upper part of the "Content" window.

Figure 23

We’ll be searching all objects and links’ attributes, including the Data attribute which contains the tcp service banner. If needed we can also search for any specific text, contained on html pages. For example, let’s search by the following key words: ftp, remote, printer, mysql, netbios.

Figure 24

Searching for 230 login will show us all FTPs which can be accessed anonymously.

Figure 25

Searching for Mysql will find databases. We can select some specific database and view it on our schema right away.

Figure 26 Figure 27

We can also establish what software is more popular on the Net. To do this we should create a Software BreakDown schema for the request results that we already have by selecting the appropriate option in the "Schema" drop-down list of the "Requests" window menu.

Figure 28

Then we match the size of the vertices to the number of their links, like we did before, by clicking the Button 1 button in the left menu of the schema

Figure 29

and change the layout of our graph to hierarchical by clicking Button 2 and choosing "Hierarchical" in the drop-down list.

Figure 30

This is the result that we get:

Figure 31

What we got here is not only what versions of services are running on what nodes but also which versions are the most popular. In the "Content" window we can also take a look at the CPE statistics for services.

Figure 32

On the Software BreakDown schema by right-clicking a certain service and choosing "Select neighbors" in the context menu, we can select all IP addresses which use the same version of this service.

Figure 33

The same way we can take a look at the services’ ports, by creating a Ports Breakdown schema and changing the size of the vertices and the layout.

Figure 34 Figure 35

Many Services objects contain location data. We can visualize these objects on a map by selecting the appropriate request results and clicking "GIS map" and then - Infrastructure.

Figure 36

After performing these actions, the user will get a map with all the objects, which have geo reference.

Figure 37

To view the captions to our objects we select them on the map, right click any of them and choose "Show captions". The captions contain information on the addresses and ports and also on the names and versions of the software.

Figure 38 Figure 39 Figure 40

If needed we can switch into the heat map mode in the "Explorer" window. For this we open the "Explorer" window in the "Windows" main menu and tick the corresponding option in the Gis-map tree-list.

Figure 41

This is what we get:

Figure 42

As you see we have acquired a lot of information on the services of the mit.edu and MIT LNS subdomains. But in your own research you can go even further and get a broader picture by searching by the company's name. In our case here it would be "Massachusetts Institute of Technology". And if you continued here you'd start with the corresponding Company object on our first schema and find even more addresses this way or another way connected to this company.

If we talk about informational security, getting data only on the network infrastructure is not enough. Information security requires not only technical solutions, as people may also become a vulnerability. In the course of the following research, we'll collect some MIT company e-mail addresses from open sources and we’ll use it as initial data for obtaining additional info on the people working there.

Let’s copy the mit.edu host from any of our schemas and into a new one by selecting it and pressing Ctrl+N on the keyboard or by simply right-clicking it and choosing "Copy onto new schema". On the new schema, we right click our object again and launch the Harvest emails request.

Figure 43

We’ll get these results:

Figure 44

Let’s select all the E-mail objects (by the object type in the "Content" window or by right-clicking our HOST and selecting all its neighbors in the context menu).

Figure 45

Then we right click any of them and choose "To requests". In this list we tick what we need to be performed (see the screenshot below) and click "Execute", thus searching for accounts connected to the e-mail addresses selected.

Figure 46

After our requests are done, we can visualize the results (one by one for now) on a new schema:

Figure 47

In addition to the accounts found, we can check if these e-mails are contained in the "HaveIBeenPwned.com" database of the biggest password leaks. We select all e-mails on our schema, right click any of them and choose haveibeenpwned request in the context menu. When it’s done we change the layout to hierarchical:

Figure 48

What we got to our schema are object leaks and objects, which show on what text exchange websites (for example, "Pastebin" or other) these emails are shared. This information will also show us where these e-mails were registered with.

Each link - between any e-mail address and the data leak - contains information on which data was leaked and when. This can also be analyzed in the "Content" window.

Figure 49

If we match the size of the vertices on our graphs to the number of their links we’ll see whose passwords were leaked more often.

Figure 50

Applying the same scenario when analyzing your own company network will help you locate the weak spots, which require your security assistance.

And Lampyre will certainly be of great help for you here.