A Key Skill for Malware Analysis and Threat Hunting

Attack pattern matching (APM) and writing a signature in order to detect and hunt a threat is an extremely valuable skill and require and desirable for malware analysis and threat hunting and as well as incident response like position. If someone understand and able to write an APM signature by using a programming language for instance python, bash scripting or C++,these skills will help him to be a successful malware analyst and also in reverse engineering.

Professional credibility and job retention

Having a strong skills sets is the key element for the professional strength and it provides the security to your current job and it makes you the right person in the eyes of your employer for the organization and organizations always want to keep those figures on every cost. Skills like APM are some of the most demanding one which every company is looking for those dealing with malware analysis.

What is an attack pattern matching (APM)?

Attack Pattern Matching (APM) is a generic and open signature algorithm that allows you to describe relevant log events in a straightforward manner. The rule format is very flexible, easy to write and applicable to any type of log file. The main purpose is to define and design set of rules which will fingerprint different attack patterns in the form of rules for both files and network traffic. (MVS GROUP INTERNSHIP DOC)

To the best of our knowledge and understanding, generating signature using the technique of Attack pattern matching (APM) is a new concept, there is similar signature generation tool called YARA rules is present for threat detection but the problem with that method is the slower process. Processing for the 10 TB logs files takes hours with YARA rules/signatures. Instead, APM signatures takes very less amount of time for processing the same amount of logs.

Sample example of an APM signature:

Title:

Description: Describe the rule in one-two sentences

Author: Give credit to yourselves

References: List any you referred to

Log source: List what kind of log you think it is

Detection: Unique identifiers or patterns

False positives: Conditions that would call it a false positive

Level: informational/low/medium/high/critical (MVS GROUP INTERNSHIP DOC)

Issues while working on project:

Technical:

a. Lack of information available: Attack pattern matching (APM) is relatively a new approach for threat hunting and securing the systems and network from outside attacks.

Solution: As I mentioned above, the APM is a new concept for threat hunting, in the beginning we did not find enough material for our project and we were not able to move further. At this point, our professor provided the best guidance and encouragement in order to proceed in the right way. The other most important factor by which we got motivation and started our journey and able to design the road map for our project was the Workshop conducted by Mr. Ali. He provided us the knowledge and tools which were required to accomplish our capstone project in very short period.

b. Lack of programming skills: Attack pattern matching (APM) needs programming skills for generating signatures.

Solution: Analyzing malwares and writing signatures requires the core knowledge of programming languages. However, both of us have programming background but for long time not working in an environment where writing/reading programming codes is the primary job. Both of us took this problem as a challenge and started working to refresh our programming skills by reading, watching videos and utilized all available resources for achieving required level of competency.

People:

a. Different schedule

Solution: We both are married and have responsibilities of our dependents, after school hours, it was very difficult to sit together and work on our project. It was very difficult to take out with common time frame for the project but after some struggle we manage a time frame which was acceptable for both of us.

b. Not heading towards same direction

Solution: While working on the project we found that some time we are going on the different direction.

At this situation, we were always respectful to each other as well as we were listening other point of view openly. The above strategy helped us to be working in the same direction.

Resources/Technologies:

During the project, we used no of free open source softwares/tools for analyzing the Logs file and generating APM signatures, these softwares, tools and websites are listed as follows:

1. Logs files provided by MVS group
2. Notepad++
3. MS Excel
4. MS Word
5. Host machine Windows 10
6. VMWare workstation15 pro
7. VM machine Windows7
8. VM machine Windows 10
9. VM machine RED Hat Linux
10. Bash scripting
11. Virus total (website)
12. Otxaleinvault (website)
13. Decode and Encode URL (website)
14. Critical Logs review cheat sheet by SANS (website)

Design and Implementation:

In this section, we will provide explanations of the preparation steps, design and the implementation phase of the capstone project in detail.

Step 1:

The first step is to make machines which are required to perform the task for the capstone project.

As mentioned in the resources section, we are using VMware workstation 15 for virtualization and Windows 10 as a host machine. We made the following three virtual machines by using trial versions;

1. VM machine Windows7
2. VM machine Windows 10
3. VM machine RED Hat Linux

Step 2:

The second step is analyzing the logs file which was provided by the MVS group by using the notepad++ Microsoft Excel and Microsoft Access. The main purpose of the analyzing logs is to find the suspicious logs, for example the logs with attached suspicious IP addresses, URL, DNS, File path, port and specific string etc.

Steps 3:

The next step is to analyze the suspicious logs and information attached with them. There are several paid tools available for analyzing the suspicious logs, but in our case, we are using basically two websites for our research which are virustotal and otxalienvault. We also used google extensively while our research for analyzing suspicious logs.

Furthermore, we wrote some bash script for filtering suspicious logs by providing malicious key words including IP addresses, URL, DNS, File path, port and specific string etc. The bash script helps detecting malicious logs based on providing strings and save time.

Interestingly, we can take our work one step ahead by doing automation for searching malicious logs by virustotal and otxalienvault, if we integrate the API’s of the above mentioned sites in our bash script. Unfortunately, lack of funding we are not able to perform the task at this point.

Step 4:

This the most important for our capstone project in which we have to write an APM signature, we plan to develop several codes which can generate an APM signature based on providing different keywords.

To best of our knowledge, there is no such type of work done before, we believe that this is a very unique task we have chosen for our capstone project.

Explanation for the bash script which will generate the APM signatures is coming in the evaluation section.

Conclusion and Future Work

We started this project with one primary goal how to write APM signature for detecting different threats, while we were analyzing different logs and also at the time of writing signatures based on logs analysis, we found that if we go for a tool or a script which have capacity to write an APM signature based on providing data would be an amazing achievement.

At this time, our script is not fully functional because of some limitations such as time and lack of paid tools, but we believe that it is a good starting point for us. We or any other group can be benefited by our research work and efforts in future and be able to develop a fully functional tool which can generate the APM automatically.

Additionally, our work can be extended with different programming language like python or C++ for added functionality. Furthermore, our work is completely open-source and it can be easily integrated with other tools and languages.

Acknowledgements

We would like to acknowledge and thank to our honorable Professor Jeffery, through his continuous guidance and supervision we are able to work on a relatively new and sophisticated technology.

Similarly, we also thankful to MVS group who gave us this opportunity by providing the resource logs file and other support for analyzing and working on APM.