A Key Skill for Malware Analysis and Threat Hunting

Attack pattern matching (APM) and writing a signature to detect and hunt threats is a super valuable skill for malware analysis, threat hunting, and incident response roles. If you understand how to write an APM signature using programming languages like Python, Bash scripting, or C++, these skills will set you up for success as a malware analyst and in reverse engineering too.

Professional credibility and job retention

Having strong skills is key for professional strength. It secures your job and makes you the go-to person for your employer. Organizations always want to keep valuable employees, no matter the cost. Skills like APM are in high demand in companies dealing with malware analysis.

What is attack pattern matching (APM)?

Attack Pattern Matching (APM) is a flexible and open signature algorithm that helps describe relevant log events simply. The rule format is easy to write and works with any log file. The main goal is to create rules to identify different attack patterns in files and network traffic. (MVS GROUP INTERNSHIP DOC)

Generating signatures with APM is a new concept. A similar tool, YARA rules, exists for threat detection, but it’s slower. Processing 10 TB of log files takes hours with YARA. In contrast, APM signatures process the same amount of logs much faster.

Sample example of an APM signature:

Title:

Description: Briefly describe the rule

Author: Who created it

References: Any sources you used

Log source: Type of log

Detection: Unique identifiers or patterns

False positives: Conditions that would be false positives

Level: informational/low/medium/high/critical (MVS GROUP INTERNSHIP DOC)

Issues while working on the project:

Technical:

a. Lack of information: APM is a new approach for threat hunting and securing systems and networks from attacks.

Solution: Since APM is new, we initially lacked material for our project. Our professor guided and encouraged us, helping us proceed correctly. The workshop by Mr. Ali was also crucial. He provided the knowledge and tools we needed for our capstone project.

b. Lack of programming skills: APM requires programming skills for generating signatures.

Solution: Analyzing malware and writing signatures requires programming knowledge. Though we have programming backgrounds, we hadn't worked with code for a while. We took this as a challenge, refreshed our skills by reading, watching videos, and used all available resources to achieve the needed competency.

People:

a. Different schedules

Solution: Both of us are married with responsibilities. After school hours, finding common time was tough. With some struggle, we managed to agree on a timeframe that worked for both of us.

b. Not heading in the same direction

Solution: Sometimes, we were moving in different directions. We always respected each other’s viewpoints and listened openly. This strategy helped us work together in the same direction.

Resources/Technologies:

We used various free, open-source software/tools for analyzing log files and generating APM signatures. Here’s a list:

1. Log files from MVS group
2. Notepad++
3. MS Excel
4. MS Word
5. Host machine: Windows 10
6. VMWare Workstation 15 Pro
7. VM machine: Windows 7
8. VM machine: Windows 10
9. VM machine: Red Hat Linux
10. Bash scripting
11. Virus Total (website)
12. Otxalienvault (website)
13. Decode and Encode URL (website)
14. Critical Logs Review Cheat Sheet by SANS (website)

Design and Implementation:

Here, we explain the preparation steps, design, and implementation of our capstone project in detail.

Step 1:

The first step was setting up the machines needed for the project. Using VMware Workstation 15 and Windows 10 as a host, we created three virtual machines:

1. VM machine: Windows 7
2. VM machine: Windows 10
3. VM machine: Red Hat Linux

Step 2:

The second step was analyzing the log files from the MVS group using Notepad++, Microsoft Excel, and Access. The goal was to find suspicious logs, like those with suspicious IP addresses, URLs, DNS, File paths, ports, and specific strings.

Step 3:

The next step was analyzing the suspicious logs and the information attached to them. Though there are paid tools for this, we used two main websites: Virus Total and Otxalienvault. We also used Google extensively during our research.

We wrote some bash scripts to filter suspicious logs by providing malicious keywords like IP addresses, URLs, DNS, File paths, ports, and specific strings. These scripts help detect malicious logs quickly, saving time.

We could automate this process by integrating APIs from Virus Total and Otxalienvault into our bash scripts. However, due to funding limitations, we couldn't do this yet.

Step 4:

This step was the most important for our project: writing an APM signature. We planned to develop several codes that can generate APM signatures based on different keywords.

To our knowledge, this type of work hasn't been done before, making our task unique for our capstone project.

We’ll explain the bash script that generates APM signatures in the evaluation section.

Conclusion and Future Work

We started this project with the goal of writing APM signatures to detect different threats. During our analysis of various logs and while writing signatures, we realized that creating a tool or script to write APM signatures based on provided data would be an amazing achievement.

Our script isn't fully functional yet due to time and funding limits, but it's a good start. Others can build on our research and efforts to develop a fully functional tool for generating APM signatures automatically.

Our work can also be extended with different programming languages like Python or C++ for added functionality. It’s open-source and can be integrated with other tools and languages easily.

Acknowledgements

We want to thank our professor, Jeffery, for his continuous guidance and supervision, which helped us work on this new and sophisticated technology.

We also thank the MVS group for providing the resource logs and other support for analyzing and working on APM.