Behavioral Analysis

Overview

Behavioral analysis monitors a malware sample's runtime actions — filesystem changes, registry modifications, process creation, and system-level activity. By observing what the malware actually does during execution, analysts can identify capabilities that static analysis alone cannot reveal, especially for packed, encrypted, or obfuscated samples.

Linux Behavioral Monitoring

File System Monitoring with inotifywait

# inotify-tools
# https://github.com/inotify-tools/inotify-tools

# Monitor filesystem changes in real-time
inotifywait -m -r /tmp /var/tmp /home -e create,modify,delete,move

# Monitor with timestamp and output to file
inotifywait -m -r --timefmt '%Y-%m-%d %H:%M:%S' --format '%T %w%f %e' \
  /tmp /etc /home -e create,modify,delete > fs_changes.log &

# Monitor specific events
# create  — file/directory creation
# modify  — file modification
# delete  — file/directory deletion
# move    — file/directory rename or move
# attrib  — permission or ownership change
# open    — file opened
# access  — file read

Process Monitoring

# Monitor new processes as they spawn
# procps (ps)
# https://gitlab.com/procps-ng/procps

# Watch process list in real-time
watch -n 1 'ps auxf'

# Monitor process creation events via audit subsystem
# auditd
# https://github.com/linux-audit/audit-userspace
sudo auditctl -a always,exit -F arch=b64 -S execve -k process_exec
sudo ausearch -k process_exec --start recent

# List open files by a specific process
ls -la /proc/<pid>/fd/

# Check process memory maps
cat /proc/<pid>/maps

# Check process environment
cat /proc/<pid>/environ | tr '\0' '\n'

# Check process command line
cat /proc/<pid>/cmdline | tr '\0' ' '

Comparing System State (Before/After)

# Snapshot approach: capture system state before and after execution

# Snapshot file listing
find / -type f -newer /tmp/timestamp_marker 2>/dev/null > new_files.txt

# Create timestamp marker before execution
touch /tmp/timestamp_marker

# After execution, find all new/modified files
find / -type f -newer /tmp/timestamp_marker 2>/dev/null

# Snapshot crontabs
crontab -l > /tmp/cron_before.txt
# (execute malware)
crontab -l > /tmp/cron_after.txt
diff /tmp/cron_before.txt /tmp/cron_after.txt

# Snapshot network listeners
ss -tlnp > /tmp/listeners_before.txt
# (execute malware)
ss -tlnp > /tmp/listeners_after.txt
diff /tmp/listeners_before.txt /tmp/listeners_after.txt

Windows Behavioral Monitoring

Process Monitor (Procmon)

Process Monitor captures real-time filesystem, registry, and process/thread activity. This is the most important single tool for Windows behavioral analysis.

Key Procmon filters for malware analysis:

Process Name    is    <malware_name.exe>    Include
Operation       is    WriteFile             Include
Operation       is    RegSetValue           Include
Operation       is    Process Create        Include
Operation       is    TCP Connect           Include
Path            contains    \Run\           Include
Path            contains    \Services\      Include
Path            contains    \Startup\       Include
Result          is    ACCESS DENIED         Include

Procmon usage workflow: 1. Start Procmon (it begins capturing immediately) 2. Set filters to focus on the malware process 3. Execute the malware 4. Let it run for 5-10 minutes 5. Stop capture and analyze events 6. Export as CSV or PML for offline analysis

Regshot — Registry Comparison

Regshot takes before and after snapshots of the Windows registry, then shows exactly what changed.

1. Open Regshot
2. Click "1st Shot" → "Shot" (captures baseline)
3. Execute the malware
4. Wait for activity to settle
5. Click "2nd Shot" → "Shot" (captures modified state)
6. Click "Compare" to see all registry changes

Changes to look for: - New Run/RunOnce keys (persistence) - New services registered - Modified security settings - New scheduled tasks - Browser or proxy settings changed

Autoruns — Persistence Enumeration

Autoruns shows all auto-start locations in Windows:
- Registry Run keys
- Scheduled tasks
- Services
- Drivers
- Boot execute
- Logon scripts
- Browser extensions
- WMI event subscriptions

Compare Autoruns output before and after malware execution to identify
new persistence mechanisms.

Process Explorer

Process Explorer provides detailed process information: - Process tree (parent-child relationships) - Loaded DLLs for each process - Open handles (files, registry keys, mutexes) - Network connections per process - Thread details and stack traces - Digital signature verification

Key observations: - Child processes spawned by the malware - Injected DLLs in legitimate processes - Suspicious mutex names (single-instance checks) - Handles to sensitive files or registry keys

Behavioral Indicators

Persistence Mechanisms

Location Technique
HKCU\Software\Microsoft\Windows\CurrentVersion\Run Registry Run key
HKLM\Software\Microsoft\Windows\CurrentVersion\Run System-wide Run key
%APPDATA%\Microsoft\Windows\Start Menu\Programs\Startup Startup folder
Scheduled tasks schtasks /create
Services sc create
WMI event subscriptions Persistent event consumer
DLL search order hijacking Drop DLL in application directory

File System Activity

Activity Significance
Dropping files to %TEMP% Staging secondary payloads
Copying self to %APPDATA% or System32 Persistence via relocation
Creating .bat or .ps1 scripts Secondary execution stages
Modifying hosts file DNS redirection
Accessing credential stores Credential theft
Encrypting user files Ransomware behavior

Process Activity

Activity Significance
Spawning cmd.exe or powershell.exe Command execution
Creating remote threads Process injection
Accessing lsass.exe Credential dumping
Spawning child processes in unusual locations Evasion
Process hollowing (creating suspended process) Code injection

Automated Sandbox Analysis

Cuckoo Sandbox

Cuckoo is an open-source automated malware analysis system that executes samples in VMs and generates comprehensive behavioral reports.

Cuckoo provides:
- Behavioral analysis (API calls, file/registry/network activity)
- Network traffic capture
- Memory dumps
- Screenshots during execution
- YARA signature matching
- VirusTotal integration
- Automated reporting (HTML, JSON, PDF)

References

Tools