test procedure
Scripts such as vbs, js or ms Office macros can execute and install a file-less backdoor on victims' systems and create a Control Channel (C2) to the attacker, who is usually in a different physical location, and maybe even in a different country. Apart from these well-known scenarios, it is possible to deliver malware using exploits, remote calls (psexec, wmic), task scheduler, registry entries, Arduino hardware (USB rubberducky) and wmı calls. This can be done with built-in Windows Tools like PowerShell. These methods load the actual malware directly from the ınternet into the target System's Memory, and continue to expand further into the local area network with native OS Tools. They may even become persistent on machines in this way. This year's test does not make use of portable executable (pe) malware. However, as the nature of Advanced persistent threats continues to evolve, we may introduce One or two samples of these in the future if appropriate.
fileless attacks
In the field of malware there are many (possibly overlapping) classification categories, and amongst other things a distinction can be made between file-based and fileless malware. Since 2017, a significant increase in fileless threats has been recorded. One reason for this is the fact that such attacks have proved Very successful from the attackers' point of view. One factor in their effectiveness is the fact that fileless threats Opera'te only in the Memory of the compromised System, making it harder for Security solutions to recognize them. It is important that fileless threats are recognised by consumer Security programs as well as by business products, for the reasons given below.
attack vectors and targets
In penetration tests, we see that certain attack vectors may not yet be well covered by Security programs, and many popular av products still provide insufficient protection. Some business Security products are Now making improvements in this area, and providing better protection in some scenarios. As mentioned above, we believe that consumer products also need to improve their protection against such malicious attacks; non-business users can be, and are, attacked in the same way. Anyone can be targeted, for a variety of reasons, including “doxing” (publishing confidential personal information) as an act of revenge. Attacking the Home computers of businesspeople is also an obvious route into accessing their Company data.
attack methods
In the Advanced threat protection test, we also include several different command-line stacks, CMD/PS commands, which can download malware from the network directly into RAM (staged) or base64 encoded calls. These methods completely avoid disk Access, which is (usually) well guarded by Security products. We sometimes use simple concealment measures, or change the method of the stager call as well. Once the malware has loaded its second stage, an http/https connection to the attacker will be established. This Inside-out mechanism has the Advantage of establishing a C2 Channel to the attacker that is beyond the protection measures of the majority of NAT and firewall products. Once the C2 tunnel has been established, the attacker can use all known Control mechanisms of the common C2 products (meterpreter, PowerShell empire, etc.). These include e. G. File uploads/downloads, screenshots, keylogging, Windows Shell (GUI), and Webcam snapshots. All the Tools used are freely available. Their Source code is open and created for research purposes. However, the bad guys often abuse these Tools for criminal purposes.
false positive (false alarm) test
A Security product that blocks 100% of malicious attacks, but also blocks legitimate (non-malicious) actions, can be hugely disruptive. Consequently, we conduct a false-positives test as part of the Advanced threat protection test, to check whether the tested products are able to distinguish malicious from non-malicious actions. Otherwise a Security product could easily block 100% of malicious attacks that e. G. Use email attachments, scripts and macros, simply by blocking such functions. For many users, this could make it impossible to carry out their normal daily tasks. Consequently, false-positive scores are taken into account in the product's test score.
We also Note that warning the user against e. G. Opening harmless email attachments can lead to a “boy who cried wolf” scenario. Users who encounter a number of unnecessary warnings will sooner or later assume that all warnings are false alarms, and thus ignore a genuine warning when it comes along.
testcases
We used five different
Initial Access Phases, distributed among the 15 test cases (e. G. 3 testcases came VIA email/spear-phishing attachment).
- trusted relationship: “adversaries may Breach or otherwise leverage organizations who have Access to intended victims. Access through trusted Third-party relationship exploits an existing connection that may not be protected or receives less scrutiny than standard mechanisms of gaining Access to a network.” (Source: Trusted Relationship, Technique T1199 - Enterprise | MITRE ATT&CK®)
- valid accounts: “adversaries may steal the credentials of a specific user or service account using credential Access techniques or capture credentials earlier in their reconnaissance process through social engineering […].“ (Source: Valid Accounts, Technique T1078 - Enterprise | MITRE ATT&CK®)
- replication through removable Media: “adversaries may move onto systems […] by copying malware to removable Media […] and renaming it to look like a legitimate file to trick users into executing it on a separate System. […]“ (Source: Replication Through Removable Media, Technique T1091 - Enterprise | MITRE ATT&CK®)
- spearphishing attachment: “spearphishing attachment is […] employs the use of malware attached to an email. […]” (Source: https://attack.mitre.org/techniques/T1193/)
- spearphishing link: “spearphishing with a link […] employs the use of links to download malware contained in email […].“ (Source: https://attack.mitre.org/techniques/T1192/)
The 15 test scenarios used in this test are Very briefly described below:
1) this threat is introduced VIA trusted relationship. Mshta launches an HTML application, which executes a staged
Empire PowerShell payload.
2) this threat is introduced VIA trusted relationship. A PowerShell script containing an amsı bypass and a PowerShell
Empire stager was executed.
3) this threat is introduced VIA trusted relationship. Windows scripting host was used to download a PowerShell payload VIA'a integrated
Empire PowerShell stager, combined with an amsı bypass.
4) this threat is introduced through valid accounts. The trusted Windows Utility Microsoft Build Engine was used to proxy the execution of an
Empire macro payload, which opens a command and Control Channel.
5) this threat is introduced through valid accounts. A vbscript which spawns a PowerShell process and executes an
Empire payload has been used.
6) this threat is introduced through valid accounts. A batch file was used to execute an obfuscated PowerShell stager, download an obfuscated
PoshC2
7) this threat is introduced VIA removable Media (USB). A JavaScript executes an obfuscated PowerShell stager, which downloads and executes a
PoshC2 PowerShell payload.
8) this threat is introduced VIA removable Media (USB). MSHTA.exe executes a PowerShell stager which launches a base64-encoded
PoshC2 staged PowerShell payload.
9) this threat is introduced VIA removable Media (USB). A malicious Microsoft Office macro executes a
PoshC2 PowerShell payload.
10) this threat is introduced VIA spearphishing attachment. Vbscript downloads and executes an xsl
PoshC2
11) this threat is introduced VIA spearphishing attachment. A HTML application downloads and executes an obfuscated PowerShell payload. This test case was created with
Metasploit Meterpreter.
12) this threat is introduced VIA spearphishing attachment. Vbscript downloads and executes an xsl payload. This test case was created with
Metasploit Meterpreter.
13) this threat is introduced VIA spearphishing link. MSHTA.exe downloads and executes an obfuscated xsl payload. This test case was created with
Metasploit Meterpreter.
14) this threat is introduced VIA spearphishing link. A JavaScript downloads and executes an obfuscated PowerShell payload. This test case was created with
Metasploit Meterpreter.
15) this threat is introduced VIA spearphishing link. Exe downloads and executes a PowerShell stager which downloads and executes an encrypted PowerShell
Empire staged PowerShell payload, combined with an amsı bypass.
false alarm test: Various false-alarm scenarios were used in order to see if any product is over-blocking certain actions (e. G. By blocking by policy email attachments, communication, scripts, etc.). None of the tested products showed over-blocking behaviour in the false-alarm test scenarios used.
If during the course of the test, we were to observe products adapting their protection to our test environment, we would use countermeasures to evade these adaptations, to ensure that each product can genuinely detect the attack, as opposed to the test situation.
what is covered by the various testcases?
Our tests use a subset of the ttp (tactics, techniques, procedures) listed in the
MITRE ATT&CK framework. This year, the above 15 testcases cover the items shown in the table below:
For reference purposes, the Full mıtre att&ck Framework for Windows can be seen here:
Matrix - Enterprise | MITRE ATT&CK®
In our opinion, the goal of every AV/EPP/EDR System should be to detect and prevent attacks or other malware as soon as possible. In other words, if the attack is detected before, at or soon after execution, thus preventing e. G. The opening of a command and Control Channel, there is no need to prevent post-exploitation activities. A good burglar alarm should GO off when somebody breaks into your House, not wait until they start stealing things.
A product that blocked certain legitimate functions (e. G. Email attachments or scripts) would be categorised only as “tested”.
observations on consumer products
In this section, we report some additional information which could be of interest to readers.
Detection/Blocking stages
Pre-execution (pre): When the threat has not been run, and is inactive on the System.
On-execution (on): İmmediately after the threat has been run.
Post-execution (post): After the threat has been run, and its actions have been recognised.
Bundan sonraları Av'leri kendi düşüncelerine göre sıralıyorlar.
Buyurun.
Sesiniz, sedanız kesildi. Üzücü.
AV-Comparatives released their Enhanced Real-World Test 2020 for Consumer security products, testing the protection against advanced attacks