Detecting or preventing proxyshell/proxylogon attack against Microsoft Exchange Server
The cyber drill scenario simulated by BGD-eGOV CIRT where one of Bangladesh’s prominent financial organizations, named FIN, is worried about a potential cybersecurity breach because one of the IT personnel found a threatening text file in their Active Directory Domain Service (AD DS) and Exchange Server Administrator profile (in the desktop of both servers).
The network of the simulated environment was quite simple (Figure 1). Both the Domain Controller and the Exchange Server were inside the demilitarized zone. The Exchange Server had Internet connectivity and ports have been forwarded from the Internet Gateway Router to facilitate email communication.
I just tried to simplify this in more easier way. Hope you will like it. Its a methodical walkthrough rather than question solving. What you need to do when real attack happens, this walkthrough will tell you that hypothetical concept.
Here is the related technical information:
Server: Domain Controller (DC)
Operating System: Windows Server 2016
Role: Active Directory Domain Service (AD DS)
Domain: FIN.LOCAL
IP address: 10.10.10.1/24 [Not routable to the Global Internet]
Server: EXCHANGE2016
Operating System: Windows Server 2016
Role: Mail server (Exchange Server 2016)
Domain: FIN.LOCAL
IP address 1: 10.10.10.10/24 [Not routable to the Global Internet]
IP address 2:192.168.85.210/24 [Assume these IP address network was routable to the Global Internet]Login credential for Administrator (privileged user) and non-privileged user ITsupport was provided.
We have categorized the overall challenge into five sections:
- Conformation of the incident.
- Persistence.
- Lateral movement.
- Privilege escalation to domain administrator.
- Stealthy advanced persistent threat.
Validating the cyber incident
First of all we need to find out that, is the attack surface is internal or external. If the server is internet exposed then chances are it is external. Then we need to find out that whether the attacker exploited the server successfully or not. First of all you need to find out what are the services running on this server. If it is WordPress related server then try to find out what are the WordPress exploitable vulnerabilities present on the server using vulnerability scanner like Nessus, nmap, OpenVAS etc. Then read article on internet that, for exploitable vulnerabilities what are the leftover files or ioc’s attacker left on the system. Sometimes vendor of the product or service; also tell you the steps to go through to find out you are already hacked or not (like what Microsoft did in exchange proxy logon vulnerability). You need to keep an eye on the threat news portal for latest vulnerability and their exploitation news.
A global wave of cyber attacks and data breaches began in January 2021 after four zero-day exploits were discovered in on-premises Microsoft Exchange Servers, giving attackers full access to user emails and passwords on affected servers, administrator privileges on the server, and access to connected devices on the same network. Microsoft named Hafnium as the group responsible for the 2021 Microsoft Exchange Server data breach. Microsoft identified the following vulnerabilities as being exploited: CVE-2021–26855, CVE-2021–26857, CVE-2021–26858, and CVE-2021–27065. In our scenario, we need to validate whether our Exchange 2016 system is vulnerable to this attack and if it has already been exploited. Microsoft has provided detection tools and guidance to assist users needing to do this. By running the Exchange On-premises Mitigation Tool with MSERT in detect-only mode, we will be able to determine whether our Exchange 2016 system is vulnerable:
Using the Microsoft provided script, we can detect that server is vulnerable or not. We can also perform a VA against the server to find out vulnerabilities using Nessus or Qualys or nmap or nikto or OpenVAS tools.
Once we come to know that server is vulnerable then we need to know that server is exploited or not. By looking at the threatning text on the server or seeing some unwanted files in the server directory we can say that it is exploited. But still if we have tool that specifically tell us whether the server is compromise or not, we should check with that tool.
We can easily now assume that the server has been compromised.
Now note the time. It is very much important during any IR or hunting activity. By seeing the time, we can say that it is UTC format. We need to convert it to BD time. Bangladesh is 6 hours ahead than the UTC time. Meaning if UTC time is 10:55:41 AM then BDT time will be 4:55:41 PM
2021-10-05T10:55:41.116Z, is written in ISO 8601 format, which is a standard for representing dates and times in a machine-readable format.
Breaking it down:2021: the year
10: the month (October)
05: the day of the month
T: separates the date from the time
10: the hour (in 24-hour format) so it is morning time
55: the minute
41: the second
.116: the fraction of a second (in this case, 116 milliseconds)
Z: indicates that this time is in the UTC time zone (also known as "Zulu" time)
So the time represented by 2021-10-05T10:55:41.116Z is October 5th, 2021 at 10:55:41 and 116 milliseconds in the UTC time zone.
Now you need to search for logs in relevant log files at around 4:55:41 PM in BDT time to find anomaly.
Persistence:
Now the attacking part. Generally attacker try to put a backdoor or webshell on the server so that calling that shell they can again get access on the server. So in order to catch that you need to note the time when attack happened and based on that you need to look in the relevant ids/ips, security solutions logs as well as web server access logs (irrespective of linux or windows). Shells are generally created with random name in the exploitation phase and again removed by the attacker sometimes when the exploitation is done. That's why its common case when you try to find that shell in the server, you get nothing. In this Exchange vulnerability this is exactly what happened. We will see that soon.
Note: On the IIS server shell extension is .aspx and linux server shell extension is .php
Note: This kinds of popular exploited vulnerabilities can be exploited using Metasploit framework.
You need to know where the IIS logs are written. As the attack took place on 5th October, so you should search for the shell in the log file of 5th October. You already know that what time the attack has happened. So in the log files find that time and search for suspicious shell.
Let me explain the log entry. WgUP.aspx is the malicious backdoor that is created at the time of exploitation of this vulnerability and it is deleted again once the exploitation works successfully. 192.168.85.210 is our exchange server ip and 192.168.85.130 is the attacker ip. Say attacker using kali linux with metasploit from that ip. So attacker initiates attack from 192.168.85.130 ip and creates a random name backdoor with .aspx extension (i.e. WgUP.aspx) and place that under /owa/auth/WgUP.aspx
Then attacker force the exchange server to execute that via GET request and send the output back to the attacker.
The above walkthrough also help you to understand this.
We can see that using exchange_proxylogon_rce exploit, the attacker is attacking the server.
I already told you that attacker sometimes delete the backdoor once the vulnerability is exploited. And each time different name backdoor is created.
We can see that attacker got the shell. Now attacker is operating the system using nt authority\system user who is the highest permissive level user in windows system who can do anything on the system. getuid command will tell you that.
As the exploitation was successful so we should always check if attacker plant a backdoor or not that will execute every time the system reboots. Microsoft Sysinternals Autoruns tool is a great tool to check that.
The vb script is designed in such a way that it continuously tries to connect to the attacker server.
There was a session running because of that vb script backdoor. Attacker just need to listen on a port like nc -lnvp 631
Then within a second the shell again reconnects. To see you the demo, we just exit it from the shell.
Now we are again listening on it.
The shell is bind with this attacker ip.
See attacker again got a shell with exchange server. So as an IR specialist, you need to remove that from the system so that attacker does not get any shell on the server again once the system reboots.
You can also check with process explorer sysinternals tool to find out malicious process running on the system.
Lateral Movement:
Now we need to find out how attacker laterally move throught the network. We can check in the server that what are files or folders that are newly created after the attack has happend. Attacker in this case generally download mimikatz tool to dump the password in clear text or password hash from lsass memory. Then attacker initiate pass the hash attack or if found the password in clear text then try to log on using RDP protocol. Defender can catch the mimikatz so attacker can disable defender as well as attacker has the nt authority\system level access on the system. Running mimikatz also requires admin rights. So attacker can also do that as well.
We can search on these folders to find more suspicious things. We can see that mimikatz, wget is already downloaded there. We can query this to the powershell logs to find out what commands attacker executed. Again check the time and filter powershell logs based on that. If you can forward this logs to any siem then your task will be easier. I will try to show that in my next video walkthrough.
By entering EXCHANGEADMIN folder,
We can see that in order to move laterally, attacker did the above things like dump the hashes, decrypt the ITsupport user credential from that hash (not showing here the hash decryption stage), query to the DC ip to see it is live or not etc.
We can see that more users are created.
Lets investigate powershell logs.
Powershell logs are found in two places.
- Applications and Services Logs > Windows Powershell
- Applications and Services Logs > Microsoft > Windows > Powershell > Operational
From here we can filter logs with date and time along with required event id.
We can filter with event id 4104 also we can filter with keyword (Use the find option right side in the event viewer). As we saw previously that invoke-mimikatz.ps1 and wget.exe was present on the C: drive so we can query using those keyword in powershell logs using find feature.
4104 event id telling us this powershell script has been executed.
Also powershell event id 400 and 403 tell us what tools attacker downloaded, its something like http/https related. We filtered using Mimikatz and wget keyword that we found earlier on the C drive and we got this results.
It is downloaded on 17.49.58 time and executed on 18.01.42
Things are starts getting clearing now. Now you can tell me like we saw mimikatz and wget ioc/ioa on the C drive that's why we query this to powershell logs to ease the investigation. Now what if attacker remove those tools then how can you detect those in the powershell logs? First of all you need to have clear understandings of what attackers next steps could be. You need to build a strong hypothesis. As we get a threatening text from DC also so no doubt that attacker successfully moved laterally and compromise DC as well. In order to move laterally attacker needs mimikatz or other password dumping tool to dump the hash, then decrypt password from the hash or play pass the hash attack.
There are some other tools that will help you to find anomaly from your event logs or powershell logs. I will walkthrough those in other of my video walkthrough.
Investigating DC RDP terminal service logs we can see that attacker successfully logs in.
Privilege escalation to domain administrator
Attacker successfully login to DC system using the ITsupport account. This account is low priv account so attacker now needs to escalate his privilege to become domain admin. Attacker can search for privilege escalation vulnerability locally or remotely from exchange server using any type of vulnerability scanner.
Lets go to our exchange server and check powershell logs 800 to see attacker execute any script or not to find out vulnerability in the remote system.
- Example of Event ID 800: Suppose you have enabled PowerShell transcripts on your system, and you start a PowerShell session by launching the PowerShell console. As soon as you start the session, PowerShell will begin capturing all input and output from the session and logging it to a file. You can see that transcription has started by checking the Windows PowerShell event log, where you will see an event with ID 800 indicating that transcription has begun.
- Example of Event ID 4104: Suppose you have a PowerShell script that runs daily to perform some administrative task on your system. When the script runs, it will generate an event in the PowerShell Operational event log with ID 4104 indicating that the script has been executed. This event will include information such as the name of the script, Commandline parameters, the user account under which it was executed, and the time at which it ran.
A little bit of googling will tell you the script is used to discover PrintNightMare vulnerability on the remote system by checking spooler service is on or not !!!!. And attacker found that the service is on.
Attacker will see that the service is on in DC and Exchange2016 system.
Now its clear that as attacker found the service is on to the DC system, so attacker will exploit this.
By investigating the DC powershell logs 4104 telling us what script attacker execute to escalate its privilege. Now studying a little about printnightmare vulnerability will tell you what are the ioc’s you could see on the exploited system.
https://github.com/calebstewart/CVE-2021-1675
When this vulnerability is exploited then a new user is created and that user is automatically added to the local administrator group.
Seeing the below SS, ADADMIN user is created and the user is under administrator group now.
Attacker then add this user to domain admin group.
The arrow is pointing to the Domain Admins Group.
You can investigate the Microsoft-Windows-PrintService/Operational logs on DC to find out more ioc’s.
How attacker deliver the payload to the DC?
We can see that a share is being created on the exchange server and accessed by anonymous user.
Via this share payload is delivered to the DC.
So attacker created a share folder on exchange server and from there attacker transfer the privilege escalation payload to the DC. Anonymous user from DC can access the exchange share folder to fetch the payload from exchange server.
Stealthy APT
How attacker maintain domain wide persistent?
As an IR, we should always check if attacker created any latest firewall rules or not.
We found a rule created in firewall. The rule is something like if anyone hit on exchange server over port 3340, forward that request to 10.10.10.1 over port 3389.
This is the rule configuration.
This is the DC system. And because of that firewall rule, the traffic is forwarded to DC ip over port 3389.
This is it. Hope you like the walkthrough. Please subscribe to the following.
LinkedIn:
https://www.linkedin.com/in/md-mahimbin-firoj-7b8a5a113/
YouTube:
https://www.youtube.com/@mahimfiroj1802/videos
Thanks.