[Exchange Online] Phishing email was not caught

[Exchange Online]Phishing email was not caught

    

Title: 

Phishing email should not have been caught

Desc: 

This customer is a legal firm An email from specific sender to our internal users with the subject Polo Horse Ownership was caught as a high confidence phishing email. Our customer is potentially liable to legal recourse due to this email not being actioned. We need immediate support regarding this in order to understand why this has happened and provide detailed responses to the client, who may be required to give evidence to the court for this matter

Possible Solution / Q&A:

• For your initial query:

 

    Per checking with our escalation team with the dedicated files, they have replied:

 

                ML is detecting this as phish via body model and header model, the message was false-positively caught by the Dynamic model. 

                For detailed root cause, we’re afraid this information is not exposed to public and all are dynamically maintained or adjusted by backend algorithm or AI.  Even our internal staff cannot see all details, not to mention our customers.

                If the issue is still reproducing in customer’s end, we can manually add morganenglish.com.au in the reputation allow list. While this may need the sending domain morganenglish.com.au to create a ticket under the affected tenant.

 

 

                Please find the response below:

• While we can both agree that this email      was flagged as false positive, is it possible to provide some details how      it was flagged as false positive so as we can avoid this in the future

Response from escalation team:

 

The backend algorithm is not exposed to us as well, but we do know there had been a change in the code in earlier this year.

Due to large amount Phishing emails being received with PDF attachment, Microsoft has changed the code so our FT(Front door) server, also known as Exchange Online Protection server, will be more strict on messages sent with PDF attachment.

This year, as reported by our front line engineers, the number of false-positive messages caught increased about 30%.

No pain, no gain. Please let our customer understand that Microsoft is trying to lower the number of false-positive events as well as increasing the security in EOP.

Our AI needs more time to extend and correct its own PHISH/SPAM/BULK models.

 

• Can you please confirm there is no      misconfiguration on the simpsons.com.au Office 365 and Exchange Online      tenant that would’ve caused this

Response from me, as they won’t handle this part:

 

To answer this query, you will need to analyze the header along with the extended trace.

 

From the extended trace we can see that it triggered the SPAM filter policy:  64c566dd-ad94-47ec-a8f4-f9b331f0caf8

 

And the message was identified as HPHISH, the related settings in the SPAM filter is below:

Name	block [email protected]
ID	64c566dd-ad94-47ec-a8f4-f9b331f0caf8
OrganizationId	AUSPR01A003.PROD.OUTLOOK.COM/Microsoft   Exchange Hosted Organizations/simpsonssolicitors.onmicrosoft.com -   AUSPR01A003.PROD.OUTLOOK.COM/ConfigurationUnits/simpsonssolicitors.onmicrosoft.com/Configuration
PhishSpamAction	Quarantine
PhishZapEnabled	TRUE

 

Well, in order to deliver these false-positive emails to quarantine, you have other options (E,g deliver to Junk)…

You may change the PhishSpamAction, but this does not solve the original issue, because whether a message is identified as PHISH/HPHISH or not, totally depends on our EOP AI’s message model.

 

What you can do, is set PhishZapEnabled to $false, which will lower the number of false-positive caught events, but the disadvantage is, some latest PHISH models won’t be added into your EOP message model list (Your tenant will have a higher chance to be affected by latest phishing models ).

 

-PhishZapEnabled

The PhishZapEnabled parameter enables or disables zero-hour auto purge (ZAP) to detect phishing messages in delivered messages in Exchange Online mailboxes. Valid values are:

·  $true: Phish ZAP is enabled. This is the default value. The result depends on the spam filtering verdict action: MoveToJmf = Read and unread phish messages are moved to the Junk Email folder. Delete, Redirect, or Quarantine: Read and unread phish messages are quarantined. AddXHeader or ModifySubject = no action is taken on the message.

·  $false: Phish ZAP is disabled.

Note: Use this parameter instead of the ZapEnabled parameter. The ZapEnabled parameter will be deprecated in a future release. During the transition, the value of this parameter is inherited from the ZapEnabled parameter. After you use the PhishZapEnabled parameter or the corresponding phish ZAP setting in the admin center on an existing spam filter policy, the ZapEnabled parameter value is ignored for phish ZAP.

            Original link: https://docs.microsoft.com/en-us/powershell/module/exchange/set-hostedcontentfilterpolicy?view=exchange-ps

 

• Is there any additional information that      can be provided to us and to our client Simpsons Solicitors to understand      the this issue and to confirm our understanding that this was caused by a      false-positive event and not changes made by Microsoft or Interconnekt      which resulted in this unfortunate event.

 

I guess the above answers may help on this query.

The root cause was Exchange Online Protection had updated the message model, and PDF security filter was more strict this year, which resulted a false-positive event.

The evidence of this issue is not caused by changes made in your tenant, is that the SPAM filter policy block [email protected] has not been changed since 2019-12-17.

 

WhenChanged

2019-12-17 03:44:57Z

 

• For your latest query:

 

The Email in question is SYCP282MB0144EBA2DECD66DA6D812FB2EB7B0@SYCP282MB0144.AUSP282.PROD.OUTLOOK.COM, attached with extended report and the original email, the email was sent to 3 recipients at the simpsons.com.au domain 1/3 users received the email which obviously begs the question of what happened, I performed an analysis of the email and email flow and while I can’t determine the root cause why it was flagged as phishing email I have found in the email header that, 2 hops within the email header is blacklisted by backscatter which I believe might be a contributing factor, these 2 hops are

SYCP282MB0144.AUSP282.PROD.OUTLOOK.COM fe80::6c12:39be:eb53:cfa5

AUS01-SY3-obe.outbound.protection.outlook.com 52.100.186.223

As you can see they’re both Microsoft owned hops, can you please investigate and advise on this situation, attached is the powershell output of hostedcontentfilterpolicy, original email message and extended report of this message via security.office.com.

 

I discovered that the mail was sent from Microsoft Exchange Online HRDP(High Risk Delivery Pool).

*CAT:OSPM refers to Outbound SPAM

*X-Forefront-Antispam-Report-Untrusted: Refers to the auth result from other O365 tenant which took part in the mail flow

 

The IP pool is our official servers for outbound SPAM messages, once the mail was identified as SPAM by the sender’s EOP Outbound SPAM filter, the message will be delivered via this pool.

 

Since your tenant received message from HRDP, the message will always be marked as SPM regardless of the message content.

               

                In that scenario, your tenant SPAM filter setting about the PhishZapEnabled didn’t affect the result, this is totally related with sender’s tenant.

 

About HRDP:

 

                Why would EOP will route a sender’s message via HRDP?

• EOP will scan every      outbound message to protect its reputation.

• Every sender’s      reputation is recorded at backend with a score, the score the percentage      based.

• The formula: (The SPAM count of user’s outbound messages) / (The      total count of user’s outbound messages )

• Once a specific sender      in your tenant has reached around 40%(This number is changing based on      AI’s dynamic model), all sender’s future emails within next 24 hours will      be delivered via HRDP

 

What if the sender is keep sending SPAM messages?

• Once the percentage has      reached around 70%(This number is also dynamic), the sender will be      considered as compromised potential, and will be blocked for sending      outbound messages, admins can release the sender in https://protection.office.com/restrictedusers

 

What kind of messages will be identified as OSPM?

• Bulk mails

• Newsletters

• Email with pure images      (E.g jpg,png…)

• Other Emails that      matches our OSPM message models

 

Troubleshooting in the sender’s tenant:

From backend I didn’t see the sender has any restriction history, which indicates that he didn’t touch the 70% SLA.

To further investigate the issue, user would have to export the extended trace regarding last 24 hours, and analyze from where it triggered the HDRP. (you can search for the keywords “DI=SO” in sender’s extended trace.)

               

                Reference articles:

                https://social.technet.microsoft.com/wiki/contents/articles/36402.exchange-online-troubleshooting-high-risk-delivery-pool.aspx

 

https://docs.microsoft.com/en-us/microsoft-365/security/office-365-security/high-risk-delivery-pool-for-outbound-messages?view=o365-worldwide#:~:text=High%2Drisk%20delivery%20pool,-To%20prevent%20this&text=The%20high%20risk%20delivery%20pool,outbound%20email%20from%20sending%20spam.