Office 365 – Analyze the security settings for your tenant

UPDATE 24/11/2015 – The script has been already updated has some bugs have been found. So please ensure you are always using the latest version by downloading it from the website

Microsoft is currently testing (early beta stage) an Office 365 security analyzer to help administrators to understand their current security configuration and potential risks they may have missed.

As said this is an early beta, so this will be slightly changed when released.

So, to have an early view, you need to connect to the following site


As first action, you need to download a bunch of PowerShell scripts to gather the data; you will have to sign in with your Office 365 administrator account to download the files

To help you prepare the client on which you will run these scripts, you need to have:

Once you are ready, using a command prompt – as usual always use the run as administrator – run the securityscore.bat file you have extracted from the ZIP file downloaded earlier

You may also have a multiple security prompts to run the PowerShell


Then you will be asked to authenticate to your Office 365 tenant – if your admin account uses MFA, the script will fail.


Then the script is looking if you meet the prerequisites (PowerShell modules)


If you missed some the script is going to connect automatically to the related download page and stops


If everything is looking good, the script connects to Azure AD

You will be asked to confirm your participation as the script will gather data about how your end-users


If you have issue to connect to your tenant (ERROR:  FAILED To connect to MSO Service) – as per below, try to edit the CollectAndScoreSecurityData.ps1 and the SecurityScorer.psm1 files to:

  • Cut the following lines from the SecurityScorer file

Write-Log "Connecting to Azure Active Directory"
$connectMso = Connect-MsolService -Credential $userCredential

  • Remove the following lines from the SecurityScorer file (just after the previous lines you have cut)

if ($connectMso -eq $null) {
    Write-Error " FAILED To connect to MSO Service"
    Throw "Failed to Connect to MSO Service" }

  • Past the cut lines to the CollectAndScoreSecurityData file after the lines, and replace the $userCredential variable with $AdminCredential

Write-Log " Set up the Service Sessions" 2
Get-ServiceSessions $AdminCredential


Then the script is connecting to both Exchange Online, Skype for Business and SharePoint Online.

NOTE 1 it is recommended to assign all the license to the account used for the scoring


NOTE 2 you may got some errors for some accounts during the Exchange Online analysis as some of your accounts may not have an Exchange Online license

Then the script analyze the password policy for each users found


Continue with DLP and Admin roles


Checks for illicit logon against Azure AD, suspicious malware or suspicious activities on delegated mailbox


You may be asked to validate there is no suspicious activities from time to time.

Please note that some activity reports displayed in the console may be quite complicated to read Smile

At the end you will get a security scoring and upload the results onto the web site


Leave a Comment

Your email address will not be published. Required fields are marked *

The reCAPTCHA verification period has expired. Please reload the page.