Configuration Review
Configuration Review of Microsoft 365
Microsoft 365 is where most organisations keep everything and where most business email compromise begins. It is also configured once during migration and rarely revisited, so the settings reflect what was convenient during cutover rather than what is appropriate now. The review concentrates on identity and sharing, because those are the two surfaces that actually get exploited.
Methodology
- 01
Tenant inventory
Licences, enabled services, domains, directory size, administrative role assignments and the tenant's own secure score baseline recorded before assessment.
- 02
Identity and conditional access review
Every conditional access policy evaluated for coverage gaps, exclusions, report-only policies never enforced, and legacy authentication paths that bypass them entirely.
- 03
Privileged access review
Global administrator count, standing versus eligible assignments, break-glass account configuration, and whether privileged accounts are licensed and protected differently from standard ones.
- 04
Exchange Online review
Mail flow rules, external forwarding, anti-phishing and anti-spoofing policies, SPF, DKIM and DMARC alignment, and mailbox auditing coverage.
- 05
SharePoint and OneDrive review
External sharing defaults, anonymous link policy and expiry, site-level overrides, and the actual population of shared links rather than the policy that governs them.
- 06
Teams and collaboration review
Guest access, external federation, meeting policies and app permission governance.
- 07
Defender and logging review
Safe Links and Safe Attachments coverage, alert policies, unified audit log retention, and whether anyone reviews what is generated.
- 08
Reporting and retest
Technical report and executive summary together, with changes sequenced by user impact, then a retest confirming closure.
Approach to testing
- Read-only assessment via a dedicated reviewer account with Global Reader and Security Reader. Nothing is changed and no user is affected.
- Conditional access is assessed by effect, not by count. Fourteen policies with an exclusion group that covers half the organisation is weaker than three with none.
- Actual sharing is enumerated, not only the policy. A tenant configured to prevent anonymous links may still hold thousands created before the policy changed.
- Legacy authentication is tested for reachability rather than read from a setting, because a blocked protocol that still accepts a connection somewhere is not blocked.
- Recommendations state user impact explicitly. Enforcing MFA on every account is correct and will generate service desk volume, and a report that hides that will not be actioned.
Types of assessment
Full tenant review (default)
Every workload against the CIS benchmark. The complete picture and the usual scope.
Identity-focused review
Entra ID, conditional access and privileged access only. The highest-value subset if budget or time is limited.
Data exposure review
SharePoint, OneDrive and Teams sharing, enumerating what is actually shared externally rather than what policy permits.
Post-incident review
Run after a business email compromise, focused on persistence — forwarding rules, OAuth grants, delegate access and app registrations.
Frameworks and standards
- CIS Microsoft 365 Foundations Benchmark
- The primary baseline, assessed control by control at the level appropriate to the client's licensing.
- Microsoft Secure Score
- Used as a tracking metric between reviews, with the caveat that it rewards some actions disproportionately.
- Microsoft Cloud Security Benchmark
- Where the tenant is assessed alongside Azure resources.
- MITRE ATT&CK for Cloud
- Technique mapping for the detection review.
- NIST SP 800-53
- Control mapping where the client has a compliance obligation.
Tools used
Tooling is where testing starts, not where it ends. Every automated result is reproduced by hand before it reaches a report.
Microsoft 365 DSC / Graph API
Read-only export of tenant configuration for offline analysis.
ScubaGear
CISA's automated baseline assessment for M365, used as a cross-check.
Monkey365 / Maester
Automated CIS benchmark evaluation and reporting.
ROADrecon
Entra ID enumeration and relationship analysis.
PowerShell modules
Exchange Online, SharePoint and Teams administration modules for enumeration the Graph does not expose.
Checklist approach
The checklist is the floor, not the ceiling. It guarantees coverage so nothing standard is missed; the findings that matter usually come from what a tester does after it is complete.
Identity and authentication
- MFA coverage across all accounts, including service and shared mailboxes
- Legacy authentication blocked and verified unreachable
- Conditional access coverage, exclusions and report-only policies
- Password policy, banned password list and writeback
- Guest user lifecycle and access review
- Risky sign-in and risky user policy configuration
Privileged access
- Global administrator count against Microsoft's guidance
- Privileged Identity Management eligible versus standing assignment
- Break-glass accounts: exclusion, monitoring and credential storage
- Administrative accounts separate from day-to-day identities
- Application and service principal consent grants
Email security
- SPF, DKIM and DMARC published and aligned, with DMARC enforcement level
- Anti-phishing policy coverage and impersonation protection
- External forwarding blocked or restricted
- Mail flow rules reviewed for bypass and exfiltration paths
- Safe Links and Safe Attachments coverage across all users
- Mailbox auditing enabled and retained
Data sharing
- Tenant and site-level external sharing settings
- Anonymous link creation, expiry and permission level
- Enumeration of existing anonymous links across the tenant
- Sensitivity labels and their application
- DLP policy coverage for regulated data
- Guest access to Teams and the content behind them
Application governance
- User consent settings for third-party applications
- Existing OAuth grants reviewed for over-permissive scopes
- App registration ownership and credential expiry
- Admin consent workflow
Logging and detection
- Unified audit log enabled and retention period
- Alert policies for privileged actions and mass download
- Defender alert routing and who acts on it
- Log export to a SIEM where one exists
How findings are scored
Every finding is scored on CVSS 3.1 and placed in one of five levels. The executive summary adds a sixth band — Compliant — so components that passed appear on the same chart as those that did not.
- Critical
- Immediate measures must be taken. These vulnerabilities can allow an attacker to take complete control of the application or server — stealing user data, tricking users into supplying sensitive information, or defacing the site.
- High
- Maximum risk associated with a specific vulnerability instance. May enable an attacker to compromise the application and its data, partially or completely, or to modify application behaviour beyond its intended purpose. To be handled with utmost priority.
- Medium
- Considerable risk. May enable an attacker to exploit the application to a particular level, gaining low-level information that can be used to craft more specific attacks.
- Low
- Lowest risk. May allow an attacker to gain some information about the application that was not intended to be known, without an exploitation technique currently available at that instance.
- Informational
- A functionality or component is missing best-practice implementation. Not a risk today, but may become one as the application changes or as exploitation techniques, policy or legal requirements evolve.
Scan types selected
- Safe Checks
- Standard / OWASP Top 10
- Destructive
- SANS Top 25
- Business Logic Vulnerability Testing
Standard toolset by stage
- OSINT
- Datasploit, Google Dorks, Shodan
- Enumeration & Scanning
- Nmap, Wfuzz, Unicornscan
- Domain Enumeration
- Nikto, DnsRecon, Knock
- Crawling & Fuzzing
- Burp Suite, Acunetix, Netsparker
- Vulnerability Analysis
- OpenSSL, sqlmap, CVE-Details
- Exploitation
- Metasploit, Netcat, Exploit-DB
How CSS tests
A unified swarm of agents, for blind spot detection
AI agents drive several testing tracks against the same target at once, then cross-check each other. A single tester works one hypothesis at a time; parallel agents cover the space a sequential pass leaves behind.
Conditional access is evaluated by effect across every user, application and condition combination. A tester checks the policies; agents compute which accounts actually end up unprotected, which is the only number that matters.
Anonymous sharing links accumulate over years. Enumerating every one across every site is machine work, and the policy setting says nothing about what already exists.
OAuth grants across hundreds of applications and thousands of users hide the one over-permissive consent that reads all mailboxes.
Mail flow rules interact. Evaluating the combined effect of forty rules on a message is a simulation problem rather than a reading problem.
Every agent finding is validated by a human reviewer. The assessment is read-only throughout: no policy is modified, no user affected, nothing enabled or disabled during the review.
Why this differs
What CSS does that most vendors do not
Every one of these is checkable. Ask any vendor for the same and compare the answers.
Effect, not settings
Most reviews list settings against a benchmark. CSS computes which accounts are actually unprotected once exclusions, licensing gaps and report-only policies are resolved — usually a much shorter and more alarming list.
What is shared, not what may be shared
The review enumerates existing anonymous links and guest access rather than reporting the policy that governs future ones. The historical population is where the exposure is.
Both reports, always
Technical report and executive summary together, plus a change list sequenced by user impact.
Fixation, not a backlog
CSS works the change sequence with the IT team, including which changes need communication to users, and retests to evidence closure.
Reporting
Two documents, two audiences
Both are produced for every engagement. They are not the same document at two lengths — they answer different questions and are written separately. The structure below is the one CSS actually issues.
Technical assessment report
For the engineers who will fix it
- Disclaimer, and Limitations on Disclosure and Use
- Risk Level & Description — the five levels above, scored on CVSS 3.1
- Scan Type — which of the five assessment types were selected
- Assessment Scope — the control areas covered
- Assessment Date — the exact testing window
- Objective of the Assessment — objectives listed against completion status
- Tools Utilization — manual and automated tooling by stage
- Summary of the Assessment
- Overall Recommendations, split into Must Have and Should Have
- Vulnerability Overall Classifications as per Organization
- Security Issues Highlighted
- The Key Findings — each with evidence and detailed recommendation
- Summary of Findings & Conclusion
For this assessment specifically
- Scope: tenant, licensing, workloads and the review date
- Every finding against the CIS Microsoft 365 Benchmark with control references
- Conditional access effectiveness analysis: which accounts are actually unprotected and why
- Enumerated external sharing: links, guests and the content reachable through them
- OAuth grant inventory with over-permissive scopes flagged
- Prioritised change list with user impact for each change
- Retest results appended against each original finding
Executive summary
For the people who will fund the fix
- Objectives, each against a completion status
- Overall Finding of the Assessment — total threats identified, broken down by component and severity
- Summary of the Assessment
- Artefacts of the Assessment — the key findings as a numbered register with severity
- Observation of the Assessment — the major attacks the organisation should be prepared for, given what was found
- Overall Recommendation, including a Business Enabling Recommendation sequence
- Must Have and Should Have actions
For this assessment specifically
- How many accounts could be taken over today, and what that would reach
- Benchmark compliance as a percentage with the material gaps named
- The three changes that most reduce exposure, with their user impact stated
- Regulatory position where the tenant holds regulated data
- Remediation timeline and retest date
- One page
Case studies
What this finds in practice
Representative engagement patterns. Sector and scale only — no client is named, and no detail is included that could identify one.
A professional services firm of around 700 staff.
- Finding
- Conditional access appeared comprehensive with eleven policies. An exclusion group created for a 2022 migration still contained 94 accounts, including four global administrators, none of which required MFA from any location.
- Recommendation
- Empty the exclusion group, replace blanket exclusions with time-bound, individually justified ones, and alert on any addition to a conditional access exclusion.
- Outcome
- Group emptied over two weeks after per-account verification. Alerting on exclusion changes has since caught two additions made during support incidents.
An engineering consultancy using SharePoint for project delivery with clients.
- Finding
- Anonymous link creation had been disabled eighteen months earlier, but 11,400 links created before that change remained live and unexpired. Sampling found drawings, contracts and staff personal data reachable by anyone holding a URL.
- Recommendation
- Expire all historical anonymous links, apply a default expiry to any future link, and re-issue access through authenticated guest accounts where sharing is still required.
- Outcome
- Links expired in batches over three weeks with a client communication plan. 340 were re-issued as authenticated guest access, which indicated how many of the rest were genuinely abandoned.
A logistics company, post-incident review after a business email compromise.
- Finding
- The original intrusion had been remediated by a password reset. The review found a surviving OAuth grant to a third-party application holding Mail.Read across the tenant, and an inbox rule forwarding finance mail to an external address, neither of which a password reset affects.
- Recommendation
- Revoke the grant, remove the rule, restrict user consent to verified publishers with low-impact permissions, and add alerting on new forwarding rules.
- Outcome
- Both persistence mechanisms removed. The consent restriction is what closed the class; the incident response process was updated to include OAuth and forwarding review as standard.
Next
Scope this assessment
Most scopes are settled in one call. Tell us what the application does and who uses it, and we will tell you what testing it properly involves.