What a sales demo can't show you about an MDR provider
By the time you have two or three managed detection and response finalists, the demos all look alike. Every vendor shows a clean console, a well-written sample incident and an analyst who answers right away. None of that tells you what happens when a real alert fires in your environment at 2 a.m. on a Sunday and the on-call analyst has to choose between isolating your finance director's laptop and sending you an email.
A structured MDR proof of concept answers that question before you commit to a contract that often runs one to three years. The plan below takes 30 days and targets four things demos hide: escalation quality, containment authority, after-hours response and false-positive noise. It ends with a weighted scoring sheet so your CISO, CFO and risk manager can compare finalists on the same numbers.
Run it on at least two finalists at once, and don't skip it because a vendor ranks well with analyst firms. Rankings measure the vendor in general. A proof of concept measures the vendor working in your environment.
Looking to streamline vendor RFP selection?
Access pre-negotiated tier discounts through our independent institutional network.
Steps 1–3: Scope, success criteria and written containment authority
Step 1: Pick a representative slice of your environment. Deploy each provider to roughly 50–200 endpoints that match your real mix: a few servers, finance and executive laptops, one cloud workload and your identity provider (Microsoft Entra ID, Okta or Google Workspace). If a vendor can only ingest its own EDR agent, learn that now. A test group made up only of IT staff machines produces almost no realistic signal.
Step 2: Write success criteria before deployment. Agree on measurable targets with each vendor in writing, for example:
- Time from a high-severity detection to a human-confirmed escalation (detection plus triage)
- Time from escalation to containment action (your mean time to respond)
- Share of escalations that include a root cause, affected assets and a recommended action
- Number of escalations per week that turn out to be benign
Step 3: Grant containment authority in writing, and check data handling. Decide which actions the provider may take without calling you first: isolating a host, disabling an account, killing a process or blocking a hash. Put this in a short rules-of-engagement document. Some vendors only advise during a trial and contain only after a contract is signed. If so, you're testing a different service from the one you'll buy, and you should score it that way.
Use the same document to confirm where telemetry is stored and who can access it. Data-protection rules vary by region: the EU and UK GDPR, India's DPDP Act, Brazil's LGPD, Saudi Arabia's PDPL and Singapore's PDPA all bear on cross-border transfers and processor agreements. Have privacy counsel approve the data processing terms before any agent is installed, even for a trial.
A 30-day timeline that fits around normal operations
The schedule below assumes one internal owner who can give the test about 4–6 hours a week. Finalists run in parallel on separate endpoint groups so they face the same conditions.
| Days | Activity | What you're measuring |
|---|---|---|
| 1–5 | Agent rollout, log integrations, rules of engagement signed | Deployment effort, onboarding responsiveness, integration gaps |
| 6–10 | Baseline period, no simulations | Organic alert volume, tuning questions, early false positives |
| 11–20 | Unannounced attack simulations, including two outside business hours | Detection coverage, escalation quality, mean time to respond |
| 21–25 | One live tabletop with the provider's incident lead | Communication under pressure, containment judgment |
| 26–30 | Report review, reference calls, scoring | Reporting clarity, commercial terms, final weighted score |
On cost: some providers run a trial free of charge for qualified buyers. Others charge for a paid pilot, often somewhere in the low-to-mid four figures up to the low tens of thousands of dollars depending on scope, and frequently credit it against year one. Full MDR contracts for mid-market buyers commonly land in a broad range of roughly $8–$30 per endpoint per month, usually with minimum annual fees. These are indicative 2026 market ranges that vary by contract and region, and local pricing in Europe, India, the Gulf and Latin America often differs from headline rates, so confirm with each vendor.
Steps 4–5: Attack simulation testing, including the 2 a.m. test
Step 4: Run unannounced simulations mapped to MITRE ATT&CK. Tell the vendor's account team that testing will happen during the window. Don't tell the SOC when. Open-source frameworks such as Atomic Red Team, or commercial breach-and-attack simulation tools, let you repeat the same techniques on every finalist. A practical set of five to eight scenarios:
- Credential dumping from LSASS on a workstation
- A suspicious PowerShell download cradle
- A new local admin account plus a persistence mechanism
- Impossible-travel or token-theft sign-in against your identity provider
- Mass file renames on a test share that look like early ransomware
- A cloud API key used from an unfamiliar location
Step 5: Fire at least two scenarios outside business hours. Choose a weekend and a public holiday in your main operating country. A 24/7 claim means little if the overnight shift is thin or covers several regions from one location. Record exact timestamps: when the technique ran, when the provider detected it, when a human contacted you and when containment happened. Pull timestamps from your own logs as well as the vendor's portal.
If you have an incident response retainer or a cyber insurer with a breach panel, tell them about the simulations in advance. Some policies and panel arrangements carry notification expectations, and a test shouldn't look like a real claim.
Looking to streamline vendor RFP selection?
Access pre-negotiated tier discounts through our independent institutional network.
Steps 6–7: Grading escalation quality and containment decisions
Step 6: Score each escalation on content. A fast escalation that says "suspicious activity on HOST-114, please investigate" just hands the work back to you. A good one says what happened, how the provider knows, which accounts and hosts are affected, what it has already done and what you need to decide. Grade every escalation from 1 to 5 against those five elements, and have two people score them independently.
Step 7: Test judgment, not only speed. In the day 21–25 tabletop, give the provider's incident lead a messy scenario. Say a 300-person manufacturer finds ransomware precursors on a server that runs its production line, and isolating the server stops the plant. Watch whether the lead asks about business impact before isolating, offers narrower options such as disabling the account or blocking the process, and states clearly who makes the final call.
Red flags at this stage:
- Escalations sent by email only, with no phone call for high severity
- Analysts who can't explain why an alert fired
- Containment that waited for your approval even though you pre-authorized it
- A different contact on every escalation and no named incident lead
- Reluctance to put response-time targets in the contract
Step 8: Counting false-positive noise and the work it pushes back to you
Noise is the hidden cost in MDR. A provider that escalates dozens of benign events a week is using your team as its triage layer. Track each finalist's escalations across the full 30 days and sort them into three buckets: true positive, benign but explained, and unexplained or wrong.
Then estimate the internal hours. Hypothetical example: if Provider A sends 25 escalations a week and each takes your analyst 20 minutes to close, that's more than eight hours a week, roughly a fifth of a full-time role. Depending on country, a loaded analyst cost can run from tens of thousands to well over $100,000 a year, so that time can cancel out a lower per-endpoint price.
Also note how each provider handles tuning. The better ones ask questions during the baseline period, suppress known-good activity with your approval and show you the suppression list. A provider that quietly lowers alert volume without telling you what it suppressed creates a different risk: blind spots you can't audit.
Step 9: The scoring sheet for comparing finalists
Set the weights before you see any results so scores can't be bent to justify a favorite. The weights below suit most mid-size buyers. Regulated firms in financial services or healthcare may want to raise reporting and data handling.
| Criterion | Weight | How to score (1–5) |
|---|---|---|
| Detection coverage across simulations | 20% | Share of scenarios detected and escalated |
| Mean time to respond, including after-hours | 20% | Against the targets agreed in Step 2 |
| Escalation quality | 15% | Average of independent 1–5 grades |
| Containment authority and judgment | 15% | Acted within agreed rules; tabletop performance |
| False-positive noise | 10% | Benign escalations per week and internal hours |
| Integration and deployment effort | 10% | Coverage of identity, cloud and existing tools |
| Reporting, data residency and contract terms | 10% | Clarity, regional compliance, SLA commitments |
Use the sheet as a gate as well as a ranking. Disqualify any provider that missed an after-hours simulation completely, however well it scores elsewhere. Among those still in, pick the highest weighted score unless the gap is under 0.3 points. In that case, pricing and contract flexibility should decide it. Before you sign, write the response-time targets you measured into the service schedule, and have your broker check whether the chosen provider meets any MDR or EDR requirements in your cyber insurance policy.