Protected Chat ·
Privacy Test Results
We tested how well Protected Chat masks identifiers and blocks sensitive content using 100 synthetic scenarios.
Latest full run
96 of 100 scenarios passed.
- 96 passed
- 1 failed
- 3 inconclusive
What the failed case showed
The gateway masked a phone number but missed the fictional caregiver’s name beside it. We added a rule for phrases such as “Caregiver [name] called.” All four affected cases passed in a targeted replay. The full 100-case suite has not been rerun since that fix.
Results by test group
| Test group | Passed / tested |
|---|---|
| Clean notes that should pass unchanged | 30 / 30 |
| Identifier masking | 31 / 32 |
| Sensitive content policy | 23 / 26 |
| Privacy across chat turns | 12 / 12 |
How to read the results
Each scenario counted once. A case passed when every turn produced the expected privacy decision and protected the specified identifiers. The three inconclusive cases returned a detector error and sent nothing.
We inspected the requests the gateway prepared for the cloud with a simulated cloud connection. The updated gateway also passed 15 of 15 fresh synthetic scenarios in a separate evaluation. These results describe the tested examples; they do not establish accuracy on real patient records.