When Incident Records Train AI Models: New Risks Emerge
Security incident records traditionally served a single critical purpose: documenting organizational decisions and providing evidence for accountability. Vendors now introduce a secondary use—leveraging these records for AI model development and validation. Microsoft Security Copilot exemplifies this pattern, offering customers a setting that allows Microsoft to capture interaction data to enhance its security-specific model (built on Azure OpenAI). The captured data includes whatever Microsoft plug-ins and agents have used during product interactions. Microsoft’s documentation states this data does not train its foundation model or transfer to OpenAI. Data-sharing defaults vary by provisioning method, with sharing enabled by default except for Microsoft 365 E5 and E7 auto-provisioning. Customers who haven’t reviewed settings pages may not know which default applies to them.
Large collections of incident records reveal organizational security patterns and decision thresholds without requiring external access to individual ticket contents. Record patterns expose which signals the organization treats as meaningful, which alerts it suppresses, acceptable alert volumes, which detections analysts mark benign, and where detection gaps exist. This creates the article’s central concern: these records now exist in two places—the organizational archive and the vendor’s copies—each subject to different threats.
Dual use creates accountability and security risks. Vendor breaches could expose records outside organizational control; legal demands might force disclosure without the organization’s involvement. Original retention and access rules were designed for organizational archives, not third-party copies. The article recommends contracts should explicitly separate evidence retention from model development, requiring distinct permissions and preventing generic AI clauses from covering rule content, closure notes, and ticket text.
Read the full article
Continue reading at Dark Reading →This is an AI-generated summary. Read the original for the full story.