How Microsoft Is Taking Down AI Hackers Who Create Harmful Images Of Celebrities And Others
Microsoft, Monday, December 29th, 2025
It was a slow Friday afternoon in July when a seemingly isolated problem appeared on the radar of Phillip Misner, head of Microsoft's AI Incident Detection and Response team. Someone had stolen a customer's unique access code for an AI image generator and was going around safeguards to create sexualized images of celebrities.
Misner and his coworkers revoked the code but soon saw more stolen customer credentials, or API keys, pop up on an anonymous message board known for spreading hateful material. They escalated the issue into a company-wide security response in what has now become Microsoft's first legal case to stop people from creating harmful AI content.
'We take the misuse of AI very seriously and recognize the harm of abusive images to victims,' Misner says.