Provisioning Microsoft Services Issues

On 23 November, 2025, between 06:03 – 12:38 UTC, customers attempting to purchase Microsoft subscriptions through the BSS and Storefront applications experienced intermittent errors. The provisioning service responsible for handling Microsoft subscription requests was partially degraded due to a hardware fault on one of the backend servers. The faulty server caused inconsistent processing outcomes and disrupted some customer transactions.

The root cause of the incident was a disk failure on one of the servers used by the Microsoft provisioning service. The failing disk disrupted the server’s ability to reliably record and process provisioning-related information. Although the server remained partially operational, the corrupted disk behavior caused inconsistent responses, which manifested as intermittent failures during customer purchase attempts.

During this period, users experienced intermittent failures and difficulty to complete any purchase of Microsoft subscriptions. Service was fully restored at 12:38 UTC, and since then all systems are operating normally.

Early alerts did not correlate strongly with the provisioning failures until additional customer reports surfaced.

The issue was resolved by isolating the problematic server from production traffic. No customer data was lost or compromised. Preventive actions are being identified to ensure similar failures are automatically contained before affecting customers.

Impact

• Temporary inability for some customers to complete subscription purchases
• Some provisioning requests stalling or failing on first attempt
• No data was lost.
• The issue was resolved by our staff within 6h 35m

What We’re Doing Next

To prevent this from happening again, we are are going to:

• automatically remove the affected server from the service pool in case any degradation is detected without requiring manual intervention
• update alerting logic to better distinguish between intermittent provisioning failures and full service failures

Current Status

The issue has been fully resolved, and the service has been running normally since the fix.

We apologize for the inconvenience this caused and thank you for your patience and understanding.