Degraded performance in the EU-West-2 (Paris) region

Incident Report for Wasabi Technologies

Postmortem

On Sunday 17 November at approximately 00:14 UTC, an incident occurred in the eu-central-2 (Paris) region, resulting in elevated HTTP 5XX error responses and slow response times to buckets impacting some customers.

During an upgrade to the region, an error caused the service to allocate more connections than needed resulting in an overload to one of the service components. This caused an increase in errors returned to a subset of customer buckets which were served by this component. To address the situation, the Operations Team reverted the update to restore service.  By 15:11 UTC on 17 November 2024 service in the eu-central-2 region was fully restored to normal operational status.  The error was corrected and the update was  subsequently re-deployed successfully.

Posted 4 months ago. Dec 06, 2024 - 11:32 UTC

Resolved

This incident is now resolved.
Posted 5 months ago. Nov 17, 2024 - 02:14 UTC

Monitoring

We have identified the issue and are making changes to address the traffic. We are continuing to monitor the system as well
Posted 5 months ago. Nov 17, 2024 - 01:21 UTC

Update

Our Operations team continues to work with the network traffic team to isolate the problem, we will update this page with more information soon
Posted 5 months ago. Nov 17, 2024 - 00:48 UTC

Investigating

We are currently investigating degraded performance in the EU-West-2 (Paris) region
Posted 5 months ago. Nov 17, 2024 - 00:06 UTC
This incident affected: EU-West-2 (Paris).