Intermittent Unavailability of Web Frontend

Incident Report for Scrutinizer

Resolved

This incident has been resolved.
Posted Oct 25, 2019 - 18:14 CEST

Update

The recovery process is still running, and we expect it to take another 2 hours before completing. Certain git operations are still degraded during this time. We will continuing to post status updates here.
Posted Oct 25, 2019 - 15:15 CEST

Update

We are continuing to monitor for any further issues.
Posted Oct 25, 2019 - 11:25 CEST

Monitoring

The recovery process is still running. All data is already available, but at a reduced redundancy. Due to the ongoing recovery process and restoring of full data redundancy, we are currently seeing git operations to take longer than usual.

This can affect individual pages which display source code or access other git information.
Posted Oct 25, 2019 - 10:28 CEST

Update

We have identified a degradation in the storage that serves git data for various parts of the platform. A recovery process has been triggered and is currently running.
Posted Oct 25, 2019 - 08:44 CEST

Identified

The issue has been identified and a fix is being implemented.
Posted Oct 25, 2019 - 08:32 CEST

Investigating

We are currently investigating this issue.
Posted Oct 25, 2019 - 08:01 CEST
This incident affected: Web.