GitHub Investigating Critical Performance Degradation Across Multiple Services

This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available.

We are continuing to apply mitigations to address sporadic Copilot authentication failures in some applications. We expect full recovery within the next 30 minutes. Copilot usage via the GitHub CLI and GitHub App are unaffected.

Issues is operating normally.

We are continuing to investigate sporadic failures affecting Copilot authentication in some applications. Copilot usage via the GitHub CLI and GitHub App are unaffected.

We are continuing to investigate sporadic authentication failures. We have partially disabled authentication token retries and have seen improvement, and we are monitoring impact before fully applying this mitigation.

API Requests is operating normally.

API Requests is experiencing degraded availability. We are continuing to investigate.

The degradation affecting Git Operations has been mitigated. We are monitoring to ensure stability.

We identified the problematic component and have taken corrective actions, but we are seeing residual impact in the form of sporadic authentication failures. We are continuing to apply additional mitigations and investigate the remaining impact.

Issues is experiencing degraded performance. We are continuing to investigate.

We identified the problematic component and have taken corrective actions, but we are seeing residual impact across numerous services. We are continuing to apply additional mitigations and investigate the remaining impact.

Git Operations is experiencing degraded performance. We are continuing to investigate.

The degradation affecting API Requests, Actions, Git Operations, Issues, Pages, Pull Requests and Webhooks has been mitigated. We are monitoring to ensure stability.

We identified the problematic component and have taken corrective actions. There are strong signs of recovery but we are still working to completely restore service, with error rates still remaining slightly elevated. We will post further updates as recovery continues.

We are experiencing high error rates around 20% for web experiences and api traffic. Archive downloads and raw repository content downloads are experiencing an approximate 50% error rate. SAML and OIDC authentication, SCIM, and Team Sync are also impacted. We are still working to identify the root cause and will continue to post updates as we learn more and perform mitigation.

We are experiencing high error rates around 20% for web experiences and api traffic. Archive downloads and raw repository content downloads are experiencing an approximate 50% error rate. SAML and OIDC authentication, SCIM, and Team Sync are also impacted. We are currently performing mitigations and will post updates as we progress.

Webhooks is experiencing degraded performance. We are continuing to investigate.

Git Operations is experiencing degraded performance. We are continuing to investigate.

Pages is experiencing degraded performance. We are continuing to investigate.

API Requests is experiencing degraded availability. We are continuing to investigate.

Webhooks is experiencing degraded availability. We are continuing to investigate.

We are experiencing high error rates around 20% for web experiences and api traffic. Archive downloads and raw repository content downloads are experiencing an approximate 50% error rate. SAML and OIDC authentication, SCIM, and Team Sync are also impacted. We are currently performing mitigations based on our investigation thus far and are monitoring for improvement.

Actions is experiencing degraded availability. We are continuing to investigate.

Pull Requests is experiencing degraded availability. We are continuing to investigate.

Issues is experiencing degraded availability. We are continuing to investigate.

Pull Requests is experiencing degraded availability. We are continuing to investigate.

Copilot is experiencing degraded availability. We are continuing to investigate.

We are experiencing high error rates around 20% for web experiences and api traffic. Archive downloads and raw repository content downloads are experiencing an approximate 50% error rate. SAML and OIDC authentication, SCIM, and Team Sync are also impacted. Investigations are on-going and we will continue to provide updates as we discover more information.

We are experiencing high error rates around 20% for web experiences and api traffic. Archive downloads and raw repository content downloads are experiencing an approximate 50% error rate. Investigations are on-going into the root cause, and updates will continue to be provided as we investigate.

Pull Requests is experiencing degraded performance. We are continuing to investigate.

Issues is experiencing degraded performance. We are continuing to investigate.

We are seeing an approximate 20% error rate across numerous experiences including Pull Requests, Issues, and others. Investigations are currently under way and we will be posting updates as they become available

Webhooks is experiencing degraded performance. We are continuing to investigate.

Actions is experiencing degraded performance. We are continuing to investigate.

API Requests is experiencing degraded performance. We are continuing to investigate.

We are investigating reports of impacted performance for some GitHub services.