LaunchDarkly reports multiple service issues

We are continuing to monitor for any further issues, see our previous update for recovery steps.

Customers continue to successfully reestablish connections to our streaming services. We recommend reviewing your application logs for the incident window. Affected SDKs emit messages—varying by language—that contain any of the following: "giving up permanently," "Invalid SDK key," "unauthorized," "not authorized," or "401." Any service that logged these messages may be unable to receive flag updates or send event data until it is restarted. Here are the recommended actions for each affected application: - If you use server-side SDKs, please restart the application to re-establish its connection to LaunchDarkly. - If you use the Relay Proxy, please restart the Relay Proxy to re-establish connections. It should not be necessary to restart the applications behind it. - Client-side SDKs will reconnect automatically as they restart. - If this does not resolve the issue for an affected application (e.g. you still see the log messages from above), please log a support ticket and we will follow up with you directly to assist with remediation.

Customers continue to successfully reestablish connections to our streaming services. We recommend reviewing your application logs for the incident window. Affected SDKs emit messages—varying by language—that contain any of the following: "giving up permanently," "Invalid SDK key," "unauthorized," "not authorized," or "401." Any service that logged these messages may be unable to receive flag updates or send event data until it is restarted. Here are the recommended actions for each affected application: - If you use server-side SDKs, please restart the application to re-establish its connection to LaunchDarkly. - If you use the Relay Proxy, please restart the Relay Proxy to re-establish connections. It should not be necessary to restart the applications behind it. - Client-side SDKs will reconnect automatically as they restart. - If this does not resolve the issue for an affected application (e.g. you still see the log messages from above), please log a support ticket and we will follow up with you directly to assist with remediation.

Our Flag Delivery services have recovered and we're observing an increase in customer traffic successfully reestablish connections to our streaming services. Customers who continue to encounter 401 connection errors in their SDKs or Relay Proxy instances are advised to restart their applications and/or Relay Proxy instances.

Our Flag Delivery services have recovered and we're observing an increase in customer traffic successfully reestablish connections to our streaming services. Customers who continue to encounter 401 connection errors in their SDKs or Relay Proxy instances are advised to restart their applications and/or Relay Proxy instances.

We are investigating continued impact to customers' server SDK caused by the previous incident (https://status.launchdarkly.com/incidents/chywwz01ptb0). If customers are receiving SDK or Relay Proxy 401s please re-establish connections.

We are investigating continued impact to customers' mobile and server SDK caused by the previous incident (https://status.launchdarkly.com/incidents/chywwz01ptb0). If customers are receiving SDK or Relay Proxy 401s please re-establish connections.

We are investigating continued impact to customers' mobile and server SDK caused by the previous incident (https://status.launchdarkly.com/incidents/chywwz01ptb0). If customers are receiving SDK or Relay Proxy 401s please re-establish connections.

All service availability has been fully restored. Customers using server-side SDKs might need to restart their applications to re-enable the SDKs' connection to LaunchDarkly. Customers using the Relay Proxy may also need to restart their Relay Proxy; in this case, the Relay Proxy should be restarted before restarting their applications that connect to it. To determine if an application needs to be restarted, customers can look at their application logs. Affected SDKs will emit (depending on language) logs containing one of "giving up permanently", "Invalid SDK key", "unauthorized", or "not authorized", or "401". Services which emitted logs such as these before 9:13AM PT may be unable to receive flag updates and will be unable to send events to LaunchDarkly until they are restarted.

Production service impact has been restored. However, customers might need to restart their applications to fully reconnect to LaunchDarkly's services. Our engineering team is working on guidance that can be provided to customers.

We are continuing to work on a fix for this issue.

The web application availability has recovered. Flag delivery and event ingestion continue to be impacted and are actively recovering. Our engineering team continues to work on remediation.

We have identified the root cause and are implementing a fix.

The LaunchDarkly web application (app.launchdarkly.com) was unavailable and is actively recovering at the moment. The Flag Delivery network was also impacted, causing an elevated failure rate with retrieving flag values. Customers using client-side or server-side SDKs should continue to see the last known flag values if a local cache exists, or fall back to in-code values. Customers using our Relay Proxy should continue to see last known flag values if a local cache exists. Customers using our Edge SDKs should continue to see last known flag values. Some event data loss may have occurred.

We are continuing to work on a fix for this issue.

The issue has been identified and a fix has been implemented

The issue has been identified and a fix has been implemented

We are continuing to investigate this issue.

We are continuing to investigate this issue.

We are currently investigating this issue

We are currently investigating this issue

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We have identified the issue and are working towards releasing a fix.

We are currently investigating an issue where experiments using frequentist analysis cannot be saved or started.