Intermittent errors

Incident Report for Connect CRM APIs

Resolved

CRM and API performance have returned to normal levels.
Posted Feb 24, 2020 - 15:14 CST

Monitoring

After confirming CRM stability, we are now taking the Contact API out of maintenance mode. We continue to monitor the CRM's performance for slowness and will take corrective actions as necessary.
Posted Feb 24, 2020 - 13:50 CST

Identified

CRM performance has improved significantly for clients; however, we will leave the Contact endpoint in maintenance mode while engineering and I/T bring online other processes which were disabled during troubleshooting. We will remove maintenance mode once we receive the all clear that supporting processes are again online.
Posted Feb 24, 2020 - 12:02 CST

Update

We have placed the Contact API in maintenance mode while teams troubleshoot it. Requests to this endpoint will be responded to with HTTP 503 - Service Unavailable, with an additional header "Scheduled Maintenance."
Posted Feb 24, 2020 - 11:20 CST

Update

We are also investigating timeouts associated with requests to the Lead endpoints.
Posted Feb 24, 2020 - 10:01 CST

Investigating

We are currently investigating reports of intermittent HTTP 500 / 504 responses affecting primarily the Contact endpoints. Multiple teams are engaged as this is a system-wide issue affecting the CRM.
Posted Feb 24, 2020 - 09:55 CST
This incident affected: APIs (Contacts API, Leads API).