Incidents | Modal Labs Incidents reported on status page for Modal Labs https://status.modal.com/ https://d1lppblt9t2x15.cloudfront.net/logos/4aed1028d24910fc147a2962adc30011.png Incidents | Modal Labs https://status.modal.com/ en Sandbox exec processes fail after 5 minutes https://status.modal.com/incident/611946 Mon, 30 Jun 2025 19:59:00 -0000 https://status.modal.com/incident/611946#1a9a91367ef8891dd40b29a664baf265df654e11eda0f816e1a17b2225558534 Bug has been reverted out of production. All new exec processes will now function correctly. Sandbox exec processes fail after 5 minutes https://status.modal.com/incident/611946 Mon, 30 Jun 2025 19:40:00 -0000 https://status.modal.com/incident/611946#2406618c0d02472fef8dc37e3963b8b24052744bf0d0ae8f214fec548c76b856 Due to an incorrectly expired internal channel, operations against a Sandbox exec process which are older than 5 minutes fail. We are rolling back the change. Web endpoints and Web API are down https://status.modal.com/incident/609934 Fri, 27 Jun 2025 03:08:12 -0000 https://status.modal.com/incident/609934#3eabbcbde2a6b441c2ab153b82e2986c371f2dcc061930e4f962f6a1c4582ecd Web API recovered. Service incident https://status.modal.com/incident/609942 Fri, 27 Jun 2025 03:08:00 -0000 https://status.modal.com/incident/609942#72292064395e879fd271e28477378a591483c54a93ef75774e335e6acd3333cf We have rolled back the affected component and services have recovered. Service incident https://status.modal.com/incident/609942 Fri, 27 Jun 2025 03:08:00 -0000 https://status.modal.com/incident/609942#72292064395e879fd271e28477378a591483c54a93ef75774e335e6acd3333cf We have rolled back the affected component and services have recovered. Service incident https://status.modal.com/incident/609942 Fri, 27 Jun 2025 03:08:00 -0000 https://status.modal.com/incident/609942#72292064395e879fd271e28477378a591483c54a93ef75774e335e6acd3333cf We have rolled back the affected component and services have recovered. Service incident https://status.modal.com/incident/609942 Fri, 27 Jun 2025 03:08:00 -0000 https://status.modal.com/incident/609942#72292064395e879fd271e28477378a591483c54a93ef75774e335e6acd3333cf We have rolled back the affected component and services have recovered. Service incident https://status.modal.com/incident/609942 Fri, 27 Jun 2025 03:08:00 -0000 https://status.modal.com/incident/609942#72292064395e879fd271e28477378a591483c54a93ef75774e335e6acd3333cf We have rolled back the affected component and services have recovered. Service incident https://status.modal.com/incident/609942 Fri, 27 Jun 2025 03:08:00 -0000 https://status.modal.com/incident/609942#72292064395e879fd271e28477378a591483c54a93ef75774e335e6acd3333cf We have rolled back the affected component and services have recovered. Web endpoints and Web API are down https://status.modal.com/incident/609934 Fri, 27 Jun 2025 03:06:14 -0000 https://status.modal.com/incident/609934#919ef97c210791c292882844321cf56f9d3cba860ce3c114e7ba164fc3e90560 Web endpoints recovered. Web endpoints and Web API are down https://status.modal.com/incident/609934 Fri, 27 Jun 2025 02:57:16 -0000 https://status.modal.com/incident/609934#f1d58d982eff54e6e4cad85f73256f2108af69df17505a0c0ed7d5809e0c8b25 Web endpoints went down. Web endpoints and Web API are down https://status.modal.com/incident/609934 Fri, 27 Jun 2025 02:53:25 -0000 https://status.modal.com/incident/609934#a9922cd77e943561d19f6b5275eeeb5c85ae7810e341650f2866fe8cd1c5e9b4 Web API went down. Service incident https://status.modal.com/incident/609942 Fri, 27 Jun 2025 02:51:00 -0000 https://status.modal.com/incident/609942#3890bc20bf2fe03dc6b2133c02a348f1f3ab9709a3755e542cd3fb16165ea405 Multiple services are unavailable. Service incident https://status.modal.com/incident/609942 Fri, 27 Jun 2025 02:51:00 -0000 https://status.modal.com/incident/609942#3890bc20bf2fe03dc6b2133c02a348f1f3ab9709a3755e542cd3fb16165ea405 Multiple services are unavailable. Service incident https://status.modal.com/incident/609942 Fri, 27 Jun 2025 02:51:00 -0000 https://status.modal.com/incident/609942#3890bc20bf2fe03dc6b2133c02a348f1f3ab9709a3755e542cd3fb16165ea405 Multiple services are unavailable. Service incident https://status.modal.com/incident/609942 Fri, 27 Jun 2025 02:51:00 -0000 https://status.modal.com/incident/609942#3890bc20bf2fe03dc6b2133c02a348f1f3ab9709a3755e542cd3fb16165ea405 Multiple services are unavailable. Service incident https://status.modal.com/incident/609942 Fri, 27 Jun 2025 02:51:00 -0000 https://status.modal.com/incident/609942#3890bc20bf2fe03dc6b2133c02a348f1f3ab9709a3755e542cd3fb16165ea405 Multiple services are unavailable. Service incident https://status.modal.com/incident/609942 Fri, 27 Jun 2025 02:51:00 -0000 https://status.modal.com/incident/609942#3890bc20bf2fe03dc6b2133c02a348f1f3ab9709a3755e542cd3fb16165ea405 Multiple services are unavailable. Infrastructure Upgrade https://status.modal.com/incident/597829 Sat, 07 Jun 2025 05:00:21 +0000 https://status.modal.com/incident/597829#40d54170bfb7e7054c954408a0ad2c2032a279d0363188b0b62a29c9451e548e Maintenance completed Infrastructure Upgrade https://status.modal.com/incident/597829 Sat, 07 Jun 2025 05:00:21 +0000 https://status.modal.com/incident/597829#40d54170bfb7e7054c954408a0ad2c2032a279d0363188b0b62a29c9451e548e Maintenance completed Infrastructure Upgrade https://status.modal.com/incident/597829 Sat, 07 Jun 2025 05:00:21 +0000 https://status.modal.com/incident/597829#40d54170bfb7e7054c954408a0ad2c2032a279d0363188b0b62a29c9451e548e Maintenance completed Infrastructure Upgrade https://status.modal.com/incident/597829 Sat, 07 Jun 2025 05:00:21 +0000 https://status.modal.com/incident/597829#40d54170bfb7e7054c954408a0ad2c2032a279d0363188b0b62a29c9451e548e Maintenance completed Infrastructure Upgrade https://status.modal.com/incident/597829 Sat, 07 Jun 2025 05:00:21 +0000 https://status.modal.com/incident/597829#40d54170bfb7e7054c954408a0ad2c2032a279d0363188b0b62a29c9451e548e Maintenance completed Infrastructure Upgrade https://status.modal.com/incident/597829 Sat, 07 Jun 2025 05:00:21 +0000 https://status.modal.com/incident/597829#40d54170bfb7e7054c954408a0ad2c2032a279d0363188b0b62a29c9451e548e Maintenance completed Infrastructure Upgrade https://status.modal.com/incident/597829 Sat, 07 Jun 2025 05:00:21 +0000 https://status.modal.com/incident/597829#40d54170bfb7e7054c954408a0ad2c2032a279d0363188b0b62a29c9451e548e Maintenance completed Infrastructure Upgrade https://status.modal.com/incident/597829 Sat, 07 Jun 2025 03:59:21 -0000 https://status.modal.com/incident/597829#ffb6ca123dd16abf89389d1b4bf0519581c5c6930b68fd7cfa25e82f3dadae23 We're performing a scaling update to a critical infrastructure component. There may be a short window (estimated 2 minutes) of downtime. We don't anticipate that workloads (i.e. functions, webhooks) will be affected. However, there may be UI issues and task scheduling delay. Infrastructure Upgrade https://status.modal.com/incident/597829 Sat, 07 Jun 2025 03:59:21 -0000 https://status.modal.com/incident/597829#ffb6ca123dd16abf89389d1b4bf0519581c5c6930b68fd7cfa25e82f3dadae23 We're performing a scaling update to a critical infrastructure component. There may be a short window (estimated 2 minutes) of downtime. We don't anticipate that workloads (i.e. functions, webhooks) will be affected. However, there may be UI issues and task scheduling delay. Infrastructure Upgrade https://status.modal.com/incident/597829 Sat, 07 Jun 2025 03:59:21 -0000 https://status.modal.com/incident/597829#ffb6ca123dd16abf89389d1b4bf0519581c5c6930b68fd7cfa25e82f3dadae23 We're performing a scaling update to a critical infrastructure component. There may be a short window (estimated 2 minutes) of downtime. We don't anticipate that workloads (i.e. functions, webhooks) will be affected. However, there may be UI issues and task scheduling delay. Infrastructure Upgrade https://status.modal.com/incident/597829 Sat, 07 Jun 2025 03:59:21 -0000 https://status.modal.com/incident/597829#ffb6ca123dd16abf89389d1b4bf0519581c5c6930b68fd7cfa25e82f3dadae23 We're performing a scaling update to a critical infrastructure component. There may be a short window (estimated 2 minutes) of downtime. We don't anticipate that workloads (i.e. functions, webhooks) will be affected. However, there may be UI issues and task scheduling delay. Infrastructure Upgrade https://status.modal.com/incident/597829 Sat, 07 Jun 2025 03:59:21 -0000 https://status.modal.com/incident/597829#ffb6ca123dd16abf89389d1b4bf0519581c5c6930b68fd7cfa25e82f3dadae23 We're performing a scaling update to a critical infrastructure component. There may be a short window (estimated 2 minutes) of downtime. We don't anticipate that workloads (i.e. functions, webhooks) will be affected. However, there may be UI issues and task scheduling delay. Infrastructure Upgrade https://status.modal.com/incident/597829 Sat, 07 Jun 2025 03:59:21 -0000 https://status.modal.com/incident/597829#ffb6ca123dd16abf89389d1b4bf0519581c5c6930b68fd7cfa25e82f3dadae23 We're performing a scaling update to a critical infrastructure component. There may be a short window (estimated 2 minutes) of downtime. We don't anticipate that workloads (i.e. functions, webhooks) will be affected. However, there may be UI issues and task scheduling delay. Infrastructure Upgrade https://status.modal.com/incident/597829 Sat, 07 Jun 2025 03:59:21 -0000 https://status.modal.com/incident/597829#ffb6ca123dd16abf89389d1b4bf0519581c5c6930b68fd7cfa25e82f3dadae23 We're performing a scaling update to a critical infrastructure component. There may be a short window (estimated 2 minutes) of downtime. We don't anticipate that workloads (i.e. functions, webhooks) will be affected. However, there may be UI issues and task scheduling delay. Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 20:00:28 +0000 https://status.modal.com/incident/575767#bfaaf0a4bbd31a04078d5c92c0a45e60a71d5dd69268bf11dcd4cebe5e662312 Maintenance completed Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 20:00:28 +0000 https://status.modal.com/incident/575767#bfaaf0a4bbd31a04078d5c92c0a45e60a71d5dd69268bf11dcd4cebe5e662312 Maintenance completed Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 20:00:28 +0000 https://status.modal.com/incident/575767#bfaaf0a4bbd31a04078d5c92c0a45e60a71d5dd69268bf11dcd4cebe5e662312 Maintenance completed Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 20:00:28 +0000 https://status.modal.com/incident/575767#bfaaf0a4bbd31a04078d5c92c0a45e60a71d5dd69268bf11dcd4cebe5e662312 Maintenance completed Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 20:00:28 +0000 https://status.modal.com/incident/575767#bfaaf0a4bbd31a04078d5c92c0a45e60a71d5dd69268bf11dcd4cebe5e662312 Maintenance completed Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 20:00:28 +0000 https://status.modal.com/incident/575767#bfaaf0a4bbd31a04078d5c92c0a45e60a71d5dd69268bf11dcd4cebe5e662312 Maintenance completed Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 20:00:28 +0000 https://status.modal.com/incident/575767#bfaaf0a4bbd31a04078d5c92c0a45e60a71d5dd69268bf11dcd4cebe5e662312 Maintenance completed Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 20:00:28 +0000 https://status.modal.com/incident/575767#bfaaf0a4bbd31a04078d5c92c0a45e60a71d5dd69268bf11dcd4cebe5e662312 Maintenance completed Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 20:00:28 +0000 https://status.modal.com/incident/575767#bfaaf0a4bbd31a04078d5c92c0a45e60a71d5dd69268bf11dcd4cebe5e662312 Maintenance completed Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 20:00:28 +0000 https://status.modal.com/incident/575767#bfaaf0a4bbd31a04078d5c92c0a45e60a71d5dd69268bf11dcd4cebe5e662312 Maintenance completed Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 18:00:28 -0000 https://status.modal.com/incident/575767#1e500f850b9e735ebff37a9174deb7a91f6a0f908bca8c1dd679368a40ec0772 We will be conducting internal networking updates that affect our main API server. Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 18:00:28 -0000 https://status.modal.com/incident/575767#1e500f850b9e735ebff37a9174deb7a91f6a0f908bca8c1dd679368a40ec0772 We will be conducting internal networking updates that affect our main API server. Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 18:00:28 -0000 https://status.modal.com/incident/575767#1e500f850b9e735ebff37a9174deb7a91f6a0f908bca8c1dd679368a40ec0772 We will be conducting internal networking updates that affect our main API server. Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 18:00:28 -0000 https://status.modal.com/incident/575767#1e500f850b9e735ebff37a9174deb7a91f6a0f908bca8c1dd679368a40ec0772 We will be conducting internal networking updates that affect our main API server. Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 18:00:28 -0000 https://status.modal.com/incident/575767#1e500f850b9e735ebff37a9174deb7a91f6a0f908bca8c1dd679368a40ec0772 We will be conducting internal networking updates that affect our main API server. Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 18:00:28 -0000 https://status.modal.com/incident/575767#1e500f850b9e735ebff37a9174deb7a91f6a0f908bca8c1dd679368a40ec0772 We will be conducting internal networking updates that affect our main API server. Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 18:00:28 -0000 https://status.modal.com/incident/575767#1e500f850b9e735ebff37a9174deb7a91f6a0f908bca8c1dd679368a40ec0772 We will be conducting internal networking updates that affect our main API server. Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 18:00:28 -0000 https://status.modal.com/incident/575767#1e500f850b9e735ebff37a9174deb7a91f6a0f908bca8c1dd679368a40ec0772 We will be conducting internal networking updates that affect our main API server. Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 18:00:28 -0000 https://status.modal.com/incident/575767#1e500f850b9e735ebff37a9174deb7a91f6a0f908bca8c1dd679368a40ec0772 We will be conducting internal networking updates that affect our main API server. Internal networking update https://status.modal.com/incident/575767 Thu, 22 May 2025 18:00:28 -0000 https://status.modal.com/incident/575767#1e500f850b9e735ebff37a9174deb7a91f6a0f908bca8c1dd679368a40ec0772 We will be conducting internal networking updates that affect our main API server. Web API is down https://status.modal.com/incident/575650 Thu, 22 May 2025 14:09:32 -0000 https://status.modal.com/incident/575650#6b4033b679ef7b176fef68d8c604de6ddce16bf0de1fbb7bd31d37b9de08645f Web API recovered. Web API is down https://status.modal.com/incident/575650 Thu, 22 May 2025 14:06:57 -0000 https://status.modal.com/incident/575650#607e7f817d0789c8eeff7940a374da7763e840ed4b3df80dcbf33d7d79732e64 Web API went down. Web endpoints is down https://status.modal.com/incident/566071 Thu, 15 May 2025 15:09:26 -0000 https://status.modal.com/incident/566071#07ee3e88fafc54e4e9b21da03a93c29029ea06fcfe76d8de641b42b0150727cd Web endpoints recovered. Web endpoints is down https://status.modal.com/incident/566071 Thu, 15 May 2025 15:04:35 -0000 https://status.modal.com/incident/566071#cb127589cd2d455056ae57b6591a441fe9a042f0c690b277497b65d6faffe7b0 Web endpoints went down. Sandbox scheduling is down https://status.modal.com/incident/558881 Thu, 08 May 2025 20:15:00 -0000 https://status.modal.com/incident/558881#0ad7384d3b86fc61be776e4e859276934922d5cd9aa4c177f84ed618ebd42501 Fully recovered Sandbox scheduling is down https://status.modal.com/incident/558881 Thu, 08 May 2025 19:22:00 -0000 https://status.modal.com/incident/558881#621ed1d5b7ab59956f356f6118fdac9e135afda2e1c07b5c29b4bbfd7bd5bce7 Scheduling for sandboxes is degraded GCP networking issue degrading GCP-pinned Modal containers https://status.modal.com/incident/540813 Sun, 06 Apr 2025 17:03:00 -0000 https://status.modal.com/incident/540813#693d0a8126bafbbfa1e96a028dc6a399fdd47a2499aaa8f67468f871b18f3b46 Due to network degradation in GCP, Modal is compromised in its ability to maintain Worker capacity and container health. If you have GCP-pinned (cloud="gcp") Functions or Sandboxes, unpinning will result in avoiding this issue. Cloudflare outage in us-east-1 causing Modal degradation in us-east-1 https://status.modal.com/incident/535641 Thu, 27 Mar 2025 20:03:00 -0000 https://status.modal.com/incident/535641#1952ec8f8799ddf0ac6c75a9caa32dc84762330d6397a820420ed0e49a2ea033 Update: The incident is now resolved. Cloudflare incident: https://www.cloudflarestatus.com/incidents/t8mq7d1yjqm4 We are remediating this by shifting Modal Workers away from us-east-1. Cloudflare outage in us-east-1 causing Modal degradation in us-east-1 https://status.modal.com/incident/535641 Thu, 27 Mar 2025 20:03:00 -0000 https://status.modal.com/incident/535641#1952ec8f8799ddf0ac6c75a9caa32dc84762330d6397a820420ed0e49a2ea033 Update: The incident is now resolved. Cloudflare incident: https://www.cloudflarestatus.com/incidents/t8mq7d1yjqm4 We are remediating this by shifting Modal Workers away from us-east-1. Cloudflare outage in us-east-1 causing Modal degradation in us-east-1 https://status.modal.com/incident/535641 Thu, 27 Mar 2025 20:03:00 -0000 https://status.modal.com/incident/535641#1952ec8f8799ddf0ac6c75a9caa32dc84762330d6397a820420ed0e49a2ea033 Update: The incident is now resolved. Cloudflare incident: https://www.cloudflarestatus.com/incidents/t8mq7d1yjqm4 We are remediating this by shifting Modal Workers away from us-east-1. Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 19:45:00 -0000 https://status.modal.com/incident/535052#7445b80ea4dfa7230ccdb453bcfabfbbbc2b903175556624a987c0bebaeb400f We have scaled up capacity successfully and redirected traffic to work around the Oracle Cloud regional networking outage in Ashburn, VA. Newly created sandboxes are fully operational. Some sandboxes created during the incident may not have been able to spin up. Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 19:45:00 -0000 https://status.modal.com/incident/535052#7445b80ea4dfa7230ccdb453bcfabfbbbc2b903175556624a987c0bebaeb400f We have scaled up capacity successfully and redirected traffic to work around the Oracle Cloud regional networking outage in Ashburn, VA. Newly created sandboxes are fully operational. Some sandboxes created during the incident may not have been able to spin up. Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 19:45:00 -0000 https://status.modal.com/incident/535052#7445b80ea4dfa7230ccdb453bcfabfbbbc2b903175556624a987c0bebaeb400f We have scaled up capacity successfully and redirected traffic to work around the Oracle Cloud regional networking outage in Ashburn, VA. Newly created sandboxes are fully operational. Some sandboxes created during the incident may not have been able to spin up. Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 19:45:00 -0000 https://status.modal.com/incident/535052#7445b80ea4dfa7230ccdb453bcfabfbbbc2b903175556624a987c0bebaeb400f We have scaled up capacity successfully and redirected traffic to work around the Oracle Cloud regional networking outage in Ashburn, VA. Newly created sandboxes are fully operational. Some sandboxes created during the incident may not have been able to spin up. Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 19:26:00 -0000 https://status.modal.com/incident/535052#ea9ce3ffd33de16985dbb6202f107f1f8d25e3d82af5c6443a9a52eff3d805a3 Oracle Cloud is mitigating the issue. In the meantime, we're trying to help the autoscaler scale up instances as fast as possible in other clouds. Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 19:26:00 -0000 https://status.modal.com/incident/535052#ea9ce3ffd33de16985dbb6202f107f1f8d25e3d82af5c6443a9a52eff3d805a3 Oracle Cloud is mitigating the issue. In the meantime, we're trying to help the autoscaler scale up instances as fast as possible in other clouds. Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 19:26:00 -0000 https://status.modal.com/incident/535052#ea9ce3ffd33de16985dbb6202f107f1f8d25e3d82af5c6443a9a52eff3d805a3 Oracle Cloud is mitigating the issue. In the meantime, we're trying to help the autoscaler scale up instances as fast as possible in other clouds. Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 19:26:00 -0000 https://status.modal.com/incident/535052#ea9ce3ffd33de16985dbb6202f107f1f8d25e3d82af5c6443a9a52eff3d805a3 Oracle Cloud is mitigating the issue. In the meantime, we're trying to help the autoscaler scale up instances as fast as possible in other clouds. Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 19:19:00 -0000 https://status.modal.com/incident/535052#0503719bfd267185eb2bb7269d1fb6cde48bc0edf23495737a6b5b85121fa9b1 We have identified the cause of the networking issue in Oracle Cloud. Oracle Cloud has confirmed an ongoing NOC outage in Oracle Cloud in us-ashburn-1. We are moving capacity to other cloud providers right now. Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 19:19:00 -0000 https://status.modal.com/incident/535052#0503719bfd267185eb2bb7269d1fb6cde48bc0edf23495737a6b5b85121fa9b1 We have identified the cause of the networking issue in Oracle Cloud. Oracle Cloud has confirmed an ongoing NOC outage in Oracle Cloud in us-ashburn-1. We are moving capacity to other cloud providers right now. Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 19:19:00 -0000 https://status.modal.com/incident/535052#0503719bfd267185eb2bb7269d1fb6cde48bc0edf23495737a6b5b85121fa9b1 We have identified the cause of the networking issue in Oracle Cloud. Oracle Cloud has confirmed an ongoing NOC outage in Oracle Cloud in us-ashburn-1. We are moving capacity to other cloud providers right now. Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 19:19:00 -0000 https://status.modal.com/incident/535052#0503719bfd267185eb2bb7269d1fb6cde48bc0edf23495737a6b5b85121fa9b1 We have identified the cause of the networking issue in Oracle Cloud. Oracle Cloud has confirmed an ongoing NOC outage in Oracle Cloud in us-ashburn-1. We are moving capacity to other cloud providers right now. Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 19:04:00 -0000 https://status.modal.com/incident/535052#f41c7ac044098f284131f6d1819c41491e72addad6fb734b6464379878948f12 We have identified networking issues with generalized user impact Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 19:04:00 -0000 https://status.modal.com/incident/535052#f41c7ac044098f284131f6d1819c41491e72addad6fb734b6464379878948f12 We have identified networking issues with generalized user impact Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 19:04:00 -0000 https://status.modal.com/incident/535052#f41c7ac044098f284131f6d1819c41491e72addad6fb734b6464379878948f12 We have identified networking issues with generalized user impact Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 19:04:00 -0000 https://status.modal.com/incident/535052#f41c7ac044098f284131f6d1819c41491e72addad6fb734b6464379878948f12 We have identified networking issues with generalized user impact Networking problems in Oracle Cloud https://status.modal.com/incident/535052 Wed, 26 Mar 2025 18:41:00 -0000 https://status.modal.com/incident/535052#91ccfd1b7643b90acf126e41d9f44b9b3cabd28f6aaaff80721e3211481ce488 We are investigating networking issues in some of our hardware, which is causing errors connecting TCP sockets on sandboxes. Volumes and Image builds are down https://status.modal.com/incident/532418 Fri, 21 Mar 2025 22:45:14 -0000 https://status.modal.com/incident/532418#c60073c7203d2d366f08832bc7da04e7573b2f063fd1ea420b6e4fe1623a28ba Volumes and Image builds recovered. Volumes and Image builds are down https://status.modal.com/incident/532418 Fri, 21 Mar 2025 22:45:14 -0000 https://status.modal.com/incident/532418#c60073c7203d2d366f08832bc7da04e7573b2f063fd1ea420b6e4fe1623a28ba Volumes and Image builds recovered. Volumes and Image builds are down https://status.modal.com/incident/532418 Fri, 21 Mar 2025 21:52:57 -0000 https://status.modal.com/incident/532418#fdf8a907930593089cd49e1f59edc06f78b36622b8a0a5dcab478748381d8efd Volumes went down. Volumes and Image builds are down https://status.modal.com/incident/532418 Fri, 21 Mar 2025 21:50:27 -0000 https://status.modal.com/incident/532418#53d3a52a229438c6dd7d8db3647b14bdfe670885a6638319b8acf0fc72fc3443 Image builds went down. Image builds is down https://status.modal.com/incident/530407 Tue, 18 Mar 2025 21:50:53 -0000 https://status.modal.com/incident/530407#2ebfcc89a48ef8284d64a07fd109cdcd032639fc915ff56c39704781c7a4fb78 Image builds recovered. Image builds is down https://status.modal.com/incident/530407 Tue, 18 Mar 2025 21:47:14 -0000 https://status.modal.com/incident/530407#21cafeba84a786d0381cef105edc3ade90a9427a2cac59b4644e12ad2ac5b94d Image builds went down. Some services are down https://status.modal.com/incident/518359 Mon, 24 Feb 2025 22:44:00 -0000 https://status.modal.com/incident/518359#91beaa076d30a3a8b42368e23523b1378192f5e35835ab0f7cd79122a1e81794 Everything is back up. Some services are down https://status.modal.com/incident/518359 Mon, 24 Feb 2025 22:44:00 -0000 https://status.modal.com/incident/518359#91beaa076d30a3a8b42368e23523b1378192f5e35835ab0f7cd79122a1e81794 Everything is back up. Some services are down https://status.modal.com/incident/518359 Mon, 24 Feb 2025 22:31:00 -0000 https://status.modal.com/incident/518359#77ae8f4a4847fa40f12b81a6761d48f4c8a86bd418cc5db915e69dd0f7322227 Image builds and some dashboard pages are currently not loading due issues with the metrics database. We are investigating. Some services are down https://status.modal.com/incident/518359 Mon, 24 Feb 2025 22:31:00 -0000 https://status.modal.com/incident/518359#77ae8f4a4847fa40f12b81a6761d48f4c8a86bd418cc5db915e69dd0f7322227 Image builds and some dashboard pages are currently not loading due issues with the metrics database. We are investigating. CPU functions, GPU functions, and 1 other service are down https://status.modal.com/incident/514631 Tue, 18 Feb 2025 03:37:48 -0000 https://status.modal.com/incident/514631#7767e45b8aaabbfae457ca1c6a506493ebfe2b81cd37f6ceaad56a48ef1f0cd1 CPU functions, GPU functions, and Image builds recovered. CPU functions, GPU functions, and 1 other service are down https://status.modal.com/incident/514631 Tue, 18 Feb 2025 03:37:48 -0000 https://status.modal.com/incident/514631#7767e45b8aaabbfae457ca1c6a506493ebfe2b81cd37f6ceaad56a48ef1f0cd1 CPU functions, GPU functions, and Image builds recovered. CPU functions, GPU functions, and 1 other service are down https://status.modal.com/incident/514631 Tue, 18 Feb 2025 03:37:48 -0000 https://status.modal.com/incident/514631#7767e45b8aaabbfae457ca1c6a506493ebfe2b81cd37f6ceaad56a48ef1f0cd1 CPU functions, GPU functions, and Image builds recovered. CPU functions, GPU functions, and 1 other service are down https://status.modal.com/incident/514631 Tue, 18 Feb 2025 03:36:31 -0000 https://status.modal.com/incident/514631#e7cd8ab92e003cd992974de449ef44c96830eadab049c628614e34d4fed9741b GPU functions and Image builds went down. CPU functions, GPU functions, and 1 other service are down https://status.modal.com/incident/514631 Tue, 18 Feb 2025 03:36:31 -0000 https://status.modal.com/incident/514631#e7cd8ab92e003cd992974de449ef44c96830eadab049c628614e34d4fed9741b GPU functions and Image builds went down. CPU functions, GPU functions, and 1 other service are down https://status.modal.com/incident/514631 Tue, 18 Feb 2025 03:35:27 -0000 https://status.modal.com/incident/514631#d7f3e5d8d7d7add48067777b250969f334527b56b280e1050417fe5353a5add8 CPU functions went down. Web endpoints, Frontend, and 2 other services are down https://status.modal.com/incident/511178 Tue, 11 Feb 2025 18:16:00 -0000 https://status.modal.com/incident/511178#24b2f296ecf2c0e9ec852d304a231f3880911e6b28b12a8e49254b28793d75d7 Web endpoints, Frontend, Web API, and 1 other resource recovered. Web endpoints, Frontend, and 2 other services are down https://status.modal.com/incident/511178 Tue, 11 Feb 2025 18:16:00 -0000 https://status.modal.com/incident/511178#24b2f296ecf2c0e9ec852d304a231f3880911e6b28b12a8e49254b28793d75d7 Web endpoints, Frontend, Web API, and 1 other resource recovered. Web endpoints, Frontend, and 2 other services are down https://status.modal.com/incident/511178 Tue, 11 Feb 2025 18:16:00 -0000 https://status.modal.com/incident/511178#24b2f296ecf2c0e9ec852d304a231f3880911e6b28b12a8e49254b28793d75d7 Web endpoints, Frontend, Web API, and 1 other resource recovered. Web endpoints, Frontend, and 2 other services are down https://status.modal.com/incident/511178 Tue, 11 Feb 2025 18:16:00 -0000 https://status.modal.com/incident/511178#24b2f296ecf2c0e9ec852d304a231f3880911e6b28b12a8e49254b28793d75d7 Web endpoints, Frontend, Web API, and 1 other resource recovered. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:13:00 -0000 https://status.modal.com/incident/511180#fe0874fa7b50462cd5974cc5ad0bb0d4cd7f0d2a13bc5ddf9fb9b6dbea1f9c07 Modal systems are back up. We're monitoring closely. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:13:00 -0000 https://status.modal.com/incident/511180#fe0874fa7b50462cd5974cc5ad0bb0d4cd7f0d2a13bc5ddf9fb9b6dbea1f9c07 Modal systems are back up. We're monitoring closely. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:13:00 -0000 https://status.modal.com/incident/511180#fe0874fa7b50462cd5974cc5ad0bb0d4cd7f0d2a13bc5ddf9fb9b6dbea1f9c07 Modal systems are back up. We're monitoring closely. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:13:00 -0000 https://status.modal.com/incident/511180#fe0874fa7b50462cd5974cc5ad0bb0d4cd7f0d2a13bc5ddf9fb9b6dbea1f9c07 Modal systems are back up. We're monitoring closely. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:13:00 -0000 https://status.modal.com/incident/511180#fe0874fa7b50462cd5974cc5ad0bb0d4cd7f0d2a13bc5ddf9fb9b6dbea1f9c07 Modal systems are back up. We're monitoring closely. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:13:00 -0000 https://status.modal.com/incident/511180#fe0874fa7b50462cd5974cc5ad0bb0d4cd7f0d2a13bc5ddf9fb9b6dbea1f9c07 Modal systems are back up. We're monitoring closely. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:13:00 -0000 https://status.modal.com/incident/511180#fe0874fa7b50462cd5974cc5ad0bb0d4cd7f0d2a13bc5ddf9fb9b6dbea1f9c07 Modal systems are back up. We're monitoring closely. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:13:00 -0000 https://status.modal.com/incident/511180#fe0874fa7b50462cd5974cc5ad0bb0d4cd7f0d2a13bc5ddf9fb9b6dbea1f9c07 Modal systems are back up. We're monitoring closely. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:13:00 -0000 https://status.modal.com/incident/511180#fe0874fa7b50462cd5974cc5ad0bb0d4cd7f0d2a13bc5ddf9fb9b6dbea1f9c07 Modal systems are back up. We're monitoring closely. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:13:00 -0000 https://status.modal.com/incident/511180#fe0874fa7b50462cd5974cc5ad0bb0d4cd7f0d2a13bc5ddf9fb9b6dbea1f9c07 Modal systems are back up. We're monitoring closely. Web endpoints, Frontend, and 2 other services are down https://status.modal.com/incident/511178 Tue, 11 Feb 2025 18:09:53 -0000 https://status.modal.com/incident/511178#f56aa733f70e0afd2a6679aaa84a315ae7ba3f65f660481eb3b79702cc49b73d Frontend and Web API went down. Web endpoints, Frontend, and 2 other services are down https://status.modal.com/incident/511178 Tue, 11 Feb 2025 18:09:53 -0000 https://status.modal.com/incident/511178#f56aa733f70e0afd2a6679aaa84a315ae7ba3f65f660481eb3b79702cc49b73d Frontend and Web API went down. Web endpoints, Frontend, and 2 other services are down https://status.modal.com/incident/511178 Tue, 11 Feb 2025 18:09:22 -0000 https://status.modal.com/incident/511178#7fa53413a8c3af0f6cf17306b6d45e84cda610ab09da7d94eb01b4a55147cdde Web endpoints went down. Web endpoints, Frontend, and 2 other services are down https://status.modal.com/incident/511178 Tue, 11 Feb 2025 18:09:00 -0000 https://status.modal.com/incident/511178#0b6651ad5a52348562865df49224160a9baf95babb8fa31287ba084435759453 OIDC Endpoint went down. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:09:00 -0000 https://status.modal.com/incident/511180#507e98f84d510595ca2789a136b785dd30f361c344f391e7d86cb59a49c95216 All Modal services are affected. We're working on rolling back the change. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:09:00 -0000 https://status.modal.com/incident/511180#507e98f84d510595ca2789a136b785dd30f361c344f391e7d86cb59a49c95216 All Modal services are affected. We're working on rolling back the change. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:09:00 -0000 https://status.modal.com/incident/511180#507e98f84d510595ca2789a136b785dd30f361c344f391e7d86cb59a49c95216 All Modal services are affected. We're working on rolling back the change. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:09:00 -0000 https://status.modal.com/incident/511180#507e98f84d510595ca2789a136b785dd30f361c344f391e7d86cb59a49c95216 All Modal services are affected. We're working on rolling back the change. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:09:00 -0000 https://status.modal.com/incident/511180#507e98f84d510595ca2789a136b785dd30f361c344f391e7d86cb59a49c95216 All Modal services are affected. We're working on rolling back the change. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:09:00 -0000 https://status.modal.com/incident/511180#507e98f84d510595ca2789a136b785dd30f361c344f391e7d86cb59a49c95216 All Modal services are affected. We're working on rolling back the change. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:09:00 -0000 https://status.modal.com/incident/511180#507e98f84d510595ca2789a136b785dd30f361c344f391e7d86cb59a49c95216 All Modal services are affected. We're working on rolling back the change. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:09:00 -0000 https://status.modal.com/incident/511180#507e98f84d510595ca2789a136b785dd30f361c344f391e7d86cb59a49c95216 All Modal services are affected. We're working on rolling back the change. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:09:00 -0000 https://status.modal.com/incident/511180#507e98f84d510595ca2789a136b785dd30f361c344f391e7d86cb59a49c95216 All Modal services are affected. We're working on rolling back the change. Modal services are offline https://status.modal.com/incident/511180 Tue, 11 Feb 2025 18:09:00 -0000 https://status.modal.com/incident/511180#507e98f84d510595ca2789a136b785dd30f361c344f391e7d86cb59a49c95216 All Modal services are affected. We're working on rolling back the change. Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 19:30:39 +0000 https://status.modal.com/incident/508457#9015d39d2ebca815c8b022f6d1b410b273174109700135ca0e1ea1fed793220c Maintenance completed Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 19:30:39 +0000 https://status.modal.com/incident/508457#9015d39d2ebca815c8b022f6d1b410b273174109700135ca0e1ea1fed793220c Maintenance completed Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 19:30:39 +0000 https://status.modal.com/incident/508457#9015d39d2ebca815c8b022f6d1b410b273174109700135ca0e1ea1fed793220c Maintenance completed Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 19:30:39 +0000 https://status.modal.com/incident/508457#9015d39d2ebca815c8b022f6d1b410b273174109700135ca0e1ea1fed793220c Maintenance completed Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 19:30:39 +0000 https://status.modal.com/incident/508457#9015d39d2ebca815c8b022f6d1b410b273174109700135ca0e1ea1fed793220c Maintenance completed Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 19:30:39 +0000 https://status.modal.com/incident/508457#9015d39d2ebca815c8b022f6d1b410b273174109700135ca0e1ea1fed793220c Maintenance completed Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 19:30:39 +0000 https://status.modal.com/incident/508457#9015d39d2ebca815c8b022f6d1b410b273174109700135ca0e1ea1fed793220c Maintenance completed Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 19:30:39 +0000 https://status.modal.com/incident/508457#9015d39d2ebca815c8b022f6d1b410b273174109700135ca0e1ea1fed793220c Maintenance completed Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 19:30:39 +0000 https://status.modal.com/incident/508457#9015d39d2ebca815c8b022f6d1b410b273174109700135ca0e1ea1fed793220c Maintenance completed Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 19:30:39 +0000 https://status.modal.com/incident/508457#9015d39d2ebca815c8b022f6d1b410b273174109700135ca0e1ea1fed793220c Maintenance completed Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 18:00:39 -0000 https://status.modal.com/incident/508457#6db191a5e4d8bd835e7c585c2714933e5e2c406718bd0022c2d4b52ebe51b6be During this time, Modal is expected to work. There may be potential degraded performance (hopefully short-lived). Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 18:00:39 -0000 https://status.modal.com/incident/508457#6db191a5e4d8bd835e7c585c2714933e5e2c406718bd0022c2d4b52ebe51b6be During this time, Modal is expected to work. There may be potential degraded performance (hopefully short-lived). Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 18:00:39 -0000 https://status.modal.com/incident/508457#6db191a5e4d8bd835e7c585c2714933e5e2c406718bd0022c2d4b52ebe51b6be During this time, Modal is expected to work. There may be potential degraded performance (hopefully short-lived). Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 18:00:39 -0000 https://status.modal.com/incident/508457#6db191a5e4d8bd835e7c585c2714933e5e2c406718bd0022c2d4b52ebe51b6be During this time, Modal is expected to work. There may be potential degraded performance (hopefully short-lived). Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 18:00:39 -0000 https://status.modal.com/incident/508457#6db191a5e4d8bd835e7c585c2714933e5e2c406718bd0022c2d4b52ebe51b6be During this time, Modal is expected to work. There may be potential degraded performance (hopefully short-lived). Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 18:00:39 -0000 https://status.modal.com/incident/508457#6db191a5e4d8bd835e7c585c2714933e5e2c406718bd0022c2d4b52ebe51b6be During this time, Modal is expected to work. There may be potential degraded performance (hopefully short-lived). Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 18:00:39 -0000 https://status.modal.com/incident/508457#6db191a5e4d8bd835e7c585c2714933e5e2c406718bd0022c2d4b52ebe51b6be During this time, Modal is expected to work. There may be potential degraded performance (hopefully short-lived). Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 18:00:39 -0000 https://status.modal.com/incident/508457#6db191a5e4d8bd835e7c585c2714933e5e2c406718bd0022c2d4b52ebe51b6be During this time, Modal is expected to work. There may be potential degraded performance (hopefully short-lived). Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 18:00:39 -0000 https://status.modal.com/incident/508457#6db191a5e4d8bd835e7c585c2714933e5e2c406718bd0022c2d4b52ebe51b6be During this time, Modal is expected to work. There may be potential degraded performance (hopefully short-lived). Network infrastructure change https://status.modal.com/incident/508457 Thu, 06 Feb 2025 18:00:39 -0000 https://status.modal.com/incident/508457#6db191a5e4d8bd835e7c585c2714933e5e2c406718bd0022c2d4b52ebe51b6be During this time, Modal is expected to work. There may be potential degraded performance (hopefully short-lived). Dashboard is unavailable https://status.modal.com/incident/501380 Fri, 24 Jan 2025 18:50:00 -0000 https://status.modal.com/incident/501380#76310c5fea622a43d6eb5aa32e8ff736319eeedf768671d30caad1602aa621af We have resolved the issue. The modal.com dashboard has been back online since about 1:40 PM ET, 10 minutes ago. Deployments are stable again. We are continuing to monitor. Dashboard is unavailable https://status.modal.com/incident/501380 Fri, 24 Jan 2025 18:50:00 -0000 https://status.modal.com/incident/501380#76310c5fea622a43d6eb5aa32e8ff736319eeedf768671d30caad1602aa621af We have resolved the issue. The modal.com dashboard has been back online since about 1:40 PM ET, 10 minutes ago. Deployments are stable again. We are continuing to monitor. Dashboard is unavailable https://status.modal.com/incident/501380 Fri, 24 Jan 2025 18:16:00 -0000 https://status.modal.com/incident/501380#9774060e8e6b161b965028880be33fd211def8d877934aba47f1dd19c614a375 The modal.com web dashboard is unavailable and returning 502s. We are investigating and believe we have diagnosed the problem. Dashboard is unavailable https://status.modal.com/incident/501380 Fri, 24 Jan 2025 18:16:00 -0000 https://status.modal.com/incident/501380#9774060e8e6b161b965028880be33fd211def8d877934aba47f1dd19c614a375 The modal.com web dashboard is unavailable and returning 502s. We are investigating and believe we have diagnosed the problem. Web endpoints is down https://status.modal.com/incident/495966 Wed, 15 Jan 2025 01:27:22 -0000 https://status.modal.com/incident/495966#20f5b92241e5eecbbf5f3b7c6e621971e119f7e280c096e3afe21faff5337c4f Web endpoints recovered. Web endpoints is down https://status.modal.com/incident/495966 Wed, 15 Jan 2025 01:08:27 -0000 https://status.modal.com/incident/495966#799b138de5157043f19d207ad9cf3355136561090d375bce4da3af9e8a7ddb2e Web endpoints went down. Performance degradation across Functions, Sandboxes, and web endpoints https://status.modal.com/incident/493416 Thu, 09 Jan 2025 19:35:00 -0000 https://status.modal.com/incident/493416#85513c2b80fc600f0b634c8147c06e4841c52a8b5523f38ea6a132fcbc137d97 Systems have fully recovered. Performance degradation across Functions, Sandboxes, and web endpoints https://status.modal.com/incident/493416 Thu, 09 Jan 2025 19:35:00 -0000 https://status.modal.com/incident/493416#85513c2b80fc600f0b634c8147c06e4841c52a8b5523f38ea6a132fcbc137d97 Systems have fully recovered. Performance degradation across Functions, Sandboxes, and web endpoints https://status.modal.com/incident/493416 Thu, 09 Jan 2025 19:35:00 -0000 https://status.modal.com/incident/493416#85513c2b80fc600f0b634c8147c06e4841c52a8b5523f38ea6a132fcbc137d97 Systems have fully recovered. Performance degradation across Functions, Sandboxes, and web endpoints https://status.modal.com/incident/493416 Thu, 09 Jan 2025 19:35:00 -0000 https://status.modal.com/incident/493416#85513c2b80fc600f0b634c8147c06e4841c52a8b5523f38ea6a132fcbc137d97 Systems have fully recovered. Performance degradation across Functions, Sandboxes, and web endpoints https://status.modal.com/incident/493416 Thu, 09 Jan 2025 19:35:00 -0000 https://status.modal.com/incident/493416#85513c2b80fc600f0b634c8147c06e4841c52a8b5523f38ea6a132fcbc137d97 Systems have fully recovered. Performance degradation across Functions, Sandboxes, and web endpoints https://status.modal.com/incident/493416 Thu, 09 Jan 2025 19:35:00 -0000 https://status.modal.com/incident/493416#85513c2b80fc600f0b634c8147c06e4841c52a8b5523f38ea6a132fcbc137d97 Systems have fully recovered. Performance degradation across Functions, Sandboxes, and web endpoints https://status.modal.com/incident/493416 Thu, 09 Jan 2025 19:00:00 -0000 https://status.modal.com/incident/493416#99b0d8dac3190a63edf78d99dc8888e13509beb5fcb3e72f0ef0827388a72baa We are experiencing performance degradation across all Functions and web endpoint calls. Performance degradation across Functions, Sandboxes, and web endpoints https://status.modal.com/incident/493416 Thu, 09 Jan 2025 19:00:00 -0000 https://status.modal.com/incident/493416#99b0d8dac3190a63edf78d99dc8888e13509beb5fcb3e72f0ef0827388a72baa We are experiencing performance degradation across all Functions and web endpoint calls. Performance degradation across Functions, Sandboxes, and web endpoints https://status.modal.com/incident/493416 Thu, 09 Jan 2025 19:00:00 -0000 https://status.modal.com/incident/493416#99b0d8dac3190a63edf78d99dc8888e13509beb5fcb3e72f0ef0827388a72baa We are experiencing performance degradation across all Functions and web endpoint calls. Performance degradation across Functions, Sandboxes, and web endpoints https://status.modal.com/incident/493416 Thu, 09 Jan 2025 19:00:00 -0000 https://status.modal.com/incident/493416#99b0d8dac3190a63edf78d99dc8888e13509beb5fcb3e72f0ef0827388a72baa We are experiencing performance degradation across all Functions and web endpoint calls. Performance degradation across Functions, Sandboxes, and web endpoints https://status.modal.com/incident/493416 Thu, 09 Jan 2025 19:00:00 -0000 https://status.modal.com/incident/493416#99b0d8dac3190a63edf78d99dc8888e13509beb5fcb3e72f0ef0827388a72baa We are experiencing performance degradation across all Functions and web endpoint calls. Performance degradation across Functions, Sandboxes, and web endpoints https://status.modal.com/incident/493416 Thu, 09 Jan 2025 19:00:00 -0000 https://status.modal.com/incident/493416#99b0d8dac3190a63edf78d99dc8888e13509beb5fcb3e72f0ef0827388a72baa We are experiencing performance degradation across all Functions and web endpoint calls. Snapshot restores (beta) is down https://status.modal.com/incident/473022 Thu, 05 Dec 2024 18:04:36 -0000 https://status.modal.com/incident/473022#009d7106ba28f2a790dfae8497a0649b636a643375ae2b484fcdfdc5f37a64fb Snapshot restores (beta) recovered. Snapshot restores (beta) is down https://status.modal.com/incident/473022 Thu, 05 Dec 2024 18:02:30 -0000 https://status.modal.com/incident/473022#2f256780dc8ee8db894f631a523cfdae0c40bac7d187933b23ecce59784d6be6 Snapshot restores (beta) went down. Upgrading system components https://status.modal.com/incident/462246 Wed, 04 Dec 2024 18:00:00 +0000 https://status.modal.com/incident/462246#309105fa6bbce5b268f156435d29342316b382ec97250609b82c7cc9785c37e4 Maintenance completed Upgrading system components https://status.modal.com/incident/462246 Wed, 04 Dec 2024 18:00:00 +0000 https://status.modal.com/incident/462246#309105fa6bbce5b268f156435d29342316b382ec97250609b82c7cc9785c37e4 Maintenance completed Upgrading system components https://status.modal.com/incident/462246 Wed, 04 Dec 2024 18:00:00 +0000 https://status.modal.com/incident/462246#309105fa6bbce5b268f156435d29342316b382ec97250609b82c7cc9785c37e4 Maintenance completed Upgrading system components https://status.modal.com/incident/462246 Wed, 04 Dec 2024 18:00:00 +0000 https://status.modal.com/incident/462246#309105fa6bbce5b268f156435d29342316b382ec97250609b82c7cc9785c37e4 Maintenance completed Upgrading system components https://status.modal.com/incident/462246 Wed, 04 Dec 2024 18:00:00 +0000 https://status.modal.com/incident/462246#309105fa6bbce5b268f156435d29342316b382ec97250609b82c7cc9785c37e4 Maintenance completed Upgrading system components https://status.modal.com/incident/462246 Wed, 04 Dec 2024 16:00:00 -0000 https://status.modal.com/incident/462246#0fd064f54fd807dc30b225c6b61d9c37387dd0055b48bc1934c535f8c7425f75 We will be performing a scheduled upgrade. Upgrading system components https://status.modal.com/incident/462246 Wed, 04 Dec 2024 16:00:00 -0000 https://status.modal.com/incident/462246#0fd064f54fd807dc30b225c6b61d9c37387dd0055b48bc1934c535f8c7425f75 We will be performing a scheduled upgrade. Upgrading system components https://status.modal.com/incident/462246 Wed, 04 Dec 2024 16:00:00 -0000 https://status.modal.com/incident/462246#0fd064f54fd807dc30b225c6b61d9c37387dd0055b48bc1934c535f8c7425f75 We will be performing a scheduled upgrade. Upgrading system components https://status.modal.com/incident/462246 Wed, 04 Dec 2024 16:00:00 -0000 https://status.modal.com/incident/462246#0fd064f54fd807dc30b225c6b61d9c37387dd0055b48bc1934c535f8c7425f75 We will be performing a scheduled upgrade. Upgrading system components https://status.modal.com/incident/462246 Wed, 04 Dec 2024 16:00:00 -0000 https://status.modal.com/incident/462246#0fd064f54fd807dc30b225c6b61d9c37387dd0055b48bc1934c535f8c7425f75 We will be performing a scheduled upgrade. modal.com DNS updates https://status.modal.com/incident/453461 Thu, 31 Oct 2024 15:45:00 +0000 https://status.modal.com/incident/453461#01b52e459dec11224a4d591956b7599d758473127887cdf8ec61f945892dcd67 Maintenance completed modal.com DNS updates https://status.modal.com/incident/453461 Thu, 31 Oct 2024 15:00:18 -0000 https://status.modal.com/incident/453461#2e4bb5317e460618c4c5b0cec71e277d96d193ebd29f825c8bc8040f66ecc818 We are updating the DNS records of our website at modal.com to point to a new CDN deployment. We expect this to be a seamless transition with no observable downtime, but we are declaring a maintenance window as a precaution. (Your website will just get faster!) Volumes is down https://status.modal.com/incident/453967 Thu, 31 Oct 2024 14:52:01 -0000 https://status.modal.com/incident/453967#e3f5564cc58b678ff56700b8c31d9c62fff336949cc07f3059ffd55fae5e38d1 Volumes recovered. Volumes is down https://status.modal.com/incident/453967 Thu, 31 Oct 2024 14:30:37 -0000 https://status.modal.com/incident/453967#e5e8ab8db7d0ba242839c167bf0b07b095d9f92bd2aaff362bb1771ea5f1a9ca Volumes went down. Image builds is down https://status.modal.com/incident/437215 Mon, 30 Sep 2024 21:17:34 -0000 https://status.modal.com/incident/437215#381b1fe087599e235db55914957947640956ce5b55e205aa7a03de64fff826df Image builds recovered. Image builds is down https://status.modal.com/incident/437215 Mon, 30 Sep 2024 21:14:48 -0000 https://status.modal.com/incident/437215#3b6ecb2015a267c40b9e3713b130aeb7b2cb0477ecb0f2299ac8a07ba7919677 Image builds went down. CPU container performance degradation from over-scheduling on GCE n1-standard instance class https://status.modal.com/incident/435873 Tue, 24 Sep 2024 17:51:00 -0000 https://status.modal.com/incident/435873#ed7dcb28e0259bcfeca74e18db56dd36486c5474811e906794d9cf97bd6a503f Between Monday 23rd 15:54 UTC and 15:36 UTC a change was made to make GCP workers poll more quickly for containers and this caused them to pick up more CPU-only containers. Between 13:48 UTC and 16:18 UTC on September 24th CPU containers were allowed to be scheduled on GCP NVIDIA T4 workers which are the `n1` class of GCE VMs. Both of these changes combined to cause over-scheduling of CPU work onto GCP T4 workers and significantly degrade the performance of the hosted containers (whether CPU-only or T4 GPU-enabled). Container filesystem 'file not found' bug https://status.modal.com/incident/428812 Fri, 13 Sep 2024 17:59:00 -0000 https://status.modal.com/incident/428812#900c60fffe241a3d019100cc00a95775712007ce7dd4608bdf8350c7f335fb68 At 12:57 UTC a container filesystem change rolled out across workers which caused spurious 'file not found' (ENOENT) errors against the container's rootfs filesystem. We identified the bad commit and rolled back to prevent new workers from picking up the bug. We then worked to roll over all running production workers which were affected by the bad change and scale up new workers. All affected workers have been quarantined and thus new containers will start on new, healthy workers. We apologize for the disruption. Container filesystem 'file not found' bug https://status.modal.com/incident/428812 Fri, 13 Sep 2024 17:59:00 -0000 https://status.modal.com/incident/428812#900c60fffe241a3d019100cc00a95775712007ce7dd4608bdf8350c7f335fb68 At 12:57 UTC a container filesystem change rolled out across workers which caused spurious 'file not found' (ENOENT) errors against the container's rootfs filesystem. We identified the bad commit and rolled back to prevent new workers from picking up the bug. We then worked to roll over all running production workers which were affected by the bad change and scale up new workers. All affected workers have been quarantined and thus new containers will start on new, healthy workers. We apologize for the disruption. Ephemeral apps and image builds are degraded. https://status.modal.com/incident/403407 Thu, 25 Jul 2024 06:50:00 -0000 https://status.modal.com/incident/403407#f80f17e70dd761e03b779899c356178392246e169b67b81b618c58ea9b9c78c7 Trigger processing issues have caused issues in ephemeral Apps and modal.Image builds. We're investigating. Ephemeral apps and image builds are degraded. https://status.modal.com/incident/403407 Thu, 25 Jul 2024 06:50:00 -0000 https://status.modal.com/incident/403407#f80f17e70dd761e03b779899c356178392246e169b67b81b618c58ea9b9c78c7 Trigger processing issues have caused issues in ephemeral Apps and modal.Image builds. We're investigating. Modal function executions are degraded https://status.modal.com/incident/399044 Tue, 16 Jul 2024 18:43:00 -0000 https://status.modal.com/incident/399044#9c5278190d01f0fa78ce0813c0c185ab566dc171188bb7ad1a126c3c5066594f The system has recovered, and function executions should be proceeding as normal. The team is on standby for any further issues. Modal function executions are degraded https://status.modal.com/incident/399044 Tue, 16 Jul 2024 18:43:00 -0000 https://status.modal.com/incident/399044#9c5278190d01f0fa78ce0813c0c185ab566dc171188bb7ad1a126c3c5066594f The system has recovered, and function executions should be proceeding as normal. The team is on standby for any further issues. Modal function executions are degraded https://status.modal.com/incident/399044 Tue, 16 Jul 2024 18:43:00 -0000 https://status.modal.com/incident/399044#9c5278190d01f0fa78ce0813c0c185ab566dc171188bb7ad1a126c3c5066594f The system has recovered, and function executions should be proceeding as normal. The team is on standby for any further issues. Modal function executions are degraded https://status.modal.com/incident/399044 Tue, 16 Jul 2024 18:42:00 -0000 https://status.modal.com/incident/399044#0539c22c8abe20caaf6acc34aa91106cfcd506684f496dc334eb74b07081b060 We have identified the problem and are recovering at the moment. Modal function executions are degraded https://status.modal.com/incident/399044 Tue, 16 Jul 2024 18:42:00 -0000 https://status.modal.com/incident/399044#0539c22c8abe20caaf6acc34aa91106cfcd506684f496dc334eb74b07081b060 We have identified the problem and are recovering at the moment. Modal function executions are degraded https://status.modal.com/incident/399044 Tue, 16 Jul 2024 18:42:00 -0000 https://status.modal.com/incident/399044#0539c22c8abe20caaf6acc34aa91106cfcd506684f496dc334eb74b07081b060 We have identified the problem and are recovering at the moment. Modal function executions are degraded https://status.modal.com/incident/399044 Tue, 16 Jul 2024 17:53:00 -0000 https://status.modal.com/incident/399044#3a901b892d4e6aad1f0b172096c7ef535210679d570b5746ce4b7aa225d0ab71 We are seeing degraded performance on Modal function executions and are investigating. Modal function executions are degraded https://status.modal.com/incident/399044 Tue, 16 Jul 2024 17:53:00 -0000 https://status.modal.com/incident/399044#3a901b892d4e6aad1f0b172096c7ef535210679d570b5746ce4b7aa225d0ab71 We are seeing degraded performance on Modal function executions and are investigating. Modal function executions are degraded https://status.modal.com/incident/399044 Tue, 16 Jul 2024 17:53:00 -0000 https://status.modal.com/incident/399044#3a901b892d4e6aad1f0b172096c7ef535210679d570b5746ce4b7aa225d0ab71 We are seeing degraded performance on Modal function executions and are investigating. CPU functions on GCP, Web endpoints, and 1 other service are down https://status.modal.com/incident/397339 Fri, 12 Jul 2024 23:35:09 -0000 https://status.modal.com/incident/397339#b3634f16065029d0cd8a2db5ad9454be4b1c17eb4bf0401ebc54c6c37f8bdd4b Web endpoints recovered. CPU functions on GCP, Web endpoints, and 1 other service are down https://status.modal.com/incident/397339 Fri, 12 Jul 2024 23:29:58 -0000 https://status.modal.com/incident/397339#9620b91bdaedde3c63229ec065147658982849401caebe6ce02f1c0d43d063ea CPU functions on GCP and Volumes recovered. CPU functions on GCP, Web endpoints, and 1 other service are down https://status.modal.com/incident/397339 Fri, 12 Jul 2024 23:29:42 -0000 https://status.modal.com/incident/397339#07706af992b9dfc3b020323accc702991ab370062a774b4bf81c8e7ff90fb810 Volumes went down. CPU functions on GCP, Web endpoints, and 1 other service are down https://status.modal.com/incident/397339 Fri, 12 Jul 2024 23:28:41 -0000 https://status.modal.com/incident/397339#1d58c510c17c3423d1cd4aaf519967762713e952f3cbcbb08ae5ebcebfc1e7ca Web endpoints went down. CPU functions on GCP, Web endpoints, and 1 other service are down https://status.modal.com/incident/397339 Fri, 12 Jul 2024 16:56:13 -0000 https://status.modal.com/incident/397339#49cb8b61cc9f150e04f6884924031c8de1bd59756e8e66db81e804b97fddf0a1 Web endpoints recovered. CPU functions on GCP, Web endpoints, and 1 other service are down https://status.modal.com/incident/397339 Fri, 12 Jul 2024 16:50:11 -0000 https://status.modal.com/incident/397339#c6f9add039a8c2def062c27555dd17f73e411ab3f94b0161421d8bfc381a4721 Web endpoints went down. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/384368 Fri, 14 Jun 2024 14:47:03 -0000 https://status.modal.com/incident/384368#8a57b7409f0f90df75fc460aad856f879789ee2977436ddd494ce8a951965f5a Web API recovered. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/384368 Fri, 14 Jun 2024 14:46:47 -0000 https://status.modal.com/incident/384368#ae932190df420d12678e43edd66e396bebdcaabe3ab48a111ab054186a729910 Web endpoints recovered. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/384368 Fri, 14 Jun 2024 14:41:15 -0000 https://status.modal.com/incident/384368#35101176735e17cd71c9bcbdae693651c490a345abdf31cae11fd7aab9f72621 Volumes recovered. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/384368 Fri, 14 Jun 2024 14:40:18 -0000 https://status.modal.com/incident/384368#7f16eccb7c3ab2d244c2ecfed2cfdf34301e0f03c8d39c721363ade985511626 CPU functions on AWS recovered. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/384368 Fri, 14 Jun 2024 14:39:13 -0000 https://status.modal.com/incident/384368#07136e9099192a1341fd77b90ce6771730b85fa3311fed4ea7a82e78c6e3d197 CPU functions on AWS, CPU functions on OCI, and Volumes went down. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/384368 Fri, 14 Jun 2024 14:39:13 -0000 https://status.modal.com/incident/384368#07136e9099192a1341fd77b90ce6771730b85fa3311fed4ea7a82e78c6e3d197 CPU functions on AWS, CPU functions on OCI, and Volumes went down. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/384368 Fri, 14 Jun 2024 14:34:29 -0000 https://status.modal.com/incident/384368#bbbf51ea8a0841d5f60f0ac47de315fee13034ac128476727b63d0dbc9b66aaf Web API went down. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/384368 Fri, 14 Jun 2024 14:30:08 -0000 https://status.modal.com/incident/384368#75b6086b1cb51ffa0cb4a814f363651b437c9cb8957ec8079deb58e2204feb6f Web API recovered. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/384368 Fri, 14 Jun 2024 14:21:57 -0000 https://status.modal.com/incident/384368#0aef0635cb9e173df0f80033ef143785f95a92f3d6406a4454ec70717a509766 CPU functions on AWS and Volumes recovered. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/384368 Fri, 14 Jun 2024 14:21:57 -0000 https://status.modal.com/incident/384368#0aef0635cb9e173df0f80033ef143785f95a92f3d6406a4454ec70717a509766 CPU functions on AWS and Volumes recovered. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/384368 Fri, 14 Jun 2024 14:21:37 -0000 https://status.modal.com/incident/384368#41e332dc4fa3f3fb1a928dfc9b2db6c2159d7aa1edd945e12dd7a54039b535a2 CPU functions on AWS went down. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/384368 Fri, 14 Jun 2024 14:20:48 -0000 https://status.modal.com/incident/384368#3a50c791a09e9cc0c6dc1358424aee386237a07d8552ee0ef887eda663689338 Volumes went down. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/384368 Fri, 14 Jun 2024 14:17:57 -0000 https://status.modal.com/incident/384368#a76eaed96f78a7cd6d6d37b9538e08f857997f9d7a10fbb788cb54cf646c0d65 Web endpoints went down. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/384368 Fri, 14 Jun 2024 14:16:04 -0000 https://status.modal.com/incident/384368#0b6f0d5f217ef9efb02da7a53dab97ff8e7c053a2ab4b55f7a5bb39086e095ae Web API went down. GPU functions on GCP and Web endpoints are down https://status.modal.com/incident/379898 Wed, 05 Jun 2024 22:44:34 -0000 https://status.modal.com/incident/379898#0bdcdd6f4ef09b9554e9571dadf640a7ec711e9c8b2d540023209a0699e06787 Web endpoints recovered. GPU functions on GCP and Web endpoints are down https://status.modal.com/incident/379898 Wed, 05 Jun 2024 22:44:14 -0000 https://status.modal.com/incident/379898#874808f46c89f2bccfc697680884eb57f260bf2286dede4e5a559c6bac13c369 Web endpoints went down. GPU functions on GCP and Web endpoints are down https://status.modal.com/incident/379898 Wed, 05 Jun 2024 22:41:04 -0000 https://status.modal.com/incident/379898#e430380395c72dc143fc4dfeefb1acf5ddb92b0c37d085475dbcbf0b9c49da4e Web endpoints recovered. GPU functions on GCP and Web endpoints are down https://status.modal.com/incident/379898 Wed, 05 Jun 2024 22:36:40 -0000 https://status.modal.com/incident/379898#96d3b3fe383fb1263563bd450a47caf5f7873ac5b620f7caf18bdc8d02a27f43 Web endpoints went down. modal.com network ingress rollover https://status.modal.com/incident/377009 Thu, 30 May 2024 18:27:21 +0000 https://status.modal.com/incident/377009#012d31e038688cade10a7ccd94a1b7a8d88b5825a2fd336283f85de595151356 Maintenance completed modal.com network ingress rollover https://status.modal.com/incident/377009 Thu, 30 May 2024 18:27:21 -0000 https://status.modal.com/incident/377009#2ff6722f0aa027456a643d8ebf80638eeefdcc412cae3335811edb2f35af212b We're performing an infrastructure change on our website (modal.com). The dashboard may be inaccessible to some users for up to a minute. modal.com network ingress rollover https://status.modal.com/incident/377009 Thu, 30 May 2024 18:27:21 +0000 https://status.modal.com/incident/377009#012d31e038688cade10a7ccd94a1b7a8d88b5825a2fd336283f85de595151356 Maintenance completed modal.com network ingress rollover https://status.modal.com/incident/377009 Thu, 30 May 2024 18:27:21 -0000 https://status.modal.com/incident/377009#2ff6722f0aa027456a643d8ebf80638eeefdcc412cae3335811edb2f35af212b We're performing an infrastructure change on our website (modal.com). The dashboard may be inaccessible to some users for up to a minute. modal.com network ingress rollover https://status.modal.com/incident/377009 Thu, 30 May 2024 18:00:00 -0000 https://status.modal.com/incident/377009#2ff6722f0aa027456a643d8ebf80638eeefdcc412cae3335811edb2f35af212b We're performing an infrastructure change on our website (modal.com). The dashboard may be inaccessible to some users for up to a minute. modal.com network ingress rollover https://status.modal.com/incident/377009 Thu, 30 May 2024 18:00:00 -0000 https://status.modal.com/incident/377009#2ff6722f0aa027456a643d8ebf80638eeefdcc412cae3335811edb2f35af212b We're performing an infrastructure change on our website (modal.com). The dashboard may be inaccessible to some users for up to a minute. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/371971 Mon, 20 May 2024 07:42:07 -0000 https://status.modal.com/incident/371971#a888776a29d52c183297c091f9e0cae90ce8c4b6f9394573914b070752e70e71 Web endpoints recovered. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/371971 Mon, 20 May 2024 07:41:26 -0000 https://status.modal.com/incident/371971#41b188a70d07138ef9e3dd3c0b8d9b03637960e41ffa1bcc2925b953964b9319 Web API recovered. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/371971 Mon, 20 May 2024 07:40:00 -0000 https://status.modal.com/incident/371971#ea7487272cf4010570080222c454a8401a897a1bebd48d7447cbbb72d775812a Volumes recovered. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/371971 Mon, 20 May 2024 07:39:40 -0000 https://status.modal.com/incident/371971#f1d59632c91913741f9401f1a34ef921863cf571d5aa6eff8625ec48e3e5b2ca CPU functions on AWS recovered. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/371971 Mon, 20 May 2024 07:38:52 -0000 https://status.modal.com/incident/371971#12777789ff73aff2dd1959d98c03ab718f550bf4230c6c312c5e6051da39a31e Web API went down and CPU functions on GCP and CPU functions on OCI recovered. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/371971 Mon, 20 May 2024 07:37:26 -0000 https://status.modal.com/incident/371971#43022979242338f8f95598dbf2625b4358d2eee711d0a284f75a723b1461c9ac CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, and 2 other resources went down. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/371971 Mon, 20 May 2024 07:37:26 -0000 https://status.modal.com/incident/371971#43022979242338f8f95598dbf2625b4358d2eee711d0a284f75a723b1461c9ac CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, and 2 other resources went down. CPU functions on AWS, CPU functions on GCP, and 4 other services are down https://status.modal.com/incident/371971 Mon, 20 May 2024 07:37:26 -0000 https://status.modal.com/incident/371971#43022979242338f8f95598dbf2625b4358d2eee711d0a284f75a723b1461c9ac CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, and 2 other resources went down. Database I/O load https://status.modal.com/incident/359140 Mon, 22 Apr 2024 15:41:00 -0000 https://status.modal.com/incident/359140#1a6faa8de85fcc2f99f4bb41d00b98c5cd72de56ba0fc85325747f6704285662 We have resolved the issue. Thank you for your patience! We are continuing to monitor. Database I/O load https://status.modal.com/incident/359140 Mon, 22 Apr 2024 15:41:00 -0000 https://status.modal.com/incident/359140#1a6faa8de85fcc2f99f4bb41d00b98c5cd72de56ba0fc85325747f6704285662 We have resolved the issue. Thank you for your patience! We are continuing to monitor. Database I/O load https://status.modal.com/incident/359140 Mon, 22 Apr 2024 15:41:00 -0000 https://status.modal.com/incident/359140#1a6faa8de85fcc2f99f4bb41d00b98c5cd72de56ba0fc85325747f6704285662 We have resolved the issue. Thank you for your patience! We are continuing to monitor. Database I/O load https://status.modal.com/incident/359140 Mon, 22 Apr 2024 15:41:00 -0000 https://status.modal.com/incident/359140#1a6faa8de85fcc2f99f4bb41d00b98c5cd72de56ba0fc85325747f6704285662 We have resolved the issue. Thank you for your patience! We are continuing to monitor. Database I/O load https://status.modal.com/incident/359140 Mon, 22 Apr 2024 15:36:00 -0000 https://status.modal.com/incident/359140#667cf6524fe166e4e78ac5c4f4aacc41229310d39e60a0461300bade58379d70 We are rebooting the database to apply an urgent configuration change. Minor disruption for up to 1 minute is expected. Database I/O load https://status.modal.com/incident/359140 Mon, 22 Apr 2024 15:36:00 -0000 https://status.modal.com/incident/359140#667cf6524fe166e4e78ac5c4f4aacc41229310d39e60a0461300bade58379d70 We are rebooting the database to apply an urgent configuration change. Minor disruption for up to 1 minute is expected. Database I/O load https://status.modal.com/incident/359140 Mon, 22 Apr 2024 15:36:00 -0000 https://status.modal.com/incident/359140#667cf6524fe166e4e78ac5c4f4aacc41229310d39e60a0461300bade58379d70 We are rebooting the database to apply an urgent configuration change. Minor disruption for up to 1 minute is expected. Database I/O load https://status.modal.com/incident/359140 Mon, 22 Apr 2024 15:36:00 -0000 https://status.modal.com/incident/359140#667cf6524fe166e4e78ac5c4f4aacc41229310d39e60a0461300bade58379d70 We are rebooting the database to apply an urgent configuration change. Minor disruption for up to 1 minute is expected. Database I/O load https://status.modal.com/incident/359140 Mon, 22 Apr 2024 15:14:00 -0000 https://status.modal.com/incident/359140#87f71548af4da8b932800e679726988e224eba828631a962914e3370711d21d8 We are still experiencing degraded performance and reliability due to database load. However, we have found the root cause of the issue and are continuing to work toward resolution. Database I/O load https://status.modal.com/incident/359140 Mon, 22 Apr 2024 15:14:00 -0000 https://status.modal.com/incident/359140#87f71548af4da8b932800e679726988e224eba828631a962914e3370711d21d8 We are still experiencing degraded performance and reliability due to database load. However, we have found the root cause of the issue and are continuing to work toward resolution. Database I/O load https://status.modal.com/incident/359140 Mon, 22 Apr 2024 15:14:00 -0000 https://status.modal.com/incident/359140#87f71548af4da8b932800e679726988e224eba828631a962914e3370711d21d8 We are still experiencing degraded performance and reliability due to database load. However, we have found the root cause of the issue and are continuing to work toward resolution. Database I/O load https://status.modal.com/incident/359140 Mon, 22 Apr 2024 15:14:00 -0000 https://status.modal.com/incident/359140#87f71548af4da8b932800e679726988e224eba828631a962914e3370711d21d8 We are still experiencing degraded performance and reliability due to database load. However, we have found the root cause of the issue and are continuing to work toward resolution. Database I/O load https://status.modal.com/incident/359140 Mon, 22 Apr 2024 14:02:00 -0000 https://status.modal.com/incident/359140#a827ca6227a5f80c5468b3f96def2a1260e44583355584d72ccc1caaf90b8f84 We’re experiencing database I/O load issues at the moment, which is affecting responsiveness on the modal.com dashboard and execution of function calls. We’re actively working on resolving the issue. Database I/O load https://status.modal.com/incident/359140 Mon, 22 Apr 2024 14:02:00 -0000 https://status.modal.com/incident/359140#a827ca6227a5f80c5468b3f96def2a1260e44583355584d72ccc1caaf90b8f84 We’re experiencing database I/O load issues at the moment, which is affecting responsiveness on the modal.com dashboard and execution of function calls. We’re actively working on resolving the issue. Database I/O load https://status.modal.com/incident/359140 Mon, 22 Apr 2024 14:02:00 -0000 https://status.modal.com/incident/359140#a827ca6227a5f80c5468b3f96def2a1260e44583355584d72ccc1caaf90b8f84 We’re experiencing database I/O load issues at the moment, which is affecting responsiveness on the modal.com dashboard and execution of function calls. We’re actively working on resolving the issue. Database I/O load https://status.modal.com/incident/359140 Mon, 22 Apr 2024 14:02:00 -0000 https://status.modal.com/incident/359140#a827ca6227a5f80c5468b3f96def2a1260e44583355584d72ccc1caaf90b8f84 We’re experiencing database I/O load issues at the moment, which is affecting responsiveness on the modal.com dashboard and execution of function calls. We’re actively working on resolving the issue. False alerts for downtime https://status.modal.com/incident/352145 Sun, 07 Apr 2024 16:50:00 -0000 https://status.modal.com/incident/352145#766c399ed867c146ade7c413a403c81dc3d734139ff14150a2ee16ad212affd7 We hit a limit on our internal workspace used to run synthetic monitors, which caused our downtime alerts to fire inaccurately. We have raised the limits now. No users were affected. We're sorry for all the spam for status page subscribers! False alerts for downtime https://status.modal.com/incident/352145 Sun, 07 Apr 2024 16:50:00 -0000 https://status.modal.com/incident/352145#766c399ed867c146ade7c413a403c81dc3d734139ff14150a2ee16ad212affd7 We hit a limit on our internal workspace used to run synthetic monitors, which caused our downtime alerts to fire inaccurately. We have raised the limits now. No users were affected. We're sorry for all the spam for status page subscribers! Database migration https://status.modal.com/incident/351005 Fri, 05 Apr 2024 08:00:00 +0000 https://status.modal.com/incident/351005#b5d0544d956a65ed7d9640e2d80cc42f70f5629b940547e5d06efcc03a7db150 Maintenance completed Database migration https://status.modal.com/incident/351005 Fri, 05 Apr 2024 08:00:00 +0000 https://status.modal.com/incident/351005#b5d0544d956a65ed7d9640e2d80cc42f70f5629b940547e5d06efcc03a7db150 Maintenance completed Database migration https://status.modal.com/incident/351005 Fri, 05 Apr 2024 08:00:00 +0000 https://status.modal.com/incident/351005#b5d0544d956a65ed7d9640e2d80cc42f70f5629b940547e5d06efcc03a7db150 Maintenance completed Database migration https://status.modal.com/incident/351005 Fri, 05 Apr 2024 08:00:00 +0000 https://status.modal.com/incident/351005#b5d0544d956a65ed7d9640e2d80cc42f70f5629b940547e5d06efcc03a7db150 Maintenance completed Database migration https://status.modal.com/incident/351005 Fri, 05 Apr 2024 08:00:00 +0000 https://status.modal.com/incident/351005#b5d0544d956a65ed7d9640e2d80cc42f70f5629b940547e5d06efcc03a7db150 Maintenance completed Database migration https://status.modal.com/incident/351005 Fri, 05 Apr 2024 08:00:00 +0000 https://status.modal.com/incident/351005#b5d0544d956a65ed7d9640e2d80cc42f70f5629b940547e5d06efcc03a7db150 Maintenance completed Database migration https://status.modal.com/incident/351005 Fri, 05 Apr 2024 04:00:00 -0000 https://status.modal.com/incident/351005#b60083880152e9f54358ed05de27e48736e4a25d6015ccbbe5ff2938cc155c18 We will upgrade one of our databases to reduce latency and improve reliability. Database migration https://status.modal.com/incident/351005 Fri, 05 Apr 2024 04:00:00 -0000 https://status.modal.com/incident/351005#b60083880152e9f54358ed05de27e48736e4a25d6015ccbbe5ff2938cc155c18 We will upgrade one of our databases to reduce latency and improve reliability. Database migration https://status.modal.com/incident/351005 Fri, 05 Apr 2024 04:00:00 -0000 https://status.modal.com/incident/351005#b60083880152e9f54358ed05de27e48736e4a25d6015ccbbe5ff2938cc155c18 We will upgrade one of our databases to reduce latency and improve reliability. Database migration https://status.modal.com/incident/351005 Fri, 05 Apr 2024 04:00:00 -0000 https://status.modal.com/incident/351005#b60083880152e9f54358ed05de27e48736e4a25d6015ccbbe5ff2938cc155c18 We will upgrade one of our databases to reduce latency and improve reliability. Database migration https://status.modal.com/incident/351005 Fri, 05 Apr 2024 04:00:00 -0000 https://status.modal.com/incident/351005#b60083880152e9f54358ed05de27e48736e4a25d6015ccbbe5ff2938cc155c18 We will upgrade one of our databases to reduce latency and improve reliability. Database migration https://status.modal.com/incident/351005 Fri, 05 Apr 2024 04:00:00 -0000 https://status.modal.com/incident/351005#b60083880152e9f54358ed05de27e48736e4a25d6015ccbbe5ff2938cc155c18 We will upgrade one of our databases to reduce latency and improve reliability. Database migration https://status.modal.com/incident/343672 Fri, 22 Mar 2024 11:00:00 +0000 https://status.modal.com/incident/343672#0bf2722fb355c166df5910744acc41c11e57ced01444303e27406accc45927ff Maintenance completed Database migration https://status.modal.com/incident/343672 Fri, 22 Mar 2024 11:00:00 +0000 https://status.modal.com/incident/343672#0bf2722fb355c166df5910744acc41c11e57ced01444303e27406accc45927ff Maintenance completed Database migration https://status.modal.com/incident/343672 Fri, 22 Mar 2024 11:00:00 +0000 https://status.modal.com/incident/343672#0bf2722fb355c166df5910744acc41c11e57ced01444303e27406accc45927ff Maintenance completed Database migration https://status.modal.com/incident/343672 Fri, 22 Mar 2024 11:00:00 +0000 https://status.modal.com/incident/343672#0bf2722fb355c166df5910744acc41c11e57ced01444303e27406accc45927ff Maintenance completed Database migration https://status.modal.com/incident/343672 Fri, 22 Mar 2024 11:00:00 +0000 https://status.modal.com/incident/343672#0bf2722fb355c166df5910744acc41c11e57ced01444303e27406accc45927ff Maintenance completed Database migration https://status.modal.com/incident/343672 Fri, 22 Mar 2024 11:00:00 +0000 https://status.modal.com/incident/343672#0bf2722fb355c166df5910744acc41c11e57ced01444303e27406accc45927ff Maintenance completed Database migration https://status.modal.com/incident/343672 Fri, 22 Mar 2024 07:00:00 -0000 https://status.modal.com/incident/343672#b9ab3ac77f43ceaae843d8c6e0076ef0fc56cc57931ea21bc04d8bf63b59d168 We are migrating a database for better performance and reliability. Database migration https://status.modal.com/incident/343672 Fri, 22 Mar 2024 07:00:00 -0000 https://status.modal.com/incident/343672#b9ab3ac77f43ceaae843d8c6e0076ef0fc56cc57931ea21bc04d8bf63b59d168 We are migrating a database for better performance and reliability. Database migration https://status.modal.com/incident/343672 Fri, 22 Mar 2024 07:00:00 -0000 https://status.modal.com/incident/343672#b9ab3ac77f43ceaae843d8c6e0076ef0fc56cc57931ea21bc04d8bf63b59d168 We are migrating a database for better performance and reliability. Database migration https://status.modal.com/incident/343672 Fri, 22 Mar 2024 07:00:00 -0000 https://status.modal.com/incident/343672#b9ab3ac77f43ceaae843d8c6e0076ef0fc56cc57931ea21bc04d8bf63b59d168 We are migrating a database for better performance and reliability. Database migration https://status.modal.com/incident/343672 Fri, 22 Mar 2024 07:00:00 -0000 https://status.modal.com/incident/343672#b9ab3ac77f43ceaae843d8c6e0076ef0fc56cc57931ea21bc04d8bf63b59d168 We are migrating a database for better performance and reliability. Database migration https://status.modal.com/incident/343672 Fri, 22 Mar 2024 07:00:00 -0000 https://status.modal.com/incident/343672#b9ab3ac77f43ceaae843d8c6e0076ef0fc56cc57931ea21bc04d8bf63b59d168 We are migrating a database for better performance and reliability. Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 20:37:35 +0000 https://status.modal.com/incident/343666#286f4e7ff974f8f46fadbd828d89e5f8c86c936dc1b381bf43b2eb2eadc2e513 Maintenance completed Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 20:37:35 -0000 https://status.modal.com/incident/343666#fd9fee505341f832c7b8a9c57be2011191aed09a570d2e4ac8bc61c6cfe07340 We will be upgrading one of our databases to improve performance and reliability. Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 20:37:35 +0000 https://status.modal.com/incident/343666#286f4e7ff974f8f46fadbd828d89e5f8c86c936dc1b381bf43b2eb2eadc2e513 Maintenance completed Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 20:37:35 -0000 https://status.modal.com/incident/343666#fd9fee505341f832c7b8a9c57be2011191aed09a570d2e4ac8bc61c6cfe07340 We will be upgrading one of our databases to improve performance and reliability. Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 20:37:35 +0000 https://status.modal.com/incident/343666#286f4e7ff974f8f46fadbd828d89e5f8c86c936dc1b381bf43b2eb2eadc2e513 Maintenance completed Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 20:37:35 -0000 https://status.modal.com/incident/343666#fd9fee505341f832c7b8a9c57be2011191aed09a570d2e4ac8bc61c6cfe07340 We will be upgrading one of our databases to improve performance and reliability. Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 20:37:35 +0000 https://status.modal.com/incident/343666#286f4e7ff974f8f46fadbd828d89e5f8c86c936dc1b381bf43b2eb2eadc2e513 Maintenance completed Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 20:37:35 -0000 https://status.modal.com/incident/343666#fd9fee505341f832c7b8a9c57be2011191aed09a570d2e4ac8bc61c6cfe07340 We will be upgrading one of our databases to improve performance and reliability. Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 20:37:35 +0000 https://status.modal.com/incident/343666#286f4e7ff974f8f46fadbd828d89e5f8c86c936dc1b381bf43b2eb2eadc2e513 Maintenance completed Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 20:37:35 -0000 https://status.modal.com/incident/343666#fd9fee505341f832c7b8a9c57be2011191aed09a570d2e4ac8bc61c6cfe07340 We will be upgrading one of our databases to improve performance and reliability. Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 20:37:35 +0000 https://status.modal.com/incident/343666#286f4e7ff974f8f46fadbd828d89e5f8c86c936dc1b381bf43b2eb2eadc2e513 Maintenance completed Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 20:37:35 -0000 https://status.modal.com/incident/343666#fd9fee505341f832c7b8a9c57be2011191aed09a570d2e4ac8bc61c6cfe07340 We will be upgrading one of our databases to improve performance and reliability. Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 19:00:00 -0000 https://status.modal.com/incident/343666#fd9fee505341f832c7b8a9c57be2011191aed09a570d2e4ac8bc61c6cfe07340 We will be upgrading one of our databases to improve performance and reliability. Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 19:00:00 -0000 https://status.modal.com/incident/343666#fd9fee505341f832c7b8a9c57be2011191aed09a570d2e4ac8bc61c6cfe07340 We will be upgrading one of our databases to improve performance and reliability. Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 19:00:00 -0000 https://status.modal.com/incident/343666#fd9fee505341f832c7b8a9c57be2011191aed09a570d2e4ac8bc61c6cfe07340 We will be upgrading one of our databases to improve performance and reliability. Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 19:00:00 -0000 https://status.modal.com/incident/343666#fd9fee505341f832c7b8a9c57be2011191aed09a570d2e4ac8bc61c6cfe07340 We will be upgrading one of our databases to improve performance and reliability. Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 19:00:00 -0000 https://status.modal.com/incident/343666#fd9fee505341f832c7b8a9c57be2011191aed09a570d2e4ac8bc61c6cfe07340 We will be upgrading one of our databases to improve performance and reliability. Database upgrade https://status.modal.com/incident/343666 Wed, 20 Mar 2024 19:00:00 -0000 https://status.modal.com/incident/343666#fd9fee505341f832c7b8a9c57be2011191aed09a570d2e4ac8bc61c6cfe07340 We will be upgrading one of our databases to improve performance and reliability. Web endpoints is down https://status.modal.com/incident/337389 Thu, 07 Mar 2024 04:02:29 -0000 https://status.modal.com/incident/337389#193da43e76c4c92450378e9aeff979eb549bb2c065cd3fb791bf8c9ef2e21729 Web endpoints recovered. Web endpoints is down https://status.modal.com/incident/337389 Thu, 07 Mar 2024 03:57:23 -0000 https://status.modal.com/incident/337389#180c43fd88f96ec797f1037b6a4621cebdaa49ef0a7426f76ccb746512905761 Web endpoints went down. Web endpoints is down https://status.modal.com/incident/332362 Tue, 27 Feb 2024 00:39:51 -0000 https://status.modal.com/incident/332362#83feecb0b29fb8b6cf9be6f999d62b7cd7d949a0ff6c00d70b3bf09c5e292829 Web endpoints recovered. Web endpoints is down https://status.modal.com/incident/332362 Tue, 27 Feb 2024 00:32:39 -0000 https://status.modal.com/incident/332362#ab88e123d335ee203fa1478cb85dc94e88cfb00525a6c887d3052eb52beac604 Web endpoints went down. DNS issues causing website downtime and degradation of the service https://status.modal.com/incident/326722 Wed, 14 Feb 2024 12:20:00 -0000 https://status.modal.com/incident/326722#42b25bc4d52f8ed004f81bda75e8492fdb14573f06fd773e8d5c0aad32c1fdce We experienced a problem with our DNS records for modal.com and modal.run, meaning that the website and webpoints were unreachable for a period, unless DNS records were cached by your local resolver. This also affected workloads running on modal to some degree, as some workloads were failing to communicate with our API server. The issue has been resolved but there might be lingering effects due to negative DNS caches in your local DNS resolver (e.g. your ISP resolvers). DNS issues causing website downtime and degradation of the service https://status.modal.com/incident/326722 Wed, 14 Feb 2024 12:20:00 -0000 https://status.modal.com/incident/326722#42b25bc4d52f8ed004f81bda75e8492fdb14573f06fd773e8d5c0aad32c1fdce We experienced a problem with our DNS records for modal.com and modal.run, meaning that the website and webpoints were unreachable for a period, unless DNS records were cached by your local resolver. This also affected workloads running on modal to some degree, as some workloads were failing to communicate with our API server. The issue has been resolved but there might be lingering effects due to negative DNS caches in your local DNS resolver (e.g. your ISP resolvers). DNS issues causing website downtime and degradation of the service https://status.modal.com/incident/326722 Wed, 14 Feb 2024 12:20:00 -0000 https://status.modal.com/incident/326722#42b25bc4d52f8ed004f81bda75e8492fdb14573f06fd773e8d5c0aad32c1fdce We experienced a problem with our DNS records for modal.com and modal.run, meaning that the website and webpoints were unreachable for a period, unless DNS records were cached by your local resolver. This also affected workloads running on modal to some degree, as some workloads were failing to communicate with our API server. The issue has been resolved but there might be lingering effects due to negative DNS caches in your local DNS resolver (e.g. your ISP resolvers). DNS issues causing website downtime and degradation of the service https://status.modal.com/incident/326722 Wed, 14 Feb 2024 12:20:00 -0000 https://status.modal.com/incident/326722#42b25bc4d52f8ed004f81bda75e8492fdb14573f06fd773e8d5c0aad32c1fdce We experienced a problem with our DNS records for modal.com and modal.run, meaning that the website and webpoints were unreachable for a period, unless DNS records were cached by your local resolver. This also affected workloads running on modal to some degree, as some workloads were failing to communicate with our API server. The issue has been resolved but there might be lingering effects due to negative DNS caches in your local DNS resolver (e.g. your ISP resolvers). DNS issues causing website downtime and degradation of the service https://status.modal.com/incident/326722 Wed, 14 Feb 2024 12:20:00 -0000 https://status.modal.com/incident/326722#42b25bc4d52f8ed004f81bda75e8492fdb14573f06fd773e8d5c0aad32c1fdce We experienced a problem with our DNS records for modal.com and modal.run, meaning that the website and webpoints were unreachable for a period, unless DNS records were cached by your local resolver. This also affected workloads running on modal to some degree, as some workloads were failing to communicate with our API server. The issue has been resolved but there might be lingering effects due to negative DNS caches in your local DNS resolver (e.g. your ISP resolvers). Database upgrade https://status.modal.com/incident/320535 Thu, 01 Feb 2024 10:00:00 +0000 https://status.modal.com/incident/320535#4e15e233cf5c7deaae02cff7fd5eac1fad94a877fbe3534ae16d1b0892b7a21d Maintenance completed Database upgrade https://status.modal.com/incident/320535 Thu, 01 Feb 2024 10:00:00 +0000 https://status.modal.com/incident/320535#4e15e233cf5c7deaae02cff7fd5eac1fad94a877fbe3534ae16d1b0892b7a21d Maintenance completed Database upgrade https://status.modal.com/incident/320535 Thu, 01 Feb 2024 10:00:00 +0000 https://status.modal.com/incident/320535#4e15e233cf5c7deaae02cff7fd5eac1fad94a877fbe3534ae16d1b0892b7a21d Maintenance completed Database upgrade https://status.modal.com/incident/320535 Thu, 01 Feb 2024 10:00:00 +0000 https://status.modal.com/incident/320535#4e15e233cf5c7deaae02cff7fd5eac1fad94a877fbe3534ae16d1b0892b7a21d Maintenance completed Database upgrade https://status.modal.com/incident/320535 Thu, 01 Feb 2024 10:00:00 +0000 https://status.modal.com/incident/320535#4e15e233cf5c7deaae02cff7fd5eac1fad94a877fbe3534ae16d1b0892b7a21d Maintenance completed Database upgrade https://status.modal.com/incident/320535 Thu, 01 Feb 2024 10:00:00 +0000 https://status.modal.com/incident/320535#4e15e233cf5c7deaae02cff7fd5eac1fad94a877fbe3534ae16d1b0892b7a21d Maintenance completed Database upgrade https://status.modal.com/incident/320535 Thu, 01 Feb 2024 09:00:00 -0000 https://status.modal.com/incident/320535#80be12c3078e8f8ca469843d39cb95536cda914535f8f1ba6579453dcbbba126 We are upgrading an internal database. The Modal service should remain available during this time, although response latencies and error rates might be somewhat elevated. Database upgrade https://status.modal.com/incident/320535 Thu, 01 Feb 2024 09:00:00 -0000 https://status.modal.com/incident/320535#80be12c3078e8f8ca469843d39cb95536cda914535f8f1ba6579453dcbbba126 We are upgrading an internal database. The Modal service should remain available during this time, although response latencies and error rates might be somewhat elevated. Database upgrade https://status.modal.com/incident/320535 Thu, 01 Feb 2024 09:00:00 -0000 https://status.modal.com/incident/320535#80be12c3078e8f8ca469843d39cb95536cda914535f8f1ba6579453dcbbba126 We are upgrading an internal database. The Modal service should remain available during this time, although response latencies and error rates might be somewhat elevated. Database upgrade https://status.modal.com/incident/320535 Thu, 01 Feb 2024 09:00:00 -0000 https://status.modal.com/incident/320535#80be12c3078e8f8ca469843d39cb95536cda914535f8f1ba6579453dcbbba126 We are upgrading an internal database. The Modal service should remain available during this time, although response latencies and error rates might be somewhat elevated. Database upgrade https://status.modal.com/incident/320535 Thu, 01 Feb 2024 09:00:00 -0000 https://status.modal.com/incident/320535#80be12c3078e8f8ca469843d39cb95536cda914535f8f1ba6579453dcbbba126 We are upgrading an internal database. The Modal service should remain available during this time, although response latencies and error rates might be somewhat elevated. Database upgrade https://status.modal.com/incident/320535 Thu, 01 Feb 2024 09:00:00 -0000 https://status.modal.com/incident/320535#80be12c3078e8f8ca469843d39cb95536cda914535f8f1ba6579453dcbbba126 We are upgrading an internal database. The Modal service should remain available during this time, although response latencies and error rates might be somewhat elevated. Frontend and Web API are down https://status.modal.com/incident/314550 Thu, 18 Jan 2024 23:07:56 -0000 https://status.modal.com/incident/314550#a9912f924dd156beb917af6d963682069c409afab981dbd502079ff92587fa5d Frontend recovered. Frontend and Web API are down https://status.modal.com/incident/314550 Thu, 18 Jan 2024 23:06:39 -0000 https://status.modal.com/incident/314550#592b3174d51ee8735848aa1251bb468db684a28060e807036a7f2bb9ca6363e6 Frontend went down. Frontend and Web API are down https://status.modal.com/incident/314550 Thu, 18 Jan 2024 22:55:02 -0000 https://status.modal.com/incident/314550#264ad9817cada0afcf0a89c88c33d3fa7434b6a765a33792da63ea0f4cb55516 Frontend recovered. Frontend and Web API are down https://status.modal.com/incident/314550 Thu, 18 Jan 2024 22:49:41 -0000 https://status.modal.com/incident/314550#abe9d717fc5359360ee875bca6e8f692b44a667b54137a8a6822136e7381163c Frontend went down. Frontend and Web API are down https://status.modal.com/incident/314550 Thu, 18 Jan 2024 15:50:03 -0000 https://status.modal.com/incident/314550#5bd6a4dd3981afaee0373d9058820e567202111d4bb5dbb9019765d7bf4fd6ba Web API recovered. Frontend and Web API are down https://status.modal.com/incident/314550 Thu, 18 Jan 2024 15:46:58 -0000 https://status.modal.com/incident/314550#21bc18e63bab7be5cd656585547ff3cc882a7c1672fc6ff7f73c19c3b6a6e32c Web API went down. Frontend, Web API, and 1 other service are down https://status.modal.com/incident/314230 Thu, 18 Jan 2024 08:52:00 -0000 https://status.modal.com/incident/314230#7e83790d5041ce0ae2c2a3de06ad5fdaad755daa774f899f0e30219f4b06ccb2 According to our investigation this was a false positive. Independent internal monitoring reports no issues and we failed to reproduce any problems. We're continuing to investigate the root cause. Frontend, Web API, and 1 other service are down https://status.modal.com/incident/314230 Thu, 18 Jan 2024 08:34:00 -0000 https://status.modal.com/incident/314230#dd43eac832a250405a64d2c0dd226b64020fd2cadaf5fa5f1372d1c781403a4f Frontend recovered. Frontend, Web API, and 1 other service are down https://status.modal.com/incident/314230 Thu, 18 Jan 2024 08:29:24 -0000 https://status.modal.com/incident/314230#d3da0463ab1e131b04d01fc680e3e3eed4622833bddda27534244dc1d18c38d9 Web API recovered. Frontend, Web API, and 1 other service are down https://status.modal.com/incident/314230 Thu, 18 Jan 2024 07:20:24 -0000 https://status.modal.com/incident/314230#ca017d13262a0b666671ac482b5c92fa64bc95715997e4d640269db540cd4b40 Web API went down. Frontend, Web API, and 1 other service are down https://status.modal.com/incident/314230 Thu, 18 Jan 2024 07:10:48 -0000 https://status.modal.com/incident/314230#b195ee758ac18fd889eccf195020dbbc0957a08ced77840922e929ea0bab98d7 Frontend went down. Frontend, Web API, and 1 other service are down https://status.modal.com/incident/314230 Thu, 18 Jan 2024 03:07:12 -0000 https://status.modal.com/incident/314230#5ee90e22382f6d2f4ccc9310e38783e4ef9be888dfeac32789ba4a44b17a0362 Web endpoints recovered. Frontend, Web API, and 1 other service are down https://status.modal.com/incident/314230 Thu, 18 Jan 2024 02:44:44 -0000 https://status.modal.com/incident/314230#7c09066ee3a3c19cb403a1631ec99ef9d25601fd27c42bf65519ee445a6307e5 Web endpoints went down. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 09:27:27 +0000 https://status.modal.com/incident/311846#cf327e991891a114eb6ee088b6cc5aa2db8643c77603ddfea62e3f49c0767e8d Maintenance completed Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 09:27:27 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 09:27:27 +0000 https://status.modal.com/incident/311846#cf327e991891a114eb6ee088b6cc5aa2db8643c77603ddfea62e3f49c0767e8d Maintenance completed Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 09:27:27 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 09:27:27 +0000 https://status.modal.com/incident/311846#cf327e991891a114eb6ee088b6cc5aa2db8643c77603ddfea62e3f49c0767e8d Maintenance completed Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 09:27:27 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 09:27:27 +0000 https://status.modal.com/incident/311846#cf327e991891a114eb6ee088b6cc5aa2db8643c77603ddfea62e3f49c0767e8d Maintenance completed Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 09:27:27 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 09:27:27 +0000 https://status.modal.com/incident/311846#cf327e991891a114eb6ee088b6cc5aa2db8643c77603ddfea62e3f49c0767e8d Maintenance completed Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 09:27:27 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 09:27:27 +0000 https://status.modal.com/incident/311846#cf327e991891a114eb6ee088b6cc5aa2db8643c77603ddfea62e3f49c0767e8d Maintenance completed Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 09:27:27 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 08:53:11 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 08:53:11 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 08:53:11 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 08:53:11 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 08:53:11 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 08:53:11 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 07:45:22 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 07:45:22 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 07:45:22 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 07:45:22 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 07:45:22 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 07:45:22 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 04:59:00 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 04:59:00 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 04:59:00 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 04:59:00 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 04:59:00 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Database upgrades https://status.modal.com/incident/311846 Wed, 17 Jan 2024 04:59:00 -0000 https://status.modal.com/incident/311846#f45233cbe1e038dfbc8ba09f821c164b682f8d21932678a73558fb42acb9eeb5 We'll conduct database upgrades for improved performance, stability, and security. Web API and Web endpoints are down https://status.modal.com/incident/310464 Tue, 09 Jan 2024 20:52:47 -0000 https://status.modal.com/incident/310464#b2cda63669022cd0d9950f83d4ffef6f126a00799e9d77101a8041435dff83e4 Web endpoints recovered. Web API and Web endpoints are down https://status.modal.com/incident/310464 Tue, 09 Jan 2024 20:49:49 -0000 https://status.modal.com/incident/310464#5741555f6f80aa4499bfda8a2b311f3f90fa446b2410c7339bbf3bb0e718367a Web endpoints went down and Web API recovered. Web API and Web endpoints are down https://status.modal.com/incident/310464 Tue, 09 Jan 2024 20:49:49 -0000 https://status.modal.com/incident/310464#5741555f6f80aa4499bfda8a2b311f3f90fa446b2410c7339bbf3bb0e718367a Web endpoints went down and Web API recovered. Web API and Web endpoints are down https://status.modal.com/incident/310464 Tue, 09 Jan 2024 20:42:19 -0000 https://status.modal.com/incident/310464#e1eaca6b210b75cdc227939db6eb7d778232f8e1bdd8f720e9485b5f75c86a62 Web API went down. Web API and Web endpoints are down https://status.modal.com/incident/310464 Tue, 09 Jan 2024 18:36:06 -0000 https://status.modal.com/incident/310464#f25a5aa9354e72c90d2ab3c646529713a24c14c420ff6693f6dedc13208a8304 Web endpoints recovered. Web API and Web endpoints are down https://status.modal.com/incident/310464 Tue, 09 Jan 2024 18:29:27 -0000 https://status.modal.com/incident/310464#f493c58c276561037685e4877fc987578cc77156c171d3feca6b4aa84ebb9773 Web endpoints went down. Web endpoints is down https://status.modal.com/incident/310037 Mon, 08 Jan 2024 19:38:32 -0000 https://status.modal.com/incident/310037#b1bf0dd7f96d5ea32347ca88fde630884edd606d76ac1e6392bdbd963b4f2807 Web endpoints recovered. Web endpoints is down https://status.modal.com/incident/310037 Mon, 08 Jan 2024 19:04:53 -0000 https://status.modal.com/incident/310037#b514051f1c34c9901df23476ce32e3447b1fe64384084f1cb4d2f98a3b53969c Web endpoints went down. Web endpoints is down https://status.modal.com/incident/304923 Tue, 26 Dec 2023 15:48:16 -0000 https://status.modal.com/incident/304923#c4007ba195540938dfc2259600289ccf851443f1d39a4ae2ffbd88745dc0d7a4 Web endpoints recovered. Web endpoints is down https://status.modal.com/incident/304923 Tue, 26 Dec 2023 15:34:34 -0000 https://status.modal.com/incident/304923#ca71fd291c01d276096f92d73c63ae67c9ca8b0392e1799a5d62d427a2f77949 Web endpoints went down. Web endpoint incident resolved https://status.modal.com/incident/304971 Tue, 26 Dec 2023 01:40:00 -0000 https://status.modal.com/incident/304971#61fe8a00e8812d9df94254f84e60688a0f4781a71c6ef7c3c329a3d7fc168502 From 23:30 UTC - 01:40 UTC, web endpoints deployed on Modal had degraded performance and transient "400 Bad Request" status codes. We responded to the issue and published a fix. The root cause was increased load on our web endpoint system from user traffic, which tripped an internal rate limit. Web endpoints is down https://status.modal.com/incident/304923 Tue, 26 Dec 2023 01:29:16 -0000 https://status.modal.com/incident/304923#61d1d0506e0dafe34f8286b3290d613c6abdc2438de66e69bdf783d5fc81fd55 Web endpoints recovered. Web endpoints is down https://status.modal.com/incident/304923 Tue, 26 Dec 2023 01:25:08 -0000 https://status.modal.com/incident/304923#9f640c87454f4b304a7548da692088a697647dfc230a9572cb1c36a5c3c10549 Web endpoints went down. Web endpoints is down https://status.modal.com/incident/304923 Tue, 26 Dec 2023 01:05:24 -0000 https://status.modal.com/incident/304923#d95f761571b4b303c02f13e84e924261bb9677eb6bf18fe1a36b86e90462f450 Web endpoints recovered. Web endpoints is down https://status.modal.com/incident/304923 Tue, 26 Dec 2023 00:55:04 -0000 https://status.modal.com/incident/304923#abf4a2ca4b6adf1336dcb410ce5c82447aa59f2d82d8d432f63828d442e54134 Web endpoints went down. Web endpoints is down https://status.modal.com/incident/304923 Tue, 26 Dec 2023 00:04:47 -0000 https://status.modal.com/incident/304923#8962734724a81cd1495e0ccd499f97ca5188938da84440ec9ef418c71c7d294a Web endpoints recovered. Web endpoints is down https://status.modal.com/incident/304923 Mon, 25 Dec 2023 23:56:22 -0000 https://status.modal.com/incident/304923#a6dd5780cf1b3f5d06927e5e3cecc6f6ce7351ead0e09a0c6fdc81043d5b5daa Web endpoints went down. CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, Web endpoints, and 2 other services are down https://status.modal.com/incident/287108 Tue, 14 Nov 2023 16:46:52 -0000 https://status.modal.com/incident/287108#f6880f571f7a902fb6ac9900a152f8be0f63f32dc7c60894a1142cc8d21ee8f1 Web endpoints recovered. CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, Web endpoints, and 2 other services are down https://status.modal.com/incident/287108 Tue, 14 Nov 2023 16:41:59 -0000 https://status.modal.com/incident/287108#8bb05180a3303b14263a05af95dacd9425a0a5c60f74f1a64baed7b720fa26a7 CPU functions on AWS, CPU functions on GCP, and CPU functions on OCI recovered. CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, Web endpoints, and 2 other services are down https://status.modal.com/incident/287108 Tue, 14 Nov 2023 16:39:30 -0000 https://status.modal.com/incident/287108#c9f726afead35689147f4441905d6b102900f10e744d50e413734568f950d53d CPU functions on AWS and Web endpoints went down. CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, Web endpoints, and 2 other services are down https://status.modal.com/incident/287108 Tue, 14 Nov 2023 16:39:30 -0000 https://status.modal.com/incident/287108#c9f726afead35689147f4441905d6b102900f10e744d50e413734568f950d53d CPU functions on AWS and Web endpoints went down. CPU functions on GCP, CPU functions on OCI, Web endpoints, and 1 other service are down https://status.modal.com/incident/285072 Fri, 10 Nov 2023 02:44:22 -0000 https://status.modal.com/incident/285072#4f04b098c0023085c26f21d67f8edc9383939b6c48c97299140e8ed5d3ea62dc Web endpoints recovered. CPU functions on GCP, CPU functions on OCI, Web endpoints, and 1 other service are down https://status.modal.com/incident/285072 Fri, 10 Nov 2023 02:39:58 -0000 https://status.modal.com/incident/285072#0874cf4566b76ff173c6c5b9ea6ec9e7859c411f80d3d1c43ad8661565086d75 Web endpoints went down. GPU functions on GCP, GPU functions on OCI, API, Web endpoints, and 2 other services are down https://status.modal.com/incident/278544 Fri, 27 Oct 2023 14:35:35 -0000 https://status.modal.com/incident/278544#5b4b6913bebef1410e2fa47983f59d594918607010366f6797354e88645ef947 Web endpoints recovered. GPU functions on GCP, GPU functions on OCI, API, Web endpoints, and 2 other services are down https://status.modal.com/incident/278544 Fri, 27 Oct 2023 13:56:29 -0000 https://status.modal.com/incident/278544#e9c403f57334acbb59b84f52ffc46f0be9518f0a1bea1065655f3d88e1abb7dd API recovered. GPU functions on GCP, GPU functions on OCI, API, Web endpoints, and 2 other services are down https://status.modal.com/incident/278544 Fri, 27 Oct 2023 13:39:54 -0000 https://status.modal.com/incident/278544#b948212cdf52d83ba9ea0775fc8269b477f37a07a690fc4c5c145f40828a636f API went down. GPU functions on GCP, GPU functions on OCI, API, Web endpoints, and 2 other services are down https://status.modal.com/incident/278544 Fri, 27 Oct 2023 13:32:23 -0000 https://status.modal.com/incident/278544#ab6eb2d11c0b199084f747af15a0d696ddd561e85e7e7ae04940e76d2554cc66 Web endpoints went down. GPU functions on GCP, GPU functions on OCI, API, Web endpoints, and 2 other services are down https://status.modal.com/incident/278544 Fri, 27 Oct 2023 13:15:36 -0000 https://status.modal.com/incident/278544#e1a44c0e2bde45d6a61be2bb2a231cc63c8e47ce54945beff114b47ce7021f1f Web endpoints recovered. GPU functions on GCP, GPU functions on OCI, API, Web endpoints, and 2 other services are down https://status.modal.com/incident/278544 Fri, 27 Oct 2023 13:13:17 -0000 https://status.modal.com/incident/278544#60e877d30ce329e2732ef957444ed34651d241d54e00a5ec00ef3d8ba4c84a18 Web endpoints went down. Performance degradation https://status.modal.com/incident/277114 Tue, 24 Oct 2023 17:00:00 -0000 https://status.modal.com/incident/277114#1e32e4f7a3f4caa941ee9f41a313535a1b104003fbb30476e294c0a0929562b4 Issue resolved. Performance is back to normal. Performance degradation https://status.modal.com/incident/277114 Tue, 24 Oct 2023 17:00:00 -0000 https://status.modal.com/incident/277114#1e32e4f7a3f4caa941ee9f41a313535a1b104003fbb30476e294c0a0929562b4 Issue resolved. Performance is back to normal. Performance degradation https://status.modal.com/incident/277114 Tue, 24 Oct 2023 17:00:00 -0000 https://status.modal.com/incident/277114#1e32e4f7a3f4caa941ee9f41a313535a1b104003fbb30476e294c0a0929562b4 Issue resolved. Performance is back to normal. Performance degradation https://status.modal.com/incident/277114 Tue, 24 Oct 2023 17:00:00 -0000 https://status.modal.com/incident/277114#1e32e4f7a3f4caa941ee9f41a313535a1b104003fbb30476e294c0a0929562b4 Issue resolved. Performance is back to normal. Performance degradation https://status.modal.com/incident/277114 Tue, 24 Oct 2023 17:00:00 -0000 https://status.modal.com/incident/277114#1e32e4f7a3f4caa941ee9f41a313535a1b104003fbb30476e294c0a0929562b4 Issue resolved. Performance is back to normal. Performance degradation https://status.modal.com/incident/277114 Tue, 24 Oct 2023 16:47:00 -0000 https://status.modal.com/incident/277114#05679d9ad358b0ebc9f684e795996d19c363a0b283fcc323297dd2127265167b We are experiencing higher load than usual leading to performance degradation across our APIs, function, webhook executions. Performance degradation https://status.modal.com/incident/277114 Tue, 24 Oct 2023 16:47:00 -0000 https://status.modal.com/incident/277114#05679d9ad358b0ebc9f684e795996d19c363a0b283fcc323297dd2127265167b We are experiencing higher load than usual leading to performance degradation across our APIs, function, webhook executions. Performance degradation https://status.modal.com/incident/277114 Tue, 24 Oct 2023 16:47:00 -0000 https://status.modal.com/incident/277114#05679d9ad358b0ebc9f684e795996d19c363a0b283fcc323297dd2127265167b We are experiencing higher load than usual leading to performance degradation across our APIs, function, webhook executions. Upgrading system components https://status.modal.com/incident/271843 Mon, 16 Oct 2023 21:00:00 +0000 https://status.modal.com/incident/271843#3258477ce1ee3e4db2dccab1fd0209590b5bf0f17e5763de7257d4205ee43ba1 Maintenance completed Upgrading system components https://status.modal.com/incident/271843 Mon, 16 Oct 2023 21:00:00 +0000 https://status.modal.com/incident/271843#3258477ce1ee3e4db2dccab1fd0209590b5bf0f17e5763de7257d4205ee43ba1 Maintenance completed Upgrading system components https://status.modal.com/incident/271843 Mon, 16 Oct 2023 21:00:00 +0000 https://status.modal.com/incident/271843#3258477ce1ee3e4db2dccab1fd0209590b5bf0f17e5763de7257d4205ee43ba1 Maintenance completed Upgrading system components https://status.modal.com/incident/271843 Mon, 16 Oct 2023 21:00:00 +0000 https://status.modal.com/incident/271843#3258477ce1ee3e4db2dccab1fd0209590b5bf0f17e5763de7257d4205ee43ba1 Maintenance completed Upgrading system components https://status.modal.com/incident/271843 Mon, 16 Oct 2023 21:00:00 +0000 https://status.modal.com/incident/271843#3258477ce1ee3e4db2dccab1fd0209590b5bf0f17e5763de7257d4205ee43ba1 Maintenance completed Upgrading system components https://status.modal.com/incident/271843 Mon, 16 Oct 2023 15:00:00 -0000 https://status.modal.com/incident/271843#a3a858124dbe5cd94d74a8dee2f7d9f4d9675a2215a4c61cf170ca193cde2869 We will be performing a scheduled upgrade. Upgrading system components https://status.modal.com/incident/271843 Mon, 16 Oct 2023 15:00:00 -0000 https://status.modal.com/incident/271843#a3a858124dbe5cd94d74a8dee2f7d9f4d9675a2215a4c61cf170ca193cde2869 We will be performing a scheduled upgrade. Upgrading system components https://status.modal.com/incident/271843 Mon, 16 Oct 2023 15:00:00 -0000 https://status.modal.com/incident/271843#a3a858124dbe5cd94d74a8dee2f7d9f4d9675a2215a4c61cf170ca193cde2869 We will be performing a scheduled upgrade. Upgrading system components https://status.modal.com/incident/271843 Mon, 16 Oct 2023 15:00:00 -0000 https://status.modal.com/incident/271843#a3a858124dbe5cd94d74a8dee2f7d9f4d9675a2215a4c61cf170ca193cde2869 We will be performing a scheduled upgrade. Upgrading system components https://status.modal.com/incident/271843 Mon, 16 Oct 2023 15:00:00 -0000 https://status.modal.com/incident/271843#a3a858124dbe5cd94d74a8dee2f7d9f4d9675a2215a4c61cf170ca193cde2869 We will be performing a scheduled upgrade. API and Web endpoints are down https://status.modal.com/incident/272719 Sat, 14 Oct 2023 14:26:38 -0000 https://status.modal.com/incident/272719#e425864ab092137fb55c204468033916fc0511d511c24b038b6e5fff01590be4 Web endpoints recovered. API and Web endpoints are down https://status.modal.com/incident/272719 Sat, 14 Oct 2023 14:23:45 -0000 https://status.modal.com/incident/272719#43db5dd48a994e176058d976c1b5722c1e1018a0ffb2d647014ff4fd2a8831d6 Web endpoints went down. API and Web endpoints are down https://status.modal.com/incident/272719 Sat, 14 Oct 2023 14:18:03 -0000 https://status.modal.com/incident/272719#1b0084bfedb88a583607771f69f822d646f7330f7b15d291bfc6d958318c032a Web endpoints recovered. API and Web endpoints are down https://status.modal.com/incident/272719 Sat, 14 Oct 2023 14:12:17 -0000 https://status.modal.com/incident/272719#e79b90ab2471330da973ecb26430df815dd22928be24bb547b5babb1f467c7f0 API recovered. API and Web endpoints are down https://status.modal.com/incident/272719 Sat, 14 Oct 2023 14:09:17 -0000 https://status.modal.com/incident/272719#57741e301dc9829a83ec38bf705112289f5308315432dd5a53cd324954e2227b API went down. API and Web endpoints are down https://status.modal.com/incident/272719 Sat, 14 Oct 2023 14:07:48 -0000 https://status.modal.com/incident/272719#fc108fe45485d8fb43fb463691ad9529152ba54e2f1be4e7f5c0cb788e39ee4a Web endpoints went down. GPU functions on GCP and Web endpoints are down https://status.modal.com/incident/271342 Wed, 11 Oct 2023 17:32:52 -0000 https://status.modal.com/incident/271342#47eea1674fc830c98fa85411c205fc62d95545cff6afe9fb2494d9c43cde3eef Web endpoints recovered. GPU functions on GCP and Web endpoints are down https://status.modal.com/incident/271342 Wed, 11 Oct 2023 17:29:21 -0000 https://status.modal.com/incident/271342#f2205747ddce02bccec8b059f4427cff21ce4e627b6a8300e2bc5b5b91847388 Web endpoints went down. Web endpoints is down https://status.modal.com/incident/268359 Wed, 04 Oct 2023 19:58:08 -0000 https://status.modal.com/incident/268359#760fb027839e9511d867078b88193e2c19fa29c24b55c097827fcfe2ae8a0257 Web endpoints recovered. Web endpoints is down https://status.modal.com/incident/268359 Wed, 04 Oct 2023 19:56:08 -0000 https://status.modal.com/incident/268359#6fec372bfb55f9cef20ccd82c6fcd134b896f4f47b9058b20daa00da6c7bc086 Web endpoints went down. Network maintenance https://status.modal.com/incident/265716 Fri, 29 Sep 2023 18:55:00 +0000 https://status.modal.com/incident/265716#061a3dd437c4fe4ee866ccbdd6e08d3c8cd2ead4f7e9fd5d663c2e6c97616893 Maintenance completed Network maintenance https://status.modal.com/incident/265716 Fri, 29 Sep 2023 18:00:00 -0000 https://status.modal.com/incident/265716#dd313821b4881f587846f038c79fe29c22295c8b061be5d06b9ede8acc7702f9 We are performing scheduled network maintenance related to web endpoint handlers. This is to improve overall stability of our web ingress system, and to prepare for a new system that will reduce the amount of "Bad Gateway" and "Internal Server Errors" that users experience. We don't anticipate any downtime during this maintenance window, but we will stay alert and track any issues as they occur. Degraded performance in functions https://status.modal.com/incident/262498 Thu, 21 Sep 2023 19:37:00 -0000 https://status.modal.com/incident/262498#8e31876500045b490a17483d14777e1332fe75ec6cd0f15a776c70c24148bcf8 Issue has been resolved. We are monitoring. Degraded performance in functions https://status.modal.com/incident/262498 Thu, 21 Sep 2023 19:37:00 -0000 https://status.modal.com/incident/262498#8e31876500045b490a17483d14777e1332fe75ec6cd0f15a776c70c24148bcf8 Issue has been resolved. We are monitoring. Degraded performance in functions https://status.modal.com/incident/262498 Thu, 21 Sep 2023 19:37:00 -0000 https://status.modal.com/incident/262498#8e31876500045b490a17483d14777e1332fe75ec6cd0f15a776c70c24148bcf8 Issue has been resolved. We are monitoring. Degraded performance in functions https://status.modal.com/incident/262498 Thu, 21 Sep 2023 18:00:00 -0000 https://status.modal.com/incident/262498#54d9b0e917f06698e3dd6f0047d21ff5fa745a29bc07bb6e98871281330517e2 We are investigating an issue in which Mounts are showing degraded performance. Degraded performance in functions https://status.modal.com/incident/262498 Thu, 21 Sep 2023 18:00:00 -0000 https://status.modal.com/incident/262498#54d9b0e917f06698e3dd6f0047d21ff5fa745a29bc07bb6e98871281330517e2 We are investigating an issue in which Mounts are showing degraded performance. Degraded performance in functions https://status.modal.com/incident/262498 Thu, 21 Sep 2023 18:00:00 -0000 https://status.modal.com/incident/262498#54d9b0e917f06698e3dd6f0047d21ff5fa745a29bc07bb6e98871281330517e2 We are investigating an issue in which Mounts are showing degraded performance. CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, Web endpoints, and 2 other services are down https://status.modal.com/incident/256859 Fri, 08 Sep 2023 21:59:40 -0000 https://status.modal.com/incident/256859#18db3a4d881bb3b3c74e0af624b906cfa830171c44a9afbcd76a49fdee9b6be3 Web endpoints recovered. CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, Web endpoints, and 2 other services are down https://status.modal.com/incident/256859 Fri, 08 Sep 2023 21:49:44 -0000 https://status.modal.com/incident/256859#aaa45a4de43a342a38a1cfecc24b03381528c593851d0e0012a44d763f5db5dd CPU functions on AWS recovered. CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, Web endpoints, and 2 other services are down https://status.modal.com/incident/256859 Fri, 08 Sep 2023 21:45:24 -0000 https://status.modal.com/incident/256859#b6c07cab4bc3b89a8d0204c01ffd6d77783b43041f01e2ad12829d62c0a50bd1 CPU functions on AWS went down. CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, Web endpoints, and 2 other services are down https://status.modal.com/incident/256859 Fri, 08 Sep 2023 20:39:57 -0000 https://status.modal.com/incident/256859#99d6b9d48af96276f77c782b1d5527391762cba7d1c2730773a3931128139f1a Web endpoints went down. Web endpoints is down https://status.modal.com/incident/256470 Thu, 07 Sep 2023 21:43:55 -0000 https://status.modal.com/incident/256470#f64b5f5d77c18a1256acd26ba9beb3846b6f4ff381aaa6ce32ab981f51542fb4 Web endpoints recovered. Web endpoints is down https://status.modal.com/incident/256470 Thu, 07 Sep 2023 21:37:41 -0000 https://status.modal.com/incident/256470#9ccaafa244cbf7d62e3314042871b1851d3ea0fd6f18cfcd28be0224fbbee495 Web endpoints went down. CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, GPU functions on AWS, GPU functions on GCP, GPU functions on OCI, and 4 other services are down https://status.modal.com/incident/247847 Sat, 19 Aug 2023 16:24:55 -0000 https://status.modal.com/incident/247847#62b4c46f35ac6d5770ea8868c67ec3ed03d0ecc39a597fcfe1a65b7fb188cea6 CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, and 3 other resources recovered. CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, GPU functions on AWS, GPU functions on GCP, GPU functions on OCI, and 4 other services are down https://status.modal.com/incident/247847 Sat, 19 Aug 2023 16:24:55 -0000 https://status.modal.com/incident/247847#62b4c46f35ac6d5770ea8868c67ec3ed03d0ecc39a597fcfe1a65b7fb188cea6 CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, and 3 other resources recovered. CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, GPU functions on AWS, GPU functions on GCP, GPU functions on OCI, and 4 other services are down https://status.modal.com/incident/247847 Sat, 19 Aug 2023 09:32:38 -0000 https://status.modal.com/incident/247847#a0a19b8ae74986f5f50cac9ba12f75c879b40a0b8b7be2565e321a033a8cfc9f CPU functions on AWS went down. CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, GPU functions on AWS, GPU functions on GCP, GPU functions on OCI, and 4 other services are down https://status.modal.com/incident/247847 Sat, 19 Aug 2023 09:26:35 -0000 https://status.modal.com/incident/247847#4ba4743ad067007b5acbc44c6b81c771b815d19b3b235b0b2896c229a59e2d04 CPU functions on AWS recovered. CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, GPU functions on AWS, GPU functions on GCP, GPU functions on OCI, and 4 other services are down https://status.modal.com/incident/247847 Sat, 19 Aug 2023 09:26:34 -0000 https://status.modal.com/incident/247847#d988fed53deee8f1deb48e22b520f1d7a781bbc10eaeb79685f34f3faa60bf41 GPU functions on AWS, GPU functions on GCP, and GPU functions on OCI went down. CPU functions on AWS, CPU functions on GCP, CPU functions on OCI, GPU functions on AWS, GPU functions on GCP, GPU functions on OCI, and 4 other services are down https://status.modal.com/incident/247847 Sat, 19 Aug 2023 09:14:25 -0000 https://status.modal.com/incident/247847#e5bbce309d8f3c6e76f7565eba7ff3ba1de94fdc41ec159d72ff0375a9d0ea24 CPU functions on AWS, CPU functions on GCP, and CPU functions on OCI went down. DNS issues on Oracle Cloud https://status.modal.com/incident/244486 Fri, 11 Aug 2023 09:17:00 -0000 https://status.modal.com/incident/244486#e022bd155c1c9264cdbb99ecbbfdeb1ada1435857c6fcbd1eb0e08a5c7237d87 We have identified and resolved the root cause of the issue. Function execution works as normal on Oracle Cloud. We will continue actively monitoring. DNS issues on Oracle Cloud https://status.modal.com/incident/244486 Fri, 11 Aug 2023 09:11:00 -0000 https://status.modal.com/incident/244486#314eb356dae320d22fcf42fe5f0579ef52c3d55e3281ed80fb7c0b5574f816c0 We are investigating internal DNS resolution issues on Oracle Cloud. Tasks running on other cloud providers are not affected. Network maintenance https://status.modal.com/incident/243415 Thu, 10 Aug 2023 14:00:00 +0000 https://status.modal.com/incident/243415#4e8df5c9fca8e89e8deaf6c79877e90ed6ec49ed3ef80bf830b0d353580bc2ca Maintenance completed Network maintenance https://status.modal.com/incident/243415 Thu, 10 Aug 2023 13:00:00 -0000 https://status.modal.com/incident/243415#32ee7eb65b13691f87e35498042d6d54eb069358f5406ecb6dc105b402605c7d We are removing resources that are no longer needed. Webhooks and Function scheduling and execution are down https://status.modal.com/incident/240043 Tue, 01 Aug 2023 09:22:16 -0000 https://status.modal.com/incident/240043#3d69a2c3f26c64ac2e8d2e81e3e9d698f7cb47b332e93edf1f7f239f1170232b Function scheduling and execution recovered. Webhooks and Function scheduling and execution are down https://status.modal.com/incident/240043 Tue, 01 Aug 2023 09:18:51 -0000 https://status.modal.com/incident/240043#e726b09ec576bf88ce6f52a40ed97ebb9dcf67a16b1bf07e188ecea720544186 Function scheduling and execution went down. Network maintenance https://status.modal.com/incident/238669 Fri, 28 Jul 2023 20:00:00 +0000 https://status.modal.com/incident/238669#903f64b1cfdf1ddcba938c2102ec2820fb7a24a43d0e3128b9589fbdd005db84 Maintenance completed Network maintenance https://status.modal.com/incident/238669 Fri, 28 Jul 2023 19:00:00 -0000 https://status.modal.com/incident/238669#656333f752961e75663b3fb19a1912feebf2cbb081424b94197e3c1b6b1559e9 We don't expect downtime and only minimal API instability, but we're making some performance related network changes and are declaring this window out of caution. Function scheduling and execution is down https://status.modal.com/incident/227550 Sun, 02 Jul 2023 23:10:31 -0000 https://status.modal.com/incident/227550#9ab7c50631d9c8789ce6f43d4ab828b4a3742f35ea92a94435500d848c002890 Function scheduling and execution recovered. Function scheduling and execution is down https://status.modal.com/incident/227550 Sun, 02 Jul 2023 23:09:27 -0000 https://status.modal.com/incident/227550#68179c4b6d5aa7b70672b5648f6e100045b026cbbb97b981e8aa792f22cabb40 Function scheduling and execution went down. Webhooks and Function scheduling and execution are down https://status.modal.com/incident/225937 Wed, 28 Jun 2023 22:09:18 -0000 https://status.modal.com/incident/225937#f5923a320c13828e40599e2bebc9ca1345f109a596282c2dcd266a5344c6b50c Function scheduling and execution recovered. Webhooks and Function scheduling and execution are down https://status.modal.com/incident/225937 Wed, 28 Jun 2023 22:06:25 -0000 https://status.modal.com/incident/225937#3b3bdcc22c0588149306c593c3f36703ddb984a679b40964216b9a32e410c3b4 Function scheduling and execution went down. Upgrading databases https://status.modal.com/incident/221163 Tue, 20 Jun 2023 19:00:00 +0000 https://status.modal.com/incident/221163#1e468b74a03c5927cc1e101dfa15df5cb97722e744b324356d1f7ffb8ea3c0c3 Maintenance completed Upgrading databases https://status.modal.com/incident/221163 Tue, 20 Jun 2023 15:00:00 -0000 https://status.modal.com/incident/221163#ad579e4cd76416224f3c7070d7d4f48f30950d27418a7d82199d6073e9f20b38 We'll be conducting a security upgrade to our databases. We expect minimal API instability. Webhooks and Function scheduling and execution are down https://status.modal.com/incident/222298 Mon, 19 Jun 2023 20:11:50 -0000 https://status.modal.com/incident/222298#8f117d8849616c6edc0a37e744a48c4e281a8218f66c6f5fcc2087588cc7db8d Function scheduling and execution recovered. Webhooks and Function scheduling and execution are down https://status.modal.com/incident/222298 Mon, 19 Jun 2023 20:02:40 -0000 https://status.modal.com/incident/222298#fb5dd029bbf44e30b726f7a144c066d47ef89f3b9aabf0c6a98d7f3ba605a407 Function scheduling and execution went down. Service degraded (upstream AWS outage) https://status.modal.com/incident/219776 Tue, 13 Jun 2023 20:58:00 -0000 https://status.modal.com/incident/219776#6ab273ee26fe09e5c6773ebda8902ac670e9a23d7c944098dd495f56b31a2b17 The upstream AWS issues have subsided, and we're no longer receiving errors from AWS STS. Service degraded (upstream AWS outage) https://status.modal.com/incident/219776 Tue, 13 Jun 2023 20:31:00 -0000 https://status.modal.com/incident/219776#849c2bf5de1e69308bd9b31b08667945003dc3fdbbd0ad85a2edb033d89b09a9 Modal service health is degraded due to [upstream AWS outage](https://health.aws.amazon.com/health/status). Some API requests and function calls will return 500. We're actively working on finding a workaround. Database upgrade https://status.modal.com/incident/217635 Fri, 09 Jun 2023 17:00:00 +0000 https://status.modal.com/incident/217635#e8f07f6525f08b21eed6aed957199a2ae42ce2714b8b7b94f1d257c32be81364 Maintenance completed Database upgrade https://status.modal.com/incident/217635 Fri, 09 Jun 2023 15:00:00 -0000 https://status.modal.com/incident/217635#b940a7e883e0dd9259a059237f56a482bffbdd8119115f1ddb13b451f335956e We are upgrading one of our databases to improve performance and security. Updating networking configuration https://status.modal.com/incident/211576 Thu, 25 May 2023 18:00:00 +0000 https://status.modal.com/incident/211576#6da95ceaa9bb78f963c43c51867cbcf00d48d9ae0aa94093744fe5a46b0ea0b8 Maintenance completed Updating networking configuration https://status.modal.com/incident/211576 Thu, 25 May 2023 17:00:00 -0000 https://status.modal.com/incident/211576#08046c5818e9bb95b8733db08e2ebc5f62f86960aed081d4419f84cf4809854e We'll conduct security updates to our networking stack. Upgrading network stack https://status.modal.com/incident/208165 Wed, 17 May 2023 15:00:00 +0000 https://status.modal.com/incident/208165#b2d0e44096fbd54df37078205187303750f6858cb47f6e27207289b7a0accadf Maintenance completed Upgrading network stack https://status.modal.com/incident/208165 Wed, 17 May 2023 14:30:00 -0000 https://status.modal.com/incident/208165#3fc00c3c625e2f496857cffa0e5c90264bf4e532c8345e34e735f043164c8bd2 An upgrade to our networking stack may cause slight API instability and slowness fora few minutes, but no downtime is expected. Updates logging system and main database https://status.modal.com/incident/197659 Thu, 20 Apr 2023 08:00:00 +0000 https://status.modal.com/incident/197659#f21d8bc1f1ad4ee84c2dba2f492892aa0b6b0ea01ed118a239a77fbb9bf64373 Maintenance completed Webhooks and API, Webhooks, Function scheduling and execution, and 1 other service are down https://status.modal.com/incident/198331 Thu, 20 Apr 2023 07:33:51 -0000 https://status.modal.com/incident/198331#3cd1408e63603448e3900f56fae240b15595834b0dd89e3fffced696a57969da Function scheduling and execution recovered. Webhooks and API, Webhooks, Function scheduling and execution, and 1 other service are down https://status.modal.com/incident/198331 Thu, 20 Apr 2023 07:21:18 -0000 https://status.modal.com/incident/198331#b745f38d86fcdf5357b71e4174d93570cd7d9573354ef0ab07b0f00cfdec8666 Webhooks and API and Function scheduling and execution went down. Updates logging system and main database https://status.modal.com/incident/197659 Thu, 20 Apr 2023 00:00:00 -0000 https://status.modal.com/incident/197659#1744f399d508356f7b646b96d761108bc6acf089b8191613f99572c520ae7414 We will be updating our logging backend to encrypt all logging cache at rest. We will also be updating our main application database to improve reliability and performance. Function scheduling and execution is down https://status.modal.com/incident/196093 Fri, 14 Apr 2023 04:11:52 -0000 https://status.modal.com/incident/196093#070a3cd1cea4a934f3b5b1dbadf3effedd35c480c3632bb5426067914341ae2d Function scheduling and execution recovered. Function scheduling and execution is down https://status.modal.com/incident/196093 Fri, 14 Apr 2023 03:59:19 -0000 https://status.modal.com/incident/196093#269fb4ede1155470822e7df2b38ab1b8e0cc9f662185f61c9d4320c1dc9077de Function scheduling and execution went down. Function scheduling and execution is down https://status.modal.com/incident/195602 Wed, 12 Apr 2023 22:28:40 -0000 https://status.modal.com/incident/195602#c33dfc376c15a9ebf4e084b21c2667858de38742e302c869e9ab504dbe85d332 Function scheduling and execution recovered. Function scheduling and execution is down https://status.modal.com/incident/195602 Wed, 12 Apr 2023 22:25:47 -0000 https://status.modal.com/incident/195602#77e8834f03f4be70d7966534afe5fbea10c04b718a39bc5acc6b939e2ac4aa8e Function scheduling and execution went down. Function scheduling and execution is down https://status.modal.com/incident/195602 Wed, 12 Apr 2023 22:24:34 -0000 https://status.modal.com/incident/195602#748acf28c35f55fa7759426d6db9be67b5be0af280bd621bb72cdda7d0ccdead Function scheduling and execution recovered. Function scheduling and execution is down https://status.modal.com/incident/195602 Wed, 12 Apr 2023 22:17:42 -0000 https://status.modal.com/incident/195602#b930be2cfbade7800c9a9219dd5b7a2a8e23b98a46866174cb6f3bc5f493f42f Function scheduling and execution went down. Webhooks and Function scheduling and execution are down https://status.modal.com/incident/193278 Wed, 05 Apr 2023 20:26:45 -0000 https://status.modal.com/incident/193278#3a7f144b46b1e93c726663849120d22540a19b175d065b655e460fcd30d9a709 Function scheduling and execution recovered. Webhooks and Function scheduling and execution are down https://status.modal.com/incident/193278 Wed, 05 Apr 2023 20:25:11 -0000 https://status.modal.com/incident/193278#2845f8f3df962183753b74707e2529f816dc2725b232ebd7b23a89231b88e02c Function scheduling and execution went down. Upgrading file-system encryption https://status.modal.com/incident/188809 Sun, 26 Mar 2023 18:00:00 +0000 https://status.modal.com/incident/188809#b9f9fc8463a5ce53416925c8c04ab44b86072326b285139b6fbb965d5520cc20 Maintenance completed Upgrading file-system encryption https://status.modal.com/incident/188809 Sun, 26 Mar 2023 14:00:00 -0000 https://status.modal.com/incident/188809#9f62999526d5dafe319eb548ce4334443fd4808b6aa09918b02837f60339837a We are upgrading file-system encryption in our API servers to more robust encryption at-rest. Function scheduling and execution is down https://status.modal.com/incident/187341 Tue, 21 Mar 2023 07:09:45 -0000 https://status.modal.com/incident/187341#f53d4a914154f06c8c0e6fbf87234114725dd8cd9be49308726b0a892d8afc4b Function scheduling and execution recovered. Function scheduling and execution is down https://status.modal.com/incident/187341 Tue, 21 Mar 2023 07:07:27 -0000 https://status.modal.com/incident/187341#1e948a0f77eb15aee64e7a392cf1ec64b0c708ae419aa4958eaa540a1e599bc6 Function scheduling and execution went down. Webhooks and Function scheduling and execution are down https://status.modal.com/incident/186371 Sat, 18 Mar 2023 09:22:10 -0000 https://status.modal.com/incident/186371#ea3b03bf47da644dc39e1bd27e190165b62fd22b654fa8a8d6ffd24cb6e626fe Function scheduling and execution recovered. Webhooks and Function scheduling and execution are down https://status.modal.com/incident/186371 Sat, 18 Mar 2023 09:21:11 -0000 https://status.modal.com/incident/186371#108b1fd29ffa92ed31d4c406cba3b8ba97cd362c5fe32c37b72fb14201f198d4 Function scheduling and execution went down. Webhooks and API, Webhooks, Function scheduling and execution, and 1 other service are down https://status.modal.com/incident/186218 Fri, 17 Mar 2023 22:48:13 -0000 https://status.modal.com/incident/186218#e93345c14f9db66190e3e7c9c720c3e23283ab6720d47279bfe1c79df07057c3 Function scheduling and execution recovered. Webhooks and API, Webhooks, Function scheduling and execution, and 1 other service are down https://status.modal.com/incident/186218 Fri, 17 Mar 2023 22:43:20 -0000 https://status.modal.com/incident/186218#bd9aa4eb59d1d4aebed6ff15709420516d205bb3e4082a7b118e2dda46e2e03e Function scheduling and execution went down.