<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><link rel="alternate" type="text/html" href="https://status.jumbo.live/"/><title>Issues on Jumbo Status</title><link>https://status.jumbo.live/issues/</link><description>Incident history</description><generator>github.com/cstate</generator><language>en</language><lastBuildDate>2026-04-21T20:34:26+00:00</lastBuildDate><updated>2026-04-21T20:34:26+00:00</updated><atom:link href="https://status.jumbo.live/issues/index.xml" rel="self" type="application/rss+xml"/><item><title>[Resolved] Event site availability issues</title><link>https://status.jumbo.live/issues/2026-04-21-event-sites-gateway-timeouts/</link><pubDate>Tue, 21 Apr 2026 20:34:26 +0000</pubDate><guid>https://status.jumbo.live/issues/2026-04-21-event-sites-gateway-timeouts/</guid><category>2026-04-21 21:14:34</category><description>&lt;!-- Body copy pending final narrative from Justin. Primary outage
window per UptimeRobot export for opusadvisors.events:
20:34:26 → 21:14:34 UTC = 40m 8s of sustained 522 Gateway
Timeouts. Shorter recurrences were observed in the following
hours as capacity re-balanced. --&gt;
&lt;p&gt;&lt;em&gt;Resolved&lt;/em&gt; - Event sites returning to full availability. Post-incident review underway.
&lt;span class="faded"&gt;(Apr 21, 2026)&lt;/span&gt;
&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Monitoring&lt;/em&gt; - Mitigation applied. A small number of short timeouts continue to be observed as origin capacity re-balances.
&lt;span class="faded"&gt;(Apr 21, 2026)&lt;/span&gt;
&lt;/p&gt;</description><content type="html">&lt;!-- Body copy pending final narrative from Justin. Primary outage
window per UptimeRobot export for opusadvisors.events:
20:34:26 → 21:14:34 UTC = 40m 8s of sustained 522 Gateway
Timeouts. Shorter recurrences were observed in the following
hours as capacity re-balanced. --&gt;
&lt;p&gt;&lt;em&gt;Resolved&lt;/em&gt; - Event sites returning to full availability. Post-incident review underway.
&lt;span class="faded"&gt;(Apr 21, 2026)&lt;/span&gt;
&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Monitoring&lt;/em&gt; - Mitigation applied. A small number of short timeouts continue to be observed as origin capacity re-balances.
&lt;span class="faded"&gt;(Apr 21, 2026)&lt;/span&gt;
&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Investigating&lt;/em&gt; - Event sites are intermittently returning gateway timeouts for visitors. Registration, channels, and Zoom integration remain unaffected on healthy routes.
&lt;span class="faded"&gt;(Apr 21, 2026)&lt;/span&gt;
&lt;/p&gt;
&lt;div class="jumbo-rca" markdown="1"&gt;
&lt;h2 id="post-incident-report"&gt;Post-incident report&lt;/h2&gt;
&lt;p&gt;&lt;em&gt;Published
&lt;span class="faded"&gt;(Apr 22, 2026)&lt;/span&gt;
&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;At 8:34 PM UTC on April 21, 2026, we were notified of a critical outage affecting event sites hosted in our Google Cloud us-central1 (Iowa) region. Our on-call team immediately engaged senior engineering, who remoted into the origin infrastructure, rebooted the affected server, and restored accessibility to event sites.&lt;/p&gt;
&lt;p&gt;The primary outage window lasted &lt;strong&gt;40 minutes&lt;/strong&gt;. Shorter recurrences were observed in the following hours as origin capacity re-balanced, and were resolved without further intervention. Registration, channels, and Zoom integration continued to function on healthy routes throughout.&lt;/p&gt;
&lt;p&gt;This was classified as a &lt;strong&gt;tier-3 incident&lt;/strong&gt;. Mitigations were deployed on affected sites to minimize service disruption during the recovery window.&lt;/p&gt;
&lt;p&gt;As this is one of the longest and broadest service disruptions in Jumbo’s history, we are reviewing our backup and failover planning to ensure faster recovery for future events of this nature.&lt;/p&gt;
&lt;/div&gt;</content></item><item><title>[Resolved] Registration errors on event sites</title><link>https://status.jumbo.live/issues/2025-11-18-registration-errors/</link><pubDate>Tue, 18 Nov 2025 13:00:32 +0000</pubDate><guid>https://status.jumbo.live/issues/2025-11-18-registration-errors/</guid><category>2025-11-18 14:36:31</category><description>&lt;!-- Body copy pending final narrative from Justin. Primary outage
window per UptimeRobot export for opusadvisors.events:
13:00:32 → 14:36:31 UTC = 1h 35m 59s of sustained 500 errors.
Three shorter precursor windows occurred earlier in the day
(11:35–11:53, 12:26–12:31, 12:40–12:50) and are covered by
the "Investigating" update below. --&gt;
&lt;p&gt;&lt;em&gt;Resolved&lt;/em&gt; - Registration service fully recovered. Monitoring confirms normal operation.
&lt;span class="faded"&gt;(Nov 18, 2025)&lt;/span&gt;
&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Monitoring&lt;/em&gt; - Mitigation applied. Watching for recurrence.
&lt;span class="faded"&gt;(Nov 18, 2025)&lt;/span&gt;
&lt;/p&gt;</description><content type="html">&lt;!-- Body copy pending final narrative from Justin. Primary outage
window per UptimeRobot export for opusadvisors.events:
13:00:32 → 14:36:31 UTC = 1h 35m 59s of sustained 500 errors.
Three shorter precursor windows occurred earlier in the day
(11:35–11:53, 12:26–12:31, 12:40–12:50) and are covered by
the "Investigating" update below. --&gt;
&lt;p&gt;&lt;em&gt;Resolved&lt;/em&gt; - Registration service fully recovered. Monitoring confirms normal operation.
&lt;span class="faded"&gt;(Nov 18, 2025)&lt;/span&gt;
&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Monitoring&lt;/em&gt; - Mitigation applied. Watching for recurrence.
&lt;span class="faded"&gt;(Nov 18, 2025)&lt;/span&gt;
&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Investigating&lt;/em&gt; - Investigating elevated 500 errors affecting registration flows on event sites. Earlier this morning there were three shorter windows of the same behavior; this most recent window is sustained. Session, streaming, and attendee-facing content are unaffected.
&lt;span class="faded"&gt;(Nov 18, 2025)&lt;/span&gt;
&lt;/p&gt;
&lt;div class="jumbo-rca" markdown="1"&gt;
&lt;h2 id="post-incident-report"&gt;Post-incident report&lt;/h2&gt;
&lt;p&gt;&lt;em&gt;Published
&lt;span class="faded"&gt;(Nov 19, 2025)&lt;/span&gt;
&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;On November 18, 2025, a subset of event sites running on our managed hosting provider’s legacy network experienced an issue in which automated cron jobs stopped executing. This disrupted registration flows on the affected sites, producing intermittent errors for attendees attempting normally functioning activities such as completing registration or payment. The longest sustained window lasted approximately 1 hour 36 minutes.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Sites on the Cloudflare Advanced Network were not affected.&lt;/strong&gt; Only sites still running on the provider’s legacy network were impacted. Session playback, channels, and other attendee-facing surfaces continued to operate normally throughout.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;No data loss was detected.&lt;/strong&gt; Backup registration capture kept all submitted attendee records intact and they were reconciled into the primary registration system once service was restored.&lt;/p&gt;
&lt;/div&gt;</content></item></channel></rss>