Bot - Outage

Incident Report for Ticket Tool

Postmortem

After an hour of troubleshooting and debugging the issue was located and found to be a hardware failure, A short window of time where I attempted to resolve the issue myself failed and I moved into resolving the issue by spinning up a new server to host the API & Redis cache that the dead server was responsible for.

After another hour I managed to get it all setup, DNS updated and IP addresses pointing to the new server we had the api back online.. about 10 minutes after that I had the premium bot back online..

We monitored the bot for about 15 minutes to make sure the migration didn’t have any unforeseen issues and proceeded to spin up the primary bot.

It all was solid and we’re back online.

Posted Sep 14, 2026 - 10:37 CDT

Resolved

This incident has been resolved.

Posted Sep 14, 2026 - 09:45 CDT

Update

Primary bot is coming back online

Posted Sep 14, 2026 - 09:21 CDT

Monitoring

The server has been replaced, we have started up the premium bot and are monitoring it for stability issues on the new server.. The primary bot will follow soon.

Posted Sep 14, 2026 - 09:04 CDT

Identified

We have located the issue as a hardware failure and are looking to get this fixed or replaced.

Posted Sep 14, 2026 - 08:25 CDT

Investigating

At this time Ticket Tool's Bot is experiencing outages in all servers. The Development Team has been alerted and is looking into it.

Posted Sep 14, 2026 - 07:31 CDT

This incident affected: Primary Bot, Premium Bot, API & Database, and Dashboard.