Yellow - Performance Degradation

Incident Report for PEAK 15 Systems

Postmortem

Summary

Following routine overnight system maintenance, one of our web servers did not return to a fully healthy application state. This caused embedded iframe content within the application to fail to load for some users.

The issue was isolated to a single server in our web tier. Once identified, we temporarily removed the server from service, restored normal operation, and verified full functionality.

All systems are now operating normally.

What Happened

As part of standard maintenance, our web servers undergo a scheduled restart to ensure system health. While both servers reported a successful restart, one server did not fully initialize all required web application components.

Because the server appeared online at the infrastructure level, traffic continued to route to it. However, embedded iframe-based components were not being served correctly from that node.

Impact

Users may have experienced:

  • Embedded iframe content not loading within certain application pages

No other application features, data, security, or integrations were affected.

Resolution

Once the issue was detected, our team took the following steps:

  1. Traffic was stopped to the affected web server by removing it from the load-balanced pool
  2. A follow-up IIS service restart was performed on the affected server
  3. Validation testing confirmed that iframe content was loading normally
  4. The server was returned to service and traffic routing was restored

Normal behavior was confirmed across all web servers before closing the incident.

Root Cause

The incident was caused by an incomplete application component initialization following a routine server restart. Although the server and core web services were running, a required component responsible for serving embedded iframe content did not start correctly, leaving the server in a partially functional state.

Our monitoring detected the server as available but did not detect this specific application-layer failure.

Preventive Actions

We are implementing the following improvements:

  • Enhanced post-restart health checks to validate iframe and embedded component functionality
  • Automated application-level validation as part of server startup procedures
  • Improved monitoring alerts to detect partial application failures more quickly

These measures will help us detect similar issues faster and further reduce the chance of user impact.

We apologize for the inconvenience and appreciate your patience. If you continue to experience any issues, please contact our support team.

Posted Feb 06, 2026 - 07:14 PST

Resolved

Earlier today we identified and resolved an issue that affected part of our web application experience.

One of our web servers did not restart correctly following routine overnight maintenance. As a result, some embedded content within the application (such as iframes ) may not have loaded as expected for a portion of users.
Posted Feb 06, 2026 - 07:08 PST