Skip to main content

Troubleshooting Guide: 504 Gateway Timeout Errors

Distributor troubleshooting for a 504 Gateway Timeout error in Encompass, often from heavy reports or exports: retry, cancel long-running queries, check proxy timeouts. Requires v26.01+.

B
Written by Bailey Naas-Thornton

Minimum Version Required: 26.01

Pre-Conditions

  • Requires an active internet connection.

  • Requires administrative access to your network routing, load balancer, or reverse proxy settings (if self-hosting or managing your own network infrastructure).

  • Access to application, database, and proxy log files.

Steps

  • Retry the Request: Wait a few moments and refresh the page or retry the API call. Transient network degradation or temporary spikes in downstream API latency often resolve themselves quickly.

  • Review Active Database Queries: Check for long-running SQL executions or database lock waits. If a massive report or complex data export is tying up the database, cancel the operation to free up the thread pool.

  • Check Timeout Thresholds: Inspect the configurations on your intermediate proxy layers (e.g., Nginx, Apache, load balancers, or Cloudflare). Ensure that the timeout thresholds are not configured too low for operations that are expected to take a longer time to process.

  • Monitor Server Resources: Evaluate your application servers for thread pool saturation, thread starvation, or request queueing backlogs. Consider scaling up resources or restarting the application service to clear overloaded queues.

  • Contact Support: If the issue persists after attempting the above steps, please contact support immediately. To ensure efficient troubleshooting, please gather and provide the following information:

    • Exact error messages or codes encountered (including the URL or API endpoint returning the 504 error).

    • Screenshots or video recordings of the issue.

    • Log files (specifically reverse proxy logs, application error logs, and slow query database logs).

    • The exact steps taken by the user that led to the issue (e.g., "I clicked 'Generate Report' on the Sales dashboard").

    • The date and time the issue was first observed, your local time zone, and any patterns of recurrence.

    • Any changes made to the system, network architecture, or application code before the issue appeared.

Fields And Definitions

  • 504 Gateway Timeout: An HTTP status code indicating that a server acting as a gateway or reverse proxy did not receive a timely response from an upstream server it needed to access to complete the request.

  • Reverse Proxy: A server that sits in front of web servers and forwards client requests to those backend servers.

  • Upstream Server: The backend server (like an application or database server) that the proxy is attempting to communicate with.

  • SQL Execution Block / Lock Wait: A scenario where a database query is stalled because the data it needs to read or write is currently locked by another active transaction.

  • Thread Pool Saturation: A condition where all available threads allocated to process incoming requests are currently in use, forcing new requests to wait in a queue or time out.

  • Downstream API Latency: Delays caused by third-party services or external systems that your application relies on to formulate a response.

Best Practices

  • Optimize Heavy Queries: Ensure database tables are properly indexed and optimize heavy SQL queries so they execute efficiently without causing lock waits.

  • Asynchronous Processing: Move long-running tasks (like heavy report generation or massive data imports) to background processing queues rather than tying up HTTP request threads.

  • Tune Proxy Configurations: Align the timeout settings of your intermediate proxy layers with the actual time required for your most complex application tasks to complete.

  • Implement Monitoring: Set up automated monitoring and alerting for thread pool usage, API latency, and database query times so you can identify and resolve saturation before it results in 504 errors.

Did this answer your question?