Open Closed

redis commands timeout #5847


User avatar
0
portx-dev created

Hi, we have redis commands that timeout. I asked that problem in StackExchange/StackExchange.Redis, and got the answer.

It looks like you're experiencing thread exhaustion (WORKER: (Busy=38,Free=32729,Min=8,Max=32767)), likely from long-running synchronous operations. I can't say about the rest of your application, but you for sure have some synchronous Redis calls (sync-ops: 7262, async-ops: 28127). You'll want to do all those things async to have much better utilization of the thread pool and less blocking/stalls as a result of thread exhaustion (waiting for a new one to become available).

reference: https://github.com/StackExchange/StackExchange.Redis/issues/2558

According to the analysis, what we can do for solving timeout issue in ABP Framework?


ABP Framework version: Commercial 7.2 UI type: Angular DB provider: EF Core


5 Answer(s)
  • User Avatar
    0
    Anjali_Musmade created
    Support Team Support Team Member

    Hello https://support.aspnetzero.com/QA/Questions/6559/StackExchangeRedisRedisTimeoutException-Timeout#answer-b6aac0d5-f5e1-145e-cf82-39ecb0563c4d Have you tried this syncTimeout

  • User Avatar
    0
    portx-dev created

    Thanks for sharing that. But increasing the timeout is not fundamental solution. The problem is slow commands. I doubt many redis commands dispatched at the same time affect to bad performance. One of the requests dispatched1456 redis commands. Is that normal?

  • User Avatar
    0
    Anjali_Musmade created
    Support Team Support Team Member

    Hello portx-dev,

    Actually this is a performance issue try to optimize your queries and code. For Worker Busy Thread < Min Thread

    I am providing some workaround please try this - You need to change your Min Thread for that please refer this https://learn.microsoft.com/en-us/dotnet/core/runtime-config/threading or try to add this code in Program.cs of Host

    System.Threading.ThreadPool.SetMinThreads(200, 200);

    I have given here Min Thread value as 200 you can change it as per your requirement

    Please do let me know if this helps you

    Thank you, Anjali

  • User Avatar
    0
    portx-dev created

    Thanks for providing solution. The performance was improved by increasing min threads.

    On the other side, there are still requests that contain many redis command. I think 200~ commands in a request is irregular number, isn't it?

  • User Avatar
    0
    Anjali_Musmade created
    Support Team Support Team Member

    Hello portx-dev,

    In one of our similar deployed application we are using Min Thread = 1000 and its working fine.

    Regards,

Made with ❤️ on ABP v8.2.0-preview Updated on March 25, 2024, 15:11