Open Closed

Background worker won't start #6331


User avatar
0
marco@iwell.nl created
  • ABP Framework version: v5.2
  • UI Type: Angular
  • Database System: EF Core (SQL Server)
  • Tiered (for MVC) or Auth Server Separated (for Angular): angular..
  • Exception message and full stack trace: none
  • What am I trying to achieve: I need a background worker process that runs once and keeps running (polling an Azure queue and processing messages).
  • Steps to reproduce the issue:

Setup

We're running two instances of our backend:

  1. The 'default' ABP project that provides the REST API for our frontend (Angular) and to provide an interface to our customers.
  2. Another instance that only runs periodic tasks. (implementations of AsyncPeriodicBackgroundWorkerBase) This instance is stripped off running any web middle ware or providing the REST endpoints.

These instances run in different Azure App Services, so they are not aware of each other. It's NOT a clustered environment.

Goal

In the 'worker' instance I need a new BackgroundWorker that runs once and keeps running until the application stops. Basically: while (!stoppingToken.IsCancellationRequested) { ... }

If I understand the documentation correct, that can be achieved by implementing the BackgroundWorkerBase class like this:

This works fine when I register the Worker on the rest API instance on the HttpApiHostModule.cs like this: await context.AddBackgroundWorkerAsync<IotMessageWorker>();

When I debug the restAPI instance of my application, all is okay. The debugger stops on the worker class. However, when I register my worker in the worker instance, the debugger won't hit any breakpoints. (besides a constructor, so the class is loaded)

Troubleshooting and questions

Clearly, before writing this post, I've been trying to get this to work myself. No cigar though.

AsyncPeriodicBackgroundWorkerBase We already run this setup on production for our current background workers. The commonality between these, is they are all implementations of the AsyncPeriodicBackgroundWorkerBase. These run just fine. Which dazzles me, because this is just another implementation of the BackgroundWorkerBase.

Dependencies or services My guess is, that I need another dependency in my WorkerModule. I tried a couple of obvious ones, but that doesn't seem to help. Also an option is that I miss some service registration in the DI container. I tried to match the things with the restAPI instance, but that didn't help either.

Clearly we probably stripped too much off on our worker instance, but I can not figure out what is missing. That's also my question: What makes the BackgroundWorkerBase run (and why is that different from the AsyncPeriodBackgroundWorkerBase)?

implementation of the WorkerService PortalWorkerModule

Entrypoint of Program.cs

Startup class of the WorkerService:

I hope anyone can give me a pointer on how to achieve this the correct and supported way :)


3 Answer(s)
  • User Avatar
    0
    liangshiwei created
    Support Team Fullstack Developer

    Hi,

    I noticed your version is 5.2 and you are using the Hangfire provider.

    We enhanced it in 6.0: https://github.com/abpframework/abp/pull/14608

    you can override the HangfireBackgroundWorkerManager

    [Dependency(ReplaceServices = true)]
    [ExposeServices(typeof(IBackgroundWorkerManager)]
    public class MyHangfireBackgroundWorkerManager : HangfireBackgroundWorkerManager
    {
        private bool _isDisposed;
        private readonly List<IBackgroundWorker> _backgroundWorkers;
        
        public async override Task StartAsync(CancellationToken cancellationToken = default)
        {
            await base.StartAsync(cancellationToken);
            await StartWorkersAsync(cancellationToken);
        }
        
        public async override Task AddAsync(IBackgroundWorker worker, CancellationToken cancellationToken = default)
        {
            switch (worker)
            {
                case IHangfireBackgroundWorker hangfireBackgroundWorker:
                case AsyncPeriodicBackgroundWorkerBase or PeriodicBackgroundWorkerBase:
                     await base.AddAsync(worker, cancellationToken);
                     break;
                default:
                    _backgroundWorkers.Add(worker);
    
                    if (IsRunning)
                    {
                        await worker.StartAsync(cancellationToken);
                    }
                    break;   
            }
        }
        
        private async override Task StartWorkersAsync(CancellationToken cancellationToken = default)
        {
            IsRunning = true;
    
            foreach (var worker in _backgroundWorkers)
            {
                await worker.StartAsync(cancellationToken);
            }
        }
        
        public override async Task StopAsync(CancellationToken cancellationToken = default)
        {
            IsRunning = false;
    
            foreach (var worker in _backgroundWorkers)
            {
                await worker.StopAsync(cancellationToken);
            }
        }
    }
    
  • User Avatar
    0
    marco@iwell.nl created

    Ahh thank you, that might be something I can work with. Coworkers of mine are actually right in the middle of upgrading to version 7.4 which I expect to be finished any time soon!

    Still pretty strange to me that the BackgroundWorker doesn't run and the PeriodicBackgroundWorkerBase does run in our current situation. If you can shed any light on this, that would be great!

    Right now, to continue the feature, I'm implementing the periodic version. Since it's a singleton I've implemented the 'isRunning' solution so we actually only have it running once. And also enables some kind of auto-recovery option in case the process crashes.

  • User Avatar
    0
    liangshiwei created
    Support Team Fullstack Developer

    Ahh thank you, that might be something I can work with. Coworkers of mine are actually right in the middle of upgrading to version 7.4 which I expect to be finished any time soon!

    Ok, But there's a bug in Hangfire and fixed today, we haven't released the patch version:https://github.com/abpframework/abp/issues/18477 Your worker may execute twice(Only workers inherited from BackgroundWorkerBase)

    You can avoid it in the following way:

    public class MyWorker : BackgroundWorkerBase
    {
        private bool IsRunning { get; set; }
        public override Task StartAsync(CancellationToken cancellationToken = default)
        {
            if(IsRunning)
            {
               return...
            }
            
            IsRunning = true;
            //...
        }
    
        public override Task StopAsync(CancellationToken cancellationToken = default)
        {
            //...
        }
    }
    

    Still pretty strange to me that the BackgroundWorker doesn't run and the PeriodicBackgroundWorkerBase does run in our current situation. If you can shed any light on this, that would be great!

    Because Hangfire did not support BackgroundWorkerBase before, It will be ignored instead of added to the worker list.

Made with ❤️ on ABP v8.2.0-preview Updated on March 25, 2024, 15:11