Activities of "kfrancis@clinicalsupportsystems.com"

Working on adding plugin modules, for handling logic specific to clients and I started seeing background jobs stop working suddenly (reverting to NullBackgroundJobManager).

Working on creating a repro repo.

• ABP Framework version: v8.0.2
• UI Type: MVC
• Database System: EF Core (SQL Server)
• Tiered (for MVC) or Auth Server Separated (for Angular): yes

Without changes, commenting out the options.PlugInSources.AddFolder line causes it to start working correctly again (though without the module being registered). Here's the code in which commenting it out makes it work again:

    await builder.AddApplicationAsync<CabMDWebModule>(options =>
    {
        // Hmm, works for now but what we want is a fixed location and if there are plugins, add them.
        var currentDirectory = options.Services.GetHostingEnvironment().ContentRootPath;
        var plugDllInPath = "";
        for (var i = 0; i < 10; i++)
        {
            var parentDirectory = new DirectoryInfo(currentDirectory).Parent;
            if (parentDirectory == null)
            {
                break;
            }
            if (parentDirectory.Name == "src")
            {
#if DEBUG
                plugDllInPath = Path.Combine(parentDirectory.FullName, "CabMD.ClientPlugIn.RMA", "bin", "Debug", "net8.0");
#else
            plugDllInPath = Path.Combine(parentDirectory.FullName, "CabMD.ClientPlugIn.RMA", "bin", "Release", "net8.0");
#endif
                break;
            }
            currentDirectory = parentDirectory.FullName;
        }

        if (Path.Exists(plugDllInPath))
        {
            // delete *.deps.json files before adding the folder as the dep file causes an error
            foreach (var depFile in Directory.GetFiles(plugDllInPath, "*.deps.json"))
            {
                File.Delete(depFile);
            }

            options.PlugInSources.AddFolder(plugDllInPath);
        }
    });

hi

The solution is to set IsServiceBusDisabled = true in HTTP API module.

Yes, that seems to have done the trick. At the same time, for whatever reason, we also had to produce new keys. For some reason the default key that's created when you create the service bus doesn't work while a new key (created to be the same) works. Even scoped keys work (scoped to a topic), they also work.

Also, the "manage" permission is required or this'll be what happens:

Yea, we understand that - that's by design. While we know the background job runs when it runs, the job itself pubs the distributed event - why doesn't the event get received half the time?

Ultimately, this is all just to notify a user when a job completes independent of the page they are on (as reporting jobs can take long) - if there's a better way, we're all ears - but there isn't AFAIK currently and this should work.

Will share with liming.ma@volosoft.com

There's a repro here: https://github.com/victoriatolls/Acme.BookStore.DistributedEvent

Working on using distributed events, though what we're seeing is that not all events are being received as expected (half?):

What you're seeing:

• We're triggering a background job to be run from Web
• That background job is being run within the Api.Host
• When the background job is complete, we publish an event to notify the client (for the purpose of client-side notification)
• In some cases (approximately half?) the message is received and can be dealt with properly (in SignalR hub which then causes client-side handler to ultimately call abp.notify
• In other cases, nothing is occurring.

• ABP Framework version: v7.4.2
• UI Type: MVC
• Database System: EF Core (SQL Server)
• Tiered (for MVC) or Auth Server Separated (for Angular): yes
• Steps to reproduce the issue:

Here's the options:

Configure<AbpAzureServiceBusOptions>(options =>
{
    options.Connections.Default.Admin.Retry.MaxRetries = 3;
    options.Connections.Default.Client.RetryOptions.Delay = TimeSpan.FromMilliseconds(100);
    options.Connections.Default.Client.RetryOptions.MaxDelay = TimeSpan.FromSeconds(10);
    options.Connections.Default.Client.RetryOptions.Mode = Azure.Messaging.ServiceBus.ServiceBusRetryMode.Exponential;
    options.Connections.Default.Client.RetryOptions.MaxRetries = 10;
});

• ABP Framework version: v7.4.2
• UI Type: MVC
• Database System: EF Core (SQL Server)
• Tiered (for MVC) or Auth Server Separated (for Angular): yes, tiered

I'm seeing this logo flash when first loading any page in the app, not sure how to stop the lepton logo from appearing at all.

[Dependency(ReplaceServices = true)]
public class CabMDBrandingProvider : DefaultBrandingProvider
{
    public override string AppName => "CabMD";

    public override string LogoUrl => "/images/logo/logo_blue.png";

    public override string LogoReverseUrl => "/images/logo/logo_white_large.svg";
}

Is this a side effect of how the logo is rendered?

It doesn't seem to matter if I try and override it either based on this documentation: https://docs.abp.io/en/commercial/7.4/themes/lepton-x/mvc#main-header-branding

The content doesn't change and the problem persists (the following isn't rendered, but it doesn't look like that would even help the problem as lpx-brand-logo is being changed after initial render):

<a href="" class="lpx-brand-logo" data-woah="123"></a>

Just using standard localdb for local dev:

"ConnectionStrings": {
  "Default": "Server=(localdb)\\MSSQLLocalDB;Database=ProjectDb;Trusted_Connection=True;TrustServerCertificate=True"
}

Also, if it were a general permissions issue - I would have expected far more to be wrong, but everything works except this for me.

• ABP Framework version: v7.4.1
• UI Type: MVC
• Database System: EF Core (SQL Server)
• Tiered (for MVC) or Auth Server Separated (for Angular): yes
• Exception message and full stack trace:

[2023-11-15 12:42:02.262] [Error] DEV1-PC () <Microsoft.EntityFrameworkCore.Database.Command> Failed executing DbCommand ("3"ms) [Parameters=["@__p_1='?' (DbType = Int32), @__now_0='?' (DbType = DateTime2)"], CommandType='Text', CommandTimeout='30']"\r\n""SELECT TOP(@__p_1) [a].[Id], [a].[ConcurrencyStamp], [a].[CreationTime], [a].[ExtraProperties], [a].[IsAbandoned], [a].[JobArgs], [a].[JobName], [a].[LastTryTime], [a].[NextTryTime], [a].[Priority], [a].[TryCount]\r\nFROM [AbpBackgroundJobs] AS [a]\r\nWHERE [a].[IsAbandoned] = CAST(0 AS bit) AND [a].[NextTryTime] <= @__now_0\r\nORDER BY [a].[Priority] DESC, [a].[TryCount], [a].[NextTryTime]"
[2023-11-15 12:42:02.263] [Error] DEV1-PC () <Microsoft.EntityFrameworkCore.Query> An exception occurred while iterating over the results of a query for context type '"Volo.Abp.BackgroundJobs.EntityFrameworkCore.BackgroundJobsDbContext"'."\r\n""Microsoft.Data.SqlClient.SqlException (0x80131904): Invalid column name 'ConcurrencyStamp'.\r\nInvalid column name 'ExtraProperties'.\r\nInvalid column name 'JobName'.\r\n   at Microsoft.Data.SqlClient.SqlCommand.<>c.<ExecuteDbDataReaderAsync>b__209_0(Task`1 result)\r\n   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()\r\n   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)\r\n--- End of stack trace from previous location ---\r\n   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)\r\n   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)\r\n--- End of stack trace from previous location ---\r\n   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)\r\n   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)\r\n   at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.AsyncEnumerator.InitializeReaderAsync(AsyncEnumerator enumerator, CancellationToken cancellationToken)\r\n   at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken)\r\n   at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()\r\nClientConnectionId:1067e9e4-b969-4ddb-a740-d5b5df69f130\r\nError Number:207,State:1,Class:16"
Microsoft.Data.SqlClient.SqlException (0x80131904): Invalid column name 'ConcurrencyStamp'.
Invalid column name 'ExtraProperties'.
Invalid column name 'JobName'.
   at Microsoft.Data.SqlClient.SqlCommand.<>c.<ExecuteDbDataReaderAsync>b__209_0(Task`1 result)
   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.AsyncEnumerator.InitializeReaderAsync(AsyncEnumerator enumerator, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()

From what I can tell, the columns it's complaining about do exist and there are no pending migrations.

This is what's in the model snapshot as well, which also seems correct:

modelBuilder.Entity("Volo.Abp.BackgroundJobs.BackgroundJobRecord", b =>
    {
        b.Property<Guid>("Id")
            .ValueGeneratedOnAdd()
            .HasColumnType("uniqueidentifier");

        b.Property<string>("ConcurrencyStamp")
            .IsConcurrencyToken()
            .IsRequired()
            .HasMaxLength(40)
            .HasColumnType("nvarchar(40)")
            .HasColumnName("ConcurrencyStamp");

        b.Property<DateTime>("CreationTime")
            .HasColumnType("datetime2")
            .HasColumnName("CreationTime");

        b.Property<string>("ExtraProperties")
            .IsRequired()
            .HasColumnType("nvarchar(max)")
            .HasColumnName("ExtraProperties");

        b.Property<bool>("IsAbandoned")
            .ValueGeneratedOnAdd()
            .HasColumnType("bit")
            .HasDefaultValue(false);

        b.Property<string>("JobArgs")
            .IsRequired()
            .HasMaxLength(1048576)
            .HasColumnType("nvarchar(max)");

        b.Property<string>("JobName")
            .IsRequired()
            .HasMaxLength(128)
            .HasColumnType("nvarchar(128)");

        b.Property<DateTime?>("LastTryTime")
            .HasColumnType("datetime2");

        b.Property<DateTime>("NextTryTime")
            .HasColumnType("datetime2");

        b.Property<byte>("Priority")
            .ValueGeneratedOnAdd()
            .HasColumnType("tinyint")
            .HasDefaultValue((byte)15);

        b.Property<short>("TryCount")
            .ValueGeneratedOnAdd()
            .HasColumnType("smallint")
            .HasDefaultValue((short)0);

        b.HasKey("Id");

        b.HasIndex("IsAbandoned", "NextTryTime");

        b.ToTable("AbpBackgroundJobs", (string)null);
    });

Any ideas?

I have made a sample, though this isn't using hangfire: https://github.com/kfrancis/abp-hangfire-sample and that seems to work. If you seed and login under default/admin then go to the books listing, you can see one book (one is tenanted, one is not). Click on the book row to fill in the id, then click "run". You can see in the HttpApi.Host logs that the job is running and it's pulling the name of the book to use in the log entry.

• The report job is enqueued from the BooksAppService
• DI seems to be working normally
• Data filtering seems to be working normally (using CurrentTenant.Change)
• I've added SignalR to facilitate abp.notify on the clients when the background job completes (to the specific user)

Here's what that looks like:

If you can, would you be able to take a look and see what changes need to be made to make this use Hangfire?