Open Closed

AbpBlobContainer Duplication #2707


User avatar
0
murat.yuceer created
  • ABP Framework version: v5.0.1
  • UI type: Blazor Server
  • DB provider: MongoDB
  • Tiered (MVC) or Identity Server Separated (Angular): No

Hello,

Today, I start to get error like "Sequence contains more than one element" when my hangfire job execute. I found problem when I debug method on line below

await _blobContainer.SaveAsync(agreementContainer.PartnerCopy.PdfBinaryId.ToString(), iiaAgreementResponse.Pdf);

I found the line where the error came from https://github.com/abpframework/abp/blob/e3e1779de6df5d26f01cdc8e99ac9cbcb3d24d3c/modules/blob-storing-database/src/Volo.Abp.BlobStoring.Database.Domain/Volo/Abp/BlobStoring/Database/DatabaseBlobProvider.cs#L129

Then I checked database and I've seen repetitive data formed somehow on AbpBlobContainer table (for same container name & tenant)

I using mongodb replication with 3 nodes for enable transaction.

I think it happened when two requests came in at the same time..

How it could be? Does it not lock mongo db collections until transaction finish? Or should we put lock here?


5 Answer(s)
  • User Avatar
    0
    murat.yuceer created

    Also, this is my base class for backgroundjobs

    Could this be a side effect? (Notification table like hangfire queue table to track user job status)

    using Exchanger.Roles;
    using Microsoft.AspNetCore.Identity;
    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using System.Text.Json;
    using System.Threading.Tasks;
    using Volo.Abp.BackgroundJobs;
    using Volo.Abp.Emailing;
    using Volo.Abp.Identity;
    using Volo.Abp.MultiTenancy;
    using Volo.Abp.Timing;
    using Volo.Abp.Uow;
    
    namespace Exchanger.Ewp.Notifications
    {
        public abstract class NotificationJobBase<T> : AsyncBackgroundJob<T> where T : NotificationJobArgs
        {
            public ICurrentTenant CurrentTenant { get; set; }
            public IUnitOfWorkManager UnitOfWorkManager { get; set; }
            public INotificationRepository NotificationRespository { get; set; }
            public IClock Clock { get; set; }
            public IEmailSender EmailSender { get; set; }
            public IIdentityUserRepository IdentityUserRepository { get; set; }
            public ILookupNormalizer LookupNormalizer { get; set; }
    
            protected abstract Task JobMethod(T args);
    
            public override async Task ExecuteAsync(T args)
            {
                using (CurrentTenant.Change(args.CurrentTenantId, args.CurrentTenantName))
                {
                    Exception jobException = null;
                    Notification notification = null;
    
                    using (var uow = UnitOfWorkManager.Begin(requiresNew: true, isTransactional: false))
                    {
                        //get job related notification data
                        notification = await NotificationRespository.FindAsync(args.NotificationId);
                        if (notification == null)
                            return;
    
                        if (notification.ProcessState == NotificationJobState.Cancel ||
                            notification.ProcessState == NotificationJobState.InProcess ||
                            notification.ProcessState == NotificationJobState.Completed)
                            return;
                        notification.SetState(NotificationJobState.InProcess);//to show user job working
                        await NotificationRespository.UpdateAsync(notification);
                        await uow.CompleteAsync();
                    }
    
    
    
    
    
                    bool isError = false;
                    var uowJob = UnitOfWorkManager.Begin(requiresNew: true, isTransactional: true);
                    try
                    {
                        await JobMethod(args); // <== this is actual job method overrided from parent class
    
                        notification.AddTransactionHistory(new Transaction(Clock.Now)
                        {
                            ProcessMessage = "Success",
                            IsSuccess = true
                        });
    
                        await uowJob.CompleteAsync();
                    }
                    catch (Exception ex)
                    {
                        jobException = ex;
    
                        notification.AddTransactionHistory(new Transaction(Clock.Now)
                        {
                            ProcessMessage = ex.Message,
                            ProcessMessageDetail = ex.StackTrace
                        });
    
                        isError = true;
    
                        await uowJob.RollbackAsync();
                    }
                    finally
                    {
                        uowJob.Dispose();
                    }
    
    
    
    
    
                    using (var uow = UnitOfWorkManager.Begin(requiresNew: true, isTransactional: false))
                    {
                        if (!notification.ManuallyTriggered)
                            await SendEmail(args, isError);
    
                        if (notification.TransactionHistory.Any(p => p.IsSuccess))
                        {
                            notification.SetState(NotificationJobState.Completed);
                        }
                        else
                        {
                            notification.SetState(NotificationJobState.Fail);
                        }
    
                        await NotificationRespository.UpdateAsync(notification);
                        await uow.CompleteAsync();
                    }
    
                    //throw if exception exist for hangfire retry
                    if (jobException != null)
                        throw jobException;
                }
            }
    
            private async Task SendEmail(T args, bool isError)
            {
                ...
            }
        }
    }
    
  • User Avatar
    0
    liangshiwei created
    Support Team Fullstack Developer

    Hi,

    This is a concurrency problem, can you try this?

    modelBuilder.Entity<DatabaseBlobContainer>(b =>
    {
        b.CollectionName = BlobStoringDatabaseDbProperties.DbTablePrefix + "BlobContainers";
        b.BsonMap.MapProperty(x => x.Name).SetIsRequired(true);
    });
    
  • User Avatar
    0
    murat.yuceer created

    Hi,

    This is a concurrency problem, can you try this?

    modelBuilder.Entity<DatabaseBlobContainer>(b => 
    { 
        b.CollectionName = BlobStoringDatabaseDbProperties.DbTablePrefix + "BlobContainers"; 
        b.BsonMap.MapProperty(x => x.Name).SetIsRequired(true); 
    }); 
    

    Thanks, but how it will fix problem, the name was not already empty? Shouldn't we create uniqe index with name and tenantId if we are going to solve it at database level?

  • User Avatar
    0
    liangshiwei created
    Support Team Fullstack Developer

    Hi,

    Sorry, I will check it again.

  • User Avatar
    0
    liangshiwei created
    Support Team Fullstack Developer

    Hi,

    It this work for you?: (remove duplicate keys and run the DbMigrator project)

    public override void InitializeCollections(IMongoDatabase database)
    {
        base.InitializeCollections(database);
    
        var model = new CreateIndexModel<DatabaseBlobContainer>
        (
            "{ Name: 1 }",
            new CreateIndexOptions { Unique = true }
        );
    
        database.GetCollection<DatabaseBlobContainer>(BlobStoringDatabaseDbProperties.DbTablePrefix + "BlobContainers").Indexes.CreateOne(model);
    }
    
Made with ❤️ on ABP v8.2.0-preview Updated on March 25, 2024, 15:11