Redis cache RemoveManyAsync methodu not working. #6073

0

liangshiwei created 6 months ago

Support Team Fullstack Developer

Hi,

Could you speak in English? thanks.

0

cetin.sahin created 6 months ago

We would like to perform the menu loading process with the following steps:

When a user logs in, we retrieve the menu permissions from the database and store them in Redis. We developed it as follows in ABP by overriding the MyPermissionStore.cs class.
When a user logs out, we clear the user's Redis keys using MyPermissionRedisCacheClearService, which involves using RemoveManyAsync and _cache.Remove(key) to set the values of the keys in Redis to null.

When a user logs in again, we only fetch that user's menu permissions from the database and write them back to Redis.

using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Logging;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Volo.Abp;
using Volo.Abp.Authorization.Permissions;
using Volo.Abp.Caching;
using Volo.Abp.DependencyInjection;
using Volo.Abp.PermissionManagement;
using EnzimWeb.Helpers;
using Volo.Abp.Identity;
using Volo.Abp.Users;
using Volo.Abp.Data;
using System.Reflection;
using IdentityModel;
using System.Security.Claims;
using Veri.Temel;
using System;
using Veri.Temel.Sabitler;

namespace EnzimWeb.Overrides
{
    [Dependency(ReplaceServices = true)]
    [ExposeServices(typeof(IPermissionStore))]
    public class MyPermissionStore : IPermissionStore, ITransientDependency
    {
        public new ILogger<PermissionStore> Logger { get; set; }

        protected new IPermissionGrantRepository PermissionGrantRepository { get; }

        protected new IPermissionDefinitionManager PermissionDefinitionManager { get; }

        protected new IDistributedCache<PermissionGrantCacheItem> Cache { get; }
        protected IRoleHelperRepository _roleHelperRepository { get; }
        protected IdentityUserManager _identityUserManager { get; }
        protected ICurrentUser _currentUser { get; }

        public MyPermissionStore(
        IPermissionGrantRepository permissionGrantRepository,
        IDistributedCache<PermissionGrantCacheItem> cache,
        IPermissionDefinitionManager permissionDefinitionManager,
        IRoleHelperRepository roleHelperRepository,
        IdentityUserManager userManager,
        ICurrentUser currentUser)
        {
            PermissionGrantRepository = permissionGrantRepository;
            Cache = cache;
            PermissionDefinitionManager = permissionDefinitionManager;
            Logger = NullLogger<PermissionStore>.Instance;
            _roleHelperRepository = roleHelperRepository;
            _identityUserManager = userManager;
            _currentUser = currentUser;
        }




        public virtual async Task<bool> IsGrantedAsync(string name, string providerName, string providerKey)
        {
            return (await GetCacheItemAsync(name, providerName, providerKey)).IsGranted;
        }

        protected virtual async Task<PermissionGrantCacheItem> GetCacheItemAsync(
        string name,
        string providerName,
        string providerKey)
        {
            var cacheKey = CalculateCacheKey(name, providerName, providerKey);

            Logger.LogInformation($"PermissionStore.GetCacheItemAsync: {cacheKey}");

            var cacheItem = await Cache.GetAsync(cacheKey);

            if (cacheItem != null)
            {
                Logger.LogInformation($"Found in the cache: {cacheKey}");
                return cacheItem;
            }

            Logger.LogInformation($"Not found in the cache: {cacheKey}");

            cacheItem = new PermissionGrantCacheItem(false);

            await SetCacheItemsAsync(providerName, providerKey, name, cacheItem);

            return cacheItem;
        }

        protected virtual async Task SetCacheItemsAsync(
        string providerName,
        string providerKey,
        string currentName,
        PermissionGrantCacheItem currentCacheItem)
        {
            var permissions = await PermissionDefinitionManager.GetPermissionsAsync();

            Logger.LogDebug($"Getting all granted permissions from the repository for this provider name,key: {providerName},{providerKey}");

            var grantedPermissionsHashSet = new HashSet<string>(
            (await PermissionGrantRepository.GetListAsync(providerName, providerKey)).Select(p => p.Name)
            );
            foreach (var item in await GetEnzimPermissions(providerName, providerKey))
            {
                grantedPermissionsHashSet.AddIfNotContains(item);
            }
            Logger.LogDebug($"Setting the cache items. Count: {permissions.Count}");

            var cacheItems = new List<KeyValuePair<string, PermissionGrantCacheItem>>();

            foreach (var permission in permissions)
            {
                var isGranted = grantedPermissionsHashSet.Contains(permission.Name);

                cacheItems.Add(new KeyValuePair<string, PermissionGrantCacheItem>(
                CalculateCacheKey(permission.Name, providerName, providerKey),
                new PermissionGrantCacheItem(isGranted))
                );

                if (permission.Name == currentName)
                {
                    currentCacheItem.IsGranted = isGranted;
                }
            }
            await Cache.SetManyAsync(cacheItems);

            Logger.LogDebug($"Finished setting the cache items. Count: {permissions.Count}");
        }
        public virtual async Task<MultiplePermissionGrantResult> IsGrantedAsync(string[] names, string providerName, string providerKey)
        {
            Check.NotNullOrEmpty(names, nameof(names));

            var result = new MultiplePermissionGrantResult();

            if (names.Length == 1)
            {
                var name = names.First();
                result.Result.Add(name,
                await IsGrantedAsync(names.First(), providerName, providerKey)
                ? PermissionGrantResult.Granted
                : PermissionGrantResult.Undefined);
                return result;
            }

            var cacheItems = await GetCacheItemsAsync(names, providerName, providerKey);
            foreach (var item in cacheItems)
            {
                result.Result.Add(GetPermissionNameFormCacheKeyOrNull(item.Key),
                item.Value != null && item.Value.IsGranted
                ? PermissionGrantResult.Granted
                : PermissionGrantResult.Undefined);
            }

            return result;
        }

        protected virtual async Task<List<KeyValuePair<string, PermissionGrantCacheItem>>> GetCacheItemsAsync(
        string[] names,
        string providerName,
        string providerKey)
        {
            var cacheKeys = names.Select(x => CalculateCacheKey(x, providerName, providerKey)).ToList();

            Logger.LogInformation($"PermissionStore.GetCacheItemAsync: {string.Join(",", cacheKeys)}");

            var cacheItems = (await Cache.GetManyAsync(cacheKeys)).ToList();
            if (cacheItems.All(x => x.Value != null))
            {
                Logger.LogInformation($"Found in the cache: {string.Join(",", cacheKeys)}");
                return cacheItems;
            }

            var notCacheKeys = cacheItems.Where(x => x.Value == null).Select(x => x.Key).ToList();

            Logger.LogInformation($"Not found in the cache: {string.Join(",", notCacheKeys)}");

            var newCacheItems = await SetCacheItemsAsync(providerName, providerKey, notCacheKeys);

            var result = new List<KeyValuePair<string, PermissionGrantCacheItem>>();
            foreach (var key in cacheKeys)
            {
                var item = newCacheItems.FirstOrDefault(x => x.Key == key);
                if (item.Value == null)
                {
                    item = cacheItems.FirstOrDefault(x => x.Key == key);
                }

                result.Add(new KeyValuePair<string, PermissionGrantCacheItem>(key, item.Value));
            }

            return result;
        }

        protected virtual async Task<List<KeyValuePair<string, PermissionGrantCacheItem>>> SetCacheItemsAsync(
        string providerName,
        string providerKey,
        List<string> notCacheKeys)
        {
            var permissions = (await PermissionDefinitionManager.GetPermissionsAsync())
            .Where(x => notCacheKeys.Any(k => GetPermissionNameFormCacheKeyOrNull(k) == x.Name)).ToList();

            Logger.LogDebug($"Getting not cache granted permissions from the repository for this provider name,key: {providerName},{providerKey}");

            var grantedPermissionsHashSet = new HashSet<string>(
            (await PermissionGrantRepository.GetListAsync(notCacheKeys.Select(GetPermissionNameFormCacheKeyOrNull).ToArray(), providerName, providerKey)).Select(p => p.Name)
            );
            var nCacheKeys = notCacheKeys.Select(GetPermissionNameFormCacheKeyOrNull).ToArray();

            foreach (var item in await GetEnzimPermissions(providerName, providerKey))
            {
                grantedPermissionsHashSet.AddIfNotContains(item);
            }
            Logger.LogDebug($"Setting the cache items. Count: {permissions.Count}");

            var cacheItems = new List<KeyValuePair<string, PermissionGrantCacheItem>>();

            foreach (var permission in permissions)
            {
                var isGranted = grantedPermissionsHashSet.Contains(permission.Name);

                cacheItems.Add(new KeyValuePair<string, PermissionGrantCacheItem>(
                CalculateCacheKey(permission.Name, providerName, providerKey),
                new PermissionGrantCacheItem(isGranted))
                );
            }

            await Cache.SetManyAsync(cacheItems);

            Logger.LogDebug($"Finished setting the cache items. Count: {permissions.Count}");

            return cacheItems;
        }

        protected virtual string CalculateCacheKey(string name, string providerName, string providerKey)
        {
            return PermissionGrantCacheItem.CalculateCacheKey(name, providerName, providerKey);
        }

        protected virtual string GetPermissionNameFormCacheKeyOrNull(string key)
        {
            //TODO: throw ex when name is null?
            return PermissionGrantCacheItem.GetPermissionNameFormCacheKeyOrNull(key);
        }

        private async Task<List<string>> GetEnzimPermissions(string providerName, string providerKey)
        {
            List<string> permissions = new List<string>();
            try
            {

                if (providerName.Equals("U"))
                {
                    var user = await _identityUserManager.FindByIdAsync(_currentUser.Id?.ToString());
                    if (user != null)
                    {
                        var varsayilanRol = user.GetProperty<int>("VarsayilanRol");
                        if (varsayilanRol > 0)
                        {
                            var userMenus = await _roleHelperRepository.GetMenuYetkiListAsync(varsayilanRol);
                            var userIslemYetkis = await _roleHelperRepository.GetIslemYetkiListAsync(varsayilanRol);
                            foreach (var formTipi in userMenus)
                            {
                                permissions.Add($"FormTipi.{(int)formTipi}");
                            }
                            foreach (var islemTipi in userIslemYetkis)
                            {
                                FieldInfo? islemYetkiTipleriFieldInfo = islemTipi.GetType().GetField(islemTipi.ToString());
                                if (islemYetkiTipleriFieldInfo != null)
                                {
                                    IslemYetkiTipiNitelik? attributeIslemYetkiTipleri = islemYetkiTipleriFieldInfo.GetCustomAttribute<IslemYetkiTipiNitelik>();
                                    if (attributeIslemYetkiTipleri != null)
                                    {
                                        permissions.Add($"Modul.{(int)attributeIslemYetkiTipleri.Modulu}##IslemYetkiTipi.{(int)islemTipi}");
                                    }

                                }
                            }
                        }
                    }
                }

            }

            catch (System.Exception ex)
            {
                Logger.LogException(ex);

            }
            return permissions;
        }
    }
}

We want the menu permissions to be loaded from Redis unless the user logs out.

The development works as expected on our local machines with VS 2022 and IIS Express. However, it unfortunately doesn't work on IIS 10 on Server 2019. When we check the logs, it seems that the GetCacheItemAsync and GetCacheItemsAsync methods are only accessed on the server side during the user's initial login, and afterwards, the same permissions are retained. Even after logging out and clearing Redis, the menu permissions are still retrieved as if it were the initial login.

Are you using any additional caching mechanisms, apart from Redis, within IIS for loading menus and fetching permissions?

0

liangshiwei created 6 months ago

Support Team Fullstack Developer

Are you using any additional caching mechanisms, apart from Redis, within IIS for loading menus and fetching permissions?

No, We only use Redis as distributed cache.

The development works as expected on our local machines with VS 2022 and IIS Express

Can you try to change the ASPNETCORE_ENVIRONMENT to Production to check it?

0

cetin.sahin created 6 months ago

Should I change the ASPNETCORE_ENVIRONMENT value in the launch settings of all projects?

0

liangshiwei created 6 months ago

Support Team Fullstack Developer

Hi,

Yes Just a test to find the issue, change to Production to check whether it still works as expected locally.

0

cetin.sahin created 6 months ago

Hi ; I changed ASPNETCORE_ENVIRONMENT 's value from development to Production . and I run solution on local vs 2022. And then I faced to same problem in local machine. "When I examined the code on ABP's GitHub, I saw the following settings in the AbpAspNetCoreMvcClientModule.cs class for the development environment:

options.TenantConfigurationCacheAbsoluteExpiration = TimeSpan.FromSeconds(5);
options.ApplicationConfigurationDtoCacheAbsoluteExpiration = TimeSpan.FromSeconds(5);

I also found the following settings in the IdentityModelAuthenticationService class.

await TokenCache.SetAsync(cacheKey, tokenCacheItem, new DistributedCacheEntryOptions
{
    AbsoluteExpirationRelativeToNow = AbpHostEnvironment.IsDevelopment()
        ? TimeSpan.FromSeconds(5)
        : TimeSpan.FromSeconds(configuration.CacheAbsoluteExpiration)
});

How can we change these settings? Additionally, what should we do to remove menu permissions upon logout, other than clearing Redis?"

0

liangshiwei created 6 months ago

Support Team Fullstack Developer

Hi,

You can configure the ApplicationConfigurationDtoCacheAbsoluteExpiration.

Configure<AbpAspNetCoreMvcClientCacheOptions>(options=>
{
   options.ApplicationConfigurationDtoCacheAbsoluteExpiration = ....;
})

Or you can also remove the ApplicationConfigurationDto cache: https://github.com/abpframework/abp/blob/dev/framework/src/Volo.Abp.AspNetCore.Mvc.Client/Volo/Abp/AspNetCore/Mvc/Client/MvcCachedApplicationConfigurationClient.cs#L50

0

cetin.sahin created 6 months ago

we have 3 publish project (EnzimWeb.AuthServer , EnzimWeb.Blazor , EnzimWeb.HttpApi.Host) we cant find Configure<AbpAspNetCoreMvcClientCacheOptions> in our solution. please can you tell us where are we add this code

0

liangshiwei created 6 months ago

Support Team Fullstack Developer

Configure in the Blazor project.

0

cetin.sahin created 6 months ago

Thanks . we resolved this issue by your help.

10 Answer(s)