About the kubernetes multipod support(Again) #5064

7 Answer(s)

• 

  0

  alper created one year ago

  Support Team Director

  do you have a single sign on endpoint? like https://account.abp.io/

  Save Cancel
• 

  0

  cagriceylan created one year ago

  Yes, we have. We use corporate sso endpoint https://sso.setur.com.tr but our problem is api side. We get token for example https://localhost/connect/token pod1, when loadbalancer forwards the token we get from here(pod1(kubernetes)) to a different pod(pod2(kubernetes)), we get 401(unauthorized) in api services

  For example We get token pod1 (https://localhost/connect/token) we can use this token https://localhost/api/app/configs bearer pod1_token => we get success When loadbalancer forward another pods; for example; pod2 => https://localhost/api/app/configs bearer pod1_token => we get 401 unauthorized

  Save Cancel
• 

  0

  gterdem created one year ago

  Support Team Senior .NET Developer

  Can you explain your deployment scenario in more details? You have a tiered application containing AuthServer, MVC app and API.Host.

  • Is it a local staging environment k8s cluster or production environment?
  • Are you running all of them on the same k8s network?
  • Where is the load-balancer located? Is there other load balancers between MVC app and APIs? Is there specific load balancer configurations for it?

  Related to https://support.abp.io/QA/Questions/4990#answer-3a0aefd0-52c0-7e1b-f4d5-5269a1d63ef7 did you configure all of the applications (AuthServer, MVC app and the API.Host) with data-protection?:

  var redis = ConnectionMultiplexer.Connect(configuration["Redis:Configuration"]);
  dataProtectionBuilder.PersistKeysToStackExchangeRedis(redis, "MyApp-Protection-Keys");
  

  Can you diagnose the error from getting the access_token to sent API as bearer token?

  It may be related to HTTPS headers not being forwarded.

  If that's the case, you can try adding forwarded headers configuration to your AuthServer module ConfigureServices:

  context.Services.Configure<ForwardedHeadersOptions>(options =>
  {
      options.ForwardedHeaders = ForwardedHeaders.XForwardedFor |
                                 ForwardedHeaders.XForwardedProto |
                                 ForwardedHeaders.XForwardedHost;
      options.KnownNetworks.Clear();
      options.KnownProxies.Clear();
  });
  

  and add:

  public override void OnApplicationInitialization(ApplicationInitializationContext context)
  {
      var app = context.GetApplicationBuilder();
      var env = context.GetEnvironment();
      app.UseForwardedHeaders(); // Before any HTTPS redirection midware
      ...
      
  }
  

  Save Cancel
• 

  0

  cagriceylan created one year ago

  We have no separate environment for AuthServer. We use build in openiddict to web application with api.

  Like this:

  There are 4 environment such as; playground, testing, staging and production and whole environments run k8s on the same network each.

  There are only one specific configuration load balancer is sticky session.

  There is a web application with api which runs k8s and we implement this solution on it.

  We tried seperated application(authserver, web, apihost) and there are many other problems with communication eachothers and we changed the architecture this.

  Save Cancel
• 

  0

  gterdem created one year ago

  Support Team Senior .NET Developer

  If you are using non-tiered mvc application, which is not using jwt for authentication; you can use session just fine. Sticky session is related to your load balancer configuration and ABP does not add any configuration related to it. You need to add that configuration manually like any .net application behind a load balancer. The problem seems to be related to your sticky session configuration.

  However if you use tiered application, it will be using jwt and you'll have to use redis and data protection for consistency between replicated pods.

  I don't think I can replicate your environment. You can share detailed error messages for better diagnose to see if we can help further.

  Save Cancel
• 

  0

  alper created one year ago

  Support Team Director

  This issue has been resolved. maybe @cagriceylan can explain how we solved it

  Save Cancel
• 

  0

  cagriceylan created one year ago

  Firstly thanks for your online support Mr. Alper & Mr. Halil

  As Mr. @alper said, our problem has been solved, and we have been continuing our tests for a while, and we did not see a problem.

  The steps we take are as follows;

  1- Add the Microsoft.AspNetCore.DataProtection.StackExchangeRedis package to Web project. 2- Redis is integrated into the web project to following steps.

  • Add this two following methods yourprojectnameWebModule class ConfigurePersistKeysToStackExchangeRedis ve ConfigureRedis

   private void ConfigurePersistKeysToStackExchangeRedis(ServiceConfigurationContext context, IConfiguration configuration)
      {
          ConnectionMultiplexer redis = ConnectionMultiplexer.Connect(configuration["Redis:ConnectionString"]);
          context.Services
              .AddDataProtection()
              .SetApplicationName("loyalty-web-ui")
              .PersistKeysToStackExchangeRedis(redis, "Loyalty.Web.UI-Protection-Keys");
      }
  

  private void ConfigureRedis(ServiceConfigurationContext context, IConfiguration configuration)
      {
          ConnectionMultiplexer multiplexer = ConnectionMultiplexer.Connect(configuration["Redis:ConnectionString"]);
          context.Services.AddSingleton<IConnectionMultiplexer>(multiplexer);
      }
  

  • Also add this code blocks to ConfigureServices method

      ConnectionMultiplexer multiplexer = ConnectionMultiplexer.Connect(configuration["Redis:ConnectionString"]);
  

  context.Services.AddSingleton\<IConnectionMultiplexer>(multiplexer);
  

  3- Run the following command in the web project directory for install Volo.Abp.Caching.StackExchangeRedis

  • C:\Users\username\source\repos\yourprojectname\src\Web

      abp add-package Volo.Abp.Caching.StackExchangeRedis
  

  • After this process, the [DependsOn(typeof(AbpCachingStackExchangeRedisModule))] attribute should be added yourprojectnameWebModule automatically.

  4-

  • Add the GetSigningCertificate method to yourprojectnameWebModule

  private X509Certificate2 GetSigningCertificate(IWebHostEnvironment hostingEnv, IConfiguration configuration)
      {
          var fileName = "authserver.pfx";
          var passPhrase = "32165487-6547-6544-1236-123654789654"; 
          var file = Path.Combine(hostingEnv.ContentRootPath, fileName);
          if (!System.IO.File.Exists(file)) 
          {
              throw new FileNotFoundException($"Signing Certificate couldn't found: {file}"); 
          }
          return new X509Certificate2(file, passPhrase);
      }
  

  • Add this code blocks to PreConfigureServices method in yourprojectnameWebModule

          var hostingEnvironment = context.Services.GetHostingEnvironment();
          var configuration = context.Services.GetConfiguration();
  
          if (!hostingEnvironment.IsDevelopment())
          {
              PreConfigure<AbpOpenIddictAspNetCoreOptions>(options =>
              {
                  options.AddDevelopmentEncryptionAndSigningCertificate = false;
              });
  
              PreConfigure<OpenIddictServerBuilder>(builder =>
              {
                  builder.AddSigningCertificate(GetSigningCertificate(hostingEnvironment, configuration));
                  builder.AddEncryptionCertificate(GetSigningCertificate(hostingEnvironment, configuration));
                  builder.SetIssuer(new Uri(configuration["AuthServer:Authority"]));
              });
          }
  

  • Run the following command in the web project directory

      * dotnet dev-certs https -v -ep ./authserver.pfx -p 32165487-6547-6544-1236-123654789654
  

  Save Cancel