Open Closed

Microservices on Kubernetes for Production Sites #3271


0
thedatacrew created

Hi,

Is their some guidance on the configuration of the Microservices Templates when moved to a production environment i.e. with real domain names. What apps/services/gateways need external ingress and an external domains and which chart values need changing?

I can't find anything in the documentation regarding making and deploying a production version of the software.

Thanks


15 Answer(s)
  • 0
    gterdem created
    Support Team

    Hello, We provide helm charts for microservice templates. You can also check https://github.com/abpframework/eShopOnAbp/tree/main/etc/k8s/eshoponabp sample that has both publish scripts, domain names and values.azure.yaml with the deployed azure configurations.

  • 0
    thedatacrew created

    Thank you, that's very helpful, you should reference that in the README.MD file under "Deploying in Production". How are you managing the Certificate is it using Let's Encrypt Cert Manager or is it a full-blown SAN / Wildcard Cert?

    Thanks.

  • 0
    gterdem created
    Support Team

    Thank you, that's very helpful, you should reference that in the README.MD file under "Deploying in Production". How are you managing the Certificate is it using Let's Encrypt Cert Manager or is it a full-blown SAN / Wildcard Cert?

    Thanks.

    We use Let's Encrypt in the sample. It is declared in each ingress as you can check https://github.com/abpframework/eShopOnAbp/blob/d261dd9c4f36ce68790458980d9c7b4fbe2fb268/etc/k8s/eshoponabp/charts/administration/templates/administration-ingress.yaml#L10

  • 0
    thedatacrew created

    Great - I'm using Traefik as an Ingress Controller and it's also using Let's Encrypt and it all seems to be working with the changes. When I browse to the admin app (Blazor Server), I'm not seeing the Login Screen, it's completely blank.

    I'm getting The cookie 'XSRF-TOKEN' has set 'SameSite=None' and must also set 'Secure'.

    Do I need to implement https://community.abp.io/posts/patch-for-chrome-login-issue-identityserver4-samesite-cookie-problem-weypwp3n ? Or was this fixed in 5.2 ?

    Thanks

  • 0
    gterdem created
    Support Team

    Do I need to implement https://community.abp.io/posts/patch-for-chrome-login-issue-identityserver4-samesite-cookie-problem-weypwp3n ? Or was this fixed in 5.2 ?

    It is not related to ABP so It is not something we can fix.

    We also had problems with it so we implemented it: https://github.com/abpframework/eShopOnAbp/blob/d261dd9c4f36ce68790458980d9c7b4fbe2fb268/apps/auth-server/src/EShopOnAbp.AuthServer/SameSiteCookiesServiceCollectionExtensions.cs

  • 0
    thedatacrew created

    I implemented this, but when I click login, it tries to redirect https://app.mydomain.com/Account/Login to http://auth.mydomain.com instead of https://auth.mydomain.com - I cannot find anywhere where is is configured to not use HTTPS.

  • 0
    gterdem created
    Support Team

    You can check the IdentityServerDataSeeder file. It is located under both the DbMigrator and the IdentityService.HttpApi.Host projects

    Whichever you are using to seed the data, check the appsettings.json file for IdentityServer initial data. You can examine the IdentityServerDataSeeder to learn where they are used and set.

  • 0
    thedatacrew created

    Ok, thanks

    In the dbmigrator appsettings.json there is this

    In the Chart the Environments Variables maps don't match

    I'm assuming these need to match for the container to override the config and seed the correct values.

  • 0
    gterdem created
    Support Team

    If you are using Kubernetes, yes.

    You need to check the IdentityService values to override.

  • 0
    thedatacrew created

    Hi,

    I implemented the SameSiteCookiesServiceCollectionExtensions.cs in the AuthServer using teh eshop examples - I'm assuming that this is the only place it is required?

    Browsing to https://auth.mycompany.net I still get The cookie 'XSRF-TOKEN' has set 'SameSite=None' and must also set 'Secure'.

    Also, both the https://www.mycompany.net and https://app.mycompany.net redirect the login to http://auth.mycompany.net not https which gives a 404.

    I have check the configs and the databases and the seeded data - it all looks good.

    I'm a bit flummoxed now. What can I look at next? Is their an opportunity to open a paid support ticket?

    [09:41:30 INF] Bundled __bundles/Lepton.Global.9A9449B4A1BEC7DF689B1E3C3552F66F.js (736917 bytes)
    [09:41:30 INF] Executed page /Account/Login in 3456.6196ms
    [09:41:30 INF] Executed endpoint '/Account/Login'
    [09:41:30 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/Account/Login - - - 200 - text/html;+charset=utf-8 3513.3771ms
    [09:41:30 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/__bundles/Lepton.Global.ADD5F01D11E6ABD793872CD20AFE07ED.css?_v=637920924901301785 - -
    [09:41:30 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/Abp/ApplicationConfigurationScript - -
    [09:41:30 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/__bundles/Lepton.Global.9A9449B4A1BEC7DF689B1E3C3552F66F.js?_v=637920924909157339 - -
    [09:41:30 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/Abp/ServiceProxyScript - -
    [09:41:30 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/libs/timeago/locales/jquery.timeago.en.js?_v=637920493210000000 - -
    [09:41:31 INF] Executing endpoint 'Volo.Abp.AspNetCore.Mvc.ProxyScripting.AbpServiceProxyScriptController.GetAll (Volo.Abp.AspNetCore.Mvc)'
    [09:41:31 INF] Executing endpoint 'Volo.Abp.AspNetCore.Mvc.ApplicationConfigurations.AbpApplicationConfigurationScriptController.Get (Volo.Abp.AspNetCore.Mvc)'
    [09:41:31 INF] Sending file. Request path: '/libs/timeago/locales/jquery.timeago.en.js'. Physical path: '/app/wwwroot/libs/timeago/locales/jquery.timeago.en.js'
    [09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/libs/timeago/locales/jquery.timeago.en.js?_v=637920493210000000 - - - 200 778 application/javascript 28.2478ms
    [09:41:31 INF] Route matched with {area = "Abp", action = "GetAll", controller = "AbpServiceProxyScript", page = ""}. Executing controller action with signature Microsoft.AspNetCore.Mvc.ActionResult GetAll(Volo.Abp.AspNetCore.Mvc.ProxyScripting.ServiceProxyGenerationModel) on controller Volo.Abp.AspNetCore.Mvc.ProxyScripting.AbpServiceProxyScriptController (Volo.Abp.AspNetCore.Mvc).
    [09:41:31 INF] Route matched with {area = "Abp", action = "Get", controller = "AbpApplicationConfigurationScript", page = ""}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.ActionResult] Get() on controller Volo.Abp.AspNetCore.Mvc.ApplicationConfigurations.AbpApplicationConfigurationScriptController (Volo.Abp.AspNetCore.Mvc).
    [09:41:31 INF] Sending file. Request path: '/__bundles/Lepton.Global.ADD5F01D11E6ABD793872CD20AFE07ED.css'. Physical path: 'N/A'
    [09:41:31 INF] Sending file. Request path: '/__bundles/Lepton.Global.9A9449B4A1BEC7DF689B1E3C3552F66F.js'. Physical path: 'N/A'
    [09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/__bundles/Lepton.Global.ADD5F01D11E6ABD793872CD20AFE07ED.css?_v=637920924901301785 - - - 200 507556 text/css 67.9968ms
    [09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/__bundles/Lepton.Global.9A9449B4A1BEC7DF689B1E3C3552F66F.js?_v=637920924909157339 - - - 200 738613 application/javascript 58.8700ms
    [09:41:31 INF] Executing ContentResult with HTTP Response ContentType of application/javascript
    [09:41:31 INF] Executed action Volo.Abp.AspNetCore.Mvc.ProxyScripting.AbpServiceProxyScriptController.GetAll (Volo.Abp.AspNetCore.Mvc) in 243.8057ms
    [09:41:31 INF] Executed endpoint 'Volo.Abp.AspNetCore.Mvc.ProxyScripting.AbpServiceProxyScriptController.GetAll (Volo.Abp.AspNetCore.Mvc)'
    [09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/Abp/ServiceProxyScript - - - 200 1154 application/javascript 288.5070ms
    [09:41:31 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/images/logo/logo-dark.png - -
    [09:41:31 INF] Sending file. Request path: '/images/logo/logo-dark.png'. Physical path: '/app/wwwroot/images/logo/logo-dark.png'
    [09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/images/logo/logo-dark.png - - - 200 1386 image/png 7.1821ms
    [09:41:31 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/libs/flag-icon-css/flags/1x1/gb.svg - -
    [09:41:31 INF] Sending file. Request path: '/libs/flag-icon-css/flags/1x1/gb.svg'. Physical path: '/app/wwwroot/libs/flag-icon-css/flags/1x1/gb.svg'
    [09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/libs/flag-icon-css/flags/1x1/gb.svg - - - 200 538 image/svg+xml 0.5844ms
    [09:41:33 WRN] The cookie 'XSRF-TOKEN' has set 'SameSite=None' and must also set 'Secure'.
    
  • 0
    gterdem created
    Support Team

    Also, both the https://www.mycompany.net and https://app.mycompany.net redirect the login to http://auth.mycompany.net not https which gives a 404.

    Setting auto-redirect in webserver from HTTP to HTTPS should fix this problem.

    Browsing to https://auth.mycompany.net I still get The cookie 'XSRF-TOKEN' has set 'SameSite=None' and must also set 'Secure'.

    What is this application? Angular back-office application or mvc application?

  • 0
    thedatacrew created

    It's a Blazor Server for the APP (https://app.mycompany.net) and MVC for the Public Website (https://www.mycompany.net)

  • 0
    gterdem created
    Support Team

    Browsing to https://auth.mycompany.net I still get The cookie 'XSRF-TOKEN' has set 'SameSite=None' and must also set 'Secure'.

    It seems related to HTTPS redirection.

    You can update your cookie configurations in your authserver application Module ServiceConfiguration:

    ...
    .AddCookie("Cookies", options =>
    {
        options.ExpireTimeSpan = TimeSpan.FromDays(365);
    
        options.Cookie.HttpOnly = true;
        options.Cookie.SameSite = SameSiteMode.None;
        options.Cookie.SecurePolicy = CookieSecurePolicy.Always;
    })
    ...
    

    See more about working with sameSite cookies.

    However it is recommended to use the authentication server on https.

    Redirection from http to https in authserver web-server configuration should fix this problem.

  • 0
    thedatacrew created

    The Auth Server is set to use HTTPS in the configs and chart vaules. The redirect from the web apps is going to http. It's configured the same as the eShop example.

    How does eShop auto redirect to HTTPS?

  • 0
    gterdem created
    Support Team

    Https redirection is based on the webserver you are using. eShop is hosted on azure kubernetes cluster. You can check forced ssl-redirection: https://github.com/abpframework/eShopOnAbp/blob/d261dd9c4f36ce68790458980d9c7b4fbe2fb268/etc/k8s/eshoponabp/charts/authserver/templates/authserver-ingress.yaml#L7

    You can google it if you are using IIS or Nginx for more accurate information.