Deployment production tutorial Kubernetes / Let's encrypt #5891

0

gterdem created 7 months ago

Support Team Senior .NET Developer

We don't have a full guide about how to deploy Microservice template to Azure yet. But we have a public repository with build scripts and azure helm chart values at https://github.com/abpframework/eShopOnAbp/tree/main/etc/k8s/azure

Let us know if you come across any problems.

0

carl.hallqvist created 7 months ago

Thanks gterdem,

I don't think that is related to Azure? I am no expert, but when looking at the k8s directory, it seems that most of it is in place? I guess its the installation of cert-manager and nginx ingress (for instance) and the issuer manifests. Also adding ingress part for all services.

I will try to make it work by my self, but the abp really need some deployment guides. We was failing to deploy abp framework too, and now we are trying the commercial one.

0

carl.hallqvist created 6 months ago

Hi,

I have almost managed to deploy everything. However, got some configuration failure from public web app:

[19:46:55 INF] TRIAL-INFO - You are using the trial version of ABP and your expiry date is 2023-10-26 [19:46:55 INF] Now listening on: http://[::]:80 [19:46:55 INF] Application started. Press Ctrl+C to shut down. [19:46:55 INF] Hosting environment: Production [19:46:55 INF] Content root path: /app [19:48:58 INF] Request starting HTTP/1.1 GET http://public.staging.microservice.io/ - - [19:48:58 WRN] Could not find IdentityClientConfiguration for AbpMvcClient. Either define a configuration for AbpMvcClient or set a default configuration. [19:48:58 INF] Start processing HTTP request GET https://microservice-gateway-web-public/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0 [19:48:58 INF] Sending HTTP request GET https://microservice-gateway-web-public/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0 [19:49:00 INF] Sending HTTP request GET https://microservice-gateway-web-public/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0 [19:49:04 INF] Sending HTTP request GET https://microservice-gateway-web-public/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0 [19:49:12 INF] Sending HTTP request GET https://microservice-gateway-web-public/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0 [19:49:28 INF] Sending HTTP request GET https://microservice-gateway-web-public/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0 [19:49:28 ERR] Connection id "0HMU9033L8A7L", Request id "0HMU9033L8A7L:00000001": An unhandled exception was thrown by the application. Volo.Abp.Http.Client.AbpRemoteCallException: An error occurred during the ABP remote HTTP request. (Connection refused (microservice-gateway-web-public:443)) See the inner exception for details. ---> System.Net.Http.HttpRequestException: Connection refused (microservice-gateway-web-public:443) ---> System.Net.Sockets.SocketException (111): Connection refused

This is my value file:

# auth-server sub-chart override
authserver:
  config:
    selfUrl: https://auth.staging.microservice.io   
    corsOrigins: https://gateway.staging.microservice.io,https://gateway-public.staging.microservice.io,https://identity.staging.microservice.io,https://administration.staging.microservice.io,https://basket.staging.microservice.io,https://catalog.staging.microservice.io,https://order.staging.microservice.io,https://cmskit.staging.microservice.io,https://payment.staging.microservice.io,https://admin.staging.microservice.io,https://www.staging.microservice.io
    allowedRedirectUrls: https://admin.staging.microservice.io
    disablePii: "false"
    identityService:
      connString: "Server=microservice-sqldb,1433;Database=Microservice_Identity;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
    administrationService:
      connString: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
    saasService:
      connString: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
    authServer:
      authority: http://microservice-authserver
      requireHttpsMetadata: "false"
      swaggerClientId: WebGateway_Swagger
    dotnetEnv: Production
    redisHost: microservice-redis
    rabbitmqHost: microservice-rabbitmq
    elasticsearchUrl: http://microservice-elasticsearch
  ingress:
    host: auth.staging.microservice.io
    tlsSecret: auth-staging-microservice-io-tls
  image:
    repository: "registry.digitalocean.com/microservice/microservice-app-authserver"
    tag: latest

# public-web sub-chart override
public-web:
  config:
    selfUrl: https://www.staging.microservice.io
    gatewayUrl: https://microservice-gateway-web-public
    authServer:
      authority: https://auth.staging.microservice.io
      requireHttpsMetadata: "false"
      isOnProd: "true"
      metaAddress: http://microservice-authserver
    dotnetEnv: Production
    redisHost: microservice-redis
    rabbitmqHost: microservice-rabbitmq
    elasticsearchHost: http://microservice-elasticsearch
  ingress:
    host: public.staging.microservice.io
    tlsSecret: public-staging-microservice-io-tls
  image:
    repository: "registry.digitalocean.com/microservice/microservice-app-public-web"
    tag: latest

blazor:
  config:
    selfUrl: https://blazor.staging.microservice.io
    gatewayUrl: https://microservice-gateway-web
    authServer:
      authority: https://microservice-st-authserver
      requireHttpsMetadata: "false"
      responseType: "code"
  ingress:
    host: blazor.staging.microservice.io
    tlsSecret: blazor-staging-microservice-io-tls
  image:
    repository: registry.digitalocean.com/microservice/microservice-app-blazor
    tag: latest
    pullPolicy: IfNotPresent

# identity-service sub-chart override
identity:
  config:
    selfUrl: https://identity.staging.microservice.io
    corsOrigins: https://gateway.staging.microservice.io,https://gateway-public.staging.microservice.io
    connectionStrings:
      identityService: "Server=microservice-sqldb,1433;Database=Microservice_Identity;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
      administrationService: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
      saasService: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
    authServer:
      authority: http://microservice-authserver
      requireHttpsMetadata: "false"
      swaggerClientId: WebGateway_Swagger
      swaggerClientSecret: "1q2w3e*"
    dotnetEnv: Production
    redisHost: microservice-redis
    rabbitmqHost: microservice-rabbitmq
    elasticsearchHost: http://microservice-elasticsearch  
  identityServerClients: # Seeded Clients
    webRootUrl: https://admin.staging.microservice.io/  
    publicWebRootUrl: https://www.staging.microservice.io/
    webGatewayRootUrl: https://gateway.staging.microservice.io/
    publicWebGatewayRootUrl: https://gateway-public.staging.microservice.io/
    identityServiceRootUrl: https://identity.staging.microservice.io/
    administrationServiceRootUrl: https://administration.staging.microservice.io/
    accountServiceRootUrl: https://auth.staging.microservice.io
  ingress:
    host: identity.staging.microservice.io
    tlsSecret: identity-staging-microservice-io-tls
  image:
    repository: "registry.digitalocean.com/microservice/microservice-service-identity"
    tag: latest

# administration sub-chart override
administration:
  config:
    selfUrl: https://administration.staging.microservice.io
    corsOrigins: https://microservice-gateway-web,https://microservice-st-gateway-web-public
    connectionStrings:
      administrationService: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
      saasService: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"    
    authServer:
      authority: http://microservice-st-authserver
      requireHttpsMetadata: "false"
      swaggerClientId: WebGateway_Swagger
      swaggerClientSecret: "1q2w3e*"
    remoteServices:
      abpIdentityBaseUrl: https://microservice-st-identity
      useCurrentToken: "false"   
    dotnetEnv: Production
    redisHost: microservice-redis
    rabbitmqHost: microservice-rabbitmq
    elasticsearchHost: http://microservice-elasticsearch
  synchedCommunication: # Used for server-to-server (client-credentials) communication with identityService for user permissions
    authority: http://microservice-authserver
    requireHttps: "false"
    validateIssuerName: "false"
    validateEndpoints: "false"
  ingress:
    host: administration.staging.microservice.io
    tlsSecret: administration-staging-microservice-io-tls
  image:
    repository: registry.digitalocean.com/microservice/microservice-service-administration
    tag: latest

# gateway-web sub-chart override
gateway-web:
  config:
    selfUrl: https://gateway.staging.microservice.io
    corsOrigins: https://admin.staging.microservice.io
    globalConfigurationBaseUrl: http://microservice-gateway-public
    authServer:
      authority: http://microservice-authserver
      requireHttpsMetadata: "false"
      swaggerClientId: WebGateway_Swagger
      swaggerClientSecret: "1q2w3e*"
    dotnetEnv: Production
    redisHost: microservice-redis
    rabbitmqHost: microservice-rabbitmq
    elasticsearchHost: http://microservice-elasticsearch
  ingress:
    host: gateway.staging.microservice.io
    tlsSecret: gateway-staging-microservice-io-tls
  image:
    repository: "registry.digitalocean.com/microservice/microservice-gateway-web"
    tag: latest
  reRoutes:
    accountService:
      url: http://microservice-authserver
    identityService:
      url: http://microservice-identity
    administrationService:
      url: http://microservice-administration

# gateway-web-public sub-chart override
gateway-web-public:
  config:
    selfUrl: https://gateway-public.staging.microservice.io
    authServer:
      authority: http://microservice-authserver
      requireHttpsMetadata: "false"
      swaggerClientId: WebGateway_Swagger
      swaggerClientSecret: "1q2w3e*"
    dotnetEnv: Production
    redisHost: microservice-redis
    rabbitmqHost: microservice-rabbitmq
    elasticsearchHost: http://microservice-elasticsearch
  ingress:
    host: gateway-public.staging.microservice.io
    tlsSecret: gateway-public-staging-microservice-io-tls
  image:
    repository: "registry.digitalocean.com/microservice/microservice-gateway-web-public"
    tag: latest
  reRoutes:
    accountService:
      url: http://microservice-authserver
    identityService:
      url: http://microservice-identity
    administrationService:
      url: http://microservice-administration

# dbmigrator
dbmigrator:
  config:
    connectionStrings:
      identityService: "Server=microservice-sqldb,1433;Database=Microservice_Identity;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
      administrationService: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
      saasService: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
      productService: "Server=microservice-sqldb,1433;Database=Microservice_ProductService;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
    dotnetEnv: Production
    elasticsearchUrl: http://microservice-elasticsearch
    redisHost: microservice-redis
  openIdDict:
    applications:
      authServerRootUrl: https://microservice-authserver/
      angularRootUrl: https://microservice-angular/
      webRootUrl: https://microservice-web/
      blazorRootUrl: https://microservice-blazor/
      blazorServerRootUrl: https://microservice-blazor-server/
      publicWebRootUrl: https://microservice-public-web/
      webGatewayRootUrl: https://microservice-gateway-web/
      publicWebGatewayRootUrl: https://microservice-gateway-web-public/ 
    resources:  
      accountServiceRootUrl: https://microservice-authserver/
      administrationServiceRootUrl: https://microservice-administration/
      identityServiceRootUrl: https://microservice-identity/
      saasServiceRootUrl: https://microservice-saas/
      productServiceRootUrl: https://microservice-product/
  image:
    repository: registry.digitalocean.com/microservice/microservice-db-migrator
    tag: latest
    pullPolicy: IfNotPresent  

kibana:
  config:
    elasticsearchUrl: http://microservice-elasticsearch:9200
  ingress:
    host: kibana.staging.microservice.io
    tlsSecret: kibana-staging-microservice-io-tls

# Default values for microservice.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

replicaCount: 1

image:
  repository: nginx
  pullPolicy: IfNotPresent
  # Overrides the image tag whose default is the chart appVersion.
  tag: ""

imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""

serviceAccount:
  # Specifies whether a service account should be created
  create: true
  # Annotations to add to the service account
  annotations: {}
  # The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  name: ""

podAnnotations: {}

podSecurityContext: {}
  # fsGroup: 2000

securityContext: {}
  # capabilities:
  #   drop:
  #   - ALL
  # readOnlyRootFilesystem: true
  # runAsNonRoot: true
  # runAsUser: 1000

service:
  type: ClusterIP
  port: 80

# ingress:
#   enabled: false
#   className: ""
#   annotations: {}
#     # kubernetes.io/ingress.class: nginx
#     # kubernetes.io/tls-acme: "true"
#   hosts:
#     - host: chart-example.local
#       paths:
#         - path: /
#           pathType: ImplementationSpecific
#   tls: []
#   #  - secretName: chart-example-tls
#   #    hosts:
#   #      - chart-example.local

resources: {}
  # We usually recommend not to specify default resources and to leave this as a conscious
  # choice for the user. This also increases chances charts run on environments with little
  # resources, such as Minikube. If you do want to specify resources, uncomment the following
  # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
  # limits:
  #   cpu: 100m
  #   memory: 128Mi
  # requests:
  #   cpu: 100m
  #   memory: 128Mi

autoscaling:
  enabled: false
  minReplicas: 1
  maxReplicas: 100
  targetCPUUtilizationPercentage: 80
  # targetMemoryUtilizationPercentage: 80

tolerations: []

affinity: {}

0

gterdem created 6 months ago

Support Team Senior .NET Developer

public-web sub-chart override

public-web: config: selfUrl: https://www.staging.microservice.io gatewayUrl: https://microservice-gateway-web-public ---> should be real dns like www.myapp.gateway.com

The error is related to the gateway DNS couldn't be reached over the internet since you have provided internal network service name. Please try real DNS for it and let me know if it solves your problem.

0

carl.hallqvist created 6 months ago

Thanks gterdem,

Unfortuntely, I got same error. I think that I need to get each app / service / gateway working.

auth.staging.microservice.io service seems to be ok, the login / logout is working fine.

Log from microservice-public-web

[20:23:22 INF] Sending HTTP request GET https://gateway-public.staging.microservice.io/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0
[20:23:22 INF] Received HTTP response headers after 13.0113ms - 502

So, the response from the gateway is 502, and when looking into the log in the gateway I got:

Log from geoticket-gateway-web-public:

[20:23:22 INF] Request starting HTTP/1.1 GET http://gateway-public.staging.microservice.io/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0 - -
[20:23:22 INF] requestId: 0HMUBBQVGSBOD:00000004, previousRequestId: no previous request id, message: EndpointRateLimiting is not enabled for /api/abp/{everything}
[20:23:22 INF] requestId: 0HMUBBQVGSBOD:00000004, previousRequestId: no previous request id, message: No authentication needed for /api/abp/application-configuration
[20:23:22 INF] requestId: 0HMUBBQVGSBOD:00000004, previousRequestId: no previous request id, message: /api/abp/{everything} route does not require user to be authorized
[20:23:22 WRN] requestId: 0HMUBBQVGSBOD:00000004, previousRequestId: no previous request id, message: Error Code: ConnectionToDownstreamServiceError Message: Error connecting to downstream service, exception: System.Net.Http.HttpRequestException: Connection refused (localhost:44367)
 ---> System.Net.Sockets.SocketException (111): Connection refused
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|281_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionAsync(Boolean async, CancellationToken requestCancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Ocelot.Requester.HttpClientHttpRequester.GetResponse(HttpContext httpContext) errors found in ResponderMiddleware. Setting error response for request path:/api/abp/application-configuration, request method: GET
[20:23:22 INF] Request finished HTTP/1.1 GET http://gateway-public.staging.microservice.io/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0 - - - 502 0 - 6.7145ms

This is my value file:

# public-web sub-chart override
public-web:
  config:
    selfUrl: https://public.staging.microservice.io
    gatewayUrl: https://gateway-public.staging.microservice.io
    authServer:
      authority: https://auth.staging.microservice.io
      requireHttpsMetadata: "false"
      isOnProd: "true"
      metaAddress: http://microservice-authserver
    dotnetEnv: Production
    redisHost: microservice-redis
    rabbitmqHost: microservice-rabbitmq
    elasticsearchHost: http://microservice-elasticsearch
  ingress:
    host: public.staging.microservice.io
    tlsSecret: public-staging-microservice-io-tls
  image:
    repository: "registry.digitalocean.com/microservice/microservice-app-public-web"
    tag: latest
	
###

# gateway-web-public sub-chart override
gateway-web-public:
  config:
    selfUrl: https://gateway-public.staging.microservice.io
    authServer:
      authority: http://microservice-authserver
      requireHttpsMetadata: "false"
      swaggerClientId: WebGateway_Swagger
      swaggerClientSecret: "1q2w3e*"
    dotnetEnv: Production
    redisHost: microservice-redis
    rabbitmqHost: microservice-rabbitmq
    elasticsearchHost: http://microservice-elasticsearch
  ingress:
    host: gateway-public.staging.microservice.io
    tlsSecret: gateway-public-staging-microservice-io-tls
  image:
    repository: "registry.digitalocean.com/microservice/microservice-gateway-web-public"
    tag: latest
  reRoutes:
    accountService:
      url: http://microservice-authserver
    identityService:
      url: http://microservice-identity
    administrationService:
      url: http://microservice-administration

Kind regards, Carl

0

gterdem created 6 months ago

Support Team Senior .NET Developer

[20:23:22 WRN] requestId: 0HMUBBQVGSBOD:00000004, previousRequestId: no previous request id, message: Error Code: ConnectionToDownstreamServiceError Message: Error connecting to downstream service, exception: System.Net.Http.HttpRequestException: Connection refused (localhost:44367)
 ---> System.Net.Sockets.SocketException (111): Connection refused

The error indicates that the problem is related to your gateway downstream. It seems you may have forgotten to override the re-routing for the gateways.

0

carl.hallqvist created 6 months ago

Thanks gterdem,

I did a complete reset of the helm values. This is what I got:

# auth-server sub-chart override
authserver:
  config:
    selfUrl: https://auth.staging.microservice.io
    corsOrigins: http://microservice-blazor,https://blazor.staging.microservice.io,https://gateway-web.staging.microservice.io,https://microservice-gateway-web-public,https://microservice-identity,https://microservice-administration,https://microservice-saas,https://microservice-product
    allowedRedirectUrls: http://blazor.staging.microservice.io,https://blazor.staging.microservice.io
    disablePii: "false"
    identityService:
      connString: "Server=microservice-sqldb,1433;Database=Microservice_Identity;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
    administrationService:
      connString: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
    saasService:
      connString: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
    authServer:
      authority: http://microservice-authserver
      requireHttpsMetadata: "false"
      swaggerClientId: WebGateway_Swagger
    dotnetEnv: Production
    redisHost: microservice-redis
    rabbitmqHost: microservice-rabbitmq
    elasticsearchUrl: http://microservice-elasticsearch
  ingress:
    host: auth.staging.microservice.io
    tlsSecret: staging.microservice.io
  image:
    repository: registry.digitalocean.com/microservice/microservice-app-authserver
    tag: latest

# identity-service sub-chart override
identity:
  config:
    selfUrl: https://microservice-identity
    corsOrigins: https://microservice-gateway-web
    connectionStrings:
      identityService: "Server=microservice-sqldb,1433;Database=Microservice_Identity;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
      administrationService: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
      saasService: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
    authServer:
      authority: http://microservice-authserver
      requireHttpsMetadata: "false"
    dotnetEnv: Production
    redisHost: microservice-redis
    rabbitmqHost: microservice-rabbitmq
    elasticsearchUrl: http://microservice-elasticsearch
  openIdDict: #Seed applications and resources
    applications:
      authServerRootUrl: https://microservice-authserver/
      blazorRootUrl: https://blazor.staging.microservice.io/
      publicWebRootUrl: https://microservice-public-web/
      webGatewayRootUrl: https://microservice-gateway-web/
      publicWebGatewayRootUrl: https://microservice-gateway-web-public/ 
    resources:  
      accountServiceRootUrl: https://microservice-authserver/
      administrationServiceRootUrl: https://microservice-administration/
      identityServiceRootUrl: https://microservice-identity/
      saasServiceRootUrl: https://microservice-saas/
      productServiceRootUrl: https://microservice-product/
  ingress:
    host: microservice-identity
  image:
    repository: registry.digitalocean.com/microservice/microservice-service-identity
    tag: latest

# administration-service sub-chart override
administration:
  config:
    selfUrl: https://microservice-administration
    corsOrigins: https://microservice-gateway-web,https://microservice-gateway-web-public
    connectionStrings:
      administrationService: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
      saasService: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"    
    authServer:
      authority: http://microservice-authserver
      requireHttpsMetadata: "false"
      swaggerClientId: WebGateway_Swagger
    remoteServices:
      abpIdentityBaseUrl: https://microservice-identity
      useCurrentToken: "false"   
    dotnetEnv: Production
    redisHost: microservice-redis
    rabbitmqHost: microservice-rabbitmq
    elasticsearchUrl: http://microservice-elasticsearch
  synchedCommunication: # Used for server-to-server (client-credentials) communication with identityService for user permissions
    authority: https://microservice-authserver
    requireHttps: "false"
    validateIssuerName: "false"
    validateEndpoints: "false"
  ingress:
    host: microservice-administration
  image:
    repository: registry.digitalocean.com/microservice/microservice-service-administration
    tag: latest

# saas-service sub-chart override
saas:
  config:
    selfUrl: https://microservice-saas
    corsOrigins: https://microservice-gateway-web
    connectionStrings:
      administrationService: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
      saasService: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
    authServer:
      authority: http://microservice-authserver
      requireHttpsMetadata: "false"
      swaggerClientId: WebGateway_Swagger
    dotnetEnv: Production
    redisHost: microservice-redis
    rabbitmqHost: microservice-rabbitmq
    elasticsearchUrl: http://microservice-elasticsearch
  ingress:
    host: microservice-saas
  image:
    repository: registry.digitalocean.com/microservice/microservice-service-saas
    tag: latest  

# product-service sub-chart override
product:
  config:
    selfUrl: https://microservice-product
    corsOrigins: https://microservice-gateway-web,https://microservice-gateway-web-public
    connectionStrings:
      productService: "Server=microservice-sqldb,1433;Database=Microservice_ProductService;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
      administrationService: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
      saasService: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
    authServer:
      authority: http://microservice-authserver
      requireHttpsMetadata: "false"
      swaggerClientId: WebGateway_Swagger
    dotnetEnv: Production
    redisHost: microservice-redis
    rabbitmqHost: microservice-rabbitmq
    elasticsearchUrl: http://microservice-elasticsearch
  ingress:
    host: microservice-product
  image:
    repository: registry.digitalocean.com/microservice/microservice-service-product
    tag: latest

# gateway-web sub-chart override
gateway-web:
  config:
    selfUrl: https://gateway-web.staging.microservice.io
    corsOrigins: https://blazor.staging.microservice.io
    globalConfigurationBaseUrl: http://microservice-gateway-web
    authServer:
      authority: http://microservice-authserver
      requireHttpsMetadata: "false"
      swaggerClientId: WebGateway_Swagger
    dotnetEnv: Production
    redisHost: microservice-redis
    rabbitmqHost: microservice-rabbitmq
    elasticsearchUrl: http://microservice-elasticsearch
  reRoutes:
    accountService:
      dns: https://auth.staging.microservice.io
      schema: http
      host: microservice-authserver
      port: 80
    identityService:
      dns: https://microservice-identity
      schema: http
      host: microservice-identity
      port: 80
    administrationService:
      dns: https://microservice-administration
      schema: http
      host: microservice-administration
      port: 80
    saasService:
      dns: https://microservice-saas
      schema: http
      host: microservice-saas
      port: 80
    productService:
      dns: https://microservice-product
      schema: http
      host: microservice-product
      port: 80  
  ingress:
    host: gateway-web.staging.microservice.io
    tlsSecret: staging.microservice.io
  image:
    repository: registry.digitalocean.com/microservice/microservice-gateway-web
    tag: latest
    pullPolicy: IfNotPresent

# gateway-web-public sub-chart override
gateway-web-public:
  config:
    selfUrl: https://microservice-gateway-web-public
    globalConfigurationBaseUrl: http://microservice-gateway-web-public
    authServer:
      authority: http://microservice-authserver
      requireHttpsMetadata: "false"
      swaggerClientId: WebGateway_Swagger
      swaggerClientSecret: "1q2w3e*"
    dotnetEnv: Production 
    redisHost: microservice-redis
    rabbitmqHost: microservice-rabbitmq
    elasticsearchUrl: http://microservice-elasticsearch
  reRoutes:
    accountService:
      dns: https://microservice-authserver
      schema: http
      host: microservice-authserver
      port: 80
    saasService:
      dns: https://microservice-saas
      schema: http
      host: microservice-saas
      port: 80
    administrationService:
      dns: https://microservice-administration
      schema: http
      host: microservice-administration
      port: 80
    productService:
      dns: https://microservice-product
      schema: http
      host: microservice-product
      port: 80  
  ingress:
    host: microservice-gateway-web-public
    # tlsSecret: microservice-tls
  image:
    repository: registry.digitalocean.com/microservice/microservice-gateway-web-public
    tag: latest
    pullPolicy: IfNotPresent

kibana:
  config:
    elasticsearchUrl: http://microservice-elasticsearch:9200

# dbmigrator sub-chart override
dbmigrator:
  config:
    connectionStrings:
      identityService: "Server=microservice-sqldb,1433;Database=Microservice_Identity;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
      administrationService: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
      saasService: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
      productService: "Server=microservice-sqldb,1433;Database=Microservice_ProductService;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
    dotnetEnv: Production
    elasticsearchUrl: http://microservice-elasticsearch
    redisHost: microservice-redis
  openIdDict:
    applications:
      authServerRootUrl: https://microservice-authserver/
      angularRootUrl: https://microservice-angular/
      webRootUrl: https://microservice-web/
      blazorRootUrl: https://blazor.staging.microservice.io/
      blazorServerRootUrl: https://microservice-blazor-server/
      publicWebRootUrl: https://microservice-public-web/
      webGatewayRootUrl: https://microservice-gateway-web/
      publicWebGatewayRootUrl: https://microservice-gateway-web-public/ 
    resources:  
      accountServiceRootUrl: https://microservice-authserver/
      administrationServiceRootUrl: https://microservice-administration/
      identityServiceRootUrl: https://microservice-identity/
      saasServiceRootUrl: https://microservice-saas/
      productServiceRootUrl: https://microservice-product/
  image:
    repository: registry.digitalocean.com/microservice/microservice-db-migrator
    tag: latest
    pullPolicy: IfNotPresent  

# Public Web application sub-chart override
publicweb:
  config:
    selfUrl: https://public-web.staging.microservice.io
    gatewayUrl: http://microservice-gateway-web-public
    authServer:
      authority: https://auth.staging.microservice.io # should be domain on production (https://auth.mydomain.com)
      requireHttpsMetadata: "false"
      isOnK8s: "true"
      metaAddress: http://microservice-authserver # should be container on production (http://microservice-az-authserver)
    dotnetEnv: Production 
    redisHost: microservice-redis
    rabbitmqHost: microservice-rabbitmq
    elasticsearchUrl: http://microservice-elasticsearch
  ingress:
    host: public-web.staging.microservice.io
    tlsSecret: staging.microservice.io
  image:
    repository: registry.digitalocean.com/microservice/microservice-app-public-web
    tag: latest
    pullPolicy: IfNotPresent

blazor:
  config:
    selfUrl: https://blazor.staging.microservice.io
    gatewayUrl: https://gateway-web.staging.microservice.io
    authServer:
      authority: https://auth.staging.microservice.io
      requireHttpsMetadata: "false"
      responseType: "code"
  ingress:
    host: blazor.staging.microservice.io
    tlsSecret: staging.microservice.io
  image:
    repository: registry.digitalocean.com/microservice/microservice-app-blazor
    tag: latest
    pullPolicy: IfNotPresent
# Default values for Microservice.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

replicaCount: 1

It starts to working now, but I got following question:

In the dbmigrator log I got successful report:

[19:37:09 INF] Migrating Host side...
[19:37:10 INF] Migrating SaasService database...
[19:37:19 INF] Migrating AdministrationService database...
[19:37:25 INF] Migrating IdentityService database...
[19:37:29 INF] Migrating ProductService database...
[19:37:29 INF] All databases have been successfully migrated (HOST).
[19:37:36 INF] Migrating tenants...
[19:37:36 INF] Migration completed!

But the identity log says:

[19:44:43 INF] Saving external localizations...
[19:44:43 INF] Lock is acquired for db migration and seeding on database named: IdentityService...
[19:44:46 INF] Completed to save external localizations.
[19:44:47 INF] Lock is released for db migration and seeding on database named: IdentityService...
[19:44:47 INF] Seeding IdentityServer data...
[19:44:49 ERR] Exception of type 'Volo.Abp.BusinessException' was thrown.
[19:44:49 WRN] BusinessException has been thrown. The operation will be tried 2 times more. Exception:
Exception of type 'Volo.Abp.BusinessException' was thrown.
[19:45:01 INF] Seeding IdentityServer data...
[19:45:01 ERR] Exception of type 'Volo.Abp.BusinessException' was thrown.
[19:45:01 WRN] BusinessException has been thrown. The operation will be tried 1 times more. Exception:
Exception of type 'Volo.Abp.BusinessException' was thrown.
[19:45:12 INF] Seeding IdentityServer data...
[19:45:12 ERR] Exception of type 'Volo.Abp.BusinessException' was thrown.
[19:45:12 FTL] Microservice.IdentityService.HttpApi.Host terminated unexpectedly!
Volo.Abp.AbpInitializationException: An error occurred during the initialize Volo.Abp.Modularity.OnPostApplicationInitializationModuleLifecycleContributor phase of the module Microservice.IdentityService.IdentityServiceHttpApiHostModule, Microservice.IdentityService.HttpApi.Host, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: Exception of type 'Volo.Abp.BusinessException' was thrown.. See the inner exception for details.
 ---> Volo.Abp.BusinessException: Exception of type 'Volo.Abp.BusinessException' was thrown.
   at Microservice.IdentityService.DbMigrations.OpenIddictDataSeeder.CreateApplicationAsync(String name, String type, String consentType, String displayName, String secret, List`1 grantTypes, List`1 scopes, List`1 redirectUris, List`1 postLogoutRedirectUris, List`1 permissions, String clientUri, String logoUri) in /src/services/identity/src/Microservice.IdentityService.HttpApi.Host/DbMigrations/OpenIddictDataSeeder.cs:line 422

Any ideas?

Kind regards, Carl

0

gterdem created 6 months ago

Support Team Senior .NET Developer

The problem seems to be related to your OpenIddictDataSeeder located under the IdentityService.

Normally, you can choose either running the DbMigrator to migrate all the databases or let each database migrate themselves.

If you want to fix the issue, it seems related to the CreateApplicationAsync of the OpenIddictDataSeeder. You may have forgotten to override the OpenIddict values of IdentityService appsettings.json.

0

carl.hallqvist created 6 months ago

Thanks gterdem,

I think that I got most of if working now. Great support! :-)

9 Answer(s)

public-web sub-chart override