Open Closed

EF Core PostgreSQL exception after ABP upgrade to 4.3 #1375


0
Repunjay created
  • ABP Framework version: v4.3.0. The ABP packages were upgraded from 3.0.4 to 4.3.0
  • UI type: Angular
  • DB provider: EF Core
  • Tiered (MVC) or Identity Server Separated (Angular): yes
  • Exception message and stack trace:
  • After Upgrading to Latest framework 4.3.0 and related Volo.Abp.Identity.EntityFrameworkCore and Volo.Abp.EntityFrameworkCore.PostgreSql, We are facing issues connecting to each table. for every repository call, even if it is not a relationship and plain table, we are getting the following error

Npgsql.PostgresException (0x80004005): 42P01: relation "XXX" does not exist at Npgsql.NpgsqlConnector.<ReadMessage>g__ReadMessageLong|194_0(NpgsqlConnector connector, Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage) at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken) at Npgsql.NpgsqlDataReader.NextResult() at Npgsql.NpgsqlCommand.ExecuteReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken) at Npgsql.NpgsqlCommand.ExecuteReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken) at Npgsql.NpgsqlCommand.ExecuteReader(CommandBehavior behavior) at Npgsql.NpgsqlCommand.ExecuteDbDataReader(CommandBehavior behavior) at System.Data.Common.DbCommand.ExecuteReader() at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReader(RelationalCommandParameterObject parameterObject) at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable1.Enumerator.InitializeReader(DbContext _, Boolean result) at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.Execute[TState,TResult](TState state, Func3 operation, Func3 verifySucceeded) at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable1.Enumerator.MoveNext() Exception data: Severity: ERROR SqlState: 42P01 MessageText: relation "XXX" does not exist Position: 23 File: parse_relation.c Line: 1180 Routine: parserOpenTable

  • Steps to reproduce the issue:

36 Answer(s)
  • 0
    maliming created
    Support Team

    hi Repunjay

    Steps to reproduce the issue

    You can also create a new project that using latest version.then compare the code.

    https://docs.abp.io/en/abp/latest/Migration-Guides/Index https://docs.abp.io/en/commercial/latest/migration-guides/v4_3#identity-pro-module

  • 0
    Repunjay created

    hi Repunjay

    Steps to reproduce the issue

    You can also create a new project that using latest version.then compare the code.

    https://docs.abp.io/en/abp/latest/Migration-Guides/Index https://docs.abp.io/en/commercial/latest/migration-guides/v4_3#identity-pro-module

    Hi maliming, We tried with this and still no luck. We are able to login using ABP host credentials. However, other logins for the tenant are able to login to application but not able to access data from the services created for the application. We are getting the same error as mentioned earlier. Is there any further recommendation? Can we have a meeting to discuss and resolve it on priority as this is a blocker for us?

    Thanks

  • 0
    maliming created
    Support Team

    hi

    Can you share more logs? Including normal logs before and after the error occurred.

  • 0
    Repunjay created

    hi

    Can you share more logs? Including normal logs before and after the error occurred.

    Pasting the entire log output give max length validation erorr. Is there way to attach the log file to the ticket or I can send it to your email address, please share the email address.

  • 0
    maliming created
    Support Team
  • 0
    Repunjay created

    [email protected]

    Thanks, have sent you the log files. Please confirm once you receive and advise.

  • 0
    maliming created
    Support Team

    hi

    Object reference not set to an instance of an object.

    Is this a problem? Can you share a project that can reproduce the problem. I can check it locally wiht pg.

    2021-05-24 14:39:38.911 +05:30 [ERR] Object reference not set to an instance of an object.
    System.NullReferenceException: Object reference not set to an instance of an object.
       at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.Enumerator.Dispose()
       at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
       at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
       at SCV.Litmus.ProfileManagement.Programs.ProgramAppService.GetProgramDetails(ProgramDetailsInputDto input) in D:\Projects\SCB\core-platform\SCV.Litmus\aspnet-core\modules\profile\src\SCV.Litmus.ProfileManagement.Application\Programs\ProgramAppService.cs:line 134
       at Castle.DynamicProxy.AsyncInterceptorBase.ProceedAsynchronous[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo)
       at Volo.Ab
    
  • 0
    Repunjay created

    hi

    Object reference not set to an instance of an object.

    Is this a problem? Can you share a project that can reproduce the problem. I can check it locally wiht pg.

    2021-05-24 14:39:38.911 +05:30 [ERR] Object reference not set to an instance of an object. 
    System.NullReferenceException: Object reference not set to an instance of an object. 
       at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.Enumerator.Dispose() 
       at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection) 
       at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source) 
       at SCV.Litmus.ProfileManagement.Programs.ProgramAppService.GetProgramDetails(ProgramDetailsInputDto input) in D:\Projects\SCB\core-platform\SCV.Litmus\aspnet-core\modules\profile\src\SCV.Litmus.ProfileManagement.Application\Programs\ProgramAppService.cs:line 134 
       at Castle.DynamicProxy.AsyncInterceptorBase.ProceedAsynchronous[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo) 
       at Volo.Ab 
    

    Hi,

    I've shared the details thru email. Please check and advise. Let me if it's possible to connect virtually to debug this issue.

    Thanks..

  • 0
    maliming created
    Support Team

    hi

    Maybe this is a bug of npgsql.

    https://github.com/npgsql/efcore.pg/issues/1859

    You can create a new project, and then replace the database provider to npgsql to check whether our built-in features can be works.

  • 0
    Repunjay created

    hi

    Maybe this is a bug of npgsql.

    https://github.com/npgsql/efcore.pg/issues/1859

    You can create a new project, and then replace the database provider to npgsql to check whether our built-in features can be works.

    We are using Volo.Abp packages in our projects and there is no direct reference to 'npgsql' or any other database provider nuget package in our projects. The database provider package used in our projects is 'Volo.Abp.EntityFrameworkCore.PostgreSql'. Request you to please advise on the fix. Our developer can connect with you over call to discuss on this issue. We never encountered such issue before doing ABP upgrade.

    Please note that the github issue you mentioned is also raised by our team member who is working on migrating the projects to ABP 4.3 and .Net 5.

  • 0
    maliming created
    Support Team

    hi

    You can create a new project, and then replace the database provider to npgsql to check whether our built-in features can be works

    Can you try? This can rule out the problem with npgsql.

  • 0
    Repunjay created

    hi

    You can create a new project, and then replace the database provider to npgsql to check whether our built-in features can be works

    Can you try? This can rule out the problem with npgsql.

    We created a new project and used npgsql database provider and it works fine. But the same thing is not working in our upgraded projects.

  • 0
    maliming created
    Support Team

    hi

    Can you share a project that can be run in my local? The project you shared before can't run.

  • 0
    Repunjay created

    hi

    Can you share a project that can be run in my local? The project you shared before can't run.

    Hi maliming, It's not feasible for us to share the entire solution code as it contains 60+ projects. We can explain the issue thru screen share over a call.

  • 0
    maliming created
    Support Team

    I believe that I can see the problem remotely, but it is not convenient to solve the problem remotely, and there may be network problems.

    Can you make a minimal project that reproduces the problem?

  • 0
    Repunjay created

    I believe that I can see the problem remotely, but it is not convenient to solve the problem remotely, and there may be network problems.

    Can you make a minimal project that reproduces the problem?

    Have shared the minimal project over email which should help you in reproducing the problem. Please confirm if you have received it.

  • 0
    maliming created
    Support Team

    hi

    I think it need those project.

    <ProjectReference Include="..\..\shared\SCV.Litmus.Shared\SCV.Litmus.Shared.csproj" />
    <ProjectReference Include="..\..\shared\SCV.Litmus.Utils\SCV.Litmus.Utils.csproj" />
    

    Can you try build at your local before send to avoid other build problem? Thanks

  • 0
    maliming created
    Support Team

    hi

    I received your project, Checking.

  • 0
    maliming created
    Support Team

    hi

    Although it took me a long time to run your application, it is not easy to execute. I need to delete a lot of environment information that I don’t have, But I think I found the problem. Your application uses the wrong migration.

    The SCV.Litmus.IdentityServer has own migrations file. but the migrations in SCV.Litmus.EntityFrameworkCore.DbMigrations project is the right migration that app will use.

    Everything is works when I connectiong to the database generated by SCV.Litmus.EntityFrameworkCore.DbMigrations in the SCV.Litmus.IdentityServer.

    So please try to understand and check your migrations.

  • 0
    Repunjay created

    hi

    Although it took me a long time to run your application, it is not easy to execute. I need to delete a lot of environment information that I don’t have, But I think I found the problem. Your application uses the wrong migration.

    The SCV.Litmus.IdentityServer has own migrations file. but the migrations in SCV.Litmus.EntityFrameworkCore.DbMigrations project is the right migration that app will use.

    Everything is works when I connectiong to the database generated by SCV.Litmus.EntityFrameworkCore.DbMigrations in the SCV.Litmus.IdentityServer.

    So please try to understand and check your migrations.

    hi, The Identity server and core service is working as expected for us and we are using the correct migration. As we mentioned earlier, the problem is with the profile service. If you try to invoke SampleController of profile service, you should be able to reproduce the error we are are encountering.

  • 0
    maliming created
    Support Team

    hi @Repunjay

    I can confirm that the error is because the database used by DbContext is not generated by its migration.Even if they have the same database table.

    I can't run your project correctly, can you check it yourself?

    When you share a project that reproduces the problem, you should remove all irrelevant dependencies.

  • 0
    Repunjay created

    hi @Repunjay

    I can confirm that the error is because the database used by DbContext is not generated by its migration.Even if they have the same database table.

    I can't run your project correctly, can you check it yourself?

    When you share a project that reproduces the problem, you should remove all irrelevant dependencies.

    hi maliming, I understand that you can't run the project due to depdencies but I can confirm for sure that the issue we are encountering is not due to database migration. To give you some idea -

    1. We are able to login successfully using host login credentials.
    2. We are able to login successfully using user credentials which belongs to the tenant.
    3. #1 and #2 rules out the possibility of DB migration issue of Core/Identity database (host and tenant).
    4. The issue is with ProfileManagement service which is added as a module to the project and has separate database.
    5. The repositories for Profile DB are not created/setup properly when "tenantId" is passed in the request and it throws run-time error.
    6. However, if "tenantId" is blank or not passed, the repository is created properly but no data is fetched since all tables have "tenantId" column which filters the data based on tenantid.
    7. We have tested the APIs from Postman and have noticed similar issue while debugging code.
    8. We have also re-created Profile database by executing DB migrations but still the issue persists.
    9. Our sense is that something broke at EFCore side due to which the repositories for Profile service are not created when tenantId is passed.

    Any idea what could cause #5? It's difficult for us to create such a project for you with minimal depdencies and that is why wanted to have a call with you so that you can share any inputs to fix the issue. Kindly help and advise.

    Thanks...

  • 0
    maliming created
    Support Team

    hi

    The remote check may be affected by the network speed, we can give it a try.

    Please send me an email when you're online. [email protected]

  • 0
    Repunjay created

    hi

    The remote check may be affected by the network speed, we can give it a try.

    Please send me an email when you're online. [email protected]

    hi, As discussed during meeting, we have shared the bare minimal project without depedencies for you to reproduce the issue locally. Please your email and advise on next steps.

  • 0
    maliming created
    Support Team

    we have shared the bare minimal project without depedencies for you to reproduce the issue locally.

    I can't use the project you shared before, as I said in the meeting, you should provide a project and steps that only depend on the local PostgreSQL database.

    I remember another friend had agreed to do this in the meeting,

  • 0
    Repunjay created

    we have shared the bare minimal project without depedencies for you to reproduce the issue locally.

    I can't use the project you shared before, as I said in the meeting, you should provide a project and steps that only depend on the local PostgreSQL database.

    I remember another friend had agreed to do this in the meeting,

    Yes, please check your email. We have just shared the project that depends on local PostgreSQL database. Kindly confirm.

  • 0
    maliming created
    Support Team

    HI

    I checked the inbox,trash,spam did not receive any mail,

    [email protected]

  • 0
    Repunjay created

    HI

    I checked the inbox,trash,spam did not receive any mail,

    [email protected]

    Strange you didn't received it. May be the attachment and network connection could be causing it. Could you please check it after some time and confirm. I'll also try to send it from my personal email address.

  • 0
    Repunjay created

    HI

    I checked the inbox,trash,spam did not receive any mail,

    [email protected]

    Strange you didn't received it. May be the attachment and network connection could be causing it. Could you please check it after some time and confirm. I'll also try to send it from my personal email address.

    I've resend the email and project from my official ([email protected]) and personal ([email protected]) email addresses. Please check and confirm.

  • 0
    maliming created
    Support Team

    hi

    I got your mail I will check it asap. : )

  • 0
    maliming created
    Support Team

    hi

    Here is my steps:

    1. Replace all connection strings with my locally.
    2. Run modules\litmus-core\src\SCV.Litmus.DbMigrator\SCV.Litmus.DbMigrator
    3. Run dotnet ef database update in SCV.Litmus.ProfileManagement.HttpApi.Host project.
    4. Run SCV.Litmus.IdentityServer and SCV.Litmus.ProfileManagement.HttpApi.Host.
    5. Call https://localhost:44398/profile/api/ProfileManagement/sample get success result.
    6. Add a tenant in database.
    7. Then Call https://localhost:44398/profile/api/ProfileManagement/sample get success result.

    @Repunjay Am I miss some steps?

  • 0
    Repunjay created

    hi

    Here is my steps:

    1. Replace all connection strings with my locally.
    2. Run modules\litmus-core\src\SCV.Litmus.DbMigrator\SCV.Litmus.DbMigrator
    3. Run dotnet ef database update in SCV.Litmus.ProfileManagement.HttpApi.Host project.
    4. Run SCV.Litmus.IdentityServer and SCV.Litmus.ProfileManagement.HttpApi.Host.
    5. Call https://localhost:44398/profile/api/ProfileManagement/sample get success result.
    6. Add a tenant in database.
    7. Then Call https://localhost:44398/profile/api/ProfileManagement/sample get success result.

    @Repunjay Am I miss some steps?

    Hi maliming,

    I've sent you an email with detailed steps to reproduce the error and do further debugging. Kindly check and advise.

  • 0
    maliming created
    Support Team

    The structure of the sql you shared is different from the project I received. Please try to reproduce the problem through the code, you can share the necessary code to create the data. Please keep it simple, thanks.

  • 0
    Repunjay created

    The structure of the sql you shared is different from the project I received.
    Please try to reproduce the problem through the code, you can share the necessary code to create the data. Please keep it simple, thanks.

    I'm not sure how does it prevent you to debug the issue. You can create the Litmus_Lenovo_ProfileManagement database thru code migration by executing "update-database" command and selecting "SCV.Litmus.ProfileManagement.HttpApi.Host" project. Once the DB is created, you can setup one record in Samples table using below query -

    INSERT INTO public."Samples"( "Id", "ExtraProperties", "ConcurrencyStamp", "CreationTime", "CreatorId", "LastModificationTime", "LastModifierId", "IsDeleted", "DeleterId", "DeletionTime", "TenantId", "Value") VALUES ('e42f9b05-2e27-41f4-a562-c1c00cf49d39', '{}', '', '0001-01-01 00:00:00', null, null, null, false, null, null, 'd1be844b-d3a2-031a-f036-39f5d4380239', 123);

    The host database Litmus_Admin and tenant database Litmus_Lenovo_Admin needs to be created thru script as it has the neccessary data and tenant information which is required to reproduce the issue. We cannot create all data thru code.

  • 0
    maliming created
    Support Team

    hi @Repunjay

    Our problem may be related to migration, so we shouldn't create the host database Litmus_Admin and tenant database Litmus_Lenovo_Admin through SQL.

    We need use migrations of ef core.

  • 0
    ServiceBot created
    Support Team

    This question has been automatically marked as stale because it has not had recent activity.