Open Closed

Unit testing issue with SQLite #913


User avatar
0
lalitChougule created

Check the docs before asking a question: https://docs.abp.io/en/commercial/latest/ Check the samples, to see the basic tasks: https://docs.abp.io/en/commercial/latest/samples/index The exact solution to your question may have been answered before, please use the search on the homepage.

  • ABP Framework version: v3.0.4
  • UI type: Angular
  • DB provider: EF Core
  • Tiered (MVC) or Identity Server Separated (Angular): no / yes
  • Exception message and stack trace: Yes
  • Steps to reproduce the issue: Try to implement sum on any dataType(numeric) field query / order by on decimal fields while doing query

The issue occures while doing unit test cases are as below

  1. Sum function always gives error like mentioned below while unit testing, but working properly on production
  2. SQLite doesn't support ordering and sorting on decimal fields in query.

There are few more issue I am facing where the code works fine in Production but giving error while Unit Testing, Are there any limitation for testing ? If yes then how to overcome them ?

Please find below error log for your reference. Thanks

Exception Messages :
    System.NotSupportedException : SQLite cannot order by expressions of type 'decimal'. Convert the values to a supported type or use LINQ to Objects to order the results.
  Stack Trace: 
    SqliteQueryableMethodTranslatingExpressionVisitor.TranslateThenBy(ShapedQueryExpression source, LambdaExpression keySelector, Boolean ascending)
    QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
    RelationalQueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
    MethodCallExpression.Accept(ExpressionVisitor visitor)
    ExpressionVisitor.Visit(Expression node)
    QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
    RelationalQueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
    MethodCallExpression.Accept(ExpressionVisitor visitor)
    ExpressionVisitor.Visit(Expression node)
    QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
    Database.CompileQuery[TResult](Expression query, Boolean async)
    QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
    <>c__DisplayClass12_0`1.&lt;ExecuteAsync&gt;b__0()
    CompiledQueryCache.GetOrAddQueryCore[TFunc](Object cacheKey, Func`1 compiler)
    CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
    QueryCompiler.ExecuteAsync[TResult](Expression query, CancellationToken cancellationToken)
    EntityQueryProvider.ExecuteAsync[TResult](Expression expression, CancellationToken cancellationToken)
    EntityQueryable`1.GetAsyncEnumerator(CancellationToken cancellationToken)
    ConfiguredCancelableAsyncEnumerable`1.GetAsyncEnumerator()
    EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken)
    InvoiceAppService.GetCreditNotesForInvoiceSelectionAsync(List`1 supplierCodes) line 1412
    InvoiceAppService.ApplyInvoiceLinkedCreditNotesAsync(GetQuoteInputDto input) line 1455
    AsyncInterceptorBase.ProceedAsynchronous[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo)
    CastleAbpMethodInvocationAdapterWithReturnValue`1.ProceedAsync()
    AuthorizationInterceptor.InterceptAsync(IAbpMethodInvocation invocation)
    CastleAsyncAbpInterceptorAdapter`1.InterceptAsync[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo, Func`3 proceed)
    AsyncInterceptorBase.ProceedAsynchronous[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo)
    CastleAbpMethodInvocationAdapterWithReturnValue`1.ProceedAsync()
    AuditingInterceptor.InterceptAsync(IAbpMethodInvocation invocation)
    CastleAsyncAbpInterceptorAdapter`1.InterceptAsync[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo, Func`3 proceed)
    AsyncInterceptorBase.ProceedAsynchronous[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo)
    CastleAbpMethodInvocationAdapterWithReturnValue`1.ProceedAsync()
    ValidationInterceptor.InterceptAsync(IAbpMethodInvocation invocation)
    CastleAsyncAbpInterceptorAdapter`1.InterceptAsync[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo, Func`3 proceed)
    AsyncInterceptorBase.ProceedAsynchronous[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo)
    CastleAbpMethodInvocationAdapterWithReturnValue`1.ProceedAsync()
    UnitOfWorkInterceptor.InterceptAsync(IAbpMethodInvocation invocation)
    CastleAsyncAbpInterceptorAdapter`1.InterceptAsync[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo, Func`3 proceed)
    InvoiceApplicationTests.ApplyInvoiceLinkedCreditNotesAsync() line 451
Message: 
    System.InvalidOperationException : The LINQ expression '(GroupByShaperExpression:
    KeySelector: new { 
        StatusId = (i.StatusId), 
        Code = (t.Code)
     }, 
    ElementSelector:(EntityShaperExpression: 
        EntityType: Invoice
        ValueBufferExpression: 
            (ProjectionBindingExpression: EmptyProjectionMember)
        IsNullable: False
    )
    )
        .Sum(x => x.Amount)' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync(). See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.
  Stack Trace: 
    RelationalSqlTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
    MethodCallExpression.Accept(ExpressionVisitor visitor)
    ExpressionVisitor.Visit(Expression node)
    RelationalSqlTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
    MethodCallExpression.Accept(ExpressionVisitor visitor)
    ExpressionVisitor.Visit(Expression node)
    RelationalSqlTranslatingExpressionVisitor.Translate(Expression expression)
    RelationalProjectionBindingExpressionVisitor.Visit(Expression expression)
    RelationalProjectionBindingExpressionVisitor.VisitMemberAssignment(MemberAssignment memberAssignment)
    ExpressionVisitor.VisitMemberBinding(MemberBinding node)
    RelationalProjectionBindingExpressionVisitor.VisitMemberInit(MemberInitExpression memberInitExpression)
    MemberInitExpression.Accept(ExpressionVisitor visitor)
    ExpressionVisitor.Visit(Expression node)
    RelationalProjectionBindingExpressionVisitor.Visit(Expression expression)
    RelationalProjectionBindingExpressionVisitor.Translate(SelectExpression selectExpression, Expression expression)
    RelationalQueryableMethodTranslatingExpressionVisitor.TranslateSelect(ShapedQueryExpression source, LambdaExpression selector)
    QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
    RelationalQueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
    MethodCallExpression.Accept(ExpressionVisitor visitor)
    ExpressionVisitor.Visit(Expression node)
    QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
    RelationalQueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
    MethodCallExpression.Accept(ExpressionVisitor visitor)
    ExpressionVisitor.Visit(Expression node)
    QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
    Database.CompileQuery[TResult](Expression query, Boolean async)
    QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
    &lt;&gt;c__DisplayClass9_0`1.<Execute>b__0()
    CompiledQueryCache.GetOrAddQueryCore[TFunc](Object cacheKey, Func`1 compiler)
    CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
    QueryCompiler.Execute[TResult](Expression query)
    EntityQueryProvider.Execute[TResult](Expression expression)
    Queryable.FirstOrDefault[TSource](IQueryable`1 source)
    InvoiceAppService.GetInvoiceSummaryAsync(InvoiceSummaryInputDto input) line 221
    AsyncInterceptorBase.ProceedAsynchronous[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo)
    CastleAbpMethodInvocationAdapterWithReturnValue`1.ProceedAsync()
    AuthorizationInterceptor.InterceptAsync(IAbpMethodInvocation invocation)
    CastleAsyncAbpInterceptorAdapter`1.InterceptAsync[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo, Func`3 proceed)
    AsyncInterceptorBase.ProceedAsynchronous[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo)
    CastleAbpMethodInvocationAdapterWithReturnValue`1.ProceedAsync()
    AuditingInterceptor.InterceptAsync(IAbpMethodInvocation invocation)
    CastleAsyncAbpInterceptorAdapter`1.InterceptAsync[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo, Func`3 proceed)
    AsyncInterceptorBase.ProceedAsynchronous[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo)
    CastleAbpMethodInvocationAdapterWithReturnValue`1.ProceedAsync()
    ValidationInterceptor.InterceptAsync(IAbpMethodInvocation invocation)
    CastleAsyncAbpInterceptorAdapter`1.InterceptAsync[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo, Func`3 proceed)
    AsyncInterceptorBase.ProceedAsynchronous[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo)
    CastleAbpMethodInvocationAdapterWithReturnValue`1.ProceedAsync()
    UnitOfWorkInterceptor.InterceptAsync(IAbpMethodInvocation invocation)
    CastleAsyncAbpInterceptorAdapter`1.InterceptAsync[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo, Func`3 proceed)
    InvoiceApplicationTests.GetInvoiceSummaryAsync() line 188
    --- End of stack trace from previous location where exception was thrown ---
    

10 Answer(s)
  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    hi

    You can create a DbContext in the unit test to change DecimalProperty.

    The Decimal type provides a high level of precision. If you don't need that level of precision, however, we recommend using double instead. You can use a value converter to continue using decimal in your classes.

    modelBuilder.Entity<MyEntity>()
        .Property(e => e.DecimalProperty)
        .HasConversion<double>();****
    

    https://docs.microsoft.com/en-us/ef/core/providers/sqlite/limitations#query-limitations


    For GroupByShaperExpression

    Can you share the query code?

  • User Avatar
    0
    hansmogren created

    I have a similar problem where I need to handle DateTimeOffset data and sqlite. I found this article. Where would I typically add the code? How do I create a separate DbContext for the unit tests? Today the tests use the MigrationsDbContext. I could edit the proper EntityFrameworkCore.DbContext and add code to the OnModelCreating and check if the the database is sqlite but that seems like the wrong way.

    You can create a DbContext in the unit test to change DecimalProperty.

    The Decimal type provides a high level of precision. If you don't need that level of precision, however, we recommend using double instead. You can use a value converter to continue using decimal in your classes.

    modelBuilder.Entity<MyEntity>() 
    
    .Property(e => e.DecimalProperty) 
    .HasConversion<double>();****
    
  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    hi @all

    https://github.com/abpframework/abp/commit/b73f9e2c6e91251f800d5aa3a00b1edf516ed933

  • User Avatar
    0
    hansmogren created

    hi @all

    https://github.com/abpframework/abp/commit/b73f9e2c6e91251f800d5aa3a00b1edf516ed933

    Thanks,

    After trying this I get this error when running the unit test:

    Volo.Abp.AbpInitializationException : An error occurred during the initialize Volo.Abp.Modularity.OnApplicationInitializationModuleLifecycleContributor phase of the module MyProject.MyProjectTestBaseModule, MyProject.TestBase, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: SQLite Error 1: 'no such table: AbpSettings'.. See the inner exception for details.
    ---- Microsoft.Data.Sqlite.SqliteException : SQLite Error 1: 'no such table: AbpSettings'.
    

    And what about MyProjectNameEntityFrameworkCoreTestModule.CreateDatabaseAndGetConnection? Should it use the MyProjectNameMigrationsDbContext, not the new TestDbContext?

    private static SqliteConnection CreateDatabaseAndGetConnection()
    {
        var connection = new SqliteConnection("Data Source=:memory:");
        connection.Open();
        var options = new DbContextOptionsBuilder<MyProjectNameMigrationsDbContext>()
            .UseSqlite(connection)
            .Options;
        using (var context = new MyProjectNameMigrationsDbContext(options))
        {
            context.GetService<IRelationalDatabaseCreator>().CreateTables();
        }
        return connection;
    }
    
  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    hi hansmogren

    You should use MyProjectNameMigrationsDbContext in CreateDatabaseAndGetConnection method.

    I'm using the free template, What is your app type? tiered or separate-tenant-schema?

  • User Avatar
    0
    hansmogren created

    hi hansmogren

    You should use MyProjectNameMigrationsDbContext in CreateDatabaseAndGetConnection method.

    Ok! I don't understand why the migrations context is used for testing, but anyway :)

    I'm using the free template, What is your app type? tiered or separate-tenant-schema?

    My solution is based on the commercial startup template. It's a monolith, not tiered, and no separate tenants.

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    hi @hansmogren

    Please create a demo project and send it to me, I will change it just like the free template. liming.ma@volosoft.com

  • User Avatar
    0
    ServiceBot created
    Support Team Automatic process manager

    This question has been automatically marked as stale because it has not had recent activity.

  • User Avatar
    0
    ademaygun created

    Hi @maliming ,

    Have you considered using the in-memory database in the .net core framework instead of sqllite? If so, why didn't you choose it?

  • User Avatar
    1
    maliming created
    Support Team Fullstack Developer

    hi

    Considering that sqlite is more similar to the actual database

    https://learn.microsoft.com/en-us/ef/core/testing/

Made with ❤️ on ABP v8.2.0-preview Updated on March 25, 2024, 15:11