Get SQL code from an Entity Framework Core IQueryable<T>

c# entity-framework-core

I am using Entity Framework Core and I need to see which SQL code is being generated. In previous versions of Entity Framework I could use the following:

string sql = ((System.Data.Objects.ObjectQuery)query).ToTraceString();

Where query is an IQueryable object ... But ToTraceString is not available in EF Core.

How can I do something similar in EF Core?


Solution 1:

EF core 5/6 / Net 5/6

query.ToQueryString()

See Documentation ToQueryString() and What's New in EF Core 5.0

var query = _context.Widgets.Where(w => w.IsReal && w.Id == 42);  
var sql = query.ToQueryString();

For older net core frameworks an Extension can be used.

Core 2.1.2


using System.Linq;
using System.Reflection;
using Microsoft.EntityFrameworkCore.Query;
using Microsoft.EntityFrameworkCore.Query.Internal;
using Microsoft.EntityFrameworkCore.Query.Expressions;
using Microsoft.EntityFrameworkCore.Query.Sql;
using static Microsoft.EntityFrameworkCore.DbLoggerCategory;

    public static class QueryableExtensions
    {
        private static readonly TypeInfo QueryCompilerTypeInfo = typeof(QueryCompiler).GetTypeInfo();
    
        private static readonly FieldInfo QueryCompilerField = typeof(EntityQueryProvider).GetTypeInfo().DeclaredFields.First(x => x.Name == "_queryCompiler");
        private static readonly FieldInfo QueryModelGeneratorField = typeof(QueryCompiler).GetTypeInfo().DeclaredFields.First(x => x.Name == "_queryModelGenerator");
        private static readonly FieldInfo DataBaseField = QueryCompilerTypeInfo.DeclaredFields.Single(x => x.Name == "_database");
        private static readonly PropertyInfo DatabaseDependenciesField = typeof(Database).GetTypeInfo().DeclaredProperties.Single(x => x.Name == "Dependencies");
    
        public static string ToSql<TEntity>(this IQueryable<TEntity> query)
        {
            var queryCompiler = (QueryCompiler) QueryCompilerField.GetValue(query.Provider);
            var queryModelGenerator = (QueryModelGenerator)QueryModelGeneratorField.GetValue(queryCompiler);
            var queryModel = queryModelGenerator.ParseQuery(query.Expression);
            var database = DataBaseField.GetValue(queryCompiler);
            var databaseDependencies = (DatabaseDependencies) DatabaseDependenciesField.GetValue(database);
            var queryCompilationContext = databaseDependencies.QueryCompilationContextFactory.Create(false);
            var modelVisitor = (RelationalQueryModelVisitor) queryCompilationContext.CreateQueryModelVisitor();
            modelVisitor.CreateQueryExecutor<TEntity>(queryModel);
            var sql = modelVisitor.Queries.First().ToString();
    
            return sql;
        }
    }

EF Core 3.0

        public static string ToSql<TEntity>(this IQueryable<TEntity> query)
        {
            using var enumerator = query.Provider.Execute<IEnumerable<TEntity>>(query.Expression).GetEnumerator();
            var enumeratorType = enumerator.GetType();
            var selectFieldInfo = enumeratorType.GetField("_selectExpression", BindingFlags.NonPublic | BindingFlags.Instance) ?? throw new InvalidOperationException($"cannot find field _selectExpression on type {enumeratorType.Name}");
            var sqlGeneratorFieldInfo = enumeratorType.GetField("_querySqlGeneratorFactory", BindingFlags.NonPublic | BindingFlags.Instance) ?? throw new InvalidOperationException($"cannot find field _querySqlGeneratorFactory on type {enumeratorType.Name}");
            var selectExpression = selectFieldInfo.GetValue(enumerator) as SelectExpression ?? throw new InvalidOperationException($"could not get SelectExpression");
            var factory = sqlGeneratorFieldInfo.GetValue(enumerator) as IQuerySqlGeneratorFactory ?? throw new InvalidOperationException($"could not get IQuerySqlGeneratorFactory");
            var sqlGenerator = factory.Create();
            var command = sqlGenerator.GetCommand(selectExpression);
            var sql = command.CommandText;
            return sql;
        }

see Gist from RosiOli

EF Core 3.1

using System.Linq;
using System.Reflection;
using System.Collections.Generic;
using Microsoft.EntityFrameworkCore.Query.SqlExpressions;
using Microsoft.EntityFrameworkCore.Query;

public static string ToSql<TEntity>(this IQueryable<TEntity> query) where TEntity : class
{
    using var enumerator = query.Provider.Execute<IEnumerable<TEntity>>(query.Expression).GetEnumerator();
    var relationalCommandCache = enumerator.Private("_relationalCommandCache");
    var selectExpression = relationalCommandCache.Private<SelectExpression>("_selectExpression");
    var factory = relationalCommandCache.Private<IQuerySqlGeneratorFactory>("_querySqlGeneratorFactory");

    var sqlGenerator = factory.Create();
    var command = sqlGenerator.GetCommand(selectExpression);

    string sql = command.CommandText;
    return sql;
}

private static object Private(this object obj, string privateField) => obj?.GetType().GetField(privateField, BindingFlags.Instance | BindingFlags.NonPublic)?.GetValue(obj);
private static T Private<T>(this object obj, string privateField) => (T)obj?.GetType().GetField(privateField, BindingFlags.Instance | BindingFlags.NonPublic)?.GetValue(obj);

The issue is also tracked by the EF net core team and is scheduled for the next release.

Solution 2:

This answer is for EF Core 2.1.

For EF Core 3.0 and 3.1 see the @Thom Kiesewetter's answer

For EF Core 5 there will be built-in method ToQueryString() used on IQueryable<>

Since EF 7 is renamed to Entity Framework Core I will summarize you the options for EF Core.

There are 3 approaches for logging SQL statements from IQueryable<>:

  • Using Built-in or Custom Logging. Logging the executing query using your logger of choice or the built-in Logger in .NET Core as mentioned in this tutorial.
  • Using a Profiler. Using an SQL Profiler like MiniProfiler to monitor the executing query.
  • Using Crazy Reflection Code. You can implement some custom reflection code similar to the older approach to perform the same basic concept.

Here is the crazy reflection code (extension method):

public static class IQueryableExtensions
{
    private static readonly TypeInfo QueryCompilerTypeInfo = typeof(QueryCompiler).GetTypeInfo();

    private static readonly FieldInfo QueryCompilerField = typeof(EntityQueryProvider).GetTypeInfo().DeclaredFields.First(x => x.Name == "_queryCompiler");

    private static readonly FieldInfo QueryModelGeneratorField = QueryCompilerTypeInfo.DeclaredFields.First(x => x.Name == "_queryModelGenerator");

    private static readonly FieldInfo DataBaseField = QueryCompilerTypeInfo.DeclaredFields.Single(x => x.Name == "_database");

    private static readonly PropertyInfo DatabaseDependenciesField = typeof(Database).GetTypeInfo().DeclaredProperties.Single(x => x.Name == "Dependencies");

    public static string ToSql<TEntity>(this IQueryable<TEntity> query) where TEntity : class
    {
        var queryCompiler = (QueryCompiler)QueryCompilerField.GetValue(query.Provider);
        var modelGenerator = (QueryModelGenerator)QueryModelGeneratorField.GetValue(queryCompiler);
        var queryModel = modelGenerator.ParseQuery(query.Expression);
        var database = (IDatabase)DataBaseField.GetValue(queryCompiler);
        var databaseDependencies = (DatabaseDependencies)DatabaseDependenciesField.GetValue(database);
        var queryCompilationContext = databaseDependencies.QueryCompilationContextFactory.Create(false);
        var modelVisitor = (RelationalQueryModelVisitor)queryCompilationContext.CreateQueryModelVisitor();
        modelVisitor.CreateQueryExecutor<TEntity>(queryModel);
        var sql = modelVisitor.Queries.First().ToString();

        return sql;
    }
}

After adding this extension method to your code, you can use the method as follows:

// Build a query using Entity Framework
var query = _context.Widgets.Where(w => w.IsReal && w.Id == 42);  
// Get the generated SQL
var sql = query.ToSql();

Referral: http://rion.io/2016/10/19/accessing-entity-framework-core-queries-behind-the-scenes-in-asp-net-core/ and https://gist.github.com/rionmonster/2c59f449e67edf8cd6164e9fe66c545a

Solution 3:

For anyone just trying to diagnose a one-off misfiring EF Core query or the like and not wanting to change their code, there are a couple of options:

Use SQL Server Management Studio (SSMS) SQL Profiler

If you've got SQL Server Management Studio (SSMS) installed you can just fire up the SQL Profiler from the Tools menu in SSMS:

SQL Profiler option in the Tools menu in SQL Server Management Studio (SSMS)

And then start a new trace running in SQL Profiler once it opens.

You'll then be able to see the incoming SQL request from EF, they are generally pretty well formed and easy to read.

Check the Output Window in Visual Studio

In my copy of VS2019, using EF2.2 I can change the output window to show the output from the Web Server (select the name of your app and web server in the "Show output from" combo at the top of the Output pane) and the outgoing SQL is also shown in there. I've checked my code and as far as I can see I haven't done anything to enable that, so I think it must do this by default:

enter image description here

If you want to see the parameters sent to SQL server in the queries you can switch that on when setting up the DBContext with the EnableSensitiveDataLogging method, e.g.

services.AddDbContext<FusionContext>(options => options
    .UseSqlServer(connectionString))
    //.EnableDetailedErrors()
    .EnableSensitiveDataLogging()

@Tich -- Lil3p mentions in the comments that they also needed to use a switch to turn on SQL Debugging in the Debug tab of the project's Properties page (which sets "sqlDebugging": true in LaunchSettings.json). I checked and I haven't got that switched on for any of my projects, but that may be worth experimenting with too if the above isn't working for you.

Solution 4:

My take based on @nikolay-kostov answer.

The difference is that I get the SQL command with parameters extracted instead of hard coded which is more in line with how EF Core send commands to the database. Also, if you want to edit and send the command to the database, it is a better practice to use parameters.

    private static class IQueryableUtils 
    {
        private static readonly TypeInfo QueryCompilerTypeInfo = typeof(QueryCompiler).GetTypeInfo();

        private static readonly FieldInfo QueryCompilerField = typeof(EntityQueryProvider).GetTypeInfo().DeclaredFields.First(x => x.Name == "_queryCompiler");

        private static readonly FieldInfo QueryModelGeneratorField = QueryCompilerTypeInfo.DeclaredFields.First(x => x.Name == "_queryModelGenerator");
        private static readonly FieldInfo queryContextFactoryField = QueryCompilerTypeInfo.DeclaredFields.First(x => x.Name == "_queryContextFactory");
        private static readonly FieldInfo loggerField = QueryCompilerTypeInfo.DeclaredFields.First(x => x.Name == "_logger");
        private static readonly FieldInfo DataBaseField = QueryCompilerTypeInfo.DeclaredFields.Single(x => x.Name == "_database");

        private static readonly PropertyInfo DatabaseDependenciesField = typeof(Database).GetTypeInfo().DeclaredProperties.Single(x => x.Name == "Dependencies");

        public static (string sql, IReadOnlyDictionary<string, object> parameters) ToSql<TEntity>(IQueryable<TEntity> query) where TEntity : class
        {
            var queryCompiler = (QueryCompiler)QueryCompilerField.GetValue(query.Provider);
            var queryContextFactory = (IQueryContextFactory)queryContextFactoryField.GetValue(queryCompiler);
            var logger = (Microsoft.EntityFrameworkCore.Diagnostics.IDiagnosticsLogger<DbLoggerCategory.Query>)loggerField.GetValue(queryCompiler);
            var queryContext = queryContextFactory.Create();
            var modelGenerator = (QueryModelGenerator)QueryModelGeneratorField.GetValue(queryCompiler);
            var newQueryExpression = modelGenerator.ExtractParameters(logger, query.Expression, queryContext);
            var queryModel = modelGenerator.ParseQuery(newQueryExpression);
            var database = (IDatabase)DataBaseField.GetValue(queryCompiler);
            var databaseDependencies = (DatabaseDependencies)DatabaseDependenciesField.GetValue(database);
            var queryCompilationContext = databaseDependencies.QueryCompilationContextFactory.Create(false);
            var modelVisitor = (RelationalQueryModelVisitor)queryCompilationContext.CreateQueryModelVisitor();

            modelVisitor.CreateQueryExecutor<TEntity>(queryModel);
            var command = modelVisitor.Queries.First().CreateDefaultQuerySqlGenerator()
                .GenerateSql(queryContext.ParameterValues);

            return (command.CommandText, queryContext.ParameterValues);
        }
    }

Solution 5:

Entity Framework Core 3.x

You can get it through logging.

Create the factory:

var loggerFactory = LoggerFactory.Create(builder =>
{
    builder
    .AddConsole((options) => { })
    .AddFilter((category, level) =>
        category == DbLoggerCategory.Database.Command.Name
        && level == LogLevel.Information);
});

Tell the DbContext which factory to use:

optionsBuilder.UseLoggerFactory(_loggerFactory);

From this post

You can get more information if you want to implement ILogger:

public class EntityFrameworkSqlLogger : ILogger
{
    #region Fields
    Action<EntityFrameworkSqlLogMessage> _logMessage;
    #endregion
    #region Constructor
    public EntityFrameworkSqlLogger(Action<EntityFrameworkSqlLogMessage> logMessage)
    {
        _logMessage = logMessage;
    }
    #endregion
    #region Implementation
    public IDisposable BeginScope<TState>(TState state)
    {
        return default;
    }
    public bool IsEnabled(LogLevel logLevel)
    {
        return true;
    }
    public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception exception, Func<TState, Exception, string> formatter)
    {
        if (eventId.Id != 20101)
        {
            //Filter messages that aren't relevant.
            //There may be other types of messages that are relevant for other database platforms...
            return;
        }
        if (state is IReadOnlyList<KeyValuePair<string, object>> keyValuePairList)
        {
            var entityFrameworkSqlLogMessage = new EntityFrameworkSqlLogMessage
            (
                eventId,
                (string)keyValuePairList.FirstOrDefault(k => k.Key == "commandText").Value,
                (string)keyValuePairList.FirstOrDefault(k => k.Key == "parameters").Value,
                (CommandType)keyValuePairList.FirstOrDefault(k => k.Key == "commandType").Value,
                (int)keyValuePairList.FirstOrDefault(k => k.Key == "commandTimeout").Value,
                (string)keyValuePairList.FirstOrDefault(k => k.Key == "elapsed").Value
            );
            _logMessage(entityFrameworkSqlLogMessage);
        }
    }
    #endregion
}

Related

Remove all line breaks from a long string of text
How to delete an enum type value in postgres?
How to position a table at the center of div horizontally & vertically
How can I send large messages with Kafka (over 15MB)?
Flutter: Setting the height of the AppBar
How can I install a previous version of Python 3 in macOS using homebrew?
How to draw border around a UILabel?
Can I use complex HTML with Twitter Bootstrap's Tooltip?
json.net has key method?
"for loop" with two variables? [duplicate]
jQuery using append with effects
Create database from command line