C# Casting implementation of (interface with generic argument that is also an interface) results to InvalidCastException

First in your declaration

IAnimalHandler<IAnimal>

IAnimal it's the name for a generic type parameter, not the reference to the IAnimal interface. It's confusing. Just redefine in the following way:

public interface IAnimalHandler<out T> where T : IAnimal { }

out modifier is required to be able to perform an assignment operation:

IAnimalHandler<Dog> -> IAnimalHandler<IAnimal>

Read the following article if you would like to know more about covariance in c#:

https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/covariance-contravariance/

The casting is not working because IAnimalHandler<Dog> cannot be stored in IAnimalHandler<IAnimal>. Let's see an example.

IAnimal dog = new Dog();

// Not working
IAnimalHandler<IAnimal> handler = new DogHandler();

public interface IAnimal { }

public class Dog : IAnimal { }
public class Cat : IAnimal { }


public interface IAnimalHandler<T> where T : IAnimal {
    public void Handle(T animal);
}

public class DogHandler : IAnimalHandler<Dog>
{
    public void Handle(Dog animal)
    {
        // Handle logic here...
    }
}

public class CatHandler : IAnimalHandler<Cat>
{
    public void Handle(Cat animal)
    {
        // Handle logic here...
    }
}

As you can see, it's not possible to assign DogHandler to the variable handler. A variable of type IAnimalHandler<IAnimal> means that the concrete underlying class should have a method with this signature public void Handle(IAnimal animal). This method would accept any type that implements IAnimal. The type DogHandler only accepts Dog in it's Handle method which is not any IAnimal implementation.

I don't know exactly what you are trying to do with this, but here's an example that could satisfy your needs

using System;
using System.Linq;
using System.Collections.Generic;

var handlerMap = new Dictionary<Type, IAnimalHandler>();

AppDomain.CurrentDomain
    .GetAssemblies()
    .SelectMany(a => a.GetTypes())
    .Where(t => t.GetInterfaces().Any(i => i.Name.Contains(nameof(IAnimalHandler))))
    .ToList()
    .ForEach(t =>
    {
        IAnimalHandler instance = (IAnimalHandler)Activator.CreateInstance(t);

        handlerMap[instance.HandledType] = instance;
    });

handlerMap[typeof(Dog)].Handle(new Dog());

public interface IAnimal { }

public class Dog : IAnimal { }
public class Cat : IAnimal { }


public interface IAnimalHandler
{
    Type HandledType { get; }
    public void Handle(IAnimal animal);
}

public class DogHandler : IAnimalHandler
{
    public Type HandledType => typeof(Dog);

    public void Handle(IAnimal animal)
    {
        if (animal is Dog dog)
        {
            // Handle dog here...
            Console.WriteLine("Dog handled");
        }
        else
        {
            throw new InvalidCastException($"Expected to handle a {typeof(Dog)} got {animal.GetType()}");
        }
    }
}

I hope it helped you!