What is the "base class" for C# numeric value types?

The answer is: you don't need to provide overloads for ALL the numeric types, just for Double and Decimal. All others (except maybe some very unusually large ones) will be automatically converted to these.

Not a base class but in fact that was the red herring. The base class System.ValueType doesn't help much as it includes types that are not numerics. The language reference i was reading was what got me confused in the first place :)

(I was just looking for who to attribute the answer to and it was a combination of Scarecrow and Marc Gravell, but since they were comments i have put the answer here)

There isn't one (or at least, not one that just means "numbers"). You could use:

void Foo<T>(T value) where T : struct {...}

But that allows any struct - not just numbers. If you want to do arithmetic, generic operators may be of use. Other than that; overloads it the most viable option.

What I do:

 public interface INumeric<T>
 {
     T Zero { get; }
     T One { get; }
     T MaxValue { get; }
     T MinValue { get; }
     T Add(T a, T b);
     // T Substract(....
     // T Mult...
 }  

 public struct Numeric: 
     INumeric<int>, 
     INumeric<float>,
     INumeric<byte>,
     INumeric<decimal>,
     // INumeric<other types>
 {
     int INumeric<int>.Zero => 0;
     int INumeric<int>.One => 1;
     int INumeric<int>.MinValue => int.MinValue;
     int INumeric<int>.MaxValue => int.MaxValue;
     int INumeric<int>.Add(int x, int y) => x + y;

     // other implementations...
 }

Now, you can use it in a method:

bool IsZero<TNum, T>(TNum ops, T number) 
   where TNum : INumeric<T>
{
   return number == ops.Zero;      
}

or extension method

 public static bool IsZero<TNum, T>(this TNum ops, T number)
      where TNum : INumeric<T>
 {
      return number == ops.Zero;
 }

and in your code:

 ...
 var n = new Numeric(); // can be an static prop

 Console.WriteLine(IsZero(n, 5)); // false
 Console.WriteLine(IsZero(n, 0f)); // true
 Console.WriteLine(IsZero(n, "0")); // compiler error

or, with extension method:

 Console.WriteLine(n.IsZero(5));  // false
 Console.WriteLine(n.IsZero(0f)); // true
 Console.WriteLine(n.IsZero("0")); // compiler error

The short answer is: Numeric types are value types, hence they derive from System.ValueType. The full answer is: you should read this article from MSDN. Moreover I think that you should read C# language reference :). Value type not equals numeric type, because values types include also structures and enumerations.