Interview riddle

These interview problems are sometimes weird, where notations are bad, rules are arbitrary, and they expect only one answer where several could fit.

Here is one, which could be the expected one, but probably not:

To compute $a \times b$, take the numerator of $\dfrac{ab^2}{6}$ after simplification of the fraction.

I don't see how they could argue it is wrong.

Easy, just define

$$\begin{array}{rcl}a \times b &=& \hspace{10.5pt}(a-4)(b-5)(a-5)(b-6)(a-6)(b-7)(a-7)(b-8)/72 + \\&& 25(a-3)(b-4)(a-5)(b-6)(a-6)(b-7)(a-7)(b-8)/18 + \\&& 15(a-3)(b-4)(a-4)(b-5)(a-6)(b-7)(a-7)(b-8)/8 \hspace{5.25pt}+ \\&& 49(a-3)(b-4)(a-4)(b-5)(a-5)(b-6)(a-7)(b-8)/36 + \\&&\hspace{5.5pt}7(a-3)(b-4)(a-4)(b-5)(a-5)(b-6)(a-6)(b-7)/18\end{array}$$

This might be a possible solution. For a positive integer $n$, let $\nu_2(n)$ be the largest $k$ such that $2^k|n$, and similarly, let $\nu_3(n)$ be the largest $k$ such that $3^k|n$. Finally let $$h(n)=\frac{n}{3^{\nu_3(n)}2^{1+4\lfloor \nu_2(n)/4\rfloor}}$$ If we consider $$ a\times ~ b {\buildrel \rm def\over =}~b h(ab) $$ then $(k-1)\times k$ coincides with the proposed results for $k=4,5,6,7$ and yields $224$ for $k=8$.

The left-hand-side input and the right-hand-side output can be imagined as binary numbers in a kind of truth table:

All eight output bits can be calculated from the seven input bits evaluating simple Boolean expressions.

Spoiler Alert: (I use the answer given above in the response below. If you don't want to see it, you may want to skip this answer...)

I'm replacing $\times$ by $\circ$, as the latter is more commonly used with unknown operations. I hate it when people redefine a common symbol, then "$=$" to describe a relationship.

Note that $$\begin{align}3\circ4 &= 4\cdot 2\\ 4\circ 5 &= 5\cdot 10\\ 5\circ 6 &= 6\cdot 5\\ 6\circ 7 &= 7\cdot 7 \\ 7\circ 8 &= 8\cdot 28 \\ \end{align}$$

Thus, we can define: $$a\circ b\quad{\buildrel \rm def\over =}\quad b\cdot x_a$$ Where $x_n$ is some sequence. OEIS yields three possible sequences: $$x_n = \frac{\binom{n+2}{2}\gcd(n,3)}{3},\quad n \ge 0$$ (A234041) $$x_n = \text{denominatorOf}\left(\frac{(n-2)(n+3)}{(n)(n+1)}\right)\quad n \ge 3$$ (A027626: GCD of $n$-th and $(n+1)$st tetrahedral numbers, offset by me for this problem)

The last sequence from OEIS is A145911 which is not promising at all. (It's a combination of, what appears to be, $3$ other sequences.)