How to calculate the number of distinct subsets of a set that has repeated elements?

Let $E_{m}(x)=\sum_{j=0}^m x^j/j!$ be the $m^{th}$ partial sum of the exponential series. If a multiset $M$ has $r$ distinct elements, where the first element is repeated $n_1$ times, the second $n_2$ times, etc, then the number of ways to choose an ordered list consisting of $k$ elements of $M$ is equal to $$ k![x^k]\prod_{i=1}^rE_{n_i}(x).\tag{*} $$ Here, $[x^k]f(x)$ denotes the coefficient of $x^k$ in the polynomial $f(x)$.

For example, consider the multiset $\{a,a,b,c\}$ from your post. There are $3$ distinct elements, the first, $a$, appearing $n_1=2$ times, and the latter two, $b$ and $c$, appearing $n_2=n_3=1$ time. The product of the partial exponential sums in $(*)$ is therefore \begin{align} E_{2}(x)\cdot E_1(x)\cdot E_1(x) &=(1+x+x^2/2)\cdot(1+x)\cdot(1+x) \\&=1+3x+\frac{7}2x^2+2x^3+\frac12x^4 \\&=1+\frac{\color{red}3}{1!}x+\frac{\color{red}7}{2!}x^2+\frac{\color{red}{12}}{3!}x^3+\frac{\color{red}{12}}{4!}x^4\end{align} Notice that the coefficients of this polynomial correspond to the answer to your combinatorial question $(3,7,12,12)$, divided by an appropriate factorial.

The techniques used in this post are more widely known as exponential generating functions. For more of an explanation on why this works, see generatingfunctionology by Herbert Wilf, especially chapter 3 on exponential generating functions. It is available for free online.

The best I can do is with generating functions.

Specifically, if we define $A(n;m_1,m_2,\ldots,m_k)$ as the number of distinct sequences of length $n$ from a multiset with $m_i$ copies of $i$ for $1 \leq i \leq k.$ Then $$\prod_{i=1}^k \sum_{p=0}^{m_i} \frac{x^p}{p!} = \sum_{n=0}^\infty A(n;m_1,m_2,\ldots,m_k) \frac{x^n}{n!}$$ which we probably can manipulate to find a recursive definition for $A.$