Is there a notation for being "a finite subset of"?

The usual way is to use two different notations, one of which means that $A$ is finite, and the other means it's a subset of $B$. $$A\subset B,\qquad|A|<\infty.$$

There are two options:

1. You use the notation often. Then define it properly at the beginning (or when you first need it) and use whatever you think is reasonable. I'd suggest, as others: $$ A \subset_{\mathrm{fin}} B, \quad A \sqsubset B, \quad A \mathrel{\ddot{\subset}} B, \quad A \subset\!\!\!\!\!\cdot\!\!\cdot\, B \quad \ldots $$

2. You use it one or twice. Then just spell it out:

   • ... where $A\subset B$ is finite
   • ... $(\forall A \subset B, \, A\text{ finite})$
   • ...

Of course, you can use $|A|<\infty$ or $|A|<|\mathbb N|$ or $|A|<\omega$ or whatever, just try to imagine being a reader of your text and think what is the least confusing thing.

In set theory there are two standard notations for the set of finite subsets of $X$:

1. $[X]^{<\omega}$,
2. $\mathcal P_\omega(X)$ or $\mathcal P_{\aleph_0}(X)$.

In naive set theory, you can also find $\operatorname{Fin}(X)$ quite often.

So if you'd want to write that $A$ is a finite subset of $B$, you could say that $A\in[B]^{<\omega}$ or $A\in\mathcal P_\omega(B)$ or $A\in\operatorname{Fin}(B)$.

Outside of set theory, I believe writing $|A|<\infty$ is probably one of the most accepted ways of writing that $A$ is finite. But I'm sure that in some fields of mathematics there are more or less common notations, and you should probably align yourself to the crowd which will read your work.

If you are writing for yourself, then it really doesn't matter what you're using, right?

In any case, do remember the standard advice about notation:

Notation is used to reduce clutter, not to abbreviate. Define your notations and be consistent with them. Don't force the reader to keep track of your symbols, if there's no need to do so.

If I had to invent a notation for this, the most suggestive I can think of is $$ A\in \mathcal P_{<\omega}(B). $$ One could define the RHS using another notation that in fact is more or less standard in some areas: $$ \mathcal P_{<\omega}(B) = \bigcup_{n \in \Bbb N}\binom Bn $$ which notation is derived from the that for the binomial coefficient similarly to the way the Cartesian product notation $A\times B$ is derived from that for multiplication, or $Y^X=\{\,f:X\to Y\,\}$ from exponentiation.