in a topological space only finite subsets are compact sets

A topological space is anticompact if all compact sets are finite. In this answer Stefan H pointed out that an uncountable set with the cocountable topology is an example of an anticompact $T_1$-space which is not discrete. The real line, with the topology generated by the usual open sets and the cocountable sets, is an example of an anticompact Hausdorff space which is not discrete.

The set $\mathbb N$ of natural numbers, with the topology consisting of the sets not containing $1$ and the sets of asymptotic density $1$, is an example of a countable zero-dimensional Hausdorff space which is anticompact but not discrete.

P.S. In the last example, instead of taking the sets of asymptotic density $1$ as neighborhoods of the point $1$, we could take the sets $U$ such that $\sum_{n\notin U}\frac1n\lt\infty$; maybe this example is slightly easier or more familiar.

The examples given by bof are quite nice.

On the other hand, for $T_1$ and first countable spaces, being "anticompact" is equivalent to being discrete. (If you require compact spaces to be Hausdorff, as Bourbaki does, then "$T_1$" should be replaced by "Hausdorff").

Indeed, assume that $X$ is a $T_1$, first-countable and anticompact space. To show that $X$ is discrete, it is enough to show that any point $x\in X$ has a finite neighbourhood (then, since $X$ is $T_1$, it follows that in fact every point is isolated).

Assume that some point $x\in X$ has the property that every neighbourhood of $x$ is infinite. Let $(V_n)_{n\in\mathbb N}$ be a countable basis of neighbourhoods for $x$. Since each $V_n$ is infinite, one can construct a sequence $(x_n)\subset X$ such that the $x_n$ are pairwise distinct and $x_n\in V_n$ for all $n$. Then the sequence $(x_n)$ converges to $x$, so $K=\{ x\}\cup\{ x_n;\;n\in\mathbb N\}$ is compact. This contradicts the anticompactness of $X$.