Are bimodules over a commutative ring always modules?

As Zhen Lin comments above, the converse is not true. For another example, suppose $S$ is a ring containing $R$ as a commutative subring such that $R$ is not in the center of $S$. Then $S$ is naturally an $(R,R)$-bimodule using the ring multiplication on $S$. But if $R$ is not in the center of $S$ then there exists $r \in R$, $s \in S$ such that $rs \neq sr$. So the left and right actions don't agree.

For a simple example, take $R$ to be the subring of diagonal matrices in the matrix ring $\mathbb{M}_n(k)$ for a field $k$ and $n \geq 2$.

Supplementing other answers and comments: first, I would argue against thinking that "left" and "right" have genuine content, but I would argue in favor of thinking of these as notational artifacts of our left-to-right writing system, etc. (I am reminded again of Herstein's advocacy of writing functions on the right of their arguments, at least in English.)

E.g., an $R,S$-bimodule's main property is that the $R$-action and $S$-action commute with each other, and that the order-of-multiplication in $S$ is "backward", so (in English, with left-to-right conventions) an $R,S$-bimodule is equivalently a ("left") $R\otimes S^{\rm opp}$-module, with the "opposite" ring.

It is true that sometimes the notational left and right are mnemonically helpful, but their content should not be over-estimated.

"Even" in looking at $Hom_?(M,N)$ with non-commutative $R$, $S,R$-bimodule $M$, and $R,T$-bimodule $N$, the "left/right" structures might better be called pre-composition and/or post-composition and such, refering to the more basic convention of order of composition of functions: those closest the argument are applied first. Luckily, the $M\otimes_R N$ structures are more directly correctly suggested by left-right conventions, but, still, can be converted to other expressions, as noted above.

Edit: forgot to emphasize that associativity is, or should, if done right, built into all these set-ups. So the $(rm)s=r(ms)$ principle ought not be something one is worrying about in the face of all the other issues.