Is a Culture Bomb the only way to steal territory from another civ in Civ5?

No, there are three possible ways to steal territory:

• Culture Bomb with a Great Artist (Vanilla game only)
  or
  Build a Citadel with a Great General (Gods and Kings ExPack only, see PhysicalEd's answer)

• The Traditional Method - just declare war and then capture whichever of their cities "owns" the tile in question.

• The Diplomatic Method - convince the Civ in question to give you whichever of their cities "owns" the tile in question as part of a trade or peace agreement. You may find this method easy to achieve if using the Traditional method on some of their other cities first.

With both the Traditional and Diplomatic methods once you are in control of the relevant city you can either keep the city (and so the tile), or you could raze the city so all that land becomes free to be taken (by anyone!) and then purchase the tile/expand culturally from your nearest city.

The serious point here is that once a city unlocks a tile it belongs to that city for the rest of time, regardless of who owns the city, unless the city is destroyed in which case the tiles become "free" for other cities to take ownership.

The culture bomb (or building a citadel in the G&K expansion) is the only exception to this. Doing this captures all of the tiles around the tile where the effect is triggered, and this will take tiles away from any player or city state that had owned them (and I guess it gives them to the nearest city of the bombing player?).

Unlike Civ4, you cannot override a neighbour's borders by racking up a massive culture score in a bordering city.

There is another way. The AI (Attila) just did it to me. Maybe it's new in the expansion.

Use a Great General to build a Citadel. When a Citadel is built, it claims the tile it is on and one tile around it. The citadel can be built on land belonging to the civilization building it, or on unowned land. This is well documented in the Civipedia.

So Attila was pissed at me for founding a city north of one of his. He moved a general up to unclaimed land right outside my border (and right next to my mine). He built a citadel, extending his territory to take my mine (and a couple more tiles) away. The event is reported as "so and so has stolen some of your land with a Great General". He then moved another great general up into the newly claimed space and built another citadel right next to the one he had just built, taking more of my land. I don't think there is any way to stop this other than going to war.

This is correct from a Culture standpoint. Tiles claimed by a civ are permanently theirs except if a "Culture Bomb" is used. You can leverage this in your favor by preemptively purchasing tiles that will be in contention later.

Of course, the opposing civ will likely contact you and be pissed about your land purchases right near their borders, but that's just the price you pay!