Is Gwent really difficult until you hoard up all the cards?

Basically, yes, you need a better deck to beat better opponents.

The starting deck is not really powerful without a lot of surprise cards. You don't have a lot of powerful spells or heroes to really overpower an opponent.

In the beginning, your goal should be to do a quest for zoltan where you receive 3 heroes which you can keep (you can also get a bit of money, but I highly recommend the cards!). After that, you start beating the guys which are your quest objectives. Mostly, these are innkeeper which give you powerful heroes (I highly recommend beating the innkeeper in oxenfurt, he gives you a yennefer card, 7 strenght, can revive someone from the graveyard) or good spell cards. Also, you might be able to purchase some cards from merchants.

Some tips:

• If you have multiple spys on your hand, spam them in the first round and let the enemy win. You get more cards while your enemy loses some, making it easier to win the 2 other rounds.
• Do not play your high cards early. If you play your 10 strength hero early and your enemy counters with 2 catapults and horn of command, you wasted it. Only play heroes to finish an enemy with a high number.
• Keep special effects in mind. Playing a 6 strength card might be good in the first round if you can revive it with another card.
• Weather effects. Blocking a unit type while you have none of them is a good idea. For example: If you have no melees, you can play a frost card and passing, it prevents the enemy from playing high-strength melees himself, which might force him to pass.

Just adding to the accepted answer. You can, with a bit of strategy and luck, defeat merchants with your starting deck. If it's getting too hard though, you can purchase cards from some merchants. Each time you defeat a merchant/blacksmith, they give you a card. The card may not always be from the faction you were playing, but every they stack up eventually to the point where you can build an entire deck out of the cards that you have won. Just a thing to keep in mind, one opponent only gives you one card, ever, and blacksmiths are harder to beat than merchants.

Then again, as mentioned, you'll have quests to play innkeeps as well as some other people who'll give you their special cards. If you can defeat them, that is. So, don't give up easily, it can be a lot of fun!

You have to start in the lower level area's. There are the opponents the easiest, just win as much games as you can and you will get better cards along the way, so you can face stronger opponents.