Should I train all combat skills a bit or just one skill a lot?

Leveling up your skills tied to combat will increase your overall combat level a detailed description on what this does and the benefits can be found here

But for the sake of ease I will highlight the important bits here

which skills and the formula used to calculate the combat level:

• Attack and Strength or Ranged or Magic
• Defence
• Constitution
• Prayer
• Summoning (not applied toward combat level in free worlds)

The effect of your combat level:

Normally, aggressive NPC's will stop attacking a player if the player has double the NPC's combat level +1. There are exceptions, however - for example, some monsters in the Wilderness that are normally not aggressive will be aggressive to players no matter what their combat level is. Also, normally aggressive monsters that are level 69 or above will always attack a player, since the maximum combat level for players is 138.

So to answer your question, no it is not needed if you want to but it can be recommended as it does help you.

I would suggest to take a look at this OSRS Combat Calc. As this tool will help you to determine what level for which combat level you need. So using this tool you can easily plan ahead on which combat level to lvl up first.