How to stop people from settling in our area?

If you have that much manpower you may want to explore having several shifts of scouts that patrol through your borders looking for intruders/nakeds, as well as gathering resources. This would allow you to maintain an income using these people for this purpose, as well as warning your clan before they get set up enough to be tough to dig out.

I think a wall would definitely help keep people out but it is expensive. I think you should look at defenses as deterrents instead of solutions. Seeing structures or defensive structures would probably discourage some people from settling in your land as it would be likely to be opposed, and quickly at that. Use them to 'funnel' enemies to positions that would be advantageous to you to attack/defend from. Enemies are less likely to attack defenses that they can simply move around than walls that completely prevent access (and would probably be seen as a challenge to those willing enough to test your defenses anyway). Use this to your advantage.

Another good way to discourage settling is to remove what many of them come there for: resources. If you wall off or tower up small areas around key resources, you can deny that income to enemies. Additionally, this will make your own mining/extracting safer since you will have a defensive position to fight from if attacked. This also has an added bonus of nakeds not even knowing there are resources there. There's no way for them to know if that's a base structure or a resource node just by looking at it.