What will happen if I move a villager out of his village?

In order to be a village there needs to be at least 1 villager and a door.

The Minecraft village zone is a circle that radiates from the center of the village.

The center of the village is the block where the coordinates of all the doors in the village average out. Just make a best guess about the center of your village and you won't be far off.

To calculate how far you'd need to move your villager, count how many blocks the farthest door is from the village center and go a few blocks further to be safe. If the distance to the farthest door is less than 32 blocks, then use 32 blocks as your minimum distance.

You can also dig out a hole beneath the Nitwit (the name for the green-robed villagers) and pour in lava. You will kill him and not make the rest of the villagers aggressive.