Haste or no haste? That is the question
I have found this confusing too, so I looked it up.
The OED gives a number of sayings, and a number of forms of this expression, including
The more haste the lesse spede. [1546]
With more hast, then good spede, I brought it to an ende. [1556 - "then" means "than", I believe]
It seems to me from these that the meaning of the phrase, back in the sixteenth century was not "Make haste instead of speed", but "If you make more haste, the result will be less speed". I had never thought of it this way before, and I suspect that few people have.
So while the overall meaning of the proverb has survived, I now think that the understanding of the words has altered, and this is why you find it puzzling.
The saying more haste, less speed is suggesting that a focus on speed is sometimes counter-productive.
Sometimes when we do things too quickly we make mistakes and have to do them a second time. In such cases we might have been finished sooner had we not made the first attempt in such a hurry.
Haste relates to the manner in which the work is carried out, speed relates to the overall time taken for the work. More haste causes a decrease in speed.
The expression does not actually advocate for more haste. Instead, it cautions against hasty action. It is essentially a condensed way of saying "the more hastily you do something, the longer it will take you to finish it".
This expression is memorable because it is an apparent paradox. If we take a basic understanding of the phrase, it simply claims that "faster is slower", which is clearly nonsense. The real meaning in the phrase comes from the connotations attached to the word "haste", and especially its derivatives such as "hasty" and "hastily". Often, these words have negative connotations attached to them which suggest that an action was not just performed quickly, but too quickly. The result is that details were missed or the end product is unacceptable, so extra effort must be expended in the end to make up for mistakes.