Is size inherent in the meanings of "plant" and "factory"?

Merriam-Webster says this:

"plant (noun) 2 a : the land, buildings, machinery, apparatus, and fixtures employed in carrying on a trade or an industrial business; b : a factory or workshop for the manufacture of a particular product; also : power plant; c : the total facilities available for production or service; d : the buildings and other physical equipment of an institution"

"factory (noun) 2 a : a building or set of buildings with facilities for manufacturing"

There's no difference between them in my dialect. One word's shorter than the other's all that I can see. It's probably a matter of regional dialect and local (what they say at work, plant or factory) parlance.

However, one wouldn't call a power plant a power factory.

Since

• Factory is the older term, and deals with the assembly of things, and
• Plant is a word that arose (in its modern usage) for chemical refineries and the like

you could consider a Plant more complex compared than a factory, if for no other reason than the added complexity of the refinery / reaction process(es) involved.

In fact, consider that smelting facilities are generally called a plant despite also involving factory-line types of post-process handling.

Also, I'd ponder that plant arose from plantation - which is more complex (in operations) than a factory.