When does a weapon's speed take effect?

According to the Stardew Valley wiki, a weapon's speed statistic affects how fast a player moves. Does this occur only when the weapon is selected? Or does the speed change occur even when the weapon is unequipped in the player's inventory?

I think you misread that wiki page (emphasis is mine):

Speed is a statistic that affects how fast the player moves. It can be affected in a variety of ways. Weapons have their own speed statistic, which is separate from player speed.

The weapon speed statistic seems to affect the swing speed of the weapon, not player movement speed. If you swing a very slow weapon, you'll notice that the weapon sprite "stays out" longer than one that swings quickly. Fast-swinging weapons should, in theory, give you more hits per second.