Alternative word for "media" for disks, USB storage etc

Media is the correct word for that. You could add an adjective or two and call it storage media if you thought that would help- or persistent storage media. Those are all non-technical English words that ought to be recognizable, especially in context, by any average computer user.