Is there a way to turn down my PS4 controller's speaker volume?
Try this:
Hold the Playstation Button
Select Adjust Devices
Lower the volume
(Source)
You can set the controller speaker volume in the console's system settings:
Settings > Devices > Controllers > Volume Control (Speaker for Controller)
From there, you can adjust the volume, which will apply to all games.
Edit: Timmy Jim's answer provides a much quicker and simpler way of doing this.