Should I use plural noun after ''both'' in the sentence below?
I think you have answered your own question. "... of both the Reflection and Refraction roll-outs to 10"
The emphatic and unnecessary "Both" is troublesome here. It invites us to understand "... both the Reflection and the Refraction roll-out to 10".
If you had three roll-outs the problem would not arise. "Set the ... of the Reflection, Refraction, and Diffraction roll-outs to 10". To use the singular would suggest that the three variables have one common roll-out rather than one each.