Does "English as a Second Language" literally mean 2nd or can it mean 3rd or 4th?
It is used to mean
the use of English by speakers with different native languages.
A majority of the students probably know only their native languages, and names have a tendency to stick even when semantic drift or circumstances mean they are no longer accurate at all.
When a given speaker wishes to make it clear, he can certainly say that English is his third, or fourth, or twenty-first language.