Scala boilerplate: lack of common superclass of Iterable and ParIterable
Why is Scala designed with the following irritating form of boilerplate?
It would be convenient to write
def doStuffWithInts(ints: BaseIterable[Int]): Unit = ints foreach doStuffWithInt
for a common superclass BaseIterable of Iterable and ParIterable so that we can write both
val sequentialInts: Vector[Int] = getSomeHugeVector()
doStuffWithInts(sequentialInts)
and
val parInts: ParVector[Int] = getSomeHugeParVector()
doStuffWithInts(parInts)
Yet Scala forces us to copy and paste our doStuff method, once for Iterable and once for ParIterable. Why does Scala thrust such boilerplate on us by failing to have a common superclass BaseIterator of both Iterator and ParIterator?
Solution 1:
You can use IterableOnce
but that would force you to get an Iterator
which is always sequential.
This is a conscious decision from the maintainers, you can read all the related discussions by starting here: https://github.com/scala/scala-parallel-collections/issues/101
The TL;DR; is that the maintainers agree that it is a bad idea to provide an abstraction between two; mainly because parallel collections should not be used as general collections but rather as localized optimizations. Also, the point out how easy it would be to introduce errors if you could abstract over the two (as was the case in 2.12
).
Now, if you insist you want to abstract over the two, you may create your own typeclass.
Finally, I may suggest looking at using Future.traverse
instead of parallel collections.