How can I increase the speed of filtering a list

When optimizing code the first rule is to measure, preferably with a profiler that can tell you exactly what part of the code takes most of the time.

Second rule would be to use a optimal algorithm, but unless you have a huge number of items and some reasonable way to sort or index said items, linear time is the best you can do.

Here are some guesses and suggestions of things that might be improved:

• Avoid using .ElementAt, this might create a new enumerator object, and that will take some time. Especially inside inner loops. Prefer to use indexers and/or store it in local variables instead.
• Avoid using Linq. Linq is great for readability, but it will have some overhead. So when optimizing it might be worthwhile to use regular loops to see if the overhead is significant or not.
• Try to do all processing in one go. Instead iterating over all items once to find the minimum and once to do the maximum, do both at the same time. Memory is slow, and by doing as much processing of an item as possible when it is already cached helps reduce memory traffic.
• I would consider replacing RemoveAll with a loop that copies items that pass the check to a empty list. This should help ensure items are copied at most once.

A rule of thumb when optimizing is to use low level language features. These are often easier for the jitter to optimize well. But it may make the code harder to read, so use a profiler to optimize the places that need it the most.