Is there something between summation and integration?
The short answer is yes, but it is not a very easy concept. The answer lies with the definition of the integral and measures. The idea of a measure is to assign a size to subsets of the reals, and use this idea of size to perform integration (among other things). The first thing to note is that there are different ways to assign a measure to subsets of the reals. Some measures behave the way that you would expect them to in that the size of the interval $(a,b)$ is $b-a$, regardless of what $a$ and $b$ are. Other measures behave differently. For example, you could say that the measure of a set is the number of integers contained therein. Under this definition of a measure, summation and integration become the same thing. We could also define a measure which heeds only rational numbers and perform integration with respect to that measure.
In summary, integration is an abstract concept. The standard integral taught in introductory calculus is just one type of integral, and the type of integral you describe is just a different type (as is summation).
For more information on this subject see here for information on measures, and here for an explanation of how integration is defined abstractly, using measures.
Your examples can be viewd as particular cases of a general theory of integration, namely, the Lebesgue integration. In fact:
If $X=\{m,m+1,...,n\}$, $\mu$ is the counting measure and $f$ is positive, then $$\int_X f\ d\mu=\sum_{k=m}^nf(k).$$
If $X=[a,b]$, $\mu$ is the Lebesgue measure and $f$ is continuous, then $$\int_X f\ d\mu=\int_a^bf(k)\ dk,$$ where the integral in the right side is the Riemann integral.
From this point of view, the answer to your question is yes and the desired "similar operation" will also be a particular case of the Lebesgue integration. Explicitly:
- If $X=\mathbb{Q}$, $\mu$ is some suitable measure and $f$ is measurable, then $$\int_X f\ d\mu$$ can be viewed as "something between summation and (Riemann) integration that concerns rational values".
Example: If $\mu$ is the Lebesgue measure and $f$ is Lebesgue measurable, then $$\int_\mathbb{Q} f\ d\mu=0.$$