Calculate the maximum value

Calculate the maximum value of $f(a;b)=\int_{b}^{a }{(2-x-3x^2)}dx; b>a; b,a\in R$. I have tried to transfer it into $(b-a)(2-\frac{b+a}{2}-b^2-ab-a^2)$ but then I cannot do anything else.

When looking for a critical point of a multivariate function, it must be the case that the partial derivatives with respect to each variable are zero. By the fundamental theorem of calculus:

$$\frac{\partial}{\partial a}\int_a^b(2-x-3x^2)dx=-(2-a-3a^2)$$

$$\frac{\partial}{\partial b}\int_a^b(2-x-3x^2)dx=2-b-3b^2$$

As @Robert Z showed, the expressions for these partial derivatives are zero at $a,b\in\{-1,2/3\}$. From there, it's not too hard to show that $(a,b)=(-1,2/3)$ is maximal.

I just wanted to include an addendum showing how this problem could be solved using the standard critical-point extrema-finding method.

I guess that in your definition of $f(a,b)$ the limits should be swapped (otherwise $f$ has no upper bound).

Note that $$f(a,b):=\int_{a}^{b}{(2-x-3x^2)}\,dx =\int_{a}^{b}{(x+1)(2-3x)}\,dx.$$ Therefore, for $a<b$, in order to maximize the integral of $(x+1)(2-3x)$, we select the interval $(a,b)$ where the quadratic polynomial is positive, i.e. $(-1,2/3)$: $$f(a,b)=f(a,-1)+ f(-1,2/3)+ f(2/3,b)\leq f(-1,2/3)=\frac{125}{54}$$ because $f(a,-1)+f(2/3,b)\leq 0$. The last inequality is trivial for $a\leq -1<2/3\leq b$, you can easily handle the other cases.

Technically $f(a,b)$ is a function of two variables, but it can easily be split up into $\int_a^c-(x+1)(3x-2)dx+\int_c^b-(x+1)(3x-2)dx$ for a constant $c$. For a point to be a maximum, the derivative must not be a nonzero number. That is, it's either undefined or zero. Integration is the opposite of differentiation, so the derivative of $\int_c^b-(x+1)(3x-2)dx$ is just $-(x+1)(3x-2)$, and the derivative of $\int_a^c-(x+1)(3x-2)dx$ is $(x+1)(3x-2)$, and both of them are zero at $x=-1$ and $x=\frac 23$.

Next we can move on to the second derivative test. For $b$, the second derivative is $-1-3x$. Plugging in $-1$ and $\frac 23$, we get $2$ and $-3$, respectively, so $\frac 23$ is the x-value of the maximum. For $a$, we get the $-2$ and $-3$, giving $-1$ as the x-value. So $(a,b)=(-1,\frac23)$