An endpoint of a closed interval where the derivative is zero is considered a critical point?
Solution 1:
Ultimately this boils down to definition, which Wikipedia states as
A critical point of a function of a single real variable, $f(x)$, is a value $x_0$ in the domain of $f$ where it is not differentiable or its derivative is $0$ ($f'(x_0)=0$).
If we follow this religiously, then, yes, endpoints should be considered critical points because the first derivative at those points evaluate to $0$.
Seeing that this is a Calculus I problem, I would guess that it is an extremum problem applied to some 'real-world' example? If so, then, IIRC, endpoints are usually counted as 'critical points' (though not in the same sense as the definition above; to differentiate I will call them $\textit{extremum points}$) and should be checked manually (even if the derivative exists and is not $0$). An extremum is still an extremum, even if it is on the endpoints. No one is going to complain that a firm's profits aren't actually maximized on New Year's Eve just because it is the end of the year.
Overall, however, I would suggest that you follow the convention that your textbook is using (but be aware that other definitions exist; check with your school on which definition they use to mark homework and exams).
As for why the author later added that 'we do not allow relative extrema to exist at the endpoints of intervals', I believe the emphasis is on the word $\textit{relative}$ because it suggests that the extremum is 'local' in the sense that it only cares about the values of $f$ around it (and not on the entire domain). But values on the endpoints are unable to look around in $\textit{both}$ directions. I wouldn't take this too seriously though, and, ultimately, the statement depends on the context surrounding it.