Rejecting infinity
Solution 1:
Few people these days argue that the concept of infinity is absurd per se, but there are still those who hold that it is epistemically intractable, or unnecessary, or that infinite objects simply do not exist.
Finitistic systems are generally very weak. Steve Simpson claims that primitive recursive arithmetic (PRA) is finitistic, since it involves reference to only potential infinities—that is, indefinite iteration—not completed infinities such as the set of natural numbers $\mathbb{N}$. Some finitists accept the existence of countably infinite sets: $\mathbb{N}$ is finitistically acceptable but the set of real numbers $\mathbb{R}$ is not.
One might think that this is the end of the road for finitist mathematics, but various clever dodges have been thought up by philosophers to avoid this trap. Shaughan Lavine, for example, has developed a finitary version of ZFC which is not mathematically revisionary but nonetheless allows one to maintain that there are only finitely many mathematical objects.
There are a number of considerations which motivate people to take finitist positions. The first is the desire to repudiate abstract objects. If there are no true infinities then we can be realists about mathematics without having to accept the existence of a platonic heaven or otherwise account for the existence of infinitary mathematical objects. Mathematics can be understood purely in terms of the combinatory properties of physical objects.
Alternatively, one might think that mathematical objects are merely ideas in our minds: they do not exist independently of their construction by us. Since we are finite, we cannot construct all of the natural numbers, even though in principle we can keep going forever. Because of this, there is a strong tendency among constructivists to reject the completed infinite.
Epistemology provides a different reason. We only have finite computational power available with which to derive mathematical truths, and thus we cannot grasp any infinite objects (if such things exist) in their entirety. Mathematical knowledge must thus be arrived at via finitary means.
Some concepts, like that of the natural numbers and recursive functions on them, seem basic: they are primitive ideas which do not admit of further justification. Higher mathematics, however, is incredibly useful, even if it does not admit of such straightforward justification. We might therefore want to treat all mathematics beyond arithmetic as merely a game with symbols that has only instrumental value. We can see just such an approach in Hilbert's programme, with its restriction that all proofs must be finitary.
Solution 2:
Since you have some computer science background, a good and quick introduction to some of the (ultra)finitism ideas and objections can be found in Ed Nelson's talk "Warning signs of a possible collapse of contemporary mathematics".