Solution 1:

The problem is that the function likelihood_levy is either returning a value greater than .Machine$double.xmax or within the function, a division by zero is occurring. This line may be the culprit:

sum(log(1 / (x - mu))) 

if x is equal to, or just very close to mu, the problem would occur. You could add some defensive code to check for this. However:

the first parameter must be lower than x

I don't think there is a way to do that with optim