Meaning of "the field in which I grow my [expletive]"

It is likely referring to giving a fuck, which is a derogatory way of saying to care (about). For instance, saying:

I don't give a fuck / I don't have any fucks to give

means one does not care about the subject at hand.

The sentence could be a sarcastic way of saying they don't give any fucks, if the field which they are "growing their fucks" is completely empty.

It could also imply that they have a limited number of "fucks to give", which would mean they could only care so much about certain things.