What can I find in the Sedalia police station?

Yes, there is another survivor in one of the cells, who will trade food for 10 revolver and 10 rifle ammo. You can find the food in the building where Jimmy Blake (the guy you need the batteries for) is on the roof. But depending on your playstyle, you might not need the ammo anyway.

But keep in mind that Jimmy Blake won't come with you as a survivor if you don't give him the batteries. But if I remember correctly, there isn't enough space in the car for everyone anyway. So, if you are happy with the amount of ammo you can just leave the side quest. You'll just miss some irrelevant dialogue and Jimmy as a survivor.