Apparently I’m calling random numbers but there's nothing in the call log

For the last few weeks I’ve had random calls from people saying that I called them, but there is nothing in my call log. I'm running the latest iOS.

Any suggestions as to what is happening here, and should I be alarmed enough to restore my phone?

These calls are not likely coming from your phone. Rather, some robo caller is likely spoofing the caller ID using your number. The FCC website says

What can you do if your number is being spoofed?

If you get calls from people saying your number is showing up on their caller ID, it's likely that your number has been spoofed. We suggest first that you do not answer any calls from unknown numbers, but if you do, explain that your telephone number is being spoofed and that you did not actually make any calls. You can also place a message on your voicemail letting callers know that your number is being spoofed. Usually scammers switch numbers frequently. It is likely that within hours they will no longer be using your number.

Contact you carrier/provider to verify that those calls are taking place on your account. If it turns out to be bogus, your carrier/provider probably has a procedure to deal with spoofing.