How can I tell if a system rebooted as well as the time of the reboot by looking at the system logs?

Very, very easy. The last command parses /var/log/wtmp for user activity including pseudo-user reboot.

Use the command: last reboot

See man last for more information. http://linux.die.net/man/1/last

The pseudo user reboot logs in each time the system is rebooted. Thus last reboot will show a log of all reboots since the log file was created.

If the /var/log/wtmp file was rotated since your last reboot, you may be able to see prior reboots by using last -f /var/log/wtmp.1 reboot.

It varies from distribution to distribution. Sometimes /var/log/{messages,syslog,dmesg} will include it. For one I just rebooted, my last 2 lines and the very first ones after the reboot are:

Feb 29 13:16:07 hostname kernel: Kernel logging (proc) stopped.
Feb 29 13:16:07 hostname rsyslogd: [origin software="rsyslogd" swVersion="5.8.1" x-pid="592" x-info="http://www.rsyslog.com"] exiting on signal 15.
Feb 29 13:23:15 hostname kernel: imklog 5.8.1, log source = /proc/kmsg started.
Feb 29 13:23:15 hostname rsyslogd: [origin software="rsyslogd" swVersion="5.8.1" x-pid="572" x-info="http://www.rsyslog.com"] start
Feb 29 13:23:15 hostname rsyslogd: rsyslogd's groupid changed to 103
Feb 29 13:23:15 hostname rsyslogd: rsyslogd's userid changed to 101
Feb 29 13:23:15 hostname rsyslogd-2039: Could no open output pipe '/dev/xconsole' [try http://www.rsyslog.com/e/2039 ]
Feb 29 13:23:15 hostname kernel: [    0.000000] Initializing cgroup subsys cpuset
Feb 29 13:23:15 hostname kernel: [    0.000000] Initializing cgroup subsys cpu
Feb 29 13:23:15 hostname kernel: [    0.000000] Linux version 3.0.0-12-server (buildd@crested) (gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3) ) #20-Ubuntu SMP Fri Oct 7 16:36:30 UTC 2011 (Ubuntu 3.0.0-12.20-server 3.0.4)
Feb 29 13:23:15 hostname kernel: [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.0.0-12-server root=/dev/mapper/hostname-root ro
Feb 29 13:23:15 hostname kernel: [    0.000000] KERNEL supported cpus:
Feb 29 13:23:15 hostname kernel: [    0.000000]   Intel GenuineIntel
Feb 29 13:23:15 hostname kernel: [    0.000000]   AMD AuthenticAMD
Feb 29 13:23:15 hostname kernel: [    0.000000]   Centaur CentaurHauls
Feb 29 13:23:15 hostname kernel: [    0.000000] Disabled fast string operations
Feb 29 13:23:15 hostname kernel: [    0.000000] BIOS-provided physical RAM map:

... and so-on. That shows a normal shutdown. An unexpected shutdown might not tell you the time it went offline, but you can guess based on the gap between the last message and the first depending on how noisy your system is.