Moving Server to new Hardware - SBS 2008
Solution 1:
Microsoft publishes a guide called "Migrating to Windows Small Business Server 2008 from Windows Small Business Server 2008" which describes the process of setting up an answer file and performing a migrate-install on the new hardware. You can find it here:
http://www.microsoft.com/downloads/details.aspx?FamilyID=31CBC5DD-21B1-4A6E-9A9D-740CE7605448&displaylang=en
Solution 2:
My first plan of action would be to read the documentation Nic has provided a link to. If volume based backups are a non starter, this document might explain why, if it doesn't, it's got to be worth a try.
If you have the hardware, perform the install/restore to the new server offline. If you haven't yet purchased the hardware, why not grab yourself a copy of the free version of VMWare server and perform an offline test?
Make sure you document the install procedure, because you might find going off track ever so slightly could cause you a problem when you perform the process for real.
If it fails, nothing lost, delete/format your test system and fall back to the Microsoft recommended method as per Nic's answer.
If it works, make sure you give it a damn good test, (add workstations to the domain, perform logons, check exchange access etc). Once happy, destroy the offline system, and perform the job for real, using the steps you have documented earlier.
One caveat with a volume based backup: there used to be (or possibly still is) issues with backing up Exchange on server 2008, our Exchange server is still going strong on our Server 2003 box, so I'm out of touch in this respect, however I trust this isn't an issue for you (i.e. you are backing up your exchange server aren't you?)