Didn't Apple use to provide USB stick with OS X on it?
Solution 1:
Yes, The USB recovery drive shipped first with a MacBook Air and Lion was also sold as a full retail OS on USB write only format instead of the retail DVD/CD that used to be available for purchase.
Apple discontinued this practice a few years ago and has started distributing OS X via the App Store exclusively as a digital download. You can make a bootable version of the installer from the App Store version if you have a USB stick handy.
There's a free, AppleScript-based program called DiskMaker X that will help you build a bootable USB stick for Mavericks. You'll have to download the installer from the App Store first. Just cancel the installation after the download and Mavericks will remain in your Applications folder for DiskMaker X to find and use.
Solution 2:
Apple did discontinue the practice of including install media (USB or CD) with a new Mac.
This has happened because a recovery partition and internet recovery have made it unnecessary.
A recovery partition should exist on the drive which can booted from by holding down Command+R when booting.
Additionally, Apple has also introduced Internet Recovery allows the computer to be booted from the internet even if no operating system or recovery partition exists on the drive (e.g., after the drive has been completely replaced). There are only a limited number of Macs and limited network configurations that Internet Recovery supports.
Details are here: About OS X Recovery
Solution 3:
Having just done this for my employees, I'll add to Ian C's great answer: - be sure to first check your Application's folder for any existing Maverick installer. There's probably not one, but if there is, delete it. Otherwise, the App Store will see that it's already there, and not download whatever latest version is on the App Store (no idea why it doesn't recognize that it's an earlier version). No biggie if you create a bootable USB stick from, say, 10.9.1 instead of 10.9.2, but this "delete first!" method will save you the time from having to do an upgrade after the installation.