PS Vita Import / purchase overseas
I bought my PS Vita in Taiwan on release day (23/12/2011), and live in London where it was a 2 month wait before the UK release (22/2/2012). Everything is fully region-free, you choose the language on initial setup. Everything worked perfectly, even the voice-over for Welcome Park was in English.
The PSN store is linked to the location of your PSN account, meaning if you use your Australian account you will connect to the Australian store. This applies for all other services which use PSN. As I bought mine before the UK release, it meant I couldn't access the UK store until the UK release, but now everything works as expected.
Physical games are region-free too. Enjoy!
Yes, the PSP Vita is both region-free and supports language selection, which means you will be able to buy a Japanese console and play games from anywhere in the world on it. [Source]
From the same source, however, do note the following (this was written before a world wide release):
Will I have trouble using US software and services later on?
Honestly, it's hard to say. When it comes to downloadable applications, you should be able to get most things from the local PlayStation Store, but when it comes to system software updates we're in relatively uncharted territory. We're honestly not sure if Near will let US and Japanese handheld users connect with one another, in fact. We reviewed our Japanese unit in Japan, after all.
Though this looks like it might be okay since it seems dependent on your PSN account info.