How to change default region/culture settings in Windows 2012

You can also do it from the Control Panel, without changing manually the registry. There's an option allowing you to apply the Current User settings to the System Accounts.

Open Language --> Advanced settings --> Apply language settings to...

Then click the button "Copy Settings" and check the option "Copy to Welcome Screen and System Accounts".

Done.

Regional settings are applied per user. For the system account you can change it via the regisrty, see here

Change the HKEY_USERS/.DEFAULT/International/sDecimal key

What you're looking for is the "regional settings".

In Server 2012, (at least with US English), you can access them by opening up the Start menu and typing "Regional." It's under settings, and the only item that comes up, titled Change date, time or number formats.

If you click the Additional settings... button, and then the Currency tab, you can customize the format, and change your "decimal symbol" and/or "Digit grouping symbol" to something that won't cause problems form your SQL server.

If you wish to change this via GPO, or heaven forbid, editing the actual registry values, there's a technet blog on how to do it that way as well. It's done through a Computer Group Policy Preference Registry collection (Computer Configuration -> Preferences -> Windows Settings -> Registry), and the registry subkeys you want are located under HKEY_CURRENT_USER\Control Panel\International.