How do I move to a new world and keep my stuff?

In the Terraria 1.0.5 changelog, I read that parts of the world generation algorithm have been changed.

So, I would like to create a new world and move my character there, but how can I easily transfer all my old things (mainly chest contents) to the new world?

Both your character's inventory and the contents of your piggy-bank and safe transfer across worlds.

So, go to the old world, fill up your inventory and piggy bank/safe, move to the new world, dump it all in chests, rinse and repeat!

To add to BlueRaja's answer, you can also buy a Safe when you get some money (it costs 50 gold). It acts almost like a piggy bank, but makes it easier to travel between the worlds:

• Each Safe contains the same items (when accessed by the same character), no matter where they are placed within the world. Purchasing multiple Safes will not give you more space, but will offer more points of access to your Safe contents.
• The Safe can be picked up even when it contains items. This makes it a viable way to expand your character's inventory slots. It can be removed with a Hammer to be picked up again.
• The Safe's contents are saved with your character data, instead of the World data. You keep its contents with you from world to world, and from server to server.
• The Safe's contents are exclusive to each player, so other players will not have access to items in your Safe. This makes it useful for item security on PvP servers where stealing is allowed.
• Multiple players can use the same Safes without having access to each others items. Each player will only see their own items when accessing the same Safes.