Can I change difficulties after I start?

Yes, it is possible. You need to create a LAN game and then you can specify the difficulty there. A couple of important notes: if you tick "reroll world" it will generate a whole new world but I think quest progress is still saved. Also, I've heard tell that even if you don't reroll the world that all the monsters in your LAN game will be respawned. I haven't done this yet so I can't give you my experiences.

Here is a forum thread at the official Torlight 2 forums explaining this.

There is currently no regular option to change single player difficulty for a specific character. However, you have two options for this:

1. Start a LAN game on your own. You can select any difficulty level when you host the game.

2. Use the game console (note the last part about being flagged):

   If you enable and use the console, you can change difficulty with the command "setdifficulty #" where # is a number between 0 and 3, from easy to elite.

   To enable the console, open the settings.txt file located in My Documents/My Games/Runic Games/Torchlight 2/Save Then scroll down until you see CONSOLE: 0, and change the 0 to a 1. To open the console ingame, press Ctrl+Shift+NUMPAD_0.

   If you use the console, it flags your character as a console user. Other players online will be able to see a mark next to your portrait. If you don't care about this, go ahead, otherwise I'd stick to just remaking LAN games and changing the difficulty there.
   
   Source