What happens when I collect the shiny circles in VVVVVV?

In several places around the map, I have seen these shiny circles in hard-to reach places.

They look like the thing on the right side of this image.

I have seen indications on the minimap of where more of these are likely to be, but even the ones I already know about are so hard to reach, I wonder if it worth the effort.
Will anything good happen if I collect many of these? More powers, more map areas, unlock more of the story?

Collecting trinkets unlocks songs from the jukebox.

Collecting all trinkets unlocks the trophy room, where you can view the achievements you unlocked:

• Game completion achievements:
  • GAME COMPLETE
  • FLIP MODE COMPLETE
  • MASTER OF THE UNIVERSE (no deaths mode)
    
    
• Time trial achievements: (win the respective time trial with no deaths and all trinkets)
  • SPACE STATION 1 MASTERED
  • LABORATORY MASTERED
  • THE TOWER MASTERED
  • SPACE STATION 2 MASTERED
  • WARP ZONE MASTERED
  • FINAL LEVEL MASTERED
    
    
• Number of deaths achievements:
  • Win with less than 500 deaths
  • Win with less than 250 deaths
  • Win with less than 100 deaths
  • Win with less than 50 deaths
    
    
• Super Graviton achievements:
  • Last 5 seconds on the Super Gravitron
  • Last 10 seconds on the Super Gravitron
  • Last 15 seconds on the Super Gravitron
  • Last 20 seconds on the Super Gravitron
  • Last 30 seconds on the Super Gravitron
  • Last 1 minute on the Super Gravitron

Additionally, access to a Super Graviton is unlocked. It is, however, harder than the one found through a normal playthrough. Watch a video of the Super Graviton in action.

To expand on the other answers:
Collecting trinkets unlocks time trials:
3 trinkets unlocks Violet's level
6 trinkets unlocks Victoria's level
9 trinkets unlocks Vermillion's level
12 trinkets unlocks Vitellary's level
15 trinkets unlocks Verdigris' level
18 trinkets unlocks the final level

Collecting trinkets unlocks songs in the ship's jukebox:
Passion for exploring is always available
5 trinkets unlocks Pushing onwards
8 trinkets unlocks Positive force
10 trinkets unlocks Presenting VVVVVV
12 trinkets unlocks Potential for anything
14 trinkets unlocks Pressure Cooker
16 trinkets unlocks Predestined fate
18 trinkets unlocks Popular Potpourri
20 trinkets unlocks Pipe Dream and ecrof evitisoP

Collecting all 20 trinkets allows you to talk to Victoria, triggering a cutscene that takes you to the Secret Lab area, which contains the trophy room and Super Gravitron.