What is the benefit of purely functional data structure?

Purely functional (aka persistent or immutable) data structures give you several advantages:

• you never have to lock them, which extremely improves concurrency.
• they can share structure, which reduces memory usage. For example, consider list [1, 2, 3, 4] in Haskell and some imperative language like Java. To produce new list in Haskell you only have to create new cons (pair of value and reference-to-next-element) and connect it to the previous list. In Java you have to create completely new list not to damage the previous one.
• you can make persistent data structures lazy.
• also, if you use functional style, you can avoid thinking of time and sequence of operations, and so, make your programs more declarative.
• fact, that the data structure is immutable, allows you to make some more assumptions and so expand capabilities of language. For example, Clojure uses the fact of immutability to correctly provide implementations of hashCode() method on each object, so any object may be used as a key in a map.
• with immutable data and functional style you can also freely use memoization.

There's much more advantages, in general, it is another way of modeling the real world. This and some other chapters from SICP will give you more accurate view of programming with immutable structures, its advantages and disadvantages.

In addition to shared memory safety most purely function data structures also give you persistence, and practically for free. For example, let's say I have a set in OCaml, and I want to add some new values to it I can do this:

module CharSet = Set.Make(Char)
let a = List.fold_right CharSet.add ['a';'b';'c';'d'] CharSet.empty in
let b = List.fold_right CharSet.add ['e';'f';'g';'h'] a in
...

a remains unmodified after adding the new characters (it only contains a-d), while b contains a-h, and they share some of the same memory (with a set it's kind of tricky to tell how much memory is shared since it's an AVL tree and the shape of the tree changes). I can continue doing this, keeping track of all the changes I've made to the tree allowing me to go back to a previous state.

Here's a great diagram from the Wikipedia article on Purely Functional that shows the results of insert the character 'e' into the binary tree xs: