Understanding Borel sets

I'm studying Probability theory, but I can't fully understand what are Borel sets. In my understanding, an example would be if we have a line segment [0, 1], then a Borel set on this interval is a set of all intervals in [0, 1]. Am I wrong? I just need more examples.

Also I want to understand what is Borel $\sigma$-algebra.

To try and motivate the technical answers, I'm ploughing through this stuff myself, so, people, do correct me:

Imagine Arnold Schwarzenegger's height was recorded to infinite precision. Would you prefer to try and guess Arnie's exact height, or some interval containing it?

But what if there was a website for this game, which provided some pre-defined intervals? That could be quite annoying, if say, the bands offered were $[0,1m)$ and $[1m,\infty)$. I suspect most of us could improve on those.

Wouldn't it be better to be able to choose an arbitrary interval? That's what the Borel $\sigma$-algebra offers: a choice of all the possible intervals you might need or want.

It would make for a seriously (infinitely) long drop down menu, but it's conceptually equivalent: all the members are predefined. But you still get the convenience of choosing an arbitrary interval.

The Borel sets just function as the building blocks for the menu that is the Borel $\sigma$-algebra.

First let me clear one misconception.

The set of all subintervals is not a Borel set, but rather a collection of Borel sets. Every subinterval is a Borel set on its own accord.

To understand the Borel sets and their connection with probability one first needs to bear in mind two things:

1. Probability is $\sigma$-additive, namely if $\{X_i\mid i\in\mathbb N\}$ is a list of mutually exclusive events then $P(\bigcup X_i)=\sum P(X_i)$.

   Therefore the collection of all events which we can measure their probability must have the property that it is closed under countable unions; trivially we require closure under complements (i.e. negation) and thus by DeMorgan we have also closure under countable intersections.

   If so, the set of all events which we can measure the probability of them happening is a $\sigma$-algebra.

2. We wish to extend the idea that the probability that $x\in (a,b)$, where $(a,b)$ is a subinterval of $[0,1]$ is exactly $b-a$. Namely the length of the interval is the probability that we choose a point from it.

Combine these two results and we have that the Borel sets of $[0,1]$ is a collection which is a $\sigma$-algebra, and it contains all the subintervals of $[0,1]$. Since we do not want to add more than we need, then Borel sets are defined to be the smallest $\sigma$-algebra which contains all the subintervals.

Here are some very simple examples.

1. The set of all rational numbers in $ [0,1] $ is a Borel subset of $ [0,1] $.

   More generally, any countable subset of $ [0,1] $ is a Borel subset of $ [0,1] $.

2. The set of all irrational numbers in $ [0,1] $ is a Borel subset of $ [0,1] $.

   More generally, the complement of any Borel subset of $ [0,1] $ is a Borel subset of $ [0,1] $.