Am I free to stop worrying about Colossi after I kill a few?

In this video, we can see that after losing the first three Colossus units, the Protoss player didn't have a lot of them at one time. He had them, but never three again in the same place. He was splitting his minerals between Templars and Colossi.

So I could conclude that if I kill the first group of Colossus units with Vikings, I don't need a lot of vikings anymore. Is this true?

In terms of countering the opponent, let's look at the order:

1. Your opponent goes for some Gateways.
2. You create a strong bio ball.
3. Your opponent counters them with Colossi.
4. You spend less on your now weaker bio ball, and spend that money to get vikings.
5. Your opponents Colossi is now weaker, so he could spend money to counter your vikings.

So, unless your opponents goes for an odd play style, you can conclude less Colossi are made.

Side note:

Also, you can easily suspect Colossi when creating a Bio Ball; and Vikings when going Colossi.

Just try to scout Robotics, if not, scout for Twilight Council (--> HT/DT) or a Stargate (--> Carrier).

Scouting is key to make strategical decisions supported by risk analysis, in military terms:

There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are also unknown unknowns. There are things we don't know we don't know.

— Donald Rumsfeld

Scouting and map control helps you increase the amount of known knowns. :-)

It depends on how you killed the colossi.

If you have a large bunch of vikings/corruptors and used them to kill the first few, the protoss will often not build more colossi since you already have a strong counter for them. He basically makes your air units less usefull and almost a waste of food by not building colossi, so it's a good decision.

If you killed them with your ground army and have no vikings/corruptors, he will probably continue to build colossi since they can still cause massive damage to your army.

The best way to figure out whether your opponent is making more Colossi is to scout them and look for Colossi.

On the other hand, the resource commitment that goes into quickly building 3 or so Colossi is prohibitive unless the opponent has many running expansions. If they're on one base, it's completely safe to assume they're making Colossi one at a time, or if they've abandoned all other production, perhaps two at a time.