What is 'System' on mac laptop storage?

That is what is normally referred to as macOS, or "The Operating System". If it increased in size it may be the virtual memory (or "page file"). Note that I said "may."

I would look for a utility like Omni Disk Sweeper, Daisy Disk or Disk Inventory X and run it on your HD. Those utilities allow you to see a graphic representation of your drive sorted by file size. Which should show you what is eating up your disk space.