How to limit speeds on my "Linksys Wireless-G Home Router - WRT54GH"?

What you are looking for is called Quality of Service (QoS) under Applications and Gaming.

It allows you to restrict bandwidth based on different criteria (MAC address, IP address, application, physical router port).

If you are using DHCP (do your computers get IP addresses automatically?) and have people connecting wirelessly I would recommend using MAC addresses to filter out machines. Give your computer a higher priority than the others and your issues should go away.

To find the MAC addresses of your machines (in Windows) run Command Prompt and type "ipconfig /all"

Look for the connection that maps to your router (IP address will probably be 192.168.1.xxx) and note the MAC address in the table.

edit: I have a WRT54GL loaded with DD-WRT. One of my motivating factors in switching to DD-WRT was the more robust QoS settings available.

From the Linksys WRT54GH FAQ :

Q: Does the WRT54GH support wireless QOS and Internet Access Priority QOS?

A: No.

So the answer to your question is: Not possible with your current setup.

The only solution is to change the firmware, with the risk of bricking the router. Frankly, I wouldn't advice it if you describe yourself as newbie.

However, if you can find help, then you can choose between these free firmwares: DD-WRT and Tomato. Many people recommend Tomato for better QoS support.