What's the best way to monitor Internet traffic for the entire office?
We currently have a T3 line for about 28 people and it gets deadly slow during the day so I need something to help track down why. I'm assuming someone is downloading something that they may not be aware of.
I would recommend against using wireshark to monitor traffic. You'll just get too much data, but you have a hard time analyzing the data. If you need to look at/troubleshoot the interaction between a couple machines, wireshark is great. As a monitoring tool, IMHO, wireshark is not quite the tool you need.
Profile the network traffic. Try out some actual monitoring tools: http://sectools.org/traffic-monitors.html. You're looking for Top Type of traffic (likely HTTP, but who knows), Top Talkers (should be your servers, but who knows), and potentially Malformed Traffic (large amount of TCP retransmissions, malformed packets, high rates of very small packets. Probably won't see, but who knows)
At the same time, work with your management to develop a network resource usage policy. In general, business terms, what business needs does the computer network exist to meet, and what are appropriate uses of the resource. This thing is costing money, so there has to be a business justification for its very existence. Your company has policies for handling the "petty cash" drawer, and I would bet your network infrastructure costs a lot more that. The key thing to focus on is not catching people doing bad things but rather watching for potential malicious activity that is degrading network functionality (i.e., the employees' ability to get their work done). Southern Fried Security Podcast and PaulDotCom Security Weekly cover information about creating appropriate security policies.
@John_Rabotnik idea for a proxy server was great. Implement a proxy server for web traffic. Compared to traditional firewalls, proxy servers give you much better visibility into what is going on as well as more granular control over what traffic to allow (for example, real web sites) and what traffic to block (URLs made up of [20 random characters].com)
Let people know - the network is having a problem. You are monitoring the network traffic. Give them a mechanism to register network slowdowns, and capture enough meta-data about the report so that in aggregate, you might be able to analyze network performance. Communicate with your coworkers. They want you to do a good job so that they can do a good job. You are on the same team.
As a general rule, block everything, and then allow what should be allowed. Your monitoring from step one should let you know what needs to be allowed, as filtered through your network usage/security policy. Your policy should also include a mechanism by which a manager can request new kinds of access be granted.
In summary, step one, the traffic monitoring (Nagios seems to be a standard tool) helps you figure out, in general, what is going on to stop the immediate pain. Steps 2 - 5 help prevent the problem in the future.
28 people saturating a T3? Doesn't seem likely (Everyone could use streaming media all day long, and it wouldn't come close.) You might want to check for routing loops and other types of network mis-configuration. You should also check for viruses. If you've got a little botnet running on your local network, that would easily explain the traffic.
What sort of switching/firewall do you use? You may already have some capability to monitor packet traffic.
Edit: I'm also a big fan of Wireshark (though I'm old, so I still think "Ethereal" in my head). If you're going to use it, the best way is to put a machine in-line so all traffic has to pass through it. That'll allow you to run exhaustive logging without having to switch your equipment into promiscuous mode.
And if it turns out you're in need of some traffic shaping, you'll be in a good position to set up a Snort proxy...I wouldn't start out with the intention of installing one, however. I really doubt your problem is bandwidth.