Open source firewall or commercial firewall
First your question sounds like "commercial softwares are better, but how bad is it if I go to open source". Anyway I don't think that there's any major issue for using open source firewall as long as :
1/ it matches your needs
2/ you update them on a regular basis
3/ you know what you're doing and how to configure it
It would be considered ok under any circumstances in which you had the expertise necessary to choose a solution that suits your needs and to implement it effectively. This is true of commercial firewall solutions as well. The difference is that if it has a compelling feature (such as Active Directory integration in MS ISA server) you may be inclined to pay for it... or not.
Open source firewalls, like many other applications, get easier to install, configure, and maintin all the time. Smoothwall is a good example, but easy to use ones have been around for years -- my first Linux experience was using Coyote Linux as an IPTables (actually IPChains initially) firewall frontend; it booted from a floppy disk, was easy to configure, and ran great on a 66 MHz Pentium.
I use a pair of ipcop boxen to secure a small business (~80 machines) network. They do an excellent job of firewalling, running a DMZ for our mail server (and a couple of other externally accessed machines) and VPN for external workers and site-to-site VPN. Not much linux expertise required to set-up/maintain as all is done through the excellent web interface (which also has all your security logs and traffic graphs etc.)
HTH
When I setup my company I opted to buy a small box and download Smoothwall. It worked very well protecting our first webserver for nearly three years (until a hard disk failure required the box to be rebuilt).
We use two commercial firewalls now, partly because the prices are so much lower, the hardware is physically smaller than a 1/2 rack server, uses less power, we know their capacity and the support is available from the supplier.
It basically comes down to a business decision. What's the budget, what features do you need, what's your expertise/skill base?