I was stuck with my internet and ISA2004. I got ISA 2004 that work as proxy and web filter in my organisation. I found very slow internet after installing ISA 2004. But I found faster internet speed if I bypass ISA. I have AD and DNS installed in my current work place.
Here is a clue I google and found in microsoft.com. You may experience slow internet behind ISA 2004 with SP2 or SP3. This happens when you have a DNS server in internal network and ISA 2004 try to resolve external address in that DNS server. It takes too long to browse yahoo or google or any sites. This may occur even you have configured allow HTTP/HTTPS or routing rules.
Microsoft released a script that help you to resolve this issue. Before you apply this script please make sure you dont install ISA server in a domain controller. Also dont install ISA 2004 in a server that got DNS, DHCP, IIS or WINS installed. Try to install in a seperate box. Do not install any third party application that may use port 80, 443 and other port. make sure you have correct internal and external IP configured. proper routing must be in place such as gateway. confgiure you desired allow rules in ISA. After all these if you find a slow internet then go ahead with these script. copy the script in a notepad and save it as filename.vbs in c drive and run it. it will take while to finish. wait and it will popup done. now you test your internet.
You can get the script from http://support.microsoft.com/kb/839510/en-us?spid=2108&sid=global