Well, there can still be a number or reasons on why your blue faced. Are their other computers on your network that are also downloading and uploading? You could simply be experienceing network lagg, not internet lagg, which means that you can timeout more often when simply requests like viewing a webpage, runnign a FireFox plugin, or requesting updated info form the tracker. Possibly your on a slow connection and you don't have that much bandwidth? Azureus takes up a LOT of bandwidth, regarless of the actual speed it is downloading, although limiting your downloads do help speed things up. With regards to your slow internet, it probably points to these situations, but if you have say DSL with a 512 download, you should experience to much. I'm on a 6mbps cable, and when I get Azureus dling at about 30 (uploading at same) my internet laggs a bunch also.
Blue faces means that your downloading but your not connected to the tracker. If it is a private tracker, you may have been kicked from it, but you still maintained your peer connections, which is why your slowly getting your stuff. What are the warning messages that come up? For example: under your tracker status, if it shows OK (DHT:), it means the tracker doesn't have the file you've requested, but can still link you to people who have it.
And one final note, as I didn't read above. Make sure your firewall AND router allow passage through the Azureus port. Its default is 6881 I believe. This also applies to software firewalls that are positioned behind the router, like Windows Firewall.