L4D hates you?
But seriously, do a traceroute to the L4D server's IP address and see where it seems to bottleneck (or post the entire log here for us to peruse).
Start - Run - CMD
Then press enter (or click on OK)
in the new window, type;
tracert IPADDRESS
(Obviously alter the IPADDRESS bit to the IP of the L4D server.)
Then post the log here