Apply today for a FREE subscription to CIO Magazine!
Thu, Feb 8, 2007 14:30 EST

|
Posted by: Esther Schindler in News Topic: InfrastructureBlog: You're the Boss
Current Rating: |
If the Internet seems to be running slowly, lately, don't blame it on your network admins. The problem hasn't been excessively reported (except among techies, who have generated plenty of forum traffic about it), but the Internet is currently suffering a massive slowdown because the spam bots have gone out of control. The conclusion among some admins, at least, is that it's caused by a bug in one of the zombie viruses (the ones that take over a Windows computer and turn it into a spam factory). Wherever the bots are coming from, it's slowing down—and bringing down—servers. Including yours.
The result is that email that used to arrive in a few minutes is taking hours. Your servers may have hundreds or thousands of inactive connections that just sit there and don't do anything but consume bandwidth. We've taken to calling them "barnacles," and my techie spouse has written custom software to scrape 'em off every so often. And even so, he tells me that the mail servers are running out of connections every five minutes.
As a manager or team leader, you're stuck. This isn't a problem that you can solve in-house. It isn't a matter of buying another box of technology.
Some companies are apparently coping by throwing more hardware at the problem. For example, one large midwest insurance firm (a household name) had a single mail server for quite some time. They have added three additional servers (one every week) in the last month. (That knowledge wasn't brilliant investigative journalism; it comes from reading message headers and email log files. Doesn't everyone?)
But "more hardware please!" isn't a longterm solution, because there isn't much being done to kill the source of the problem. That awful Stration virus (which I mentioned in another context some weeks ago) is the likely culprit (people who write viruses don't follow QA testing practices, I suppose), and it's evolving faster than the anti-virus tools can respond.
Assuming that such tools are installed in the first place, that is. A few weeks ago, I was called to familial tech-support duty to detoxify a nephew's system, which had 43 (!) separate infections, at least one of which was a "you're infected—click here to spend $49 on an anti-virus app" shudder-inducer. David's system is no longer contributing to the problem (I hope!) but how many other unprotected home computers are giving your admins the heebie-jeebies?
This is the point where I'd ordinarily spell out the alternatives and then ask which of them is the one you personally consider most wise. Except, in this case, I don't know what the alternatives really are. (I've discounted the spouse-suggested, "Turn off all Windows computers" as regrettably unfeasible. Besides, he's cranky because he's having to squash someone else's bugs.)
It's left me to mull over the fact that John Brunner was right in his predictions about the Internet in his 1975 novel Shockwave Rider; perhaps his solution (a worm sent out to eat the worm) is the most plausible as well.
How are you dealing with it? Or haven't you noticed this problem?
Contrary to recent comments by Mr. Gates, Macs don't get infected by viruses, the machines aren't easily penetrated and we rarely have to worry about malware issues.
Vista is supposed to be better than XP but how hard is that to do?
As for addressing the underlying problem. My best suggestion would be bounties for capturing virus writers, rewards for turning in spammers and a dedicated global program to hunting down and stopping the infections from taking place.
There is a solution which you, and many others, have completely missed. One of the more interesting aspects of malicious "spam-ware" infections is the code itself normally includes a SMTP engine. On the surface, this does not seem unusual for the world of malware but there is an unusual aspect -The SMTP engines in malware are usually not RFC compliant. The OpenBSD spamd will greylist or even blacklist non-compliant SMTP servers and feed the lists into the packet filter (pf). Game Over.
The problem is not receiving spam, but that there are hundreds of times more attempts to send that are hitting your mail server. When those attempts are just sitting there, locking down your server's connection, nobody can send or receive. If those connections were sending then your solution might have some effect (we do greylisting), but they're digital vagrants standing in the doorway, preventing others from passing in or out. You can't be insulted until someone actually says something, after all; all you can do is shove the vagrants out of the way when it's clear that they're not coming in-or-out. (And thus the barnacle scraper.)
In every mail server, port 25 has a finite number of connections. The bots are using them up. Real email can't get through. Even spam can't get through. (Spam is currently 20% of what it was a week ago, at least on the servers I see from here.)
--Esther
J.C. Roberts comments are not the cure, but are an important tool in fighting spam. When I was running my own Postfix server, I found that a huge amount of spam was eliminated just by enforcing SMTP rules (right in the mailserver). Another huge amount was then eliminated by a single, well-chosen (for lack of false positives) blacklist. The beauty of these approaches was that the connection was dropped early, before the body of the email was accepted, reducing the load on the mailserver.
I've been paying attention to spam fighting for quite some time, and (as soon as another editor is done messing with it), you'll soon see an article I wrote about techniques for dealing with the topic.
I wholeheartedly agree that enforcing the RFCs can do a huge amount to prevent spam from reaching the server and the user community. So can greylisting, and a number of other technologies. However, none of those do anything to prevent the barnacles from adhering.