IT DRILLDOWN
 
NEWSLETTERS
 

CIO.com updates, insights and advice on technology, management and your career.

 
 
 
SUBSCRIBE TO CIO
 
Are you involved in setting the direction for your company's IT budget or strategy?

Apply today for a FREE subscription to CIO Magazine!

 


Thu, Feb 8, 2007 14:30 EST

The Bots that Ate the Internet

Topic: Infrastructure

Blog: You're the Boss

Current Rating: 0 Comments: 9

If the Internet seems to be running slowly, lately, don't blame it on your network admins. The problem hasn't been excessively reported (except among techies, who have generated plenty of forum traffic about it), but the Internet is currently suffering a massive slowdown because the spam bots have gone out of control. The conclusion among some admins, at least, is that it's caused by a bug in one of the zombie viruses (the ones that take over a Windows computer and turn it into a spam factory). Wherever the bots are coming from, it's slowing down—and bringing down—servers. Including yours.

The result is that email that used to arrive in a few minutes is taking hours. Your servers may have hundreds or thousands of inactive connections that just sit there and don't do anything but consume bandwidth. We've taken to calling them "barnacles," and my techie spouse has written custom software to scrape 'em off every so often. And even so, he tells me that the mail servers are running out of connections every five minutes.

As a manager or team leader, you're stuck. This isn't a problem that you can solve in-house. It isn't a matter of buying another box of technology.

Some companies are apparently coping by throwing more hardware at the problem. For example, one large midwest insurance firm (a household name) had a single mail server for quite some time. They have added three additional servers (one every week) in the last month. (That knowledge wasn't brilliant investigative journalism; it comes from reading message headers and email log files. Doesn't everyone?)

But "more hardware please!" isn't a longterm solution, because there isn't much being done to kill the source of the problem. That awful Stration virus (which I mentioned in another context some weeks ago) is the likely culprit (people who write viruses don't follow QA testing practices, I suppose), and it's evolving faster than the anti-virus tools can respond.

Assuming that such tools are installed in the first place, that is. A few weeks ago, I was called to familial tech-support duty to detoxify a nephew's system, which had 43 (!) separate infections, at least one of which was a "you're infected—click here to spend $49 on an anti-virus app" shudder-inducer. David's system is no longer contributing to the problem (I hope!) but how many other unprotected home computers are giving your admins the heebie-jeebies?

This is the point where I'd ordinarily spell out the alternatives and then ask which of them is the one you personally consider most wise. Except, in this case, I don't know what the alternatives really are. (I've discounted the spouse-suggested, "Turn off all Windows computers" as regrettably unfeasible. Besides, he's cranky because he's having to squash someone else's bugs.)

It's left me to mull over the fact that John Brunner was right in his predictions about the Internet in his 1975 novel Shockwave Rider; perhaps his solution (a worm sent out to eat the worm) is the most plausible as well.

How are you dealing with it? Or haven't you noticed this problem?


You do not have flash or javascript support.
Average (0 votes)
0
 
 
Thu, Feb 8, 2007 17:11 EST
Anonymous user
Posted by: Erik
Rating:

Contrary to recent comments by Mr. Gates, Macs don't get infected by viruses, the machines aren't easily penetrated and we rarely have to worry about malware issues.

Vista is supposed to be better than XP but how hard is that to do?

As for addressing the underlying problem. My best suggestion would be bounties for capturing virus writers, rewards for turning in spammers and a dedicated global program to hunting down and stopping the infections from taking place.

 
Thu, Feb 8, 2007 21:02 EST
Anonymous user
Posted by: J.C. Roberts
Rating:

There is a solution which you, and many others, have completely missed. One of the more interesting aspects of malicious "spam-ware" infections is the code itself normally includes a SMTP engine. On the surface, this does not seem unusual for the world of malware but there is an unusual aspect -The SMTP engines in malware are usually not RFC compliant. The OpenBSD spamd will greylist or even blacklist non-compliant SMTP servers and feed the lists into the packet filter (pf). Game Over.

 
Fri, Feb 9, 2007 9:36 EST
Posted by: admin
Rating:

The problem is not receiving spam, but that there are hundreds of times more attempts to send that are hitting your mail server. When those attempts are just sitting there, locking down your server's connection, nobody can send or receive. If those connections were sending then your solution might have some effect (we do greylisting), but they're digital vagrants standing in the doorway, preventing others from passing in or out. You can't be insulted until someone actually says something, after all; all you can do is shove the vagrants out of the way when it's clear that they're not coming in-or-out. (And thus the barnacle scraper.)

In every mail server, port 25 has a finite number of connections. The bots are using them up. Real email can't get through. Even spam can't get through. (Spam is currently 20% of what it was a week ago, at least on the servers I see from here.)

--Esther

 
Sun, Feb 11, 2007 20:42 EST
Anonymous user
Posted by: George Dinwiddie
Rating:

J.C. Roberts comments are not the cure, but are an important tool in fighting spam. When I was running my own Postfix server, I found that a huge amount of spam was eliminated just by enforcing SMTP rules (right in the mailserver). Another huge amount was then eliminated by a single, well-chosen (for lack of false positives) blacklist. The beauty of these approaches was that the connection was dropped early, before the body of the email was accepted, reducing the load on the mailserver.

 
Mon, Feb 12, 2007 10:51 EST
Posted by: admin
Rating:

I've been paying attention to spam fighting for quite some time, and (as soon as another editor is done messing with it), you'll soon see an article I wrote about techniques for dealing with the topic.

I wholeheartedly agree that enforcing the RFCs can do a huge amount to prevent spam from reaching the server and the user community. So can greylisting, and a number of other technologies. However, none of those do anything to prevent the barnacles from adhering. 

About this Blog

How would you solve this management problem?

Start a Conversation
Click to post

Got something to say? We want to hear it! Click the Post button to get started. GO»

EXPERT ADVICE
See our roster of experts.

Advice & Opinion from more than 96 of IT's most insightful thinkers.

advertisement

  PARTNERS       PODCASTS       WEBCASTS    
 

From Laggard to Leader: Transforming the Data Center

This webcast offers an understanding of how customers are transforming their data centers, the successes and challenges of each approach, and how IT can become the driver to provide real business value and competitive advantage.

Sponsored by HP  Register for this Webcast »

 

Raising the Bar on Business Service Delivery

Applications drive every business, but as networks become more complex and dynamic, performance has become a key tenant in service delivery. In this CIO webcast, Forrester and Fluke Networks offer advice and best practices for ensuring high delivery with better application performance.

Sponsored by Fluke  Watch this webcast. »

 

The Universal Wireless Client

Learn how replacing multiple wireless clients with one Universal Wireless Client can cut support and help desk costs, increase end user satisfaction, improve security, and help implement Network Access Control.

Sponsored by Fiberlink  Read this White Paper »

Resource Alerts

Get instant email notifications by topic when white papers, webcasts, and case studies are added to our library.

NAC launch from HP Procurve Podcast with Lippis Report, Part 1

ProCurve Networking by HP joins the Lippis Report to announce major product and organizational additions to their ProActive Defense strategy.  Read More »

 

Accenture's View on Web 2.0 and its impact on business

Publisher at CIO magazine, Bob Melk, talks to Accenture's Blair Jones about the emergence of Web 2.0...  Read More »

 

A Best-Practice Framework for Virtualization

This podcast offers insights and perspective on the various issues that relate to virtualization...  Read More »

Resource Alerts

Get instant email notification when white papers, webcasts, and case studies are added to our library. Don't just be up-to-date—be up to the minute with our new Resource Alerts.

CIO Viewpoints on Exchange 2007 Risks and Mitigation Strategies

Knowing where your peers have found limits and workarounds in areas including high availability, archiving, recovery, compliance, e-Discovery and storage growth can be essential in planning your successful Exchange 2007 migration.  Read More »

 

Find out what Forrester says about mobile endpoint security and its management.

Mobility raises productivity. But IT departments are hard-pressed to protect mobile data and to manage security software, wireless clients and regulatory compliance for mobile workers...   Read More »

 

Get Forrester's take on simplifying mobility with the universal wireless client.

Mobile workers want to use all types of wireless networks: WiFi, 3G cellular networks, corporate WLANs and home wireless networks. But how can IT support...  Read More »

Resource Alerts

Get instant email notification when white papers, webcasts, and case studies are added to our library. Don't just be up-to-date—be up to the minute with our new Resource Alerts.

 
NEWSLETTER

Sign-up for the Blogs & Discussion Newsletter

 
FEATURED SPONSORS
 
 
 
SPONSORED LINKS
 

Build up or Tear down? See how UC makes sense with Nortel. Calculate your UC ROI

Predict the future with HP Insight Power Manager

Drive Business Value with Enterprise Social Computing - whitepaper

See how IBM helped Bharti create a new business model

Read how IBM helped Hughes enhance security

HP LaserJet M3035 MFP series starting at $1,599. » SHOP NOW. www.hp.com

NEW HP Color LaserJet CM3530n MFP starting at $2,499. » SHOP NOW. www.hp.com

Affordable technology-no compromise. HP server solutions

Make IT Work As One@novell.com

Learn about the software-based VoIP solution from Microsoft

CIO Starter Kit includes useful resources created by top CIOs. Free Download>>

Rolling the dice with your security? Take the Self-Assessment Test now

Request a Novell/Microsoft deployment workshop and kit

Request a Novell/Microsoft deployment kit

Compuware.com - See how we make IT rock around the world

SOA Educational Library at the TIBCO SOA Resource Center

A fresh look at the impact of customer intimacy.

The Right and Wrong Master Data Management Strategies to Start Small and Grow Big

Learn how to leverage virtualization for a 74% savings in TCO.

Find out how you can affordably consolidate applications with VMware.

ESG Research on Server and Storage Virtualization

Webcast: Mitigate Operational Risk- Real Answers for Tough Times

Laptop Security: Where Do CIOs See Weaknesses?

How RFID Improves Data Center Efficiency

Paving the Way for Trusted Collaboration

SAS a Leader in Forrester BI report. Click here to see evaluation.

Protect data-HP All-in-One and Disk-Based systems

Microsoft SQL Server 2008. Read Case Studies, Watch Demos, & Download for Free

The 2008 CEO Study: Implications for the CIO

HP LaserJet P4014n printer starting at $799 after $100 IS. www.hp.com

NEW HP Color LaserJet CP3525n printer starting at $699. » SHOP NOW. www.hp.com

Predict the future with HP Insight Power Manager

A new level of interoperability. Make IT Work As One@novell.com

Businesses Transform with VMware Virtualization

IT Service Management: Metrics That Matter

Download the free CIO Starter Kit to access useful resources created by top CIOs

Log onto Hitachi True Stories, films inspired by the next great achievement

Request a Novell/Microsoft deployment workshop

Strong Authentication. Secure USB data storage. One Device

Discover PMI's credentials and career path tools

Learn how companies are changing how they reach out to their most profitable customers.

Discover what you need to consider when evaluating virtualization.

Webcast: SOA Brings Backend Systems into the Future, Rapidly & Successfully

Find out why IDC thinks virtualization is changing operating environments.

Explore the impact virtualization can have on your bottom-line.

Save with 0% Lease Offer on HP Servers and Storage

The Customer Communications Management Platform - Key Functionality and Best Practices

Data Center ROI with RFID Asset Tracking

Learn how the new Quad-Core AMD Opteron™ processor improves performance

Get help navigating the management challenges of virtualization.