… so I’m going to make the case that you should simply be blocking them en masse. But first, a little background.

Dynamic Addressing by ISPs

Internet Service Providers (ISPs) own blocks of contiguous IP addresses (aka “netblocks”) that they in turn assign to the computers that connect to the Internet through their service. When you sign up for DSL, Cable or even dial-up service and connect your computer to the Internet, the ISP assigns your computer an IP address from within a netblock it owns.

Some ISPs provide static IP services, where your computer keeps the same IP address all the time. But the vast majority of home computers are signed up for less expensive dynamic IP services, where your computer gets a new address each time it connects to the Internet. The address is often different every time you connect. But it’s always from within the same netblock.

This allows the ISP to sign up more customers than IP addresses it owns, kind of like how a bank is allowed to loan out money it doesn’t actually have in the vault. Since only a percentage of the ISP’s customers are connected at any give time, this works fine.

How to tell if an IP is in one of these Netblocks

My personal DSL provider is DSLExtreme. When my home computer connects to the Internet (actually, it’s my router), it’s always assigned an address in the netblock 72.25.123.0/24, which includes a range of IPs from 72.25.123.0 to 72.25.123.255.

A quick WHOIS lookup tells me that, indeed, DSLExtreme owns an IP allocation that includes that netblock. In fact, DSLExtreme owns 72.25.64.0/18, which is everything from 72.25.64.0 to 72.25.127.255 (16384 addresses).

Now, given any individual IP, it’s easy to tell whether or not the computer that owns that address is on a dynamic ISP network. Most ISPs loosely follow an IETF draft RFC that dictates what a reverse DNS lookup should look like for an address of this type. The typical form of the hostname is a concatenation of the IP address, some regional network identifier and the ISPs domain name.

Here are some real world examples:

IP Address	Hostname
72.25.123.40	netblock-72-25-123-40.dslextreme.com
220.129.160.241	220-129-160-241.dynamic.hinet.net
70.51.10.55	bas2-montreal19-1177750071.dsl.bell.ca
66.61.73.36	cpe-66-61-73-36.neo.res.rr.com
24.9.156.250	c-24-9-156-250.hsd1.co.comcast.net
201.253.146.69	host69.201-253-146.telecom.net.ar

How this relates to Bots

Now let’s tie this back to bots and proxies, by taking a look at some bot and IP data. (If you need a bot jargon review, you can get it here.)

Fast flux is a DNS technique used by botnets to hide phishing and malware delivery sites behind an ever-changing network of compromised hosts acting as proxies. After a bot infects a computer, it registers with the botnet and the IP address of the computer it infected is added to the list of IPs to rotate in the fast flux domain. If a domain is fast fluxing, it’s almost always a front for a botnet.

The recent escalation in Storm Worm activity came with some fresh fast fluxing domains. Using a passive DNS replication service, I harvested a set of IPs that had been mapped to a couple of those domains. These are IP addresses of computers infected with Storm Worm bots. I then ran reverse lookups on those IPs. Here are the full lists I pulled on Jan 2 for newyearwithlove.com and uhavepostcard.com.

The hostnames reveal a who’s who of ISPs — Comcast, Mindspring, Charter, Road Runner, HiNet, BellSouth, SBC, Deutsche Telecom, Earthlink, etc.

This should come as no surprise. Let’s think about the world of home computing. The average home user is an untrained computer user using an unmanaged, unpatched Windows PC. They don’t have the benefit of having a highly skilled professional such as yourself ensuring that AV updates and Windows patches are coming down in a steady stream. They are notoriously susceptible to social engineering. They are undoubtedly running as Admin.

Zombie computers aren’t marching from managed Enterprise networks, they’re out there on the Wild West home computer networks.

Your first instinct might be to download those lists and block them in your web filter or firewall or whatever ASAP. Don’t bother. Remember that all those home computers have dynamically assigned addresses. Those bots are just going to show up on different IPs tomorrow morning.

Here’s some data to back up that claim: The German Honeynet Project analyzed the recent Storm Worm run up, including this interesting graph of infections. Note the 24 hour fluctuation cycle. Thorsten Holz, one of the founders of the Honeynet Project, told Bryan Krebs of the Washington Post that the up-and-down pattern seen throughout the graph is probably caused by the regular, daily pattern of infected machines being turned off for the evening and restarted again in the morning.

How this relates to Proxies

Anonymous proxies fit this model as well. Proxy packages are designed to be installed on user’s home computers and accessed from work (an overview of this problem is here and how we handle it is here).

More importantly, sophisticated proxy applications like Ultrasurf rely on hordes of home users to opt-in as proxy servers, like P2P apps.

We recently tore open Ultrasurf (we can block it). In that process, we harvested more than 1500 IPs of proxy servers in the Ultrasurf network. Almost all of them were on home computers using dynamic addressing. Here’s the full list, but again, don’t bother, they just move around each day.

Block those Netblocks

I think that, by default, netblocks comprised solely of unmanaged and vulnerable home computers should not be trusted. If you block traffic from your network to these netblocks, you are improving the security of your network. You’re shutting down proxies and you’re cutting off bots.

Of course that’s just an educated guess. So let’s examine the two obvious problems with it:

1. How can you be sure that you won’t be overblocking, preventing your users from getting to business critical services that are out there on those netblocks?
2. Where is a comprehensive list of said netblocks?

Re: overblocking. Unfortunately, you can’t be 100% sure. It’s yet another security vs. usability trade off.  However, you can rely on some circumstantial evidence that there shouldn’t be anything out on those netblocks worth allowing.

In their service agreements, ISPs explicitly disallow their users from running services that your users would consider critical out on those computers. They don’t allow mail servers or web servers. If you want those services, they either give you a static IP or they host it for you. Either way, the IP address assigned to that host will be dedicated and not from the netblock used for dynamic addressing. There will always be exceptions, but I would argue that in this case, your end user needs to lobby IT to be allowed and not the other way around.

In fact, the anti-spam world has been all over this for a decade. A quick way to greatly decrease the volume of spam your mail server has to handle is to block these netblocks outright. If your mail server gets an SMTP connection from a computer out on one of these netblocks, it’s always a spammer.

Re: a comprehensive list. Different folks in the security space used to maintain dial-up user lists, which were intended to be full lists of these netblocks. Ownership has changed hands, and now the two prominent sources are the anti-spam organizations SORBS and SpamHaus. They maintain the lists as blocklists.

A quick lookup on the HiNet IP 125.232.0.0 illustrates my above point on overblocking. SORBS includes language from HiNet itself instructing users not to use those IPs for mail serving (can’t link to the lookup, SORBS uses a captcha and post) . But here’s the SpamHaus blocklist entry.

Unfortunately, those lists have some big omissions and disagreements, as I uncovered in my manual research of the Ultrasurf IPs. An accurate list would require each ISP to publicize its netblock usage like HiNet. Microsoft published some recent research on identifying these netblocks using analysis of MSN logins, so that’s promising.

In the meantime, 8e6 is building a list based on our own research. I’ll send it out to anyone who wants it.

We’re also working on an enhancement to our filter to allow IP ranges in the Master Library, which would allow us to provide a category that you could block and rely on us to keep updated. We would also provide an option to let a reverse DNS lookup overturn a block decision, which would give us the flexibility to maintain more coarse grained lists while handling exceptions on the fly.

Tags: Bots, home computers, ISPs, Proxies

This entry was posted on Wednesday, January 9th, 2008 at 2:17 pm and is filed under Bots, Proxies. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.