Blacklisting blogs
Neville and I were copied on an email from Steve O’Keefe over at the IAOC in which he said he tried sending out the association’s newsletter but it kept bouncing back. The reason, he learned, was the URL for our podcast, ”For Immediate Release.” The URL was in the body of the message, but the mail server at the IAOC’s Internet Service Provider wouldn’t let the email go through as long as our URL was included. We were, it turns out, on a couple of blacklists.
Specifically, we were listed on some RBLs, which stands for “Real-time Blackhole List.” One was through a mail provider called Outblaze, the other a private service called SURBL. According to the SURBL Web site, “SURBLs differ from most other RBLs in that they’re used to detect spam based on message body URIs (usually web sites). Unlike most other RBLs, SURBLs are not used to block spam senders. Instead they allow you to block messages that have spam hosts which are mentioned in message bodies.”
Email spam is at the heart of these blacklists. That confounded Neville and me, since we don’t send any email at all from “For Immediate Release.” The site, in fact, is a blog. We have an email address so listeners can send us their comments, but when we reply we use our personal email accounts. Not one email has ever been sent from the forimmediaterelease.biz domain. Our guess: somebody spoofed our domain in a spam, although we can’t get Outblaze to let us know if that’s what happened.
Once we got it resolved, which required a flurry of emails, Neville found out his domain, nevon.net, was also blacklisted. Another round of emails was required to have his domain removed from the blacklist.
In both instances, the owner of the SURBL site suggested he’d feel better about whitelisting us if we had spam policies on our sites. It was the combination of the vigilante approach to spam coupled with the requirement to publish an email policy that raised my eyebrows. At their core, of course, blogs are web sites. But they are part of what I have taken to calling the “collaborative” or “social web,” not the “reference web” with which most people are most familiar. How many blogs distribute email of any kind? Damn few, I suspect. Should bloggers be forced to post email policies just to comply with individuals creating blacklists that ISPs use to keep spam out of their customers’ in-boxes? How many bloggers have given any thought to posting email policies? How many bloggers have even figured out whether their blogs’ URLs are on a blacklist?
Spam is a problem, to be sure. It’s one of the reasons RSS is growing some popular. And I applaud anybody who can figure out a way to deal with the spam problem. Spam has had a serious impact on the level of trust most people apply to email. But I’m troubled by individuals with no formal authority who add domains to blacklists without notifying the domain owner. If, by chance, a domain owner discovers he’s been blacklisted, rectifying the situation can be time-consuming and difficult.
I’ve added an email policy to the “For Immediate Release” blog just to avoid any further problems with vigilante anti-spammers. But there has to be a better way to go about this than penalizing innocent site owners and bloggers who have never sent a single email of any kind through their domains.
If you want to check your domain, here’s one place where you can find out if you’ve been blacklisted: http://www.rulesemporium.com/cgi-bin/uribl.cgi
Was it really so “time consuming and difficult” to send us an email so we could unlist your domain? Many other RBLs don’t even listen to or acknowledge complaints. We removed your domain within 24 hours of your contacting us (on a holiday weekend). I wonder whether a commercial service would be as responsive or helpful.
SURBL lists are being used to block literally billions of spams per day. SURBL is a voluntary effort organized over the Internet with dozens of people around the world helping to provide data or services. Many more Internet users provide reports that also drive our listings.
Our goal is to have zero errors: to list only major spammers, but perfection is seldom easy to attain. Despite that, we expend a lot of energy, resources and programming to try to keep legitimate domains off our list. We are aware of the power of our tools and are attempting to wield them responsibiliy. Instead of painting us as irresponsibile, perhaps a fairer reading would take into account our willingness and ability to delist legitimate domains such as yours.
As far as “no formal authority,” welcome to the Internet. The government is nearly powerless to stop spam, and when it tries, it comes up with seemingly worse than useless solutions like CAN SPAM which is appropriately named since it allows many spammers to send spam *legally*.
The two most effective antispam services on the Internet are SpamHaus and CBL. Both are non-govermental and both are responsible for keeping the Internet working. Without them, you would not be able to use email or possibly even the web or blogs, since the infected senders and virused PCs they list could be used for many nefarious purposes other than just attempting to send multiple billions of spams per day. Without those “vigilante” services, the Internet might very well crawl to a halt. Certainly email would be nearly unusable under the deluge of spam actually getting through. Imagine getting ten thousand spams for every ham (desired message) in your mailbox, or ten thousand blog spams for every legitimate posting, and you may get an idea of how Internet life would be without these “vigilantes.”
Some of these “vigilantes” also provide data to the FBI and other international, national and local police organizations about criminal spam gang activity. Governmental cybercrime resources are pretty severely limited, but occasionally they are able to make good use of this information, and we see spammers indicted for their crimes or put in jail:
http://www.businessweek.com/magazine/content/05_22/b3935001_mz001.htm
Cheers,
Jeff Chan
Posted by Jeff Chan on 05/30 at 04:35 PMShel, you completely miss the point.
You should be directing your anger at the criminals who spoofed your domain, not at the end users who are only trying to protect themselves from spam.
Want to fix the problem? Help put spammers in prison, instead of attacking end users who are only trying to defend themselves against the avalanches of email trash by these criminals.
You yourself had to implement spam blocking on your blog by using a graphic capcha verifier. You realize this unfairly blocks sight impaired individuals from posting to your blog? Every method of spam blocking has imperfections. Your blocking has flaws. So does SURBL. Deal with it.
Your effort would be better used attacking spammers, not those trying to defend themselves against spammers.
Posted by Dan on 05/30 at 05:02 PMThere are three problems that I see.
#1. How did you end up on Outblaze’s list? Even if someone spoofed your domain name, it should be trivial to verify it.
#2. How did you get on SURBL’s list? I may be wrong, but isn’t that for URL’s found in spam email? Did someone intentionally set you up by sending out a spam blast with your URL on it?
#3. Why was IAOC blocking your email just because of that URL? Was it just because of that URL?
Without the answers to those questions, we cannot identify the real issue. It is possible that someone, for whatever reason, decided to get your site blacklisted. There isn’t much that can be done about maliciousness such as that, if that is what happened.
Posted by Brandioch on 05/30 at 05:38 PMShel, a great deal more research could have gone into this post before applying the vigilante stamp and going after highly respected volunteer service. A great many people work on SURBL to maintain quality data, but given the huge number of domains to consider, mistakes will occur from time to time. However, as has been made clear a number of times, SURBL does not block messages, but rather helps other filtering systems such as SpamAssassin decide how to handle a message. By default, the vast majority of filtering systems including SpamAssassin will not flag a message as spam based on a single SURBL hit. Either the IAOC mail administrators adjusted the default scores upwards or your email had other elements that combined with the SURBL hit caused it to be flagged as spam. It would be great if you could do a bit more research and see if your post ought to be ammended.
Posted by Andy on 05/30 at 06:24 PMShel, I think you severely underestimate the scope of this spam problem. Our mail servers receive tens of thousands of spams every day - and without these amazing free solutions like SURBL and SpamAssassin I would probably be forced out of business. Spammers have caused us countless hours of grief over the past few years - I will do anything to stop them - anything. That means occasionally innocent sites get blacklisted. If you ask me its a small cost to pay. Your vigilante is our hero.
Posted by USA Church Admin on 05/30 at 09:07 PMThe posting script on this page blows up if you leave something out, such as your email address. Then everything you typed is gone because the BACK button on the error page does not work!
Posted by Greg on 05/30 at 09:44 PMShel,
While you’re out there whinning, posting stuff you haven’t thoroughly researched, your OH SO COOL! blog doesn’t even munge poster’s email addresses.
If there was a clueless-blogger.surbl.org I’d request you get listed NOW!Posted by SpamHater on 05/30 at 10:52 PMShel, I’m a PR consultant, and very frankly I already suggest clients using email newsletters to switch to RSS feed or at least to add RSS feed aside email newsletter, giving RSS as an opportunity. The RSS feed is by far a better solution under so many points of view; the crucial point of course is that the target community has to be open to adopt the technique. But I believe this is the future. We already offer journalists this opportunity : at the moment is only an experiment, but I’m confident.
Posted by Enrico Bianchessi on 05/31 at 02:21 AMWow. Such a lot of angst! I find it amazing that a PR blog draws the most comments about posts that had nothing to do with PR—such as podcasting software or, in this case, spam.
Did I ever suggest these services shouldn’t exist, or did readers (obviously not regular readers of this blog) fly off the handle because I dared pose any questions at all?
So let me reiterate and amplify. I thought Jeff Chan’s response time was amazingly good, particularly over a weekend. I think Jeff is performing a necessary and important public service. My assertion that rectifying a situation in which a blogger finds himself blacklisted can be difficult and time-consuming has nothing to do with Jeff’s service. It took me months to resolve a similar problem, though, with AOL. In fact, I’ve had to deal with it with multiple services, none of which ever notified me that my domain was on such a list.
I’ve re-read my post several times and simply can’t find the “attack.” It’s a recounting of an experience and a discussion of the new wrinkle related to blogs that don’t send email.
I am more aware than most of the depth of the spam issue, since I work in this area and subscribe to regular updates on the topic. (Who among the commenters reads “Michael Osterman on Messaging,” which deals almost exclusively with spam and its various effects and solutions? How many of you are aware of a new report by the Organization for Economic Cooperation and Development (OECD) that shows spam causes more grief for developing countries than others?)
My ONLY complaint with these services themselves is the failure to notify anybody that they have been blacklisted. If not for someone from the IAOC notifying Neville, we might never have known mail containing our podcast domain was being blocked. The balance of my issue addressed sites that SEND NO EMAIL. Why is this such a difficult concept? Technorati tracks 10 million blogs and I haven’t seen one of them with an email policy. This is not a criticism of Jeff (who, again, was gracious, professional, and helpful) or anyone else, but a discussion of the notion that the Web is branching into a couple distinct paths, one of which simply doesn’t involve email, and the fact that much of the Internet world hasn’t caught up, continuing to lump all sites into one big category. Do we really expect every blogger to add an email policy to their blogs?
I am working hard, by the way, to migrate the 4,000 readers of my email newsletter to an RSS feed. Email for legitimate mass distribution is dead. Spammers killed it. Yes, I despise spammers. I get about 1,000 spam messages per day in my spambox; I pay hard-earned money to SpamSoap to keep those messages out of my in-box, but still have to deal with them routinely or my spambox would fill my server. But my post was about our experience being blacklisted, not not about spam. My post was also designed to alert other bloggers that they may want to check to see if they’ve been blacklisted. They could be just as unaware of the potential as Neville and I were.
To clarify, IAOC was not blocking our email. THEY were being blocked—they could not send THEIR email newsletter as long as OUR domain appeared in the body of the message. It was THEIR ISP blocking based on our domain appearing in the SURBL list. See? I really do get it.
I apologize if the word “vigilante” caused offense. I meant it in the purest sense of the word: “One who takes or advocates the taking of law enforcement into one’s own hands.” The governor of California is applauding Texas’ border vigilantes and asking them to come to our state, for heaven’s sake. I did not mean it in any negative way.
Finally, I appreciate everyone who, while clearly upset with what I wrote, was considerate and civil in their comments. Civil discourse is what blogging should be (but isn’t always) about.
Posted by Shel on 05/31 at 04:58 AMAs a former member of SURBL, and now running another list at URIBL, I have to comment. We add thousands, yes thousands of domains a week. We have no time to inform people they are listed. We don’t get paid for this work.
We all strive to get zero false positives in our lists. People in the industry know how much Jeff cares about reducing these.
The response rates for URIBL and SURBL are faster then any other RBL out there. When we make a mistake, we fix it ASAP.
I think if you used these FREE services on your own email account you would be singing the praises, instead of labeling us as vigilantes and “individuals with no formal authority”. Your right though, we are. Because the people with authority have their head so far up the DMA’s rear, they produce crap like the CANSPAM act.
We found a solution that works. Its free. We work hard. If you knew how much work went into the list, you would be amazed. We do it to help, not hurt.
ANyway I need to end this rant before I hurt myself
May your inbox be spam free.
(I didn’t even mention how we tell people NOT TO BLOCK based on our list!)
PPS: Your subbmission says my email was invalid. Its broken. Plus, ‘+’, is a perfectly legit charachter in an email address.
Posted by Chris Santerre on 05/31 at 06:01 AMRegarding notification, while I appreciate a reasonable desire for domain owners to be notified when theirs are listed, as a practical matter it’s not really feasible. We list more than 100k domains with hundreds to thousands added each day. Most of those domains have fraudulent registration information provided by the spammers. (Often the registration information they use is from stolen credit cards or a result of identity theft, i.e. they use one of their victim’s contact information on the domain registration. Other times, the information is (changed to) meaningless gibberish.) As a point of fact, criminal spammers register many hundreds of domains per day, use them for a few days in spams, then abandon them. Trying to contact registrants would be difficult and frequently unsuccessful at best, and at worst an indication that it’s time for them to switch to another domain and attack the reporter.
If anyone can suggest a way to overcome that, we’d like to hear about it.
Posted by Jeff Chan on 05/31 at 06:12 AMAnd to give you one example as to how much Jeff cares about getting it right, I once tried to add 27,000+ domains to SURBL at once. Jeff refused because there *might* have been some legit ones in error.
And as Jeff states, 90% of the data on domains is bogus.
Removal from SURBL does not *require* an antispam policy. It just helps the process. Many sites have thanked Jeff for helping them with policies.
Posted by Chris Santerre on 05/31 at 06:17 AMIMO The issue seems to be that a domain that was *apparently* never used to send email ended up on a blocklist/blacklist.
Just as email misuse is spoiling it for legitimate users so blacklist misuse spoils it by poisoning filter lists.
If a blocklist can be “poisoned”, for example by spoofing legitimate domains then it could be rendered useless by organised spammers.
Posted by Oliver on 05/31 at 06:19 AM“Please someone police the internet, but don’t police my blog.”
Okay so these lists have to send mail to spammers to tell them “Sorry buddy, you’ve been blacklisted at so and so”. Nice way for them to know they have to change their zombie relay for their spam to go through. A little like undercover cops warning the people that they investigate them.
These lists don’t list you for no reason, they have a spam with your domain in it. You say: “Not one email has ever been sent from the forimmediaterelease.biz domain.” But you can’t be sure of that, unless you administrate your own domain mail server, and you’re your own ISP.
That way you could have seen when someone used your domain for spam. And could have done something about it in time.Posted by Dino Trudel on 05/31 at 06:23 AMJeff, I appreciate the difficulty of the situation; hence, my desire to alert bloggers who would not be aware of any of this. Here’s my position: Technorati is tracking 10 million blogs, suggesting there are far more than that. The surge in blogs’ popularity is based on the fact that you need absolutely no technical expertise to start or maintain one. Thus, any blogger who has created a domain for their blog (rather than using the default service URL) is at risk for blacklisting without having the background or expertise to be aware of the situation or ferret out the means of addressing it. Once again, I’m not criticizing your or your service—but I am trying to point out that it raises a whole new set of circumstances for this new realm of blogging.
Chris, I appreciate everyone’s efforts, but the “no time” excuse bothers me. “We process hundreds of detainees every week and have no time to make sure we haven’t falsely arrested anyone.” C’mon. That’s legit? The notion that anyone performing a valuable public service is beyond constructive criticism is disturbing. While I agree that the authorities are far less effective than the volunteers, we should be constantly striving to make the system work effectively and fairly from all sides. Surely there’s a technical solution, such as an instant, automatic notification to the postmaster for any domain blacklisted so you don’t have to actually do anything.
By the way, Chris, I use Expression Engine for my blog, so I’ll be notifying them about the plus (+) issue. You’re right; that shouldn’t cause a problem.
Posted by Shel Holtz on 05/31 at 06:24 AMI agree with you Oliver. And we refuse tons of submissions. But when you add thousands a day, the odds catch up with you and one slips thru.
I have yet to see the reason why his domain was added. But we pay very close attention to look for poisoning our lists. And have some great research tools to help us out. We often no more about the domains then the owners.
But we don’t list domains that SEND emails! So it doesn’t matter. You seem to have missed that. We list domains LINKED in UCE. We don’t care where they were sent from, because they are mostly sent from Asia or Zombiesphere.
--Chris
http://www.uribl.comPosted by Chris Santerre on 05/31 at 06:26 AMShel, i see your point, but this is exactly what we try to do. We take extreme due diligence in listing domains. Jeff is *Fanatical* about it. We have custom tools designed so we don’t make mistakes.....often
You seem to think we should never be able to make a mistake? The FP rate is something ridiculous like .09%. And Jeff isn’t happy with that rate at all!
We have a whitelist with something like 70k legit domains. This way we NEVER add someone like yahoo.com or aol.com. And if you have a list of 10k legit blog sites, guess what? Get them to us and we will whitelist them as well!
Automatic response systems do NOT work. How about domains with an MX record of 127.0.0.1? Or the fact that we would be informing the spammer so he could change the URL in his spam run? What about Joe Jobs with bogus email addresses in whois data?
Do you have an example of another RBL that informs people of listings?
Posted by Chris Santerre on 05/31 at 06:37 AMDino, I find it interesting that you put quotes around your opening salvo. Did I say that? Or suggest it? What I said (and repeated) was this: Blogging is a new facet of the Net and much of the Net hasn’t caught up to it yet. Many bloggers have no technical expertise at all. They went over to blogger.com or typepad.com, signed up for an account, and started blogging. Some may have paid the extra few bucks for their own domain. They are clueless about blacklists, whitelists, and every other issue under discussion here.
Making a phone call used to require going through a switchboard operator who knew the technology. Today, few using a phone understand the technology. The Net is moving in the same direction—as it should. It will not be the sole domain of people who “get” the technology. And all these people are at risk for blacklisting and have no idea. I’m only suggesting that THEY should be aware and that the blacklisting services—which are great, terrific, wonderful, necessary, add your adjective here—need to factor this population into their thinking.
Is anybody here suggesting that anti-spam services are beyond reproach or above criticism just because they perform a valuable and necessary service? Is it really better to beat those who dare raise an issue into submission?
Posted by Shel Holtz on 05/31 at 06:38 AMChris, I’m just suggesting that options be explored rather than the situation dismissed. Once again, my main point was to raise the awareness flag for the bloggers, not to flog the anti-spammers.
Posted by Shel Holtz on 05/31 at 06:40 AMI see that. But many of the major blog sites have already been in contact with us. Blog spam is something they also fight. And they contain....can you guess.....hold on.....URLs!
So there have been talks about merging the technologies.
Everyone and their brother has a blog these days. You are correct. And blog sites almost never come up in our listings. I still don’t know how your got added.
I’m not dismissing you opinions. You say you are trying to inform other bloggers. I’m trying to inform/educate your audience as to what exactly we are up against. THe antispam world is complex and convaluted. I don’t expect bloggers to understand everything we do.
“Is it really better to beat those who dare raise an issue into submission?” Oh this isn’t a beating
Anyone who knows me can tell you I’m going easy on ya! See the problem with antispammers is...we like data. A lot! So if anyone proposes some new process to help us, and they don’t have data to back it up, well....we get a little crazy. You’ll have to forgive us our nature.
--Chris
http://www.uribl.comPosted by Chris Santerre on 05/31 at 06:49 AMSorry about the quote marks, only my interpretation.
Sure you can question their service, and that’s why you get responses here so you can understand that it’s a free service, that not everybody uses them, and that sending a few emails and waiting patiently for them to remove you is not that big a deal. Net hiccups occur, sometimes they are machine errors, sometime human.
Posted by Dino Trudel on 05/31 at 06:55 AMSorry, Chris, the “beating into submission” jab wasn’t aimed at you. You, Jeff, and several others have been great, and this conversation is EXACTLY what blogging should be about. There have been a few comments, though, that were, um, less than civil.
Posted by Shel Holtz on 05/31 at 06:55 AMIt occurs to me, Chris and Jeff, that what you guys need is some good PR. Have you ever considered seeking some pro bono work to help raise awareness and understanding around what you do and how people should address any issues that arise?
Posted by Shel Holtz on 05/31 at 06:59 AM“There have been a few comments, though, that were, um, less than civil.”
You want me to add their domains to SURBL?
(That was a JOKE Jeff!)
In all seriousness, where can I get a listing of all these blog domains? We are interested in keeping legit blog sites off the lists.
Jeff and I have beat eachother up far more then anyone else could. But he cheats! He puts nails in a 2X4, and hits you when you’re not looking!
Posted by Chris Santerre on 05/31 at 07:01 AMChris, Technorati (http://www.technorati.com) is tracking some 10+ million blogs. Contacting Dave Sifry at Technorati would be a good start. In fact, some kind of collaborative effort between you guys and Sifry would probably gain a lot of recognition and awareness for both of your efforts.
Posted by Shel Holtz on 05/31 at 07:04 AMAs the other half of the ‘For Immediate Release’ podcast team, I’d just like to add my own comment that the issue Shel and I experienced over the weekend with our domain appearing on a blacklist was indeed resolved speedily by Jeff and Suresh at Outblaze once Shel told them of the situation. So no criticisms on response ability and speedy resolution.
Likewise with my own domain that I learned on Monday was also on a blacklist. Again, Jeff and Suresh sorted that out very quickly. Shel and I discussed our experiences in yesterday’s edition of our show.
In reading Shel’s post and everyone’s comments here, I’m left with lots of questions. What I’m really trying to understand is how both the podcast domain and my own became blacklisted. Bear with me, please, in this understanding process.
Chris mentioned automated tools. So is the whole thing an automated process? Or does a domain get added to a blacklist manually, ie, a human being makes a decision on adding one or not? Likewise getting a domain off a blacklist. With my domain problem, an email to Jeff resulted in a response from he and Suresh in effect saying, no worries, we’ve taken you off the list.
Is this how it all works?
Does it also mean that, now that our podcast domain and my own domain are in a whitelist, they are regarded as squeaky clean? Yet what happens if at some future time somebody out there starts spamming using, say, my domain as his email address? Or my domain is referenced somehow or somewhere in a spammer’s email blitz? What happens then? As I mentioned to Jeff, my domain is used purely for my blog: it doesn’t even have an email set up, and no email has ever been sent using that domain. Well, not by me at least.
I tell you, this experience has been a real eye-opener in learning something about what’s going on in a very dark world of email and spam. If I’m typical, then I’d say that the vast majority of business people out there just have no idea of what people like Jeff, Suresh and Chris do, why they do it and how they do it. Shel’s suggestion re PR, pro bono or not, makes a lot of sense to me. As does his suggestion to talk to Technorati.
This is a great conversation, by the way, and I’m glad to join in!
Posted by Neville Hobson on 05/31 at 08:37 AMNo automatian. All done by humans looking at all the domains. What we have are automated tools to help us gather lots of info quickly and easily on a domain.
Some people use automatian to pull out domains from emails that need to be looked at by a human. But it still all comes down to someone having to actually look at it. We get quite fast at it.
Sometimes all it takes it cuting and pasting the wrong list into daily submissions. I once added 200 domains incorectly this way. But removed then within 2 minutes.
As for you being removed and/or whitelisted, this is both done. When something seems ‘fishy’ we remove, but don’t whitelist. But for something completely legit, like your domain, it gets whitelisted. Which means you can’t get readded to SURBL. I didn’t whitelist you in URIBL...yet
Oh the world of spam is very interesting. I suggest reading “The spam kings”. great book. We get all sorts of things happening. I myself have received death threats from spammers. They don’t play very nice at all.
Posted by Chris Santerre on 05/31 at 09:11 AMShel’s real complaint lies with IAOC who misused the RBL, not with the RBL itself.
SURBL does not block anything. They are a database. How end users choose to (mis)use that database is up to them, just like anyone can pick up a phone book and use or misuse it.
AFAIK using URI RBLs to block email is NOT RECOMMENDED.
Why don’t you ring up IAOC’s ISP and ask them why they block on SURBL listings?
There is also no way for anyone to know whether a domain is a “blog domain which never sends any email”, or anything else. All they know for certain is if a domain exists or not, and if someone sent email claiming to be from that domain or not.
The SURBL does exactly what it was designed to do—list URLs that appear in spam emails so others can use that information. How end users choose to use that information is up to them. IAOC’s ISP chose to misuse it.
Posted by Dan on 05/31 at 09:13 AMQuestion: using RSS feed to distribute newsletter or press releases content, is any of the above reported SPAM-related issues annoying anyone ???? No ??? So what ?…
)
Posted by Enrico Bianchessi on 05/31 at 11:39 AM“Blogging is a new facet of the Net and much of the Net hasn’t caught up to it yet. Many bloggers have no technical expertise at all. They went over to blogger.com or typepad.com, signed up for an account, and started blogging. Some may have paid the extra few bucks for their own domain. They are clueless about blacklists, whitelists, and every other issue under discussion here.”
I’d say that the blogs haven’t caught up with the net. The rest of us shouldn’t have to bend over backwards just because of the influx of blogging newbies. I, for one, an growing tired of the “dumbing down” of the internet just so some bloggers (most of which I feel add no value to the internet whatsoever) can hop on. Perhaps bloggers *should* learn about whitelists/blacklists/privacy policies before they launch their blogs.
I am sad to say that I used to work for some spammers. for a blacklist to notify a domain owner that they are blacklisted is simply not an option. It’s ridiculous to even think that would be a viable option. This particular company would register 2,000 domains at a time without setting up *any* email for any of them. Then they’d use the domains to blat out *millions* of emails every night. Any attempt to notify the domain owners would simply be bounced right back anyway.
Spammers *do not care* - so sadly, we must take a harder approach. Fortunately, some of the RBL admins actually do take time to verify their lists. That is pretty rare…Blogs are also responsible for an increase in spam as it is.. it’s time for the bloggers to stop expecting the internet to dumb down to their level.
Posted by Bob on 05/31 at 03:34 PMOh, wow. Others were right. This blog software makes absolutely no attempt to mask email addresses at all. There is no excuse for that.
If blogs are the “new face of the Net” than I think it’s time we tear the whole thing apart and start over.
Posted by Bob on 05/31 at 03:39 PMShel does have a point. People who are setting up their own domains with no knowledge of rbls or uribls are unlikely to find out about them unless they become blacklisted or they are informed.
I guess Shel was trying to inform bloggers of this issue. I think that the way people are going to find out is if the major blog related sites or even domain registrars are the ones offering the information.
If i want to write a blog, and buy a domain, it would be great to have information on RBLs and the like with pointers to avoiding being listed, and perhaps a premium service of being warned if you are listed.
As previously said, the blacklist managers aren’t really in a position to effectively warn all listed domains. But if when you sign up for a domain, for a fee (or not) the registrars can provide a service to check the well known rbls and uribls against your domain/ip as well as those of your nameservers (spammers often change domain names but keep their name servers) against the various lists, and inform you if you become listed on any of them, with instructions as to how you might go about delisting.
As well as the:
http://www.rulesemporium.com/cgi-bin/uribl.cgi
tool for checking for domains in the uribl lists, there is a tool on:
http://www.dnsstuff.com/
which allows you to check ips against many different blacklists.All of these tools are fantastic, and i really appreciate the effort of those developing them, but i agree that it would be helpful to have a warning system. I think this has to come from the consumer side, checking if they are listed, rather then the list managers having to inform.
Ben
Posted by Ben on 05/31 at 04:32 PMBob’s right. The internet should remain the province of the technologically proficient and inaccessible to the average citizen who can benefit from its use. We should keep it a private club for gearheads and deny admittance to those who don’t have the time or inclination to learn about RBLs. In fact, let’s scrap the web and return to telnet. Let’s go further than that. Back in 1991, “Internet Business Journal” editor and publisher Michael Strangelove received death threats from Internet users who objected to the notion of business having any presence on the Net. Let’s go back to those days.
Or, we could make the Net user-friendly so you don’t need to learn technology to make it work for you.
The simple fact is, the blogging phenomenon is hear to stay and its momentum is increasing, along with podcasting, social networking, social tagging, and a variety of other social phenomenon that happen to use the Net as their platform. It’s not going to stop just because you liked it the old way.
So, we could all figure out ways to grow and advance and work together to make all these aspects of the Net accessible and understandable to the masses. Or we could futiley dig in our heels and resist inevitable change.
Which one do you think will work better?
Posted by Shel Holtz on 05/31 at 04:59 PMIf there’s no excuse for not masking the e-mail address in the software, stop complaining about it and contact Expression Engine. Has nobody heard of collaborative efforts? I’ll save you the trouble. I’ll post the issue to their forum.
If I sound short of patience, I just got off a five-hour flight. Probably shouldn’t be commenting without a good night’s sleep.
Posted by Shel Holtz on 05/31 at 05:00 PMThanks, Ben. It appears there are some who think that everybody who starts a blog should have awareness of technical issues zapped into their heads by osmosis. Everybody wants to deal with spam, but some of those posting here don’t feel any obligation to communicate with the masses about it.
Hey, everybody. Since this is a PR blog, wouldn’t it be a great place to start a discussion about how to conduct some PR to raise awareness around these issues, rather than just complaining that the Net is evolving?
Posted by Shel Holtz on 05/31 at 05:05 PMI’ve turned off the requirement to include an email address in a post. I found it useful for getting in touch with people one-on-one who commented, but until I find out about masking email addresses, and given that Captcha precludes comment spam, I’ve gone ahead and unchecked it.
Speaking of Captcha, it’s great for keeping comment spam out (if only it worked with trackback spam), and nobody who wants to comment finds themselves locked out.
Posted by Shel on 05/31 at 05:19 PMNobody you know about, at least. Do you know if any visually-impaired people have been turned away by the captcha requirement? (The answer is probably “no;” most likely you would not hear about it....)
I will contact Dave Sifry and see if he can give us a list of blog domains. Chances are very good that those don’t intesect spammer domains so we may just whitelist them all. We’re not in the least bit interested in blacklisting blog domains, unless they happen to be used by criminal spam gangs. That’s a minor possibility, but unlikely. And if they were used, we could still blacklist them.
IIRC Dave Sifry was my customer at Best Internet many years ago....
Regarding PR for SURBLs, thanks for your very kind offer! If you or any readers have suggestions on how we can better get our message out, we’d sincerely like to hear them.
Perhaps our web site is overly technical or complex (though we tried to keep it simple), but the core of the message is that we want to enable people to detect spam by creating a list of (web) sites domains (and seldom-used IP addresses) that are advertised in spams. Our site is mainly intended for system administrators, i.e. the folks who would actually implement the checking, though as you might suggest it also needs to address non-spamming folks whose domains were occasionally listed in error.
Perhaps adding a paragraph very early on the Quick Start page about list removal would be a good move?
Posted by Jeff Chan on 05/31 at 08:16 PMChris, thanks for your answers (27) to my questions (26). That helps a bit
Bob says (30) “Blogs are also responsible for an increase in spam as it is.. it’s time for the bloggers to stop expecting the internet to dumb down to their level.” A bit of a sweeping statement but there is definitely some fact in it, the first sentence at least - see this post by Dave Sifry in March on blogosphere growth in which he addresses the subject of growth in spam blogs: http://www.sifry.com/alerts/archives/000298.html
As for your second sentence, Bob, I don’t think anyone expects the internet to “dumb down.” But you’d expect it to be a place that anyone can use more easily, which isn’t the same thing. A reality is that more people in more places are going online. Accessibility is becoming easier for more people. Network bandwidth is increasing and access prices are dropping, adding to the appeal for more people to go online. Wireless hotpots are appearing everywhere, again making it all easy and accessible. So if you can sit in a McDonalds or Starbucks today and do your email, write your blog or record your podcast - or even click to execute an email spam campaign - while you munch your Big Mac or sip your latte, then the internet as a communication medium has indeed reached the level of a consumer commodity. And as Shel noted (33), use of blogs and other social networking tools is increasing in leaps and bounds. It’s an irresistible movement.
So helping more people understand what things like spam blacklists are and why they should care seems to me to be a good idea. As a communication professional, I’d be willing to be part of a pro bono effort to help explain about the kind of work Jeff, Chris, etc, do and the depth of that work. Maybe a call to action, Shel, amongst the PR community to see who else is willing.
Jeff, you’ve already indicated that you’d be a great ‘client’ to work with as you’ve already suggested a great first step in communication. Now that I like!
Posted by Neville Hobson on 05/31 at 11:54 PMCurrently our Quick Start is organized into a brief introduction followed by technical notes about how to add SURBL support to different spam checking programs. Can you suggest where we might add a list removal paragraph? Currently the main reference is under the Lists page.
Posted by Jeff Chan on 06/01 at 12:10 AMJeff, that’s an excellent point about the visually impaired.
I’m on the road with limited time, but when I’m back on Friday, I’ll spend some time with your Quick Start and see what comes to mind. Neville may have a chance before that. In the meantime, I can’t help thinking that the typical blogger or podcaster will never even find your site or be aware of RBLs. Everybody knows about spam, but RBLs are fairly technical in nature and rather obscure in the grand scheme of things. I think a broader awareness effort about RBLs in general would coveer the good they do, how they’re used (and misused), what to do if you’re mistakenly blacklisted…
There are a number of ways to go about this. Is there any kind of coalition among the various RBLs? An association? A single place to get information? If not, it’s worth considering. Under the banner of a unified group, you could undertake any number of communication efforts, including getting the word out through some of the more influential bloggers (Steve Rubel and Robert Scoble leap to mind). An online media kit could raise awareness among those who cover high tech for the mainstream press.
Like Neville, I’d be willing to offer some pro bono services to help. Some 500 PR professionals listen to our podcast. Would you be interested in being interviewed for our show?
Posted by Shel on 06/01 at 02:57 AMIMHO, PR is needed. RBLs and URIBLs don’t have a formal group. There are reasons why we don’t. The spam world is highly complex. And having a single group heading over all such RBLs invites that group to nice ‘chickenbone’ law suit. (Thats an actual antispam term!)
Spam is tied to trojans, spyware, viruses, organised crime, child pron, ect. It is no longer just a problem of a full mail box. As someone has stated already, most of us have worked with agencies like the FBI to gather info.
I’d gladly work with you in any way to get not only bloggers, but the general public and media to better understand the problem of spam.
I have to admit, my opinion of blogs has been nothng more then “useless online diaries”. But I see some people are using them more like message boards now. I’ll have to rethink this whole blog thing
Posted by Chris Santerre on 06/01 at 05:16 AMChris, it would indeed be worth looking at blogs from the point of view of their increasing influence in the business world. While of the total number the so-called personal diaries or online journals do make up the majority, a significant and increasing number of sites wield considerable influence now, and not just as opinion-shapers.
In the area of Shel’s and my specific interest - that is, organizational communication - you might like to review a report published recently by Intelliseek, a marketing intelligence firm, and the Edelman PR Group. This report is a pretty good analysis of blogs from the business perspective and their increasing influence.
PDF, free download here -
http://www.edelman.com/image/insights/content/ISwp_TrustMEdia_FINAL-2.pdf
Jeff, I’ll see if I can get some thoughts to you re Quick Start in the coming days.
Posted by Neville Hobson on 06/01 at 05:46 AMIn response to Bob’s observation, “This blog software makes absolutely no attempt to mask email addresses at all,” I received the following response from the Expression Engine tech support staff:
“shel, do a ‘view source’ on your page and search for ‘posted by’ to get to the comments and where the authors are listed. You’ll see that they are already encoded for you. As I mentioned, visually when you just look at the email address in the page the browser renders it so that it appears normally, but the underlying source is actually encoded. Of course, no method will be 100% effective against potential email harvesters, but your email addresses certainly aren’t sitting there as raw text ready for harvesting.”
I hope that satisfies Bob!
Posted by Shel Holtz on 06/01 at 08:46 AMQuestion for the experts here: what do you think of “Spam Poison” (http://english-2542099983.spampoison.com/).
It says:
“Fight Back Against Spammers
WWW Robots (also called wanderers, spiders, crawlers, or bots) are programs that crawl the Web continually retrieving linked pages. When a spammer’s bot visits your website, blog, forum, etc, all pages and sites linked to it will be searched looking for email addresses.
Now you can fight back against their robots!”Does anyone know if this is effective?
Posted by Charles Pizzo on 06/06 at 09:44 AMIMHO they don’t work. Spammers simply rewrite their code to detect such links. Spammers higher some very intelligent coders.
Posted by Chris Santerre on 06/06 at 10:18 AMFWIW I’ve added a SURBL List Removal section as the second section on the Quick Start (main) page.
Jeff C.
Posted by Jeff Chan on 06/06 at 05:20 PM
Next entry: The Hobson & Holtz Report - Podcast #37: May 30, 2005
Previous entry: Blogs and insider trading