About a year ago I wrote about the troubles that non-profit organizations were facing communicating with their members. SPAM and virus email is a serious problem, but most ISP responses to it have made things harder for people to get legitimate email through. Solutions like Goodmail are great for companies that can afford to pay for email to get through, but for non-profits, even with a reduced rate, it can represent a significant expense. Plus there is something flat out wrong to pay for your email to be delivered because of spammers. Shouldn’t it just be as simple as sending the email and having it get to the person intended?

Well, just because we send the email doesn’t mean it’s actually getting there. I’ve encountered all sorts fun with email delivery failures over the past few years, across all sorts of ISPs, making it difficult to get information out to our parents.

We’re a smaller soccer league, so our parent communications go out to approximately 800 email addresses. Our coaches can send emails directly to their teams using our league management system, and our officers often hold lively debates via email using a mailing list. When you register your children, reset your password, or have a match rescheduled, it’ll send an email. All this email is meant to keep our parents and coaches informed throughout the season. All of this email originates from the server hosting our league management system, so all the email originates from a single IP address. This is not unusual. I’ve taken great pains to ensure the emails have all the necessary display names, headers, etc. that make them acceptable to spam filters. We utilize SPF records for our domain and we even include some headers that Microsoft, Yahoo, and others like to be there when scanning for SPAM.

You’d think our email would actually get delivered reliably. Turns out it depends who you use:

  • AOL – Always my favorite. 69 of these addresses on our parent list. Some get through, some don’t. Some users flat out reject email by default except for people in their address book/spam whitelist. But when you ask them about this, they usually have no idea what you are talking about. At least you get a message saying ‘User xyz is not accepting mail from you’. Most undelivered email just disappears – very few bounce messages get received. Other AOL users get our email fine. Seems to be a flip of the coin.
  • Earthlink – Only 15 of these thank goodness. I’ve not had email disappear (that I know of :) ), but Earthlink is home to the infamous SpamBlocker. If you set it to the ‘highest’ setting (and most users seem to), no email gets through unless your email address is in the recipient’s personal address book. If you aren’t in their address book, you get an automated response to visit a web page, fill it out, and ask the user to accept your email from your email address. Well first, when you register for our league the first time, the system emails you a confirmation and it’s not going to fill out that form. I did it myself for a few years, but it got old in a hurry. Besides, you may get email from 8 different email addresses during the course of a season (league officials, coaches, the league management system, etc), so it’s very difficult to ensure all will be on the whitelist – i.e. the address book. On paper it sounds like a great concept, but in practice, it’s extremely annoying.
  • Hotmail – 40 of our parents use Hotmail. This one has always been fun. We use MailMan to manage our mailing lists. Every once in a while, Hotmail decides it really doesn’t like messages from mailing lists and starts rejected them outright. Mailman, being the good mailing list program it is, will monitor the number of bounced messages per email address and after receiving 3 in a row, will unsubscribe the email address as ‘invalid’. So on a day where we might send 3 messages to our parents AND Hotmail gets picky, we suddenly find ALL of the Hotmail email addresses yanked off the mailing list. I think I’ve had to re-add all the Hotmail addresses close to 8 times over the course of five years. Other times, Hotmail takes the messages and delivers them without issue. But the fun doesn’t stop there. Most of our messages don’t include attachments, but occasionally we will send an attachment out. I know we could send a weblink out to the file instead, but you’d be amazed how many more people will look at the attachment if it is attached and how many people can’t click on weblinks in their email application. So we’ll often send attachments, usually PDF format. Well Hotmail is notorious for not delivering messages with attachments. I’ve seen this firsthand. Hotmail users who normally get our emails all the time, sometimes won’t get the messages with attachments. At first I thought the users were having problems opening them. Nope, Hotmail really is ‘vanishing’ emails with attachments.
  • AT&T/Bellsouth – 42 of our parents use Bellsouth. ISPs owned by AT&T started to reject our emails because we run an independent email server as part of our league management system. However, once we got the server added to their ‘whitelist’, the email seem to be getting through no problem.
  • Yahoo – 119 of our parents have Yahoo accounts. Almost no problems here. I’ve occasionally had email not arrive, but very randomly and it is never with the same users. However, Yahoo recently announced it would be partnering with Goodmail. I have no problem with them accepting Goodmail tokens to bypass their SPAM scanners. But I worry they’ll start doing what AOL does and making it much harder for email to get through their filters without the Goodmail token (i.e. extortion). For now, Yahoo mail seems to get through OK – I hope it stays this way.
  • Gmail/Google – Only 19 of our parents currently use Gmail, but I’m seeing it more and more often. I’ve had zero problems delivering to parents with Gmail addresses. We actively encourage our parents to use Gmail.
  • Mebtel – This is the fun one. Mebtel is the local ‘Mom and Pop’ telco that has served our small town for decades. While it was absorbed into a larger corporation, it has remained mostly independent and continues to serve the telecommunication needs of most Mebane residents. They started offering Internet services in the mid 90’s and many Mebane residents have Mebtel’s DSL or dialup service. 154 of our parents use Mebtel addresses. Things were fine for years, until their email group decided to combat virus email with a club. Instead of using CPU intensive scanning to search for spam and virus email, they decided to limit the number of emails a specific IP address could send to (not just through) Mebtel’s email servers. If you exceeded this threshold, the email servers would return ‘User not found’ for EVERY email you tried to send to a mebtel.net address for the remainder of the day. With 154 parents on the list, plus coaches, plus many officers who use Mebtel, our league email server sends a LOT of email to Mebtel. The threshold seemed to be around 150, so when we sent a message to the parents, suddenly all mail delivery to Mebtel would stop until midnight when the block would reset. We called them, explained why our system was sending so much email to mebtel.net addresses (because most of our members LIVED in Mebane – ding!), and got placed on their ‘whitelist’ along with, I’m sure, email servers from places like Yahoo, Microsoft, etc. Well this never sat well with them that an individual server would have to send this much email. So it seems every year or two, we get knocked off the whitelist and have to beg to be put back on. This always seems to happen right as the Fall season is ramping up – when the email volume is at it’s highest. Sure enough, this month, we suddenly got dropped off again. So we’re struggling to communicate with close to 20% of our members. My patience is almost at an end with them and I’m ready to tell everyone to ditch Mebtel for another email provider.
  • Randoms – These have always been the most fun. At various times, I’ve been blocked by the State of NC (we have many parents who are teachers or work for our county school district who use the state’s email system), the local community college (their email server actually freaked one time and re-sent an email back out to our mailing list to EVERYone, posing as me. It was the wildest thing I had ever seen.), smaller ISPs, and a variety of others. Usually a simple email explaining who we were got us on a whitelist and the emails would go through.

What a hassle! Just to try and email parents in our soccer league. I understand SPAM is a problem for everyone. But many of the tactics being taken by the larger ISPs, and even some smaller ones, are extremely broad and end up with a lot of collateral damage. It should not be this difficult to reliably deliver email to people who are part of your organization. And before anyone asks the obvious, no – the messages weren’t stuck in their SPAM folders! I also think companies that ignore the Internet standards (RFCs) should be tossed off the Internet. Returning false responses (User not found) when they really exist is just wrong and causes too much trouble for people trying to send legitimate email. At the very least, return a valid response code saying the message is being rejected as SPAM. Even more annoying is the administrators who don’t want to deal with you as you try to get whitelisted or jump through whatever hoops they wish so your email gets through. My favorite is actually our local ISP here, Mebtel. The email admin left me a voice mail recently saying they absolutely wouldn’t whitelist my email server and when I returned his call, he refused to take it and sent me to his manager’s voice mail. I’m still waiting for THAT call to be returned…

This is becoming a serious problem. But charging for delivery of email is not the answer. I know, most people who get free email shouldn’t complain. But it’s not just the free services that can cause problems, and most people DO expect legitimate email to be delivered, not just ‘vanished’ and they certainly don’t expect their ISP to advertise that their email address doesn’t exist just because some group they are a part of sent a lot of email. Yet all too often the viewpoint of the ISP is if it works for the majority of cases and some little people get caught – oh well.

So if you run a soccer league or any other non-profit with a large membership, regardless of what software/application, etc. that you use, don’t expect that all your email is reaching the intended recipients. If you’re curious about delivery success rates, at least for the free services, create an account on each of them (and don’t forget that AOL offers free email even to non AOL users now). Subscribe all the addresses to whatever system you use to email your organization and confirm they all get through without necessarily adding things to your address book, etc. My solution for now? I’m encouraging everyone to start using Google’s Gmail service. It’s a very easy to use free email service and I’ve never had delivery problems to Gmail address with any project I’ve been involved with. That may change at some point, but for now they seem to have the most sensible spam fighting setup and the messages get through reliably. Plus users can use normal email applications to access their Gmail accounts via POP3 as well as the web interface. As a bonus they’ll have an email address for life, even if they move.

Sorry for the technical rant – but it relates to soccer at least indirectly. I’m just tired of dealing with it and thus the irate parents who missed this or that because their ISP didn’t deliver our reminders.