Verifying Email Addresses - Not So Easy - Sun, 14 Feb 2010

Before the first Alpha of Behold came out in March 2005, I had a survey up which you can still see at archive.org. (Don’t you just love the color I used to use?) Between 1997 and 2004, 685 of the people who filled in that survey said “Yes” to the question “Does the concept of a program like Behold interest you?” and also gave me their e-mail address. I promised to myself that when I release the beta of Behold, I would email them back and let them know.

Well, the beta of Behold is available, and I’m finalizing the Newsletter so I am just about ready to send out that letter. Just one thing to do: verify that those email addresses still exist. I do not want to send out 600 emails and have a lot of them bounce back. It could blacklist me and/or make my ISP very unhappy with me.

So then the journey to do that began. I found out that there is no perfect way to do so. Even sending an email may bounce back because an account is just suspended or a mailbox is full. Without sending an email, it is even trickier. The main method used that does the best job under the circumstances is to set up an SMTP call to the mailserver at the domain of the email address, and send it a HELO and RCPT command. You can sort of see how this works at a Free Email Address Verifier.

So I found a number of articles on how to do this and set up PHP code to do the verification. That took a couple of days. But when I ran it, it didn’t work. It took me another couple of days before I found out that the reason was my website host blocked port 25 where SMTP goes as a spam prevention policy. The research told me there’s no way around that. Find a host who doesn’t block it. So next I tried running it from PHP on my own machine. That would go through my ISP here in Winnipeg who gives me my internet access. Nope. Port 25 blocked here as well.

Without doing the SMTP checks, only 15 of the 685 addresses could be proven wrong. The rest were unknown. I knew there were a lot more than that. So I needed something with the SMTP checks. Downloading an email verification program didn’t help. They need port 25 as well.

Finally I was able to write a routine that accessed an online email checker and check the emails one by one. I was willing to pay for this service, but I could not find one that would do it in a batch online manner for me. So here goes, a one shot check of 685 addresses. Painful but necessary.

As I write this, I’m about half way through. Looks like about 45% of the email addresses from back then were good. That will make about 300 of them. Was all this effort worth it? Now that I look back, possibly not. But I didn’t know it at the time.

I have to wait to hear back from the SMTP mailing service I signed up for to know how to configure phplist to use them for my newsletter, so it will still be a few days before the newsletter is sent.

So tomorrow it’s back to work on Behold - let’s get that log file working.