There are many aspects to sending an email over modern networks that are both invisible to the receiver and crucial to the sender. The prime objective of this paper is clarify these aspects by describing why, when, and how they are to be used. What protocols, the types of hardware involved, and general networking details will also be outlined.
A Brief History
In 1982 the very first documentation was laid out by The Internet Engineering Task Force (IETF) first describing the Simple Mail Transfer Protocol (SMTP). This was the genesis for what we now know today as electronic mail or “Email”, as it would come to be called. There were, however, protocols in place such as the Mail Box Protocol that dates back to 1971. Therefore, SMTP was not the first of its kind, but simply the most adopted. Sendmail was one of the first implementations of this standard—for the record, this was at a time when BSD Unix was still the most widely used operating system on the internet.
Message submission was introduced by 1998, a time when SMTP servers were still internal to most organizations. They would receive mail from outside the organization, relay it to users within the organization, and vice versa. However, that model soon became impractical, as users inevitably wanted to send email outside the organization to another email address outside the organization. As an example, a developer visits a 4 day workshop to learn more of his/her programming language. This developer now wants to send email from outside the company network to a customer outside the company network, or to fellow developers who are also not currently in the company network, but at home, or on vacation.
Thus, the SMTP servers had to expand their roles. They have become both Message Submission Agents (MSA) and Mail User Agents (MUA), interacting with Message Transfer Agents (MTA).
SMTP servers were no longer simply dumb devices sitting on the network, routing packets like a simple network switch, but now had to relay the proper traffic, both inbound and outbound, respective to the proper domains within that organization. As a result of this and the overall effects of the rapid expansion and popularity of the World Wide Web, SMTP had to include specific rules and methods for relaying mail and authenticating users to prevent abuses such as relaying unsolicited email (spam).
The catalyst of email verification and authentication was its own popularity. A program that began as a purely ASCII text-based language soon had to incorporate encoding and verification.
However, the standards that were set out in that original Network Working Group paper from 1982 are still there. The rules that state what can be used in both local part and domain part of the address have been the same for more than 30 years, besides a few additions.
When working with SMTP, please remember that this standard is an ancient protocol with many additions in an attempt to modernize. There are many quirks.
How It Works – A Basic Overview
1.In this illustration, the solid lines represent the request, while the dotted ones represent the feedback.
The picture above displays the typical email scenario: Christopher wants to email Stephanie through JangoMail. However, Stephanie is on a corporate network. Because of this, the inbound mail server requires proper authentication in order to verify the sender of the message. In order to do that, it reaches out over the internet to the sender’s domain in order to see if the proper records are there. Additionally, if configured, an inbound mail server will pull an up-to-date record throughout the day from blacklisting services like SpamCop, Spamhaus, Barracude, and others. Please note, when sending to an Internet Service Provider (ISP) like Comcast, Verizon, or Time Warner Cable, these ISPs keep their own blacklists due to the extremely high volume they receive to their inbound mail servers. Likewise for Google, Hotmail, AOL, and the other email providers.
The example above illustrates how to use JangoMail proper, not our SMTP service. If using our SMTP service, the message would be received by our Relay Servers first, then passed on to our Sending Servers for processing and delivery. Also, there may be times when the receiving server is actually using Google (or similar) services, as paid for by the company that owns it. In this case, the messages would not be received by an inbound server the company owns, but one in the “cloud” that is hosted at a datacenter. For smaller companies or groups, whose budgetary concerns are not viable for actually owning an email server, this “cloud” method would also apply.
Authentication Methods Explained
As stated earlier, the reasoning behind authentication methods are to verify who is sending the email and from where they are sending it in order to prevent spam or unsolicited email. In order to achieve this, there must be changes made within the sender’s Domain Name Service (DNS) records.
For the purpose of example, the domain ecommerce.com will be used.
When you send email from JangoMail, your reply-to and from address are from ecommerce.com, but the receiving server can see it came from a "jangomail.com” server. The receiving server will go to “ecommerce.com” and ask if JangoMail is truly authorized to send emails on its behalf (by viewing the SPF record). Adding these records tells the server "Yes, it's okay for them to send for us." Without these records in place, the receiving server that perform the check replies "no, they are not authorized to send," and at that point the server is inclined to drop the message into spam or delete it entirely.
The owner of ecommerce.com generates a public key within their JangoMail account to use for signing all outgoing messages (multiple key pairs are allowed). The next step is for them to enter the public key within the DNS records of their account.
Now when the user sends from ecommerce.com, the DomainKeys-enabled receiving email server extracts the signature and claimed From: domain from the email headers and fetches the public key from the DNS of ecommerce.com (as seen below). Once obtained from the DNS of the sender’s domain server, the public key is then used by the receiving mail server to verify that the signature was generated by the matching private key. If the two match, the message is valid.
The tracking domain is a specific subdomain of the TLD (Top Level Domain) ecommerce.com. In this case, it’s going to be track.ecommerce.com.
This subdomain is used to keep track of all clicks, opens, and forward-to-friend events that happen pertaining to the emails you send.
A custom tracking domain that uses the sender’s domain instead of JangoMail’s has many advantages. With SPF and DKIM authentication already in place, along with a custom tracking domain, emails sent can be completely branded entirely by the sender. There will be no JangoMail footprints. Due to the ever-increasing amount of spam in the digital world today, this ‘fully branded’ approach is essential for sending email that lands in the inbox where it should.
In addition to enhanced deliverability, a custom tracking domain will also let you establish your own reputation, aside from a default shared server with other JangoMail users.
Also, keep in mind the visible components of the tracking domain such as unsubscribe or view on web, will be branded with your domain. This will make your recipients more confident when they are clicking on links in your email and viewing the results.
To quote directly from business.ftc.gov:
“The CAN-SPAM Act, a law that sets the rules for commercial email, establishes requirements for commercial messages, gives recipients the right to have you stop emailing them, and spells out tough penalties for violations.
Despite its name, the CAN-SPAM Act doesn’t apply just to bulk email. It covers all commercial messages, which the law defines as ‘any electronic mail message the primary purpose of which is the commercial advertisement or promotion of a commercial product or service,’ including email that promotes content on commercial websites. The law makes no exception for business-to-business email. That means all email – for example, a message to former customers announcing a new product line – must comply with the law.”
Therefore, compliance is absolutely necessary if you wish to have any sort of good delivery to your receipts. This footer must include:
- Your location
- Identify the message as an ad (if applicable)
- Tell recipients how to opt-out of receiving future email from you
- Honor opt-out requests promptly
Additionally, by law you must not include deceptive or false header information, or a misleading subject line.
Content is king—the last mile, so to speak. An email can be authenticated with SPF, DKIM verified, and be using a custom tracking domain from a good, reputable domain, but still fall flat in the junk box if the content is not good.
There are a few steps one can take when crafting a message.
First, always try to solve a problem.
Your subscribers want a way to resolve a specific problem, or are looking advice on how to improve current situations and knowledge to help them drive more business and reach their goals. Offer value.
Second, make the email less about the company, and more about what the company can do for them.
Unless the customer is also a stock holder or investor, he or she does not want to know how well the company is succeeding in the market, but rather how well it is succeeding at bringing the customer what he or she wants.
Third, avoid jargon, misspellings, capital letters, or links to outside financial firms.
If the subject line of your message has too many capital letters, misspellings, or perhaps even one too many exclamation points, you can almost guarantee it’s not going to make it to the inbox. Avoid links to all major financial institutions such as Mastercard, Visa, American Express, and other creditors. Gmail, for example, does not look too highly at this type of content.
In the same manner, avoid the common commerce, employment, business, and marketing words.
A short list would be:
- Click here!
- Subject to credit
- As seen on
- Credit / Line of credit
- Opt in
- Medical / Medical Exam
These types of words and phrases are likely to cause your message not to reach its intended audience. It is unfortunate that spammers and con artists have ruined so much of this modern messaging system for legitimate senders, but this is the current marketplace.
Making sure you use the right words, are properly authenticated and are CAN-SPAM complaint are all good avenues to follow. However, there are some additional small additions you can perform to enhance delivery.
Always make sure to include a plain text copy of your message along with the HTML version. An email that is all HTML, without a plain text copy included will have a higher likelihood of being considered spam. Also, make sure not to use one giant image for your message body. Avoiding using one image for your entire message is crucial, but you should also strike a balance. JangoMail usually recommends two lines of text to each picture used.
When it comes to attachments, try to avoid them. JangoMail lets you host files in the Files section of your account for you to link to. This is preferred to attaching a PDF or Microsoft Word file to your message. Also avoid any executables such as .EXE, .ZIP, or other archiving file types. It is best-practice to have these hosted on a third-party site like DropBox or SkyDrive to share to users via a simply link.
The process of delivering email to the inbox is not as easy as it once was. Authentication methods have become extremely important. These methods let you send messages that are verified to be from you, so there is no confusion on the part of your customers or clients. The necessity of these methods has come about due to the proliferation of spam on the internet. Most statistics point to about 70 percent of all email traffic on the internet today is spam. However, if the necessary steps are taken, you can avoid becoming a statistic and be recognized as a legitimate sender.
Yet that is only half of the puzzle. A great message is nothing if not sent using a great system—that’s where JangoMail comes in. JangoMail’s key relationships with Comcast, Google, Hotmail, Yahoo, Verizon, & AOL, just to name a few, are extremely important. Our robust hardware infrastructure, along with vigorous IP reputation monitoring ensure that your message will be delivered. We receive dozens of questions a day regarding the topics in this paper and are happy to help our customers with campaign delivery.