In the email marketing industry, there are two predominant measures of performance — open rates and click through rates. However, out of the two of these, click rates are the truer measure of the effect of an email.
Depending on how they’re analyzed, sometimes opens have no significant relationship with applications or enrollments. In other forms of analysis, they have about half as strong an effect as clicks. This depends on which variables are included and how they are aggregated.
However clicks consistently have that significant, strong relationship. While ideally both click and open rates are high, out of the two, clicks are much more important.
In a sample of 14 clients for the 2016 freshman class, this was the effect of opens and clicks.
Opened zero emails: .7% search-to-Decision Ready Applicant (DRA) rate
Opened at least 1 email: 1.3%
Opened more than 1 email: 1.4%
This effect is significant. It’s however about half the effect of a click.
Clicked through zero emails: .8% search-to-DRA rate
Clicked through at least 1 email: 2.7%
Clicked through more than 1 email: 6.3%
For opens, what matters is that a student has opened at least one email. The value of additional opens tapers off, while the value of additional clicks keeps growing. This has serious implications when looking at open rates (opens/sends) and click-through rates (clicks/sends) by campaign.
The scatterplots above show the relationship between DRA rates and open rates as well as click-through rates. The line shows the relationship between the variables.
The curved lines are confidence intervals showing that the true relationship should fall somewhere within that range. Each circle represents one campaign from the freshman 2016 class or earlier.
These charts show that at the campaign level, there is actually a negative relationship between open rates and search-to-DRA rates. This means open rates are an especially bad measure of campaign performance. Compare this to the relationship between click-through rates and campaigns. In that case, there is still a significant amount of noise — more clicks don’t always translate into more applicants — but at least the relationship is positive and therefore a useful measure of campaign performance.
Rates By Email Provider
There are many factors that can create noise in an open rate. This noise makes them hard to translate into application rates. Most of these factors have nothing to do with the content of the message. Out of these, perhaps the biggest is how email providers filter incoming emails and display images (which is how Capture is able to track opens).
Open rates vary dramatically for different email providers. The chart above shows the open rates for Capture Higher Ed students’ 10 most popular email domains. It emphasizes that different providers have consistently different open rates — Gmail is almost always lower than Yahoo. A school’s open rate is going to vary depending on whether most of those emails are going to Verizon or whether they’re going to Live. This is the case regardless of how compelling a school’s email message may be.
What does this all mean?
Putting this all together, opens do provide useful information — just not in the way we often look at them. It’s useful to know that an individual student has opened at least one email as a sign of engagement. It is also useful to consider open rates for individual email sends as a measure of the appeal of the subject line.
However, beyond that point, open rates become deceptive and actually don’t provide much information about the overall health of a campaign. Instead, click rates are a more reliable measure. Ultimately, the true measure of a campaign is how many applicants and enrolled students it brings in. The goal of an email campaign is to influence students to apply and enroll, not just to open an email.
So in measuring the health of your email campaign, look to enrolled and applied rates first, then to click rates. Only then, if necessary, consider open rates.
By John Foster, Data Analyst, Capture Higher Ed