Statistics

In This Article:

Mailtrap provides analytics for all the emails you send. You’ll find the stats under three tabs in the menu: Stats, Mailbox Providers, and Email Categories.

The data presented in each tab is similar, but each focuses on different aspects of email deliverability. As such, it makes it easier to debug and monitor emails from different angles.

  • Stats give you a general overview of all your key email metrics, with the ability to filter them by a particular sending domain.
  • Mailbox Providers show various email stats broken down by the recipients’ mailbox providers. Each provider has its own policies, and your emails may suddenly start underperforming for just one provider. That’s why it’s important to monitor them separately.
  • Email Categories show stats for different types of emails you send, split into particular categories. This is especially useful when sending marketing campaigns or when A/B testing with different iterations of your emails. Email Categories view makes it easy to compare the key metrics of different types of emails.

Navigating around the statistics dashboards

In that Stats tab, you'll find a domain selector at the top of the page. Here, you can choose to show stats for a particular domain or choose nothing so that stats for all domains are shown. 

If you’ve previously added a domain but then chose to delete it, it will appear with the (deleted) suffix as shown below:

Below, you can see five scorecards showcasing the vital stats behind your emails, as well as a comparison to the previous period. We’ll explain them in more detail below.

You select the period in the top-right corner of the screen. To the right of each metric, you'll see the comparison with the previous period so you can quickly determine how things change over time and if your actions bring the desired results.

If you were to, for example, select the 10 days between November 21st and 30th, the small percentages would recalculate to show differences between the period of November 11th and 20th. If you look at stats for June, the comparison with May will be shown, and so on.

By default, the stats are shown for the last week + today.

The two tables below the scorecards represent the top mailbox providers you send emails to and the top email types distinguished by categories.

Mailbox Providers table aggregates your stats by the recipients' providers. This is useful to keep track of as sometimes the numbers (usually the deliverability) may drop only for a specific provider. It may not be visible at first when looking at the overall stats, but it will be when you look at the numbers for a specific provider.

In the Stats tab, the six most popular (among your recipients) mailbox providers are visible. Click the See All button to the right to jump to the full list with additional stats. You may also open the Mailbox Providers tab in the menu directly to see the same stats.

To the right, you can see the list of categories, also showing the six most commonly used ones with the corresponding stats. Click the Report button to view the full list, or use the Email Categories tab in the menu to the left.

Using categories is vital for proper reporting in Mailtrap. 

You will probably send many types of emails - welcome emails, onboarding sequences, re-engagement campaigns, or simple password reset emails. Tracking the deliverability or open rate for all emails simultaneously won’t give you very actionable insights. 

Categories are a way to break down all your email sending into specific campaigns, types of emails, or any other criteria you can think of.

You can assign categories when creating an email by adding a name in the X-MT-Category header. See how to set up categories in more detail. By default, no category is assigned to an email.

If no categories are set up on your account, you’ll see the following message instead of a table:

As the first emails start flowing, you’ll notice that both the Mailbox Providers and Categories tables become more colorful. We implemented color-coding to highlight the good, bad, and average results across your metrics. 

Thresholds

The thresholds are based on our extensive cross-industry research and, at this point, can’t be edited. The current values are:

  • For bounce rate:
    • 2-5% is a warning level (yellow)
    • >5% is a critical level (red)
  • For spam rate:
    • 0.08%-0.1% is a warning level (yellow)
    • >0.1% is a critical level (red)

A word of explanation on the unique open rate. This rate is based on your account and the emails that match the filters you set (if any). It considers all emails delivered or opened during the selected reporting period.

For example, let’s say we’re looking at the Email Categories statistics. We filtered them by a specific domain (e.g., mailtrap.io) and one mailbox provider (e.g., Yahoo). We also chose a reporting period (e.g., November 1st to 10th).

In the background, the unique open rate will be calculated for all the emails that meet the mentioned criteria. Let’s say it’s 25%. In this case, the threshold levels will be as follows:

  • Critical level: <20% (0.8*25%)
  • Warning level: 20%-23.5% (0.95*25%)

In the future, we may let you adjust these thresholds so that you can make them more demanding or better tailored to your industry.

At this point, color-coding is available for opens, bounces, and spam-related metrics. In the Mailbox Providers and Email Categories tabs, you can enable or disable it by toggling the Color Coding switch.

Charts

Finally, on the Stats page, you’ll also find color-coded charts visualizing the metrics.

Some charts also feature a threshold line. For the bounce and spam rates, these are the values you should not exceed if you don’t wish to harm your email deliverability. We described the formulas for calculating them above. The unique open rate threshold indicates the unique open rate for the chosen criteria, as explained just moments ago.

Clicking on the Email Log button below either chart will take you to the list of emails meeting this specific criterion in the Email Logs. For example, clicking the button under the bounce rate chart will show you the list of all the emails that bounced.

The terminology used in stats

Statistics rely on five key metrics. Because the way we calculate them may not be identical to what you’re used to, we recommend getting yourself familiar with each of them.

Delivered

Delivered refers to the percentage of emails that were accepted by the recipient’s mailbox providers compared to all emails sent. Email is counted as delivered when a Delivery event is recorded in its Event History in Email Logs.

Note: "Delivered" status doesn’t mean that a message went straight into the recipient’s Primary folder. It may have still gone into Promotions and Updates, or it might have been automatically put into a Spam folder. In some cases, an internal policy of mailbox providers may have led to discarding an email, even after it was initially accepted.

Mailtrap has no way of tracking what happens to a message after it’s been accepted by the recipient’s mailbox provider.

An email will not be delivered if either Mailtrap or the recipient’s mailbox provider rejects a message at any stage.

Mailtrap will reject an email if a recipient is on a suppression list for a given domain. Read more about suppressions here.

On top of that, an email can be rejected on the recipient’s end for various reasons:

  • A message is considered spam, phishing, or other foul play.
  • A security policy on the recipient’s end dictates that a message should be declined.
  • A server timeout occurs (in such case, Mailtrap will retry the delivery 10 times until it eventually gives up).
  • Email authentications (SPF, DKIM, DMARC) fail.

Unique open rate

Unique open rate refers to the percentage of emails that were opened at least once compared to all emails sent.

Open tracking needs to be enabled for a domain in question in the Sending Domains tab. Only then will email opens be recorded.

Email opens are tracked with invisible pixels that Mailtrap attaches to each message. When a message is opened, an open event is recorded, and a message is counted towards the unique open rate.

It’s important to know that some email clients, browsers, and plugins block such tracking. If they do, no open event will be recorded. 

For that reason, the actual open rate is nearly always higher than what’s shown in the statistics. The margin of error depends on your audience. If you’re emailing a tech-savvy audience, there’s a good chance at least some recipients will block tracking. If you’re addressing a general audience, the open rate numbers are probably more accurate.

What’s a good open rate? 

That very much depends on what you send. Marketing emails tend to score about 20-25% unique open rate. On the other hand, password reset emails hit nearly 100% because everyone wants to complete a process they just started seconds before. Product updates, confirmation emails, or onboarding sequences are somewhere in between.

Several notes about unique open rate tracking:

  • As the name suggests, only unique opens are counted. If a message is opened multiple times, it will count once.
  • If a link is clicked in a message, we automatically count it as opened as well, even if no open event was recorded earlier (for example, because tracking was blocked).
  • The percentage of opens is counted against all emails delivered, not all that were sent. For example, if 30 emails were sent, 20 were delivered, and 12 were opened, then the unique open rate is 60% (12/20).

Each event in Mailtrap - for example, open or delivery - has a timestamp that you can look up in the Email Logs. When calculating a unique open rate, Mailtrap takes into consideration all emails that had an open event in a chosen period of time. 

Some emails are opened weeks or months after they were delivered. Selecting too short of a period can result in discrepancies or even rates higher than 100%. 

How? Let’s look at an example.

You send and deliver 100 emails on Monday and another 100 on Tuesday. No emails are opened on Monday, but 150 unique opens are recorded on Tuesday. The unique open rate will be:

  • 0% for Monday (0 opened/100 delivered).
  • 150% for Tuesday (150 opened/100 delivered).

To avoid such situations, set the time periods long enough to capture both deliveries and opens, knowing that some recipients will open messages days after a successful delivery. When configuring an email, use unique categories to filter out the results only for a specific email type.

Click rate

Click rate refers to the percentage of emails that received at least one link click compared to all delivered emails. 

When any of the links in an email are clicked, a click event is recorded. The same happens for any further clicks, even if a recipient keeps clicking on the same or different links. 

You can see the details of each click (timestamp, Recipient's IP, URL) in the Events History in the Email Logs. The total number of clicks is shown in the email list on the left:

However, the metrics such as clicked and click rate used in the statistics are calculated differently. 

Clicked metric indicates how many emails received at least one click on a link present in its body. Once a single click is recorded, the email is counted as clicked. Any further clicks on the same or another link inside this message are not counted for the purpose of calculating the clicked metric. 

As such, the email above would count only once as clicked, even though it received multiple clicks.

The click rate is basically clicked/delivered * 100%.

Let’s look at another example. Ten emails were sent and delivered, each containing several links. Five recipients did nothing about them. The other five opened their emails and clicked three times each on the links - some kept clicking on the same link, and others clicked on different URLs.

  • For such a campaign, you’ll see in Mailtrap:
  • Delivered: 10
  • Clicked: 5
  • Click rate: 50% (5/10)

As you can see, the number of clicks on each of the emails is not taken into consideration here.

You’ll see one more rate in the statistics - the click-to-open rate. This one is very straightforward. The formula is click rate / unique open rate.

As was the case for opens, click tracking needs to be enabled for a domain in question in the Sending Domains tab.

Note: Click tracking and custom domain for clicks tracking are available only for paid accounts. 

If you're a paid user and you enable click tracking, the toggle for Custom Domain for Clicks Tracking will be switched on automatically. That way, all links will be redirected through your domain (mt-link.yourdomain.com). Remember that you should verify the Domain tracking DNS record to use this feature. See the Domain Setup article for details on adding DNS records.

This redirection allows us to reliably track each click on your emails. What’s more, such tracking can’t be blocked, so the click numbers are certainly more accurate than the opens we discussed previously.

Note: For now, the links redirected through custom domains are HTTP only. 

Bounce rate

Bounce rate refers to the percentage of emails dispatched from Mailtrap that were rejected on the recipient’s end compared to all emails sent.

Emails may bounce for different reasons, most commonly:

  • Invalid email address.
  • Rejection by the recipient’s mailbox because email is deemed spam, phishing, etc.
  • The security policy of a mailbox provider that rejects emails from all or some domains.
  • Permanent connection issue.

The term bounce used in Mailtrap is also known as a hard bounce. This is different from a soft bounce - another event present in Mailtrap that refers to a temporary delivery problem. 

If an email soft bounces, Mailtrap will try to deliver it 10 more times. If there’s no positive outcome, an email will (hard) bounce and get counted towards the bounce rate.

Read more about the differences between soft and hard bounce.

When an email bounces, a recipient is added to a suppression list for this domain. You won’t be able to send any further emails to this address from this particular domain. Technically, you may still email them from other verified domains. Still, if the reason for the bounce was an invalid address, it will only hurt your deliverability in the long run. Read more about bounce categories and respective suppression logic here.

Bounces and spam reports are never good for you. They also hurt Mailtrap's reputation and our users’ ability to deliver emails. For that reason, we closely monitor excessive bounce and spam rates and take the appropriate action when necessary.

Here are some things you can do to decrease the bounce rate.

  • Verify email addresses prior to sending emails to them. There are lots of tools online for doing this. Having users confirm their email addresses or register with Google or social media accounts helps eliminate typos.
  • Test your emails with the Mailtrap Email Testing solution. Send your emails to Mailtrap and check the spam score of each template. Use the provided hints to improve the score and increase your chances of successful delivery.
  • Monitor stats for specific mailbox providers (via the Mailbox Providers tab) to quickly identify those that underperform. When issues arise, consult the providers’ deliverability guidelines or contact them directly to resolve them.

Spam complaints

Spam complaints refer to the percentage of emails that are reported as spam by recipients, as compared to all emails that were delivered.

An email can be categorized as spam at several stages:

  • When it’s dispatched and arrives at the recipient’s mailbox provider, the server’s policies may classify it as spam and decide that it won’t go any further (this happens to most of the spam on the internet, luckily). Mailbox provider rejects a message, and a bounce event is recorded in Mailtrap. 
  • An email can be accepted but may never make it to an inbox. Instead, it’s sent straight to spam by an email client or a mailbox provider. Mailtrap has no way of knowing where such emails go, so the last known event is delivery.
  • Finally, an email can arrive in the recipient’s inbox. Then, a recipient can decide to mark it as spam. A spam event is recorded in Mailtrap.

Emails are only counted towards the spam complaints metric if the latter occurs.

If a recipient reports your message as spam, their email address will be added to a suppression list for a given domain. You won’t be able to send any further emails to them from this domain. It doesn’t prevent you from emailing them from another verified domain, but, as was the case for bounces, it’s usually not a good idea.

Spam reports are very bad for your email deliverability. There are several things you can do to reduce them:

  • Make sure the recipient is happy to receive emails from you. If they signed up for a specific product or service, refrain from sending them completely unrelated emails.
  • Adjust the frequency of emails, some recipients may not be comfortable getting too many of them. Letting recipients manage their email notification settings usually does the job well.
  • Test each template using Mailtrap’s built-in spam analysis, available for both Sending and classic Testing. Also, monitor the affected emails via the Email Logs. Check the responsiveness, validate links, and make sure no annoying "Hi [name]" sneaks into a copy.

Back To Top

Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.

Still need help? Contact Us Contact Us