Statistics

In This Article:

    Mailtrap provides analytics for all your sent emails. You’ll find them under three tabs in the menu: Stats, Mailbox Providers, and Email Categories.

    The data presented in each tab is similar but each focuses on different aspects of email deliverability. As such, it makes it easier for you to debug and monitor emails from different angles.

    • Stats give you a general overview of all your key email metrics, with the ability to filter them by a particular sending domain.
    • Mailbox Providers show various email stats broken down by the recipients’ mailbox providers. Each provider has its own policies, and your emails may suddenly start underperforming for just one provider. That’s why it’s important to monitor them separately.
    • Email Categories show stats for different types of emails you send, split into particular categories. It’s particularly useful when sending marketing campaigns or when A/B testing with different iterations of your emails. Email Categories view makes it easy to compare the key metrics of different types of emails.

    In that Stats tab, you'll find a domain selector at the top of the page. Here, you can choose to show stats for a particular domain or choose nothing so that stats for all domains are shown. 

    If you’ve previously added a domain but then chose to delete it, it will appear with the (deleted) suffix as shown below:

    Below you can see five scorecards showcasing the vital stats behind your emails as well as a comparison to the previous period. We’ll explain them in more detail below.

    You select the period in the top-right corner of the screen. To the right of each metric, the comparison with the previous period is shown so you can quickly see how things change over time and if your actions bring the desired effects.

    If you were to, for example, select the 10 days between November 21st and 30th, the small percentages would recalculate to show differences between the period of November 11th and 20th. If you look at stats for June, the comparison with May will be shown, and so on.

    By default, the stats are shown for the last week + today.

    The two tables below the scorecards represent the top mailbox providers you send emails to and the top email types distinguished by categories.

    Mailbox Providers table aggregates your stats by the recipient’s providers. This is useful to keep track of as sometimes the numbers (usually the deliverability) may drop only for a specific provider. It may not be visible at first when looking at the overall stats but will be when you look at the numbers for a specific provider.

    In the Stats tab, the six most popular (among your recipients) mailbox providers are visible. Click the Report button to the right to jump to the full list with additional stats. You may also enter the Mailbox Providers tab in the menu directly to see the same stats.

    To the right, you can see the list of categories, also showing the six most commonly used ones with the corresponding stats. Click the Report button to view the full list or use the Email Categories tab in the menu to the left.

    Using categories is vital for proper reporting in Mailtrap. 

    You will probably send many types of emails - welcome emails, onboarding sequences, re-engagement campaigns, or simple password reset emails. Tracking the deliverability or open rate for all at once won’t give you very actionable insights. 

    Categories are a way to break down all your email sending into specific campaigns, types of emails, or any other criteria you can think of.

    You can assign categories when creating an email by adding a name in the X-MT-Category header. See how to set up categories in more detail. By default, no category is assigned to an email.

    If no categories are set up on your account, you’ll see the following box instead of a table:

    As the first emails start flowing, you’ll notice that both Mailbox Providers and Categories tables become more colorful. We implemented color-coding to highlight the good, bad, and average results across your metrics. 

    Thresholds

    The thresholds are based on our extensive cross-industry research and, at this point, can’t be edited. The current values are:

    • For bounce rate:
      • 2-5% is a warning level (yellow)
      • >5% is a critical level (red)
    • For spam rate:
      • 0.08%-0.1% is a warning level (yellow)
      • >0.1% is a critical level (red)
    • For open rate:
      • <95% of the unique open rate of your account is a warning level (yellow)
      • <80% of the unique open rate of your account is a critical level (red)

    A word of explanation on the unique open rate. This value is calculated for your account only, for the emails that meet the filtering criteria chosen at the moment (if any). All emails delivered or opened in the selected reporting period are considered when calculating the unique open rate.

    For example, let’s say we’re looking at the Email Categories statistics, and we filtered them by a specific domain (e.g., mailtrap.io) and one mailbox provider (e.g., Yahoo). We also chose a reporting period (e.g., December 1st to 31st).

    In the background, the unique open rate for all the emails meeting the above criteria will be calculated. Let’s say it’s 25%. In this case, the threshold levels will be as follows:

    • Critical level: <20% (0.8*25%)
    • Warning level: 20%-23.5% (0.95*25%)

    In the future, we’ll let you adjust these thresholds, and for example, make them more demanding or better tailored to your industry.

    At this point, color-coding is available for opens, bounces, and spam-related metrics. In the Mailbox Providers and Email Categories tabs, you can enable or disable it on top of the page.

    Charts

    Finally, on the Stats page, you’ll also find charts visualizing the metrics. The identical color-coding is used on some charts as was the case before. 

    Some charts also feature a threshold line. For the bounce and spam rates, these are the values you should not exceed if you don’t wish to harm your email deliverability. We described the formulas for calculating them above. The unique open rate threshold indicates the unique open rate for the chosen criteria, as explained just moments ago.

    Clicking on the Email Log button below either chart will take you to the list of emails meeting this specific criterion in the Email Logs. For example, clicking the button under the bounce rate chart will show you the list of all the emails that bounced.

    The terminology used in stats

    Statistics rely on five key metrics. Because the way we calculate them may not be identical to what you’re used to, we recommend getting yourself familiar with each of them.

    Delivered

    Deliveredrefers to the percentage of emails that were accepted by the recipient’s mailbox providers in comparison with all emails sent. Email is counted as delivered when a Delivery event is recorded in its Event History in Email Logs.

    Note: Delivered status doesn’t mean that a message went straight into the recipient’s Primary folder. A message may have still gone into Promotions, Updates, or even been automatically put into a Spam folder. In some cases, an internal policy of mailbox providers may have led to an email getting discarded, even after it was initially accepted.

    Mailtrap has no way of tracking what happens to a message after it’s been accepted by the recipient’s mailbox provider.

    An email will not be delivered if either Mailtrap or the recipient’s mailbox provider rejects a message at any stage.

    Mailtrap will reject an email if a recipient is on a suppression list for a given domain. Recipients end up there when:

    • they unsubscribe.
    • report a message as spam.
    • a hard bounce is recorded - usually because an email address doesn’t exist or doesn’t accept emails from your domain.

    When either of these happens, Mailtrap will no longer send emails to such recipients from this domain.

    On top of that, an email can be rejected on the recipient’s end for many different reasons:

    • A message is considered spam, phishing, or other foul play.
    • A security policy on the recipient’s end dictates that a message should be declined.
    • A server timeout occurs (in such case, Mailtrap will retry the delivery 10 times until it eventually gives up).
    • Email authentications (SPF, DKIM, DMARC) fail.

    Unique open rate

    Unique open rate refers to the percentage of emails that were opened at least once in comparison with all emails sent.

    Open tracking needs to be enabled for a domain in question in the Sending Domains tab. Only then will email opens be recorded.

    Tracking of email opens happens with the use of invisible pixels that Mailtrap attaches to each message. When a message is opened, an open event is recorded and a message is counted towards the unique open rate.

    It’s important to know that some email clients, browsers, and plugins block such tracking. If they do, no open event will be recorded. 

    For that reason, the actual open rate is nearly always higher than what’s shown in the statistics. By how much depends on your audience. If you’re emailing a tech-savvy audience, there’s a good chance at least some recipients will block tracking. If you’re addressing a general audience, the open rate numbers are probably more accurate.

    What’s a good open rate? 

    That very much depends on what you send. Marketing emails tend to score about 20-25% unique open rate. On the other hand, password reset emails hit nearly 100% because everyone wants to complete a process they just started seconds before. Product updates, confirmation emails, or onboarding sequences are somewhere in-between.

    Several notes about unique open rate tracking:

    • As the name would suggest, unique opens are counted. If a message is opened multiple times, it will count only once.
    • If any link is clicked in a message, we automatically count it as opened as well, even if no open event was recorded earlier (because, for example, tracking was blocked).
    • The percentage of opens is counted against all emails delivered, not all that were sent. For example, if 30 emails were sent, and of these, 20 of were delivered and 12 were opened, then the unique open rate is 60% (12/20).

    Each event in Mailtrap - for example, open or delivery - has a timestamp that you can look up in the Email Logs. When calculating a unique open rate, Mailtrap takes into consideration all emails that had an open event in a chosen period of time. 

    Some emails are opened weeks or months after they were delivered. Selecting too short of a period can result in discrepancies or even rates higher than 100%. 

    How? Let’s look at an example.

    You send and deliver 100 emails on Monday and another 100 on Tuesday. No emails are opened on Monday but 150 unique opens are recorded on Tuesday. The unique open rate will be:

    • 0% for Monday (0 opened/100 delivered).
    • 150% for Tuesday (150 opened/100 delivered).

    To avoid such situations, set the time periods long enough to capture both deliveries and opens, knowing that some recipients will open messages days after a successful delivery. To filter out the results only for a specific email type, use unique categories when configuring an email.

    Click rate

    Click rate refers to the percentage of emails that received at least one link click compared to all delivered emails. 

    When any of the links in an email are clicked, a click event is recorded. The same happens for any further clicks, regardless if a recipient keeps clicking on the same or different links. 

    You can see the details of each click (timestamp, Recipient's IP, URL) in the Events History in the Email Logs. The total number of clicks is shown in the emails list to the left:

    However, the metrics such as clicked and click rate used in the statistics are calculated differently. 

    Clicked metric indicates how many emails received at least one click on a link present in its body. Once a single click is recorded, email is counted as clicked. Any further clicks, on the same or another link inside this message, are not counted for the purpose of calculating the clicked metric. 

    As such, the email above would count only once as clicked, even though it received multiple clicks.

    The click rate is basically clicked / delivered * 100%.

    Let’s look at another example. Ten emails were sent and delivered, each containing several links. Five recipients did nothing about them. The other five opened their emails and clicked three times each on the links - some kept clicking on the same link, others clicked on different URLs.

    For such campaign, you’ll see in Mailtrap:

    • Delivered: 10
    • Clicked: 5
    • Click rate: 50% (5/10)

    As you can see, the number of clicks on each of the emails is not taken into consideration here.

    You’ll see one more rate in the statistics - click to open rate. This one is very straightforward. The formula is: click rate / unique open rate and no color coding is available at the moment.

    As was the case for opens, click tracking needs to be enabled for a domain in question in the Sending Domains tab.

    By default, all links in your emails are redirected through the t.mailtrap.live domain. For example, if you link to www.google.com in your email, the URL will look like this: https://t.mailtrap.live/cl/8Q%252FvtwOw82baL… (the actual URL is much longer). When clicked, it will then redirect to www.google.com in a matter of milliseconds.

    This redirection allows us to reliably track each click on your emails. What’s more, such tracking can’t be blocked so the click numbers are certainly more accurate than opens we discussed previously.

    If, however, you’re not comfortable redirecting through t.mailtrap.live, you can instead use the very domain that you send the emails from.

    To enable a custom domain, switch the toggle on for Use Custom Domain for Clicks Tracking in the sending domain settings. Then, add a new DNS record to your domain registrar or manager and click Verify.

    See Domain Setup article for details on adding DNS records.

    Note: For now, the links redirected through custom domains are HTTP only. 

    Bounce rate

    Bounce rate refers to the percentage of emails that were dispatched from Mailtrap but were rejected on the recipient’s end, as compared to all emails sent.

    Emails may bounce for numerous reasons, most commonly:

    • Invalid email address.
    • Rejection by the recipient’s mailbox because email is deemed spam, phishing, etc.
    • The security policy of a mailbox provider that rejects emails from all or some domains.
    • Permanent connection issue.

    The term bounce used in Mailtrap is also known as a hard bounce. This is different from a soft bounce - another event present in Mailtrap that refers to a temporary delivery problem. 

    If an email soft bounces, Mailtrap will try to deliver it 10 more times. If there’s no positive outcome, an email will (hard) bounce and will be counted towards the bounce rate.

    Read more about the differences between soft and hard bounce.

    When an email bounces, a recipient is added to a suppression list for this domain. You won’t be able to send any further emails to this address from this particular domain. Technically, you may still email them from other verified domains but if a reason for the bounce was an invalid address, it will only hurt your deliverability in the long run.

    Bounces, as well as spam reports, are never good for you. They also hurt the reputation of Mailtrap and our users’ capability to deliver emails. For that reason, we closely monitor excessive bounce and spam rates and take the appropriate actions when necessary.

    Here are some things you can do to decrease the bounce rate.

    • Verify email addresses prior to sending emails to them. There are lots of tools online for doing this. Having users confirm their email address or register with Google or social media accounts helps eliminate typos.
    • Test your emails with the classic Mailtrap’s email testing functionality. Send your emails to Mailtrap and check the spam score of each template. Use the provided hints to improve the score and increase your chance of successful delivery.
    • Monitor stats for specific mailbox providers (via Mailbox Providers tab) to quickly find the ones that underperform. When issues arise, consult the providers’ deliverability guidelines or contact them directly to resolve the issues.

    Spam complaints

    Spam complaints refers to the percentage of emails that are reported as spam by their recipients, as compared to all emails that were delivered.

    An email can be categorized as spam at several stages:

    • When it’s dispatched and arrives at the recipient’s mailbox provider, the server’s policies may classify it as spam and decide that it won’t go any further (this happens to most of the spam on the internet, luckily). Mailbox provider rejects a message and a bounce event is recorded in Mailtrap. 
    • An email can be accepted but may never make it to an inbox. Instead, it’s sent straight to spam by an email client or a mailbox provider. Mailtrap has no way of knowing where such emails go so the last known event is delivery.
    • Finally, an email can arrive in the recipient’s inbox. Then, a recipient can decide to mark it as spam. A spam event is recorded in Mailtrap.

    Emails are only counted towards the spam complaints metric if the last situation occurs.

    If a recipient reports your message as spam, their email address will be added to a suppression list for a given domain. You won’t be able to send any further emails to them from this domain. It doesn’t prevent you from emailing them from another verified domain but, as was the case for bounces, it’s usually not a good idea.

    Spam reports are very bad for your email deliverability. There are several things you can do to reduce them:

    • Make sure the recipient is happy to receive emails from you. If they signed up for a specific product or service, refrain from sending them completely unrelated emails.
    • Adjust the frequency of emails, some recipients may not be comfortable getting too many of them. Letting recipients manage their email notification settings usually does the job well.
    • Test each template using Mailtrap’s built-in spam analysis, available for both Sending and classic Testing. Also monitor the affected emails via the Email Logs. Check the responsiveness, validate links, and make sure no annoying “Hi [name]” sneaked into a copy.

Back To Top

Still need help? Contact Us Contact Us