When I know the e-mail itself looks ok, I go to a complete testing system and try to trigger the e-mail from there, with real data and think about special cases, like what happens if some optional data is there or not. There I look into the logs, if some errors happened, because of some naming mismatch in the payload that should trigger the e-mail, or if some expected non optional data is missing.

When this is all ok, the e-mail is put to the live system and sometimes ramped up just to a part of users for some time in these stages:

  • Just our team,
  • all employees,
  • 10, 20, 50, 70, 100 percentage of all users - and watching logs and load of the systems involved. </p>

    There you can see then all kinds of problems

  • special data combinations you haven't thought of, or
  • the power of servers we ask for user specific data is not sufficient, or
  • we take to long to send out to all users in one day - or
  • for daily newsletters we can't get out the e-mails until 12:00 am - the later the day, the less sense the information makes to the recipient - nothing is older than yesterdays newspaper newsletter...
    Some good about our system, when an error happens the e-mail is not send - so mostly no user facing issues :-) </p>

    Thats roughly the process for every new e-mail.

    To build yourself a quick local testing system you could take a shortcut with some assumption: - the default e-mail client of every platform uses the same HTML rendering engine as the default browser. Put the HTML somewhere accessible on a web server and visit with

  • Mac OS X + Safari,
  • Windows + IE,
  • iOS + mobile Safari,
  • Android + mobile Chrome.
    The most obvious issues are catched and are reproducible by this - which is also an important topic for the developer and you to analyse issues. </p>

    If you have the HTML of your e-mails accessible it's also possible to do an easy first regression testing, when there is a change in a component of an e-mail that is used among all your e-mails: https://github.com/BBC-News/wraith It compares screenshots of the same web path on different hosts. I like it, because if you have to identify changes in 100 types of e-mails, it identifies changes easily. For me it solves the job: Frontend-dev: 'Hey tester, I just changed our button code a little, that is used in every e-email, it should not change the layout...'

    In the next part on a part of the backend, how to track and analyse e-mail interactions: https://einmanaleiki.de/testing-e-mails-backend-part-4-33