Inbox placement tests

Browsing filters, searches and view choices are remembered in your browser tab across refreshes and navigation, separately for each user and workspace.

A placement test answers one question before your prospects do: when this mailbox sends this email, where does it land? Warmbly sends a copy from a real mailbox of yours to a panel of seed inboxes spread across providers, then reads each seed's mail to see whether the copy reached the primary inbox, a Gmail tab, spam, or nowhere at all.

Tests live under Placement tests in the dashboard. Reading results needs the View analytics permission, and starting or cancelling a test needs Send campaigns, because a test sends real mail from a real mailbox.

Every copy gets one of these verdicts:

The rates are shares of the copies that were delivered, meaning every copy that left the sender and got a verdict: inbox, both kinds of tab, spam and never arrived. Failed and cancelled copies are not in the denominator, because they say nothing about the receiving side. The inbox rate counts the primary inbox only, so a copy in Promotions counts against it.

Results are broken down by provider family: Gmail, Google Workspace, Outlook.com, Microsoft 365, Yahoo Mail, iCloud, Zoho and the rest. Gmail and Google Workspace are kept apart, as are Outlook.com and Microsoft 365, because the consumer and business services filter differently and an email can do well at one and badly at the other.

The copy is the one your campaign sends

Each copy is rendered the way a campaign send is, in the same order: merge fields and spintax for a sample lead, AI blocks for that lead, form links, the mailbox's signature, the opt-out line, the List-Unsubscribe header, attachments, and the open pixel and wrapped links when tracking is on. What a test leaves out is what only a real conversation has: threading into an earlier message, the A/B variant choice, and the campaign's CC and BCC.

When you test a campaign step, the copy renders for the campaign's first lead unless you pick another contact (on the Warmbly Cloud panel it renders for a placeholder instead, so a lead's details leave your instance only when you pick one, and AI blocks are left out), and the step is copied when the test starts, so an edit halfway through does not change what the later seeds receive. A follow-up step that threads under an earlier one has no subject of its own, so the test uses the subject of the step it threads under, without the Re:. A template you write by hand renders the way a new campaign would, with the workspace's default opt-out line and the unsubscribe header on.

Nothing is added to the message for the test's sake: no marker in the subject, no special header, no hidden text. Each copy is found again by its Message-ID, so what the seed's filter judges is exactly what a prospect would receive.

The test also runs the same content check the campaign editor uses and shows its score and issues next to the results.

Sent one at a time from the real mailbox

Copies go out one by one, each as its own send from the mailbox you chose, through the worker that mailbox normally sends from, and alternate between providers so no single provider sees a burst. How far apart they go is the test's pace:

Both vary each gap by up to a third either way, and the operator can change the spaced gap; a quick test uses the smaller of the two. Quick is for when you want an answer now, for example after fixing a DNS record. Results appear as each seed syncs the copy, usually within minutes of it being sent. Scheduled tests from a monitor always run spaced.

The sending mailbox's own copy of each probe is kept out of your unibox.

Found in the seed's mail

Seed inboxes are ordinary connected mailboxes that Warmbly syncs. As a seed's mail comes in, each copy is matched by its Message-ID and the folder and Gmail labels it arrived with become its verdict. Results appear on the test live, one copy at a time, for everyone looking at it.

A copy not seen within 120 minutes of being sent is marked never arrived. Copies on Warmbly Cloud's panel get half an hour more, because their verdict travels back from the cloud. A test finishes when every copy has a verdict. A test in which no copy was delivered at all ends as failed, with the reason.

A test runs against one panel of seed inboxes:

The page shows each panel's seed count and its mix of providers, and says why a panel is unavailable when it is.

Addresses on the instance and Warmbly Cloud panels are shown masked (a***@gmail.com), so a shared panel cannot be listed and added to a sender's allowlist. Your own seed inboxes are shown in full.

Each test uses up to 20 seeds by default (the operator can change this). They are picked round-robin across provider families, so even a small test covers every provider the panel has. Two kinds of seed are always skipped:

• Seeds on the sender's own domain. Mail between two mailboxes in the same tenant skips most of the filtering a test is there to observe, so a seed at acme.com never receives a test sent from [email protected].
• The sending mailbox itself.

If nothing is left after that, the test is refused with placement_no_seeds and nothing is sent.

Testing only some providers

On the shared and Warmbly Cloud panels, the Providers chips under the panel narrow a test to the providers you pick, for example only Outlook.com and Microsoft 365 when that is where replies dried up. The test then picks its seeds from those providers alone, still round-robin and still without showing their addresses. Leave All selected to cover every provider the panel has.

Choosing which of your seeds get a copy

On Your seed inboxes, you can pick the seeds a test goes to instead of letting Warmbly spread it across providers, for example only the Gmail and Outlook.com seeds you are about to open and check by hand. The chooser under the panel lists your seed inboxes with their provider, and a chip per provider selects all of that provider's seeds at once. Choose none and the test picks as usual.

Every seed you choose gets a copy: the usual seeds-per-test cap does not apply to seeds you picked by hand. When the sending mailbox has too little of its daily limit left for all of them, the test is refused with the numbers rather than quietly skipping some. Seeds on the sender's domain are greyed out and skipped, and a seed that is not connected cannot be chosen. Choosing individual seeds is only possible on your own panel, because addresses on the shared and Warmbly Cloud panels are masked; there you pick providers instead.

Any mailbox connected to your workspace can become a seed inbox. Mark mailboxes on the Seed inboxes tab of the placement tests page; it needs the Manage mailboxes permission, and a workspace can have up to 50.

A good seed inbox is one that behaves like a stranger's inbox:

• A separate address on a different domain from your senders. Seeds on a sender's domain are skipped, as described above.
• Personal accounts at the big providers. A Gmail, an Outlook.com and a Yahoo account cover most consumer filtering. Add a Google Workspace or Microsoft 365 mailbox on another domain you own to see the business side.
• Left alone. Do not add your senders to its contacts, mark their mail as not spam, or change its filtering. Every one of those teaches the provider that your mail is wanted, which is exactly what a prospect's inbox has not learned.

Marking a mailbox as a seed changes how Warmbly treats it:

• It never warms up. Its warmup is turned off when you mark it, and starting warmup on it is refused (mailbox_is_seed) until you unmark it. A seed that exchanged mail with the warmup pool would stop being a stranger to every sender in it.
• It never sends campaign mail. It is left out of every campaign's sending mailboxes, and it cannot send a placement test itself.
• It syncs for tests. It gets a large daily sync allowance, so copies arriving from many tests are never deferred past the 120 minute window, and it imports no history. The copies it receives are stored like any other mail it syncs.

A mailbox on the instance panel cannot also be one of your seeds, and a mailbox cannot be marked while a test is still sending from it. Unmarking a seed makes it an ordinary mailbox again; its warmup stays off until you start it.

A test needs a sending mailbox, a panel, and a copy to send. The sending mailbox has to be connected and active, and cannot itself be a seed inbox.

• Test a campaign step by choosing the campaign and the step. The step's subject and body are filled in, and the campaign's opt-out line, tracking and attachments apply. Pick a contact if you want the merge fields and AI blocks rendered for someone other than the campaign's first lead.
• Test an ad-hoc template by writing a subject and a body. This is the way to try a draft before it becomes a campaign.

Cancelling a running test stops the copies that have not been sent yet. Copies already sent keep being classified, so a cancelled test still shows where they landed.

Tracking

A comparison sends each seed both versions back to back, in a random order per seed, so both see the same filter at the same moment. The difference between the two halves is what the pixel and the wrapped links cost you. Both halves send a seed the same words: the copy for that seed is rendered once, with its spintax, merge fields, AI blocks, form links, signature and opt-out line resolved, and the tracked half only adds the pixel and wraps the links. Different seeds can still draw different spintax, as different leads would. If a seed's copy cannot be rendered, neither half is sent to that seed. It counts as two tests and sends twice as many copies. A campaign that sends plain text carries no tracking, so On and Compare are refused for it (placement_invalid_tracking).

Every copy is a send from the sending mailbox. It is charged to the mailbox's daily campaign limit, and the mailbox's campaigns see those sends in their daily budget and in their minimum gap between sends. A test to 20 seeds uses 20 sends of that day; a comparison uses 40. Copies still waiting to go out already hold their place in the day, so a campaign cannot spend them while the test sends. When the mailbox has less left than a full test needs, the test goes to fewer seeds instead; below five seeds per test it is refused with placement_daily_budget, because a smaller test is noise. Test from a mailbox with headroom, or before its campaigns run for the day.

A mailbox runs one test at a time: until every copy of a test has been sent, another test from the same mailbox is refused with placement_sender_busy.

Free tests and credits

On the hosted product, every workspace gets free tests on the shared panel each month:

These are the defaults, and the operator can change both numbers. The month is the calendar month in UTC. A tracking comparison counts as two tests. The placement tests page shows how many free tests you have used, and the new-test dialog says which free test a test uses.

A paid test never starts without you agreeing to its price. Once the free tests are used up, the dialog shows what the test costs and your balance, and you tick Pay 25 credits for this test before it can start; the button repeats the price. Through the API the same agreement is max_credits, the most you agree to pay; a price above it (because the last free test went to a teammate in the meantime, say) is refused with placement_quota_exceeded, naming the price, and nothing is charged. Credits are drawn like any other spend, monthly allowance first and top-ups after, and your spend limits apply. The charge is logged as Placement test in the credits history.

• A test that delivers nothing is refunded. When no copy of a paid test left the sender (the worker was offline, or you cancelled before the first copy went out), its credits go back to the pools they came from, logged as Placement test refund, and they stop counting against your spend limits. Monthly credits whose allowance has reset since the charge are not returned, because they would have expired with it. A free test that ended that way does not use up a free test either.
• A comparison with one free test left uses the free test for one half and pays for the other.
• Monitors never spend credits. A monitor's scheduled test uses the free tests like any other, and once they are gone it records why and tries again a day later.
• A trial cannot pay. Credits are part of a paid plan, so a trial waits for the next month.

Tests on your own seed inboxes are never counted and never cost credits. A self-hosted instance does not meter its own panel at all. Tests on Warmbly Cloud's panel use the free tests of the Warmbly Cloud workspace the instance is linked to; paying for more with credits is not offered there.

A monitor re-tests a running campaign on a schedule, so a drop in placement shows up before it shows up in your reply rate. Set it up from the campaign.

Each run tests the campaign's first email step that has both a subject and a body, rendered for its first lead, with the campaign's own tracking. The runs rotate through the campaign's sending mailboxes, one mailbox per run, because every mailbox has its own standing and the one that slipped is the one you want to find. A new or re-enabled monitor runs its first test within about five minutes.

A monitor only tests a campaign that is running. When a run cannot start (the campaign is paused, none of its mailboxes is connected, the allowance is used, or the mailbox has too little of its daily limit left), the monitor records why and tries again a day later rather than a whole interval later. Its tests count against the monthly allowance and the mailbox's daily limit like any other.

When a monitor's test lands below its threshold, everyone in the workspace who can view campaigns gets a Placement monitor alert, with the result and a link to the test. With Pause on alert on, the campaign is also paused automatically, with the reason in its activity log. Like every auto-pause, it does not resume on its own: find out what changed, then start the campaign again yourself.

A test answers where one mailbox lands. With hundreds or thousands of sending mailboxes the question is different: is the fleet healthy, and if not, is the problem one mailbox, one domain, or one provider? A placement batch runs the same test from many mailboxes and reads the results together. Start one with New batch on the placement tests page, with Test sender pool on a campaign's settings, or with Test untested mailboxes from the coverage line.

Each mailbox in a batch gets its own ordinary placement test with its own results, so everything above about how a test runs, what it costs and how to read it applies to every one of them. The batch adds shared settings, one progress view, results grouped across the fleet, and one cancel button.

Choosing the senders

A campaign's senders and all mailboxes can be narrowed by provider (Google Workspace, Microsoft 365, SMTP and the rest), by sending domain, by mailbox tag, and to mailboxes with no finished placement test in the last so many days. Disconnected mailboxes are left out unless you include them; a mailbox you tick by hand is always kept, and is reported by name if it cannot send when its turn comes. Seed inboxes never send.

The senders are resolved on the server and written down when the batch starts. A mailbox tagged or added to the campaign afterwards does not join a batch that is already running, and one removed from it does not leave.

Sampling

Testing every mailbox is thorough but spends a test and a day's sends from each. A sample tests part of the fleet instead:

How a batch runs

A batch can hold thousands of mailboxes, but only a few send at once. Warmbly starts the next mailbox when one has sent every copy of its test, so at most 20 of a workspace's batch mailboxes are sending at the same time, a batch starts at most 10 mailboxes a minute, and the instance as a whole runs at most 200 batch mailboxes at once, which keeps the shared seed panel from receiving a burst (the operator can change all three). Every batch test runs at the spaced pace; quick is for a single test. Consecutive starts rotate across sending providers and domains, so a batch never works through one provider's mailboxes back to back. The number of mailboxes in a batch has no effect on how many run together; the operator's ceiling on batch size, 10,000 by default, exists only to protect the instance.

Nothing is reserved when the batch starts. Each mailbox is checked when its turn comes, against its own daily limit, minimum gap, connection, worker and the workspace's standing, exactly as a single test is. So campaigns keep sending while a batch works through the fleet, and a batch never gets around a safeguard a single test would hit.

A mailbox that cannot run when its turn comes is handled by the batch's policy:

A mailbox still sending another test is always retried a quarter of an hour later. A mailbox with no seed it can reach (every seed is on its own domain) is skipped under either policy. One mailbox failing never fails the batch: every mailbox keeps its own result or its own reason, such as its daily limit, its connection or its worker.

A batch ends completed when every mailbox finished a test, completed with warnings when some were skipped or failed, failed when none finished, or cancelled. When the free tests or the credits you agreed to run out partway, or the campaign or step being tested is deleted, the mailboxes not yet started are skipped with that reason and the batch ends with the results it has.

Cancelling a batch stops it starting mailboxes, cancels the copies not sent yet, and leaves the copies already sent to be classified, so a cancelled batch keeps every result it had.

Reading a batch

The batch page shows its progress and the overall placement, then four views:

• Mailboxes, worst inbox rate first, with the reason for any that was skipped, deferred or failed. Each opens its own test.
• Domains, each sending domain's placement across its mailboxes, worst first. One bad mailbox on an otherwise healthy domain is a mailbox problem; a whole domain low is that domain's reputation or authentication.
• Providers, the same by sending provider, which shows a problem with one kind of mailbox.
• Recipient providers, a table of sending domains against the providers that received the copies, which shows a domain that only one provider is filtering.

For a tracking comparison the overall figure is the tracked half, the copy the campaign really sends, with the untracked half beside it.

The batch does not group by worker. A worker signs in to your mailbox provider, and the provider delivers the mail from its own servers, so recipients' filters never see which worker handled a mailbox.

What a batch costs

Before a batch starts, the dialog states the workload: how many tests, the most copies it can send (seeds per test times mailboxes, doubled for a tracking comparison), and how many mailboxes send at once. Each mailbox's daily limit can only make the real number smaller.

Every mailbox's test counts as one test against the monthly free tests (two for a comparison), exactly like a single test. When a batch needs more tests than are free, you agree up front to the most the rest can cost, which the API takes as max_credits for the whole batch; the batch never spends more than that, and a test that delivers nothing is refunded as usual. When the tests cannot be paid for (a trial, or a panel without paid tests), a batch larger than the free tests left is refused; take a smaller sample or test on your own seed inboxes, which are never counted.

Fleet coverage

The placement tests page shows how much of your connected fleet finished a placement test in the last 30 days, and how many mailboxes never did. Test untested mailboxes starts a batch of exactly those.

A placement test is a signal, not a verdict. Read it with its limits in mind:

• Seeds are not your prospects. A seed inbox has never heard from you, which is what makes it useful, but it also has none of the history a real recipient's mailbox builds up, and providers weigh each recipient's own behaviour heavily.
• Corporate gateways are not covered. Many business recipients sit behind a security gateway such as Mimecast, Proofpoint or Cisco before their Microsoft 365 or Google Workspace mailbox. A seed shows the provider's own filter, not what a gateway in front of it would do.
• One test is noise. A panel holds a handful of seeds per provider, and filters do not treat every copy of the same email alike. One copy in spam is not a problem; the same provider junking most copies across several tests is.
• Compare over several tests and providers. Test more than one sending mailbox, repeat the test on another day, and look for a pattern: a single provider going bad usually points at authentication or content at that provider, while every provider going bad points at the sender.
• Never arrived is its own signal. A copy can go missing because the provider dropped or delayed it, or because the seed had not synced it yet. One missing copy means little; most copies missing at one provider usually means that provider is refusing or deferring your mail.

Tests also feed the seed placement figures on the Deliverability page, which roll every test in the window up by provider.

Both open the test, or the batch. Change them under Settings > Notifications; see notifications.

Placement tests and their results travel in a workspace export as a record, under Delivery events (without what a paid test cost, since credits do not travel), and so do placement batches with their mailboxes and results. A batch still running when it is exported arrives cancelled, so the destination never starts sending it. Campaign placement monitors travel with the campaigns. Which mailboxes are seed inboxes does not travel, so mark them again on the destination. A test that is still sending leaves its unsent copies behind.