Waiting for an email in code: polling without a webhook

Push and pull, and what each one costs

There are only two ways for your code to find out that a message has landed. Either the other side tells you, or you ask. Everything else — a client library with a waitFor in it, an SDK that « streams » an inbox, a test helper that blocks — is one of those two with the machinery hidden, and it is worth knowing which one you are holding before you have to debug it.

Four arrangements cover nearly everything on offer.

A webhook

The service makes an HTTP request to an address you own, each time mail arrives. It is the cheapest possible wait — you do nothing at all until there is something to do — and the price is an address on the public internet, a listener that is up at the moment the mail is, a shared secret to prove the request came from them, and your own answer to what happens when your listener was not there.

A long poll

You make the request and the server holds it open until either mail arrives or a timeout expires. It needs nothing from you but an outbound connection, and it costs the server a worker per waiter — which is why every service that offers one caps both the length of the wait and the number of simultaneous waiters.

A plain poll

You ask, repeatedly, and every request is answered at once with whatever is there. It is the only arrangement that works from a laptop behind a router, from a CI runner with no inbound route, and from an agent running inside somebody else's sandbox — and it is the whole subject of this guide.

A mailbox protocol

IMAP has IDLE, which is a long poll wearing a different hat: the connection stays open and the server announces new mail on it. It is genuinely push-like, and it wants a mailbox with credentials, a client that can hold a socket open and reconnect when it drops, and a server that honours the command — which is a great deal of machinery for a job that needs one message.

Put side by side, the choice turns out to be less about elegance than about what each one demands of the machine your code is running on.

The loop everybody writes first

It is four lines, it works on the day it is written, and every one of its problems shows up later and somewhere else: in a pipeline at three in the morning, in an agent that has been « thinking » for eleven minutes, in a mailbox that answers 429 to a colleague because your loop is holding the budget.

import time
import requests

while True:
    r = requests.get("https://grabmail.io/api/v1/mailbox",
                     params={"address": "signup-42@grabmail.io"})
    if r.json()["messages"]:
        break
    time.sleep(1)

Five things are wrong with it, and only the first is obvious.

It never gives up

There is no deadline, so when the message genuinely is not coming — the form rejected the address, the sender's queue is stuck, somebody mistyped the domain — this loop does not fail. It hangs. A job that hangs is worse than one that fails, because the log ends without ever saying why.

It counts attempts and calls them seconds

Even with a limit on the number of passes, thirty attempts at « one second » is never thirty seconds: each pass also costs a request, and a request that takes 400 ms turns your thirty seconds into forty-two. Add one retry and the arithmetic stops being arithmetic.

Every runner asks on the same tick

Start twenty jobs from the same pipeline and they poll in lockstep, because they all began within a few milliseconds of each other and they all sleep the same whole second. The peak is twenty times the average, and it is the peak that gets refused.

It takes the newest message, not yours

The first entry in the list is whatever is at the top of that mailbox, which on a public address may be somebody else's mail and on a reused address is last week's. A loop that exits on the first message it sees will happily exit before the one it was waiting for has arrived.

It treats every answer as a success

Reading the message list out of a 429 or a 404 raises an error three frames away from anything that explains it, and reading it out of a 500 may raise nothing at all. The status code is the first thing to look at, not the last.

Stop on a clock, not on a count

Take the deadline once, before the first request, from a monotonic clock — one that cannot jump backwards when the machine corrects its time — and compare against it at the top of every pass. Everything else in the loop is then free to change without changing how long the wait lasts: you can widen the interval, retry a refusal, or add a second filter, and ninety seconds is still ninety seconds.

How long is long enough is a question about the sender rather than about you. Mail that a machine generates in answer to a form is usually delivered in single-digit seconds; a queue with a backlog, a receiver that greylists, or an hourly batch is a different order of magnitude, and no interval you choose makes it arrive any sooner.

The deadline is also the honest place to put your error message. « Nothing matching “Confirm your email” arrived at signup-42@grabmail.io within 90 s » names the address, the filter and the budget, which is three of the four things needed to work out what happened. The fourth — what did arrive — is worth printing too: a list of the subjects the loop saw and rejected turns « it is flaky » into « the subject line changed » in one reading.

How often to ask, and when to widen

The floor is whatever the service allows, and here it is one request per second, per address. That is not a discouragement — polling once a second is the intended pattern, there is no daily quota, no monthly quota and no burst credit to manage — but it is a floor, and a loop that asks twice inside the same second gets a 429 for the second one rather than a faster answer.

A fixed interval

One second, every time, until the deadline. Perfectly good for a wait that will be over in ten seconds, and the right default for a single test on a single runner. Its only fault is that it goes on asking at the same rate long after it has become obvious that the mail is not coming.

A widening interval

One second while the message is probably still in flight, then doubling — two, four, eight — with a ceiling. It costs a little latency on a message that arrives late and saves most of the requests in a wait that was going to fail anyway. Cap it: an interval that doubles without a ceiling spends the back half of a two-minute deadline asleep.

Jitter, added and never subtracted

Spread the passes apart by a random fraction so that twenty runners stop asking on the same tick. The usual recipe — a random value between zero and the interval — is wrong here, because half of its range falls under the one-second floor. Add the randomness on top instead: the interval is a minimum, and jitter only ever makes a pass later.

A pause that is not yours to choose

When the answer is a 429, the interval is whatever Retry-After says, and the pass that was refused was not an attempt. Count it as one and a loop that is being throttled will spend its whole deadline collecting refusals without ever having read the mailbox.

Fifteen seconds at one second, then doubling to a ceiling of eight, with jitter on top, fits almost every wait in this guide into six lines:

def delay(attempt: int) -> float:
    # One second while the message is probably still in flight, then
    # wider. Never below a second: the list endpoint allows one call
    # per second, per address, so jitter is added and never taken off.
    step = 1.0 if attempt < 15 else min(8.0, 2.0 ** (attempt - 14))
    return step + random.uniform(0.0, step / 2)

The exponent is offset so that the widening starts after the fixed stretch rather than from the first pass. Without that offset the interval has already reached eight seconds by the time a slow sign-up mail lands, and a wait that should have taken twelve seconds takes twenty.

None of this applies to the first request. Ask immediately, before any sleep: a message that was already in the mailbox when the loop started — the normal case for anything triggered before the wait began — should not cost a second of latency to notice.

Reading a refusal

Every answer from the list endpoint is JSON, and the ones that are not a mailbox share one shape: an error slug, which is stable and is the thing to branch on, and a message, which is prose and may be reworded at any time. A refusal for going too fast carries a header as well:

HTTP/1.1 429 Too Many Requests
Retry-After: 1
Content-Type: application/json; charset=utf-8

{"error":"rate_limited","message":"one request per second, per address"}

Retry-After is in whole seconds, and it is the real figure — taken from how long this address's budget actually has left, not from a constant in the documentation. Sleeping exactly that is both the politest and the fastest thing to do: a shorter sleep is refused again, a longer one is time given away. Here is everything a polling loop can meet, and what each answer is really asking of it.

Two of those seven mean stop, and they are the two worth being loud about. A loop that treats unknown_domain as « not yet » spends a full ninety seconds proving something the service told it in the first forty milliseconds.

Which message is yours

A mailbox is not a queue, and the newest thing in it is not necessarily the thing you are waiting for. On a public domain anybody who guesses the address can send to it; in a test suite the same address is often reused between runs; and one sign-up frequently sends two messages — a welcome and a confirmation — of which only one carries the code. The fix is a watermark, and it has to be taken before the thing that causes the mail.

1. Before you submit the form, list the mailbox with limit=1 and keep the id of the newest message, or nothing at all if it is empty. That id is the watermark.
2. Do the thing — submit the form, call the endpoint, click the button.
3. Poll the list. Messages come back newest first, so walk down from the top and stop the moment you meet the watermark: everything from there down is older than your action and can be ignored without being read.
4. Filter what is above it on the sender, the subject, or both. A substring is usually enough, and it should be the part that is not going to be localised — a test matching « Confirm your email » fails the day the account under test is switched to another language.
5. Then, and only then, open it. The listing carries a short preview and not the body, and the code you are after is very often past the end of it. One more request gets the whole message, and it is charged against a separate and far larger budget than the list.

In a shell, those two reads look like this — the watermark first, then the poll:

curl -sG https://grabmail.io/api/v1/mailbox \
  --data-urlencode "address=signup-42@grabmail.io" --data-urlencode "limit=1"
curl -sG https://grabmail.io/api/v1/mailbox \
  --data-urlencode "address=signup-42@grabmail.io" --data-urlencode "limit=25"

Both requests name the address in full, because here the address is the mailbox: there is no session, no cursor kept on your behalf, and nothing about one call that the next one remembers. It is also why an address is safe to watch from two places at once — reading consumes nothing, so two loops on the same mailbox both see every message and neither can take one out from under the other.

Acting exactly once

A poll that is retried can see the same message twice, and it is not a rare event: the server answers, the connection drops before the body reaches you, your HTTP client retries, and the second answer contains the message the first one already carried. If what you do with a message is click a link, confirm a payment or post into a channel, doing it twice is a bug with consequences outside your process.

Keep the ids you have already handled

A set of ids in memory is enough for a wait that lives and dies inside one function. For anything that has to survive a restart — a mailbox drained by a scheduled job, an agent working through a backlog — it has to be written down somewhere that survives with it.

Deleting is idempotent

Deleting a message answers 200 the second time as well as the first, so a retried delete never looks like a failure and never needs a special case. Delete after you have acted rather than before: a crash between the two then costs you a re-read, which is recoverable, instead of the message, which is not.

The id here is not the sender's Message-ID

The id in the API is ours: it is scoped to one mailbox, and it stops existing when the message expires. The Message-ID header is the sender's, it travels with the message, and it is the one you want if you are matching the same mail across two systems — the guide to headers says where to find it.

None of this is needed for a test that waits for one code and then throws the mailbox away. All of it is needed the moment a loop runs unattended, because the failure it prevents does not look like a failure: it looks like the work being done, twice, correctly.

When the waiting belongs to the server

There is one place here where you do not write the loop, and it exists for callers that cannot afford one. An AI agent pays for every turn it spends checking, so a tool that answers « nothing yet » nine times is nine turns of nothing. The MCP server's wait_for_message holds the request open instead, polls on our side, and answers once — either with the message, or with a plain statement that it waited and nothing came.

$ curl -sX POST https://grabmail.io/mcp -H "Content-Type: application/json" -d '{
  "jsonrpc":"2.0","id":1,"method":"tools/call",
  "params":{"name":"wait_for_message","arguments":{
    "address":"signup-42@grabmail.io","subject_contains":"code",
    "timeout_seconds":25}}}'

Four things about it are worth knowing before you build on it.

It waits 25 seconds at most

timeout_seconds can ask for less and never for more. The ceiling is not arbitrary: each waiter is a worker doing nothing but sleeping, and a request held open for minutes is a request that dies to somebody's proxy timeout long before it returns.

It filters on the way in

from_contains, subject_contains and since_id are the same three decisions as the section above, made on the server. since_id is the watermark, and it matters more here than anywhere else: without it the call returns immediately with whatever was already sitting in the mailbox.

A timeout is an answer, not an error

When nothing arrives it returns timed_out set, along with how long it actually waited, and says in as many words that calling again is how you keep waiting. Two or three calls is a normal wait for a sign-up mail: that is the loop, and it is three turns instead of ninety.

There are 8 waiting places, and no queue

When all of them are busy the call comes back at once and says so, rather than joining a line behind seven other agents. It is the right failure: an agent told « too many waits in progress » can list the mailbox and carry on, while an agent sitting in a queue can only sit.

The per-address limit still applies inside it — our loop is rate-limited exactly like yours, so a server-side wait is not a way round the floor, only a way to stop paying for it in turns. For a test suite none of this is worth the trouble: a test is already a process that is allowed to sleep, and a loop in the language the test is written in is far easier to debug than a remote one. The MCP guide covers the rest of the tools.

The whole loop, once

Everything above, in one file: a deadline from a monotonic clock, an interval that widens with one-sided jitter, Retry-After honoured and not counted as an attempt, a watermark to decide what is new, a filter on the subject, and one extra request to fetch the message that the listing only previews.

import random
import time
import requests

API     = "https://grabmail.io/api/v1"
ADDRESS = "signup-42@grabmail.io"


def delay(attempt: int) -> float:
    # One second while the message is probably still in flight, then
    # wider. Never below a second: the list endpoint allows one call
    # per second, per address, so jitter is added and never taken off.
    step = 1.0 if attempt < 15 else min(8.0, 2.0 ** (attempt - 14))
    return step + random.uniform(0.0, step / 2)


def watermark(s):
    # Read this BEFORE the form is submitted. Every id above it
    # afterwards is mail that arrived because of what you did.
    r = s.get(f"{API}/mailbox", params={"address": ADDRESS, "limit": 1})
    r.raise_for_status()
    seen = r.json()["messages"]
    return seen[0]["id"] if seen else None


def wait_for(s, subject, since, timeout=120.0):
    deadline = time.monotonic() + timeout
    attempt  = 0
    rejected = set()

    while time.monotonic() < deadline:
        r = s.get(f"{API}/mailbox", params={"address": ADDRESS, "limit": 25})

        if r.status_code == 429:
            time.sleep(int(r.headers.get("Retry-After", "1")))
            continue                      # refused, so it was not an attempt
        if r.status_code == 200:
            for m in r.json()["messages"]:        # newest first
                if m["id"] == since:
                    break                 # older than the watermark
                if subject.lower() in m["subject"].lower():
                    full = s.get(f"{API}/message/{m['id']}",
                                 params={"mailbox": ADDRESS})
                    full.raise_for_status()
                    return full.json()
                rejected.add(m["subject"])
        elif r.status_code < 500:
            raise RuntimeError(r.json().get("error", r.status_code))
        # a 5xx falls through: transient, and the deadline still governs

        time.sleep(delay(attempt))
        attempt += 1

    raise TimeoutError(
        f"nothing matching {subject!r} at {ADDRESS} in {timeout:.0f}s; "
        f"saw {sorted(rejected) or 'nothing at all'}")

It is deliberately fifty-odd lines of standard library and one HTTP client. There is nothing to install, nothing to configure and no secret anywhere in it — which is the point: the same shape moves unchanged to Node, to a shell script, or to whatever your test framework already uses to make requests.

1. Read the watermark before the action that causes the mail, never after it.
2. Ask once immediately, and only then sleep. Never sleep first.
3. Take the deadline from a monotonic clock, and test it at the top of every pass.
4. Keep the interval at or above one second per address, and add jitter upwards only.
5. Sleep exactly what Retry-After says, and do not count a refusal as an attempt.
6. Branch on the error slug: unknown_domain and invalid_address mean stop, not wait.
7. Match on sender or subject, and stop walking the list when you reach the watermark.
8. Open the message before you parse it — the listing carries a preview, not the body.
9. Fail with the address, the filter, the budget, and the subjects the loop rejected.

Nine rules, and eight of them exist because of a failure somebody had to reconstruct from a log. The one that is not about failure is the second: asking once before the first sleep is what makes a wait for a message that has already arrived take four milliseconds instead of a second — which across a suite of two hundred tests is three minutes of wall clock that nobody afterwards has to explain.

Questions