A sending server that gets a 4xx reply does not fail. It queues the message and tries again later, for hours or days, depending on policy. A test suite that allows twelve seconds is not testing the same system. The mismatch is not a bug in either one, and conflating them produces test suites that assert things they cannot possibly observe.
What 4xx actually means
In SMTP, the first digit of a reply code classifies the outcome. 4xx is a transient negative: the command was well-formed and the server understood it, but the action cannot be completed right now. The correct client behaviour is to retry later, not to report a failure to the sender.
5xx is the opposite: permanent. Retrying the identical transaction will produce the identical result, so the message should be bounced back rather than retried. RFC 5321 builds the timeout expectations around exactly this split — the standard lists timeouts for the initial greeting, MAIL, RCPT, DATA initiation and the data block, each measured in minutes for the server side.
So the protocol's own model is: server-side waits are measured in minutes, client-side retries are measured in hours. A test that waits twelve seconds is observing neither.
Greylisting, the deliberate 4xx
Greylisting is a receiving-side policy that answers the first attempt from an unknown sender with a temporary rejection, on the theory that real mail servers retry and most spam software does not. It is effective precisely because retry behaviour is a signal, and it is hostile to test suites precisely because a correct client is supposed to wait.
This creates the classic false negative: a message "never arrived" in a test, when in fact the receiving server said not yet. Nothing is broken. The test simply exited before the retry window opened.
Three questions before writing a delivery test
Most confusion dissolves if you are explicit about which of these you are asserting:
- Did my application hand a message to SMTP? This is fully testable, in milliseconds, against a local sink. No internet required.
- Did the receiving server accept it? Testable in seconds to a minute, in the sense that you observe a
250or a4xx. A4xxhere is not a failure of your code. - Did the message land in a mailbox? Only observable over a retry window you do not control, and subject to greylisting, throttling, and reputation. Not a unit test. At best a monitored integration probe.
Almost every "email test is flaky" complaint is a class-3 assertion written as if it were class-1.
How to test the retry logic itself
You can absolutely test your own queueing behaviour without waiting hours — by controlling the receiving side. A local SMTP server that answers 4xx on demand lets you assert that your application queued rather than dropped, that it did not double-send on the retry, and that it eventually gives up after the configured attempt count. That is deterministic, runs in seconds, and tests the part you actually wrote.
The wrong version: point at a real receiver, hope it accepts, assert on arrival within N seconds. That test is measuring a third party's queue depth.
The budget that does make sense
For the parts that genuinely involve real delivery, assert bounds rather than equality, and make the bound explicit:
- First attempt accepted within the SMTP timeout you configured, not "instantly."
- Message visible in a polled inbox within a stated ceiling — Inboxmil's UI polls on a 12-second interval, so 12 seconds is the honest ceiling for "appears in the UI," not for "reached the mailbox."
- Zero duplicate deliveries after a transient rejection. This is the real correctness property and it is worth more than any latency number.
The takeaway
4xx means retry later, and "later" is measured in hours by the protocol. Test the handoff locally and deterministically, assert on transient-rejection handling with a sink you control, and treat real-inbox arrival as a monitored observation rather than a unit assertion. The flake disappears the moment you stop asking a test to wait out somebody else's queue.