the post that took thirteen days: four ways my own automation lied to me about its own state

The post was finished. It went out thirteen days later.

Nobody forgot about it. It sat in a queue across four separate working sessions while I tried, four different ways, to hand Meta's Graph API a carousel in a single call. Here are the images, make me a post. It was rejected every time, never usefully enough to say why, so each session I assumed my payload shape was slightly wrong and tried a slightly different payload shape.

The real constraint is that you cannot create the carousel parent container directly at all. You create each child container on its own, flagged as a carousel item. You confirm each child individually reaches a FINISHED processing state. Then you create exactly one parent container referencing the list of verified child ids, you confirm that parent reaches FINISHED too, and only then do you publish. Five steps, in that order, no shortcut. Thirteen days of a client's scheduled post sitting in a queue because I kept trying to compress a sequential state machine into one request.

I am writing it down because the three failures that came after were the same failure in different clothes. Each one was a piece of my own automation reporting a state that was not true, and me believing it because the report came out of my own terminal.

what actually runs here

The client is HorseTonic. In March and April I moved them off a Combell-hosted WordPress and WooCommerce install onto Next.js 16 with Tailwind 4 and next-intl on Vercel, with Mollie taking Bancontact, bpost doing shipping and Supabase behind it. The live language switcher offers Dutch, English and French. The repo carries four locale files, which matters later. The catalogue is Artico Gold, Artico Support, Massive Muscle, ZEN, Lacta Ease, PH, Tonic and Elektro+.

Around that site sit the parts that run without me. Images are generated rather than photographed. Product and comparison copy lives in those parallel locale files. Carousels are assembled and pushed to the Graph API by script. Renders get scored by a vision model before I look at them.

What none of it does is publish on its own. It gets to the last step and stops, I look, and I say go or I do not. That is not a limitation I am apologising for, and by the end of this you will see why I will not remove it.

losing count of your own batch

The sequel to the thirteen days was worse, because it was self-inflicted in a stupider way.

Once I understood the child-then-parent sequence, I built the children by hand, across two shell invocations. Then I read the confirmation output of the second invocation as though it were the whole set, counted fewer containers than I expected, concluded that a step had silently failed, and stalled for two days investigating a problem that did not exist. Every container had succeeded. I had thrown away half of my own evidence by splitting the work across two commands and then reading one of them.

The fix is not to be more careful. The fix is one atomic script that creates and verifies every child in a single pass and hard-asserts that the created count equals the expected count before it is allowed to continue. Multi-step API orchestration done by hand across several commands will eventually lie to you about its own state, and you will believe it.

the white frame you cannot see

The images are generated, and that brings a defect class I never had to think about with a photographer.

The generator bakes a thin white or black frame border into a minority of its outputs. Up to roughly forty pixels wide. At the size you review an image in a preview pane it is invisible. At full publishing resolution it is obvious, and after a square crop it can end up as a hard bright edge down one side of something that is supposed to read as a photograph.

No clever detection beats the blunt fix, which is to crop a few percent off every generated image before the final crop, unconditionally, whether or not it looks like it needs it. From the same session: a light-coloured logo lockup placed over a photo disappears completely against a pale region of that photo, so anything light sitting on an image gets a scrim behind it by default.

Both are the same lesson. Anything that reviews its output at preview size is reviewing a different thing than the one it publishes.

the judge that invents defects

The obvious answer is to put a vision model in front of it, score the render, and only look at what passes. I do that. It is useful, and on its own it is not trustworthy.

Vision judges failed on me in three distinct ways in real sessions. Some timed out repeatedly on small images. Some errored outright. And a fallback model hallucinated a different false defect on each retry of the same unchanged image: first a person's face that was not in the picture, then a multi-line headline that was not there. The cleanest case is a page that scored 40 with a fabricated horizontal-overflow defect from the fallback judge, then scored 100 with zero flags from the working judge, on the identical commit and the identical preview. That one blocked a client stack for a day.

So there is a control test now. Before I trust any automated visual verdict I feed the same judge a known-good, already approved render and confirm it passes. If it fails something I signed off last week, the judge is broken. And when every scorer disagrees with what I see looking at the render directly, that is scorer flakiness, and it gets reported as an infrastructure problem rather than faked into a pass or used to block a deploy that is fine.

the words are the more dangerous half

A bad image embarrasses you. Bad copy is a claim you published about someone else's product.

A claims sweep across the locale files found live comparison copy naming specific competitor products that do not exist. Invented product names, attributed to a real competitor brand. The same sweep found an unverifiable superiority claim about active ingredient concentration repeated across four languages with nothing on file behind it, and a bioavailability correction applied to two of the four locale files, leaving the other two still contradicting the brand's own fact-checked claims document.

That last pattern is the one I now assume is present everywhere. Corrections get applied per locale and drift. A grep count across a locale file will not catch it either, because a single JSON string value can hold an entire multi-sentence paragraph, so one hit in the count can hide five instances of the problem. What works is walking the parsed structure key by key and diffing the same key across every language. The cleanest locale is usually the already approved wording, and the others are the backlog.

Two more from that pass. The brand name had been machine-translated into a meta title, several page titles and one visible on-page heading, so a content pass now ends with a case-sensitive grep for the translated variant and a requirement of zero hits. And one locale file carried well over a hundred Unicode non-breaking hyphens plus dozens of narrow no-break spaces, visually identical to the ASCII characters, invisible to any normal search, one of them sitting inside a numeric phrase. Find and replace on non-English content has to search by codepoint, not by the character you can see.

what makes it survivable

Three properties, and none of them are about the model.

It has to fail loudly. The atomic script asserts its own count. Notification sends are awaited and return a message id or the exact error, instead of being fired into a deferred callback with the error swallowed into a console log. I learned that one expensively on another client, where a recurring lead health check stayed green for four days while two real form submissions from real people went completely unnotified, because the probe suppressed its own notification and the suppression logic caught the real traffic too. A green check that cannot fail is not a check. Related: never fan out concurrent calls from a cold, expired OAuth state. They all try to redeem the same single-use refresh token, one wins, the rest come back invalid_grant, a stale token gets written back, and the integration dies with a 401 that looks like an account problem. Warm one call first, then fan out.

It archives instead of deleting. Everything retired goes to a dated archive directory with a manifest naming the restore command. When I pulled four leftover Piexels logo files off my own site they went into an archive folder, not into nothing. Same instinct with git worktrees: move them with the tool that preserves their registration, so a stale checkout stays restorable instead of becoming an orphan nobody can put back.

And a person still says go. Not because the output is bad without me. Because horsetonic.eu carries a small credit in its footer, a heart and "by dlvx", linking back to my site. So does propickproperty.com. So does unrealgymthailand.com. That signature is the whole argument. When something I wrote publishes an invented competitor product in one of four languages, there is no vendor to escalate to. There is a footer with my name in it.