Building a blog syndication backfill

I’ve been building a blog-manager Rails app to track and syndicate posts across vdaluz.com, Medium, and LinkedIn. After getting the core Medium integration working, I hit a practical problem: 72 posts already existed on Medium but my local database knew nothing about them. Everything had medium_status: not_imported. Before writing any new publishing code, I needed to bring local state in sync.

The RSS dead end

My first attempt used the RSS feed at medium.com/feed/@vdaluz. The Medium::PublishedBackfill service polled it and matched posts by URL or title. It worked, but only returned 10 posts. Medium caps public RSS at 10 items. I had 72.

Going deeper with GraphQL

Medium has an undocumented GraphQL API at medium.com/_/graphql. Logged-in sessions can query latestPostsConnection(type: POST_TYPE_PUBLIC) paginated via cursor. I used a browser surface already logged into Medium to run authenticated queries and page through all 72 posts in three requests (25+25+22).

The response gave me each post’s id, title, firstPublishedAt, and previewImage.id; exactly what I needed for the backfill.

Title matching and Unicode gotchas

I matched posts to local records in two tiers: first by medium_url (posts already linked), then by normalized title. The second tier hit an unexpected problem: Medium stores apostrophes as Unicode curly quotes (U+2019) while my markdown frontmatter uses straight ASCII apostrophes (U+0027). Titles like “What’s Next” with a curly quote wouldn’t match “What’s Next” with a straight one.

Fix was a normalization function that converts curly quotes to straight before comparison. After that, 69 of 72 posts matched cleanly. The 3 misses were very old posts with substantially different titles between platforms.

The kamal stdin trap

I wrote a Rails runner script to do the updates and tried running it via bin/kamal app exec --reuse "bin/rails runner -" < script.rb. It exited 0, took about 6 seconds, and did absolutely nothing. The script ran but with empty stdin; kamal’s app exec doesn’t forward the calling shell’s stdin into the container.

The fix: scp the script to the host, docker cp it into the container, then docker exec it directly. Obvious in hindsight, annoying to discover.

Hero images on vdaluz.com

With hero_image_url populated for 69 posts (Medium CDN URLs), the next step was getting those images onto the vdaluz.com Astro site. I downloaded all 82 images (72 published + 14 scheduled, minus 4 overlaps), generated .webp variants, added heroImage frontmatter to the corresponding markdown files, and updated three Astro templates to render them: full-width banner on post pages, thumbnail on listing cards, thumbnail on related reading cards.

The heroImage field was already wired into OG meta tags. It just wasn’t rendering visually anywhere. That was a one-line observation that turned into a three-template update.

What’s in the DB now

After the backfill: 69 posts with medium_status: published, 14 with medium_status: scheduled (scraped from the Medium UI), 2 missed (old title mismatches). The DraftCreator service already reads hero_image_url when building Medium drafts, so new posts will include their hero images automatically once that column is populated.