Migrating from Substack to self-hosted Ghost: the details

Some have asked me how I feel about Substack's recent decision to ban five (5) no-name Nazi newsletters and then say "see? we're doing what you asked! And you screeching leftists are making a big fuss over nothing!" I gave a quote to the Washington Post which I will just repeat here: "It’s honestly insulting, both to writers and readers on the platform, that they think they can shut up those of us who have serious concerns with such a meager gesture." It's completely inadequate, particularly coupled with their promises that they won't be changing their policy, and that they won't be taking any sort of proactive approach to content moderation going forward.

I had a conversation with Substack co-founder Hamish McKenzie, prior to him apparently changing his mind about this handful of newsletters, that left me very skeptical that there will be meaningful change at Substack with him at the wheel — even if he is successfully pressured to make some token gestures. That skepticism only worsened with his extremely scummy behavior in leaking conversations with Platformer to what Platformer described as a "friendlier publication" — by which they mean "anti-woke" Michael Shellenberger's The Public — and in actively soliciting the supposedly organic anti-anti-Nazi letter^a of support and then canvassing supportive authors to sign it.

Happily, I've already migrated away from the platform, so I am no longer among the whole slew of writers who are either still deciding whether to leave, or who have decided to leave and are just trying to work out how.

For those of you in the former camp, I would argue that Substack's long history of horrendous decisionmaking about what kind of content they will not only tolerate but monetize, boost, and even actively court is perfectly sufficient reason alone. But should you need a few more reasons, here are a few benefits that I am personally enjoying at the moment as a result of pulling the lever attached to my ejector seat and spiraling off into the sunset.^b

Substack no longer takes 10% of my subscriber payments.

This is honestly pretty huge. When you're first starting out, other newsletter platforms that charge flat fees like $10 or $20 a month can seem like a daunting up-front investment when you don't know if anyone will care enough about what you have to say to pay for it. Then there's Substack, where you can sign up for free and not have to pay a cent. It's tempting! But as a newsletter grows, even somewhat small numbers of subscribers can make that 10% start to look pretty big. For example, if you manage to sign up 100 subscribers paying $10/month, you're already at $100/month going to Substack — far more than flat-fee platforms charge for similarly sized newsletters.

I'm no longer publishing on an a16z-backed platform.

As a writer who has been very critical of Andreessen Horowitz and of the venture capital model more generally, it made me uncomfortable to publish on a VC-backed platform whose Series A round was led by a16z.

I have full ownership of my writing and subscriber lists.

Substack likes to brag that writers retain full ownership over their writing and membership lists, and they do to a much greater extent than some other platforms. But they still keep a lot to themselves, making truly packing up and leaving a challenging experience (as I will describe in more detail later on). Now, if I wanted to, I could dump the MySQL tables and have every single scrap of information on my platform saved to my local machine. I can run scripts to update or download or do whatever to my material as necessary, without worrying I'm falling afoul of Substack's policies against running "processes" on the site.

I can accept payments any way I like.

Substack requires authors to agree that they will only accept subscription payments via their platform. As someone who wants to make it possible for subscribers to support in whatever way or amount feels reasonable to them, this meant I couldn't allow people to — say — make a one-time contribution in exchange for a subscription, or allow them to use a different payment processor.

I can make my newsletter look how I want.

Separate footnotes and references sections? Coming right up. A font outside of Substack's five options? No problem.

I can recommend anyone I want.

Substack has a cool feature where each Substack author can "recommend" any other newsletters they enjoy, via a little pop-up that shows to new subscribers and in various other places. The catch: those writers also have to be on Substack. It helps with Substack's network effects, but it also traps people in a walled garden. "Stay or lose Substack's network effects" is a huge bargaining chip in their advantage, and I've seen several writers (myself included!) reference it as a concern over leaving.

I could probably go on, but you get my gist.

Once I had decided to leave, I had to answer the question of where to go. There are many alternatives that I've already seen various former Substack authors choose. Among them are Ghost's "Ghost(Pro)" hosted option, beehiiv, Buttondown, and so on. All of them look lovely. Many third parties also sell concierge services to add functionality on top of these of platforms, or to help with migration.

Pricing-wise, and without any concierge services, I was looking at $165/month for Ghost Pro, $84/month for beehiiv, and $139/month for Buttondown for a newsletter my size.

But quickly I realized that what I wanted was not a new platform, but rather no platform. Although most platforms do a better job of content moderation than Substack, I feared that I could find myself in the same position down the road if my new host made a similarly unconscionable decision. I'm also a little bit of a control freak, and more generally, I like having as few opportunities as possible for would-be enshittifiers to mess around with what I'm trying to do.^c So, self-hosting seemed the obvious route to take, and once that decision was made, Ghost's open source blogging software was the obvious choice.^d

The process of migrating, unfortunately, is daunting. Every newsletter is different, every destination is different, and there is no unified how-to guide that covered what I was trying to do.

So, for the benefit of anyone out there who might be trying to do what I did — that is, migrate from Substack to a self-hosted instance of the open source Ghost blogging software — here is my version of the guide I wish I had.

For those of you who don't care about wonky and fairly technical details at great length, you may want to stop here. For the rest of you brave souls, read on:

Initial setup

Server

First things first, I needed a server for Ghost. I've had a VPS with DigitalOcean for... ten years now? that hosts my mollywhite.net website and a whole bunch of other bits and bobs of code, but I knew that this newsletter was going to have higher resource demands, and didn't want to risk bringing all my other stuff offline if the newsletter got too much traffic.

So, I spun up a separate droplet specifically for the newsletter. The other advantage of doing this is that DigitalOcean offers one-click "Create Ghost Droplet" functionality that does all the setup for you, including getting the database hooked up and serving the website.

I took a shot in the dark as far as how chunky of a droplet I would need, and picked the $14/month "Premium AMD" option (2GB RAM, 50GB storage, 2TB transfer). It's pretty easy to increase droplet size down the line, so this isn't a decision you really need to sweat too much.

Once the server is provisioned, you go through a CLI set-up process. Something went wrong the first time I did this, and MySQL wasn't configured properly. Rather than try to untangle where it had gotten stuck along the way, I just destroyed the droplet and started again. No problems the second time. shrug.

After that, there's some configuration to be done in the Ghost admin interface. This is all pretty straightforward stuff: setting the name of the newsletter, uploading the logo, etc.

Domain

Next, I pointed a domain at the site.

I opted to change my newsletter domain from newsletter.mollywhite.net to citationneeded.news as a part of this migration, for a number of reasons that I won't go into here. If you are planning to keep the same custom domain you were using at Substack, you'll want to do this step at the very end to avoid site downtime as you're getting things set up. Alternatively, if you don't mind people potentially getting lost in the interim, you can disconnect your custom domain from Substack and have your old Substack newsletter live at the default yourusername.substack.com domain for a little bit. Just make sure you go into Substack preferences and disconnect the domain there before updating your DNS — otherwise you can apparently get stuck in a weird state where you can't get to your Substack because they think you've still got a custom domain.

If you didn't already have a custom domain and were using the Substack one, you'll need to buy a domain — you can't take Substack's with you.

I'd already purchased the citationneeded.news domain for about $7 from Namecheap back in November when I renamed the newsletter, so I had that sitting around just redirecting to my Substack. I removed that redirect, moved the nameservers to Cloudflare,^e and set up DNS records in Cloudflare to point at the website (if you scroll down to the email section, those are entries 1 and 3 in the DNS records screenshot). I also turned off Cloudflare's caching and DNS proxying for the time being, because I knew I was going to be actively developing the site, and caching is a nightmare for that.

Email

Next up was email. In order to send bulk emails from Ghost, you need to use Mailgun. Mailgun is actually the priciest part of my setup, at $75/month for their "Foundation 100k" plan, which will allow me to send 100,000 emails a month. After 100,000, they charge $0.001 per message. This sounds like a lot of emails, but with more than 20,000 subscribers and somewhere around five newsletters a month, I'm likely going to have to pay a bit more (but probably not a lot more) than that flat $75 — particularly because Mailgun is also used to send the magic links for sign up and sign in, and the transactional emails to welcome new subscribers or remind paid subscribers of upcoming renewals. (Update: I ultimately paid $0.37 in overage fees for the 100,366 emails I sent in January).

Hooking up Mailgun to Ghost is pretty straightforward: you just snag an API key from Mailgun and plug it into the Ghost admin interface, et voilà.

Less straightforward is doing all the configuration to reduce the chances of email providers seeing someone suddenly sending 20,000 emails all at once from a previously unused domain and going "SPAMMER!" From my research, it seems like there's some stuff you can do up front, and some of it is just a waiting game as these providers build up a reputation score for your domain/IP address and classify you from there.

I looked into setting up a dedicated IP for my mailsending, because I heard that maybe that was the thing to do. The reasoning there is that with a shared IP, you run the risk of some other Mailgun customer actually being a spammer, and then your reputation gets bogged down by theirs. However, after a conversation with Mailgun support (who are incredibly lovely and responsive, by the way), I ascertained that this actually might be a bad idea, especially as I'm just getting started:

Dedicated IP’s require consistent sending over time and if the sending isn’t consistent it may damage the sending reputation of your dedicated IP address, placing the IP on blacklists and throttling your messages.

So, shared IP it is (at least for now). With that out of the way, I set up more DNS records on the citationneeded.news domain. So many. There's the record to verify with Mailgun that you control the domain, then there's the SPF record, then there's the DMARC record and the DKIM record, then there are the MX records, and I think you're supposed to put a CNAME record in there for some reason too? I don't know, I just do what I'm told. And someone's just told me about a BIMI record, which apparently makes emails look trustworthy, so I threw one of them in there for good measure. Why not.

Learn from my mistakes, though: I failed to set up a) the MX records pointing at Mailgun, and b) the DMARC record before sending my first email. This is likely why so many of the first emails bounced.

I had seen an instruction in a Mailgun setup document that said "MX records should also be added, unless you already have MX records for your domain pointed at another email service provider (e.g. Gmail)." I had MX records set up on this domain that point at Cloudflare, which forwards any emails sent to @citationneeded.news over to my email, so I thought, "well, I wouldn't want to mess anything up by having conflicting records." This seems to have been wrong! After a whole bunch of emails from the first newsletter send bounced with the error message "Sender address rejected: domain not found", I discovered that the Mailgun MX record seems to be necessary for email providers to verify the sender. Or maybe it was because there was no MX record specifically for the mg. subdomain that Mailgun uses? Either way, I added it (number 5 below), and stopped seeing those errors.

I also added the DMARC entry (#9), which is super important to a) prevent other people from trying to spoof emails as though they are coming from citationneeded.news, and b) help email service providers trust that my emails aren't spoofed and, thus, spammy. That should've been there from the start, and would've prevented a lot of other bounces.

Altogether, I added entries 2, 5, 6, and 8–11 to my DNS records. At a minimum, you will need to add entries 9, 10, 11, and apparently 5 to get your emails to reliably land where they're supposed to.

On the very optional end of things: Entry #2, the CNAME record, is used by Mailgun to track things like email opens and clicks. I've kept it on for now just to help me with debugging email stuff, but will turn it off soon, because I've always found email open/clicktracking to be kind of invasive and not super useful to me. Entry #6, the Google Postmaster one, is not necessary to make the emails work, but lets me see a little more information about what Google thinks is spammy through their Postmaster tool.

Some other tools I found useful:

• DMARCian's record wizard helps you create DMARC records
• DMARCian's XML to human converter helps you decipher any DMARC reports you receive, if you choose to have them come straight to you
• Postmark has a free DMARC monitoring tool to help you triage the bombardment of DMARC reports you will eventually receive when you realize having them come straight to you is probably a mistake

Entries 4 and 7 were already in place from when I set up email forwarding through Cloudflare. Tons of different services can do this for you for free, and you can also have Mailgun do it for you since you're already using them. I just use Cloudflare because I'm already using it for DNS and caching. You'll probably want to set something up so that you can receive emails sent to your newsletter's domain — otherwise if people reply to your newsletter email, they'll just be shouting into the void. Alternatively, you could set your personal email in the Reply-To header on your newsletter email sends, but having not done this myself I can't speak to how effective it is in catching all stray emails.

Transactional emails

One thing Substack does for you out of the box that Ghost does not do is send transactional emails — that is, pretty much everything besides the bulk newsletter sends that go out to everyone all at once. Think "thank you for subscribing!" or "your subscription is about to renew!".

Ghost does send a very brief message to new subscribers, but it's mostly just to double-confirm their subscription, and can't be customized to add any additional information:

If you want these transactional emails, and you probably do, you'll need to set them up yourself. For those using Ghost Pro and not wanting to write any code, I'm not sure there's any easy solution to this besides using a service like Zapier, or working with a group like Outpost, who sell add-ons on top of Ghost's hosted services.

I briefly experimented with using Hookdeck, a Zapier-like project with a more generous free plan, since all I was really trying to do was glue a Ghost or Stripe webhook to Mailgun's API, but ended up being unsuccessful in configuring the webhook authentication. Womp womp.

So, instead, I coded up a basic Express server, which I stuck alongside the Ghost instance on my new DigitalOcean droplet. It's really simple — it's got one endpoint to listen for the Ghost webhook event when a new user signs up, and one that listens for Stripe's invoice.upcoming event (which signifies that a customer is about to be resubscribed). In the first case, the server makes a call to the Mailgun API to fire off a simple welcome email that's a little friendlier and informative than Ghost's automatic one. For upcoming renewals, I filter to only annual subscriptions (so monthly subscribers aren't getting this email every month). That email is a little more complex — it includes some custom variables to inform people how much they're about to be charged and when — but is largely the same process.

Over in Mailgun, I used their WYSIWYG template editor to create the templates:

I may also add an email that lets people know once their subscription has actually been renewed, since no one likes surprise credit card charges, but I'm trying to balance being communicative with not bombarding subscribers' inboxes. Same goes for people whose subscriptions are about to expire without renewal, to nudge to see if they might want to resubscribe.

One note: I had originally intended to have two different welcome emails: one for paid subscribers, and one for free subscribers. This is what I had over at Substack. I was planning to just listen for the Ghost member.added event, and then send the appropriate email based on whether the member object had status: free or status: paid. Unfortunately, I discovered that when someone signs up for a paid subscription, they are first added as a free member (triggering the added event with the free status), and then updated to a paid member once the Stripe flow is completed. I couldn't think of a great way to get around this without first sending an erroneous "free" welcome email to paid subscribers, or making the webhook server stateful so it could wait a little bit to see if the update came in before sending the email. So, for now, everyone gets the same email and I just direct people to a customized welcome page with more detail after they've completed their signup.

Import

Next up: getting all of my posts and subscribers from Substack into Ghost. Substack likes to boast about how you own your own content, and can easily migrate away from Substack if you ever want to. Unfortunately, their tooling for actually doing so leaves much to be desired.

You can download an export of all of your Substack posts, and Ghost has helpfully built in a handy little wizard (in Ghost settings > Migration tools) that will help you import both your content and your subscribers. However, I would not actually recommend using it, at least not without doing some extra work first. That said, any content import or subscriber imports that you do can be relatively easily deleted and redone, so it's safe to experiment and then erase your imports later on.

Content import

If you use Substack's content export as-is, you will lose a lot and your posts will be broken. For example, it does not actually export any images or other media embedded throughout your posts, so you end up with an HTML file that hotlinks to a ton of media still hosted on Substack, which will all 404 as soon as you take your Substack site offline. Links to other Substack posts will break, "subscribe" and "share" buttons will all still point to Substack, etc. etc. Not great.

Fortunately, Ghost has come to the rescue here with their migrate tool. It's just a bunch of Node scripts, and they are easily installed with npm or yarn. They've actually got a whole slew of them if you're trying to migrate to Ghost from a whole bunch of different platforms, but you'll be wanting the Substack (mg-substack) migration tool.

I found the documentation to be a little sparse, so to answer some questions I had: this tool is intended to be run on the ZIP file of your post content that you've exported from Substack. You don't just run the script standalone to do the exporting for you, it's meant to augment the ZIP export with everything it's missing. So, when it asks for a --pathToZip, that's the path to the Substack export file. The --url argument is the URL to your old Substack site (not your new Ghost URL, if they're different), which it will scrape.

It will do things like scrape all of the media assets from your Substack, prepare them for upload them to your Ghost site, and modify references throughout your posts to use the new paths. It will also update links you might have made within your newsletter to your past posts so they use the new URLs rather than Substack's URLs, and it can update your "subscribe" buttons and the like to go to the right place. It can even import drafts, Substack pages, and Substack threads if you want them.

The tool is quite good as-is, however there were a few other things I wanted it to do.

For one, it doesn't scrape and reupload audio voiceovers of posts, a feature I've started using over the past few months. I tweaked the script so it would (see my Github branch if you need to do the same). That's probably the only change I made that's likely to be broadly useful, but I also made the following tweaks:

• Added a script to detect when I was writing an explanatory footnote vs. citing a source, and bucket those into separate "footnotes" and "references" sections
• I'd been using an image (the blue * * *) as a separator for a while in Substack because I didn't like their default <hr> styling, so I added some functionality to the migration script to avoid downloading that one same separator image a bazillion times. Instead, I updated the script to properly replace those separator images with the <hr> tag, which I've styled how I like it with the image using CSS — much more a11y-friendly!
• Similarly, I'd been using blockquotes to style the "In the news" and "Worth a read" sections of my newsletter. This is not very a11y-friendly, since semantically they are not quoted content, so I updated the script to replace those instances with some custom HTML that looks largely identical but uses more appropriate HTML tags.
• I discovered once I switched to a font with noticeable directionality in its curly quotes that a lot of my quotation marks were pointing the wrong way. This was the case in Substack also, but because of the font over there I never noticed. I updated the script to replace all curly quotes with straight quotes, because I'll be damned if I'm going to go through and try to fix all that. If you want to do this too, just add the following to the end of the processContent function in mg-substack/process.js:
  html = html.replace(/[“”]/g, '"');
html = html.replace(/[‘’]/g, '\'');

With all this done, I ran the script, imported the content (Ghost settings > Migration tools > Universal import), and did some spot checking to make sure everything worked as designed. I was pretty pleased!

This took me a few tries, so I had to clear out all of the content and reupload a few times. If you have a lot of posts, I would recommend making a smaller test version of the import ZIP to use — otherwise you may have to wait several minutes for each import to complete. If you did an import and want to clear out all content from your Ghost blog, you can easily do so via Ghost settings > Danger zone > Delete all content. Note that this does include all posts and pages, but does not remove member data.

The only thing I missed was that the Ghost migration script apparently does not detect and update links to past posts when you've linked to a subsection of that post (which I do frequently). These are links that look like https://[newsletter-url]/i/[numerical-post-id]/[section-name] (for example, https://newsletter.mollywhite.net/i/94921341/but-your-honor-they-were-mean-to-me). A subscriber kindly informed me that those links are broken. I will be writing a script to go through all my posts and fix that issue soon.

User import

So, good news and bad news. The good news is that you can migrate your Substack subscribers (paid and free) from Substack to another service. Substack uses Stripe for payment processing, and so as long as you continue to use Stripe (used by Ghost, Buttondown, beehiiv, etc.) you can swap out the newsletter platform without any interruption to them. This is huge — as someone who migrated from Patreon way back when and had to ask everyone in my (much smaller) subscriber base to manually re-subscribe, that's a nightmare.

The bad news: as with content export, Substack does not make user export all that easy on you, despite their marketing. They do allow you to export your users in CSV files from your Substack membership management interface, but I discovered that these CSV files are formatted slightly differently than the ones Substack spits out if you use Ghost's Substack migrator tool. Critically, the ones Substack spits out from its membership interface are missing the Stripe IDs of paid subscribers, which are crucial to keeping subscriptions linked as you migrate! But the export from the members interface includes a lot of additional data I need, too, like the subset of my newsletters^f a member person has opted to receive. So, I wrote a script to mash all four exports together (paid member portal export, paid members export with the Stripe data, free member portal export, free members export with whatever other data is included from that mysterious exporter). It also deals with the fact that some of the data from Substack is just... wrong. For example, the column that purports to reflect if a member has turned off emails never shows that they have, even if they've unsubscribed from all of my newsletters.

Once I mashed that all together, I imported the users into Ghost from the CSV.

If you do this and something's wrong, and you want to start over, it is possible to bulk delete all Ghost members, even though it's not immediately apparent. You have to go to your members page and then "filter" your members list with a criterion that will match everyone on your list — you can use Label is "Import [date]", which is added automatically by the Ghost migration tool — and then click the gear icon and choose "Delete selected members".

Everything was mostly good to go at that point, except that for some reason, although Ghost does support having multiple sub-newsletters per site, and allows people to subscribe or unsubscribe from them, you can't import members' subscription preferences. This would mean that all members would receive both the Citation Needed posts and the Weekly Recap posts, even if they'd previously unsubscribed from one or the other. I whacked together a quick script to read the CSV export from Substack, which contained those preferences, and update members's subscriptions after the fact. I would share it with you on Github, except I seem to have lost it. If anyone needs this, let me know — I can probably whack it back together again pretty quickly.

At this point, everything worked great membership-wise, except... oh no.

Fixing Stripe

Every single paid subscriber appeared to have the same subscription plan, which is just whatever the first tier you add to Ghost happens to be. As you may know, I've been offering pay-what-you-want subscriptions for a while, and on Substack I accomplished this by creating a $10/month or $100/year subscription, and then nine discount links for 90% off, 80% off, 70% off, and so on. Way back when, before my pay-what-you-want hack, I'd also done a couple of one-off discounts, so there were a handful of folks with those, too. And in the very beginning, the default subscription was $5/month or $50/year. All of this... does not migrate well.

Although the subscriptions themselves didn't change, all of my paid subscribers were added to the same Ghost subscriber tier, which would make it look in their Ghost member settings as though they had the $10/month or $100/year plan. I knew this would probably spark panic in some subscribers who had signed up to pay considerably less, and it's just a terrible experience for subscribers to have to assure them "no no, even though it looks like I've totally fleeced you, I haven't, I promise!" I could've edited the tier name and price to look different, or have a huge disclaimer on it like "LOOK AT THE ABOUT PAGE WHERE I EXPLAIN WHY THIS NUMBER IS WRONG", but some people always miss these things, and it was better to just fix it.

So, first I set up my ten subscription tiers (plus the founding member tier) in Ghost. Then I did my best to whip up a script using Stripe's API that would go through each of my paid subscribers, check which tier they should be in (for example, a 40% discount off the $10/$100 subscription should have the discount removed and be reassigned to the $6/$60 tier), remove the discount as needed, and update the subscription without prorating. Altogether, this meant that nothing changes on the subscriber's end in terms of payment amount, frequency, or renewal date, but the subscriptions are technically set up a little bit differently under the hood.

This was all a little terrifying, because changing a subscription in Stripe runs the real risk that I could screw up and overcharge someone — the worst possible outcome. I tested the hell out of my script, and added a bunch of bailout cases where the script would skip trying to update a subscription if anything about it was unusual.

Unfortunately, there are a ton of things that can make subscriptions unusual. The biggest one I ran into was non-US currencies. You might think it would be as easy as looking up the current USD conversion rate for their currency, but this fluctuates over time, so one Brit who signed up recently for the $10/month subscription might pay £8/month, whereas another who signed up longer ago might pay £9/month. Ultimately, I care more about people paying what they expect to pay than I do receiving exactly the USD amount reflected in their tier, so I went through all non-US currencies to manually migrate them and reflect the original payment amounts and currencies involved. This took several hours, and was perhaps the only time I've been grateful that I don't have more people paying me.

Ghost theme tweaks

One thing I love about Ghost is you can pretty easily tweak how most things look without also having to dive into the full nuts and bolts of the Ghost mono-repository. I made some changes to the Ghost theme — some cosmetic, and some more to do with the site functionality.

In the cosmetic department, I added the full-width banner on the homepage with the illustration of me on the laptop, to either encourage people to subscribe or thank them for doing so. I also added some additional links to the site footer, removed the "feature image" from automatically displaying at the top of each post (since I often include these just for social sharing purposes), and swapped out the default fonts. Nothing too exciting there (unless you get excited about fonts like I do, I suppose).

The more substantial changes were to the signup flow, and to the welcome page.

For the welcome page, I created a custom page in Ghost by modifying the routes.yaml file (Ghost Settings > Labs > Beta features > Routes) and creating a welcome.hbs file in the theme. This takes advantage of Ghost's @member.paid flag to show different content based on whether the member is paid or free.

The signup page is also custom, with some custom JavaScript, and is set up with the same routes file. This wasn't strictly necessary, but with 12 signup options (the ten pay-what-you-want tiers, plus free and founding members), the automatic page looked pretty overwhelming.

Instead, people now choose to sign up from one of four groups, and then can pick their pay-what-you-want tier from a dropdown:

Unfortunately, I will need to dig into the Ghost mono-repo to replace the "change plan" page, which looks much like the dizzying default sign-up screenshot above. I hope to do so soon, but didn't want to delay migrating any longer with something that's mostly cosmetic. The same seems to be true if you want to make any changes to Ghost's email styling.

Leaving Substack

With my content migrated over and my subscribers in place, it was finally time to leave Substack.

First, I needed to ask Substack to disconnect themselves from my Stripe account. This is critical: if you don't do this, subscriptions created through Substack will continue to send a 10% cut to Substack, even after you've left the platform!

I emailed Substack's support team:

Hello,
I am migrating my newsletter (https://newsletter.mollywhite.net/) off of Substack due to the company's recent refusal to deplatform or demonetize Nazi content.
Can you please disable the Substack's fee on my Stripe account? Thank you,
Molly White

One minute later, on Christmas Eve no less, I received a reply. How responsive!

Hi Molly,

I understand you want to make changes to your newsletter's financial setup. However, we cannot disable the fees that are automatically taken as part of the transaction process on Substack. These fees are applied to support the services provided by the platform and are not optional.

If you're looking to disconnect your Stripe account from Substack, which will cancel and refund all paid subscriptions to your newsletter, you can do so by following these steps:
...

[record scratch]

After a brief moment of panic, my soul returned to my body and I remembered how widely Substack had advertised that all writers retain control over their subscriber lists and can take them with them if they leave.

I responded:

No, I intend to migrate my paid subscribers from Substack (as is a selling point of the platform: "A Substack is the writer’s property: the email list, content, and payment relationships (should you choose to monetize) is the writer’s and the writer can take all of it with them if they ever decided to leave the platform.") I don't wish to cancel their paid subscriptions, I just wish to remove the Substack fee as I will no longer be using the platform. See: https://ghost.org/docs/migration/substack/#removing-substack-fees

I then proceeded to spend five days in terrifying limbo as I awaited a reply (which was, in fairness, over the holidays).

Finally, a different support representative replied that they had disconnected my Stripe account from my Substack account and removed Substack's cut of my subscription fees. Phew.

I would like to assume that the original email was just a newbie support representative either misunderstanding my request or not realizing that Substack does indeed allow people to do what I was asking. However, after seeing Substack's incredibly bad faith over the last few weeks, a part of me wonders if this is standard operating procedure when people make these requests, as a way of discouraging people from leaving the platform. Perhaps that's bad faith of me to wonder, I don't know. Either way, you should know that it's possible that you will get this kind of reply, and to just be persistent if you do.

Whatever you do, do not disconnect Stripe in the way they instructed in their first reply, because all paying subscribers will be unsubscribed, and I'm pretty sure there's no undoing that. (Update: I have been informed by someone more knowledgeable than I about Stripe that this is actually undoable, so you don't need to worry as much as I originally thought!)

Since migrating, I've learned that there are ways to turn off Substack's connection to your Stripe account without involving Substack support. I haven't done it myself, but the folks over at Buttondown were kind enough to provide the how-to.

Unfortunately, when Substack disconnected my Stripe account, all my subscribers turned into free subscribers. This is obvious in hindsight, but I didn't think about it until it happened. Because of this, I could no longer do the paid subscriber export that I described above, and so any paid subscribers who signed up between when I did the first export and when Substack disconnected my account would be lost in the migration.

Fortunately, there were only a handful of people who signed up for paid subscriptions during this period, so I was able to just do the free subscriber export again, pull in the paid subscriber information from Stripe manually, and run another import to fill in the missing pieces. I think made my Substack account "private", which prevented anyone new from accidentally signing up over there.

The one thing I did lose was the list of people who had unsubscribed shortly before I migrated, many of whom cited Substack's policy regarding Nazis in their messages. I had wanted to send any of these folks who had completely removed themselves from my subscriber list an email to let them know I had moved. I was able to get this information from Substack after the fact by making another request to their support team, but it took several days.

So, word of advice: as close as you possibly can to when Substack disconnects your Stripe account, make sure to export your paid subscriber list and, if you're going to want it, your unsubscribes list.

Another thing I hadn't thought about: Substack has a number of emails they automatically send for you that you may not even know about, including one that reminds people when their subscriptions are about to renew. When Substack disconnected my Stripe account, these emails stopped being sent, because as far as Substack was concerned I didn't have any paid subscribers. It wasn't until a week or so later, when I went to hook up transactional emails that I realized oh no, those emails used to be getting sent, and were no longer. As a result, there were some people who were resubscribed without getting advance notice — something I feel terrible about. I've sent an email to that group of folks and offered to refund the subscriptions in case this was not something they wanted.

To avoid falling into this same trap, you can either make sure your transactional emails are set up before you disconnect Stripe, or toggle the setting in Stripe (Settings > Subscriptions and emails > Prevent failed payments) to "Send emails about upcoming renewals". The email will look a little different, but your subscribers will at least get the warning!

Redirecting old links

Although I had done my best to update any links to this newsletter within my own content, I knew there were links to newsletter.mollywhite.net elsewhere on the web, in readers' bookmark folders, etc. that would break once I officially moved.

To avoid this, I had to redirect all newsletter.mollywhite.net traffic to citationneeded.news. I also had to take into account that Substack inserts a /p/ into its post links, whereas Ghost does not. For example, https://newsletter.mollywhite.net/p/substackers-against-nazis becomes https://citationneeded.news/substackers-against-nazis/.

After some fiddling in my NGINX configuration on my other server (the one that hosts mollywhite.net), I came up with this server block:

server {
  server_name newsletter.mollywhite.net;
  location ~ ^/p/(?<path>.*)$ {
    return 301 https://citationneeded.news/$path;
  }
  location / {
    return 301 https://citationneeded.news$request_uri;
  }
}

Is it the most elegant way of doing it? I don't know! Does it work? Sure does.

Regarding RSS: Substack keeps RSS feeds at /feed. Ghost's are at /rss. However, Ghost automatically redirects /feed to /rss, which means that anyone who subscribed to newsletter.mollywhite.net/feed should receive the citationneeded.news/rss feed in their feedreader with no updates needed on their end.

Last-minute configuration

At this point, everything was just about ready to go. This meant it was time to prepare the server to actually "go live", and begin handling some real traffic.

First I re-enabled Cloudflare caching. For now, this is a pretty standard setup, except that I've added a caching bypass rule for any pages that match the admin panel route (citationneeded.news/ghost), which are only used by me and shouldn't be cached. I'll be tweaking it as I go to help handle traffic spikes a little more smoothly.

I also ran mysql_secure_installation on my Ghost server to harden up the MySQL server. This does a couple of things to make it harder for people to compromise your SQL database (which is initially set up in a development mode), like disabling remote login as root.

Finally, I configured ufw, Ubuntu's firewall tool.

The first email send

Finally, it was time to announce that I had moved.

Then, I penned my announcement post in the Ghost editor (which I already love), said a small prayer, and hit the big red button:

I went to my email inbox, refreshed a few times, and... nothing happened.

Uh oh.

I waited a little longer, checked my social media to see if anyone had said "hey cool, congrats on moving!" No dice. I braced myself, opened the Ghost error logs, and saw that the send had failed thanks to an authentication error with Mailgun. Turns out I'd put in my old domain name when setting up the Mailgun connection. Dammit.

The good news is, Ghost automatically retries failed sends, so once I fixed the connection, it automatically tried again and we were off to the races.

Except, a huge percentage of the emails failed to land in peoples' inboxes. And either the load from the email send or from the traffic to the site from people who received the emails caused that dinky little $14/month server to hang. This was worsened by the fact that I had shared the announcement to my various social medias, which not only increased traffic to the server, but also did what's been variously called a "MastoDDoS" or a "Mastodon stampede". I won't go into it here, but it's an unfortunate quirk of Mastodon's federated design.

Nothing like having a reader clicking a link in your email triumphantly announcing your new website only to see...

Womp, wooomp.

Since the site was down already, I took the opportunity to beef up the server a little bit (which requires taking the server offline). I saw from the server logs that the RAM was maxing out, so I doubled the memory to 4GB with 2 vCPUs with a $28/month droplet (which is otherwise identical in terms of storage and transfer).

Once it came back up, it handled the load okay, although I wasn't sure if this was because the load had diminished. More on that in a sec.

After this, I saw in my Mailgun logs that emails weren't being delivered. This was super disappointing — people weren't receiving the newsletter, and people were also having trouble receiving their magic links after signing up or signing in. This turned out to be due to (I think) a combination of the missing MX records and DMARC record.

Another thing that can impact deliverability is bogus email addresses on your list. Apparently if you try to send an email blast but a lot of the emails are undeliverable because there's no valid inbox at the other end, it can really harm your reputation. After sending my first newsletter, I ran my email list through Mailgun's Validator tool to weed out bad addresses. I was pretty conservative about this, since it seemed to flag some legitimate emails as high-risk (mostly custom domains), but I ultimately turned off email sending for around 100 or so members (out of 21,000, not bad!) where the email had no hope at all of getting through to someone. I had Substack's double opt-in turned on for the vast majority of the time I was on that platform, which probably went a long way to help this. Only three of the invalid emails were paid members, who I'll be reaching out to — for all three of them, there were very similar email addresses in my member list, so I suspect they just typoed when signing up for their paid subscription.

Finally, I tweaked some settings so that the newsletter is sent from newsletter@citationneeded.news, but one-off emails come from support@citationneeded.news. I'm hoping that this means that email providers won't throttle, say, the sign-in links even if they throttle newsletter emails. It may be that the throttling applies to the whole domain, but it's worth a shot (plus it helps people filter things a little better).

Now that I've resolved that handful of issues, things are looking way better. There are still some temporary failures from email providers who think I might be a spammer, which I think is mostly a waiting game as my sending reputation improves. I'm also encountering a few, but not many, one-off issues that I'm debugging with Mailgun support.

My second attempt at sending email went much better than the first, and the vast majority of emails went through. I also experimented with sending the email, waiting a little bit, and then posting the link to Mastodon after the initial traffic surge went back down. Unfortunately, the initial traffic surge and the MastoDDoS were both enough to cause the server to hang for a few minutes each, so I had some more tweaking to do with caching and so forth. However, between email sends, the new server is motoring along at about 10% CPU and 50% memory, so hopefully it's sized okay and I can handle the rest with configuration tweaks.

The last email from Substack

I also sent one last email from Substack to inform subscribers that I had moved. This was particularly necessary in my case, because a lot of people had not received the announcement email due to the email configuration issues I mentioned.

One word of advice: if you do this, and if you have custom email headers configured in Substack depending on if someone's a free or paid subscriber, turn those off first!

I forgot to do so, and so everyone got an email saying at the top that they were a free subscriber, because as far as Substack is concerned at this point, they are. This caused a few paid subscribers to (reasonably) believe that their subscription had not migrated properly.

After this email was sent, I deleted my Substack account. This felt nice.

The end result

After all of this, I have found myself with a roughly $103/mo setup: $28/mo for the VPS, $75/mo (plus overage tbd) for Mailgun. This is considerably cheaper than staying on Substack, and also slightly cheaper than both hosted Ghost and Buttondown.

However, more important to me than the exact price is the degree of control I have over my own not-a-platform, where I can guarantee my newsletter is not sitting on a server alongside a be-swastikaed hate screed.

Even with a few remaining hiccups to work out, this antifascist living room is feeling pretty nice.

One final note

I realize that this is... a lot. If you are a newsletter writer looking to flee the Substack ship, please don't let this discourage you. I'm already seeing other newsletters like The Sword and the Sandwich, Today in Tabs, and Disconnect moving to various other services (Buttondown, beehiiv, and Ghost Pro, respectively) at least relatively seamlessly, and hopefully with a lot less custom work than I did.

I chose the somewhat more laborious route of self-hosting, but it is far from the only route! And if you're not a technical person but you want to self-host, there are a ton of really capable people out there who you can hire to help you get something up and running.