How I Generate My RSS Feed

I only just now started supplying an RSS feed to you fine people! You can subscribe to it at www.senders.io/blog/feed.rss!

I decided rather than manually generating the file contents I’d hook into my pre-existing publish scripts to be able to generate the RSS file.

Publishing blog posts - shell scripts ftw

In My Markdown -> HTML Setup I touch on how I publish my markdown files into HTML for this blog. But what I don’t really touch on is the shell scripts that tie the whole process together.

What I have is two, now three, scripts that feed the whole process:

  1. publish-blog.sh - the main script
  2. compile-md.sh - generates the HTML output
  3. update-feed.sh - generates/appends the RSS feed

The update-feed.sh script is the new one I just added.

publish-blog.sh is the primary interface, I supply the date of the post and the path to the md file and that calls compile and update to automate the entire process.

Without going into TOO much detail you can view the latest versions of the scripts at git.senders.io/senders/senders-io/tree/.

But the gist of the scripts is I parse out the necessary details, find/replace some tokens in template files I have setup for headers and footers, and concat the outputs into the final output HTML files, and now RSS feed.

update-feed.sh

Source File: git.senders.io/senders/senders-io/tree/update-feed.sh

This script is pretty interesting. I didn’t want to deal with any XML parsers and libraries to just maintain a proper XML rss file and push items into the tree. Rather, I just follow a similar setup to my markdown generation. I leverage some temporary files to hold the contents, a static temp file for the previously generated content, and at the end swap the temp file with the real file.

I take in an input of the publish date (this is the date from the publish script), the title, and the HTML file path. These are all already variables in the publish script, but also something I can manually supply if I need to publish an older article, or something I wrote directly in HTML.

The core of the script is found here:

PUBDATE=$(date -d "$1" -R)
TITLE=$2
FILE_PATH=$3
PERMALINK=$(echo "${FILE_PATH}" | sed -e "s,${TKN_URL_STRIP},${URL_PREFIX},g")
LINK=$(echo "${PERMALINK}" | sed -e "s,${TKN_INDEX_STRIP},,g")

# Generate TMP FEED File Header

cat -s $FILE_RSS_HEADER > $FILE_TMP_FEED
sed -i -E "s/${TKN_BUILDDATE}/${BUILDDATE}/g" $FILE_TMP_FEED
sed -i -E "s/${TKN_PUBDATE}/${PUBDATE}/g" $FILE_TMP_FEED

# Generate TMP Item File

cat -s $FILE_RSS_ITEM_HEADER > $FILE_TMP_ITEM
sed -i -E "s~${TKN_TITLE}~${TITLE}~g" $FILE_TMP_ITEM
sed -i -E "s/${TKN_PUBDATE}/${PUBDATE}/g" $FILE_TMP_ITEM
sed -i -E "s,${TKN_PERMALINK},${PERMALINK},g" $FILE_TMP_ITEM
sed -i -E "s,${TKN_LINK},${LINK},g" $FILE_TMP_ITEM
sed -n "/<article>/,/<\/article>/p" $FILE_PATH >> $FILE_TMP_ITEM
cat -s $FILE_RSS_ITEM_FOOTER >> $FILE_TMP_ITEM

# Prepend Item to items list and overwrite items file w/ prepended item
## In order to "prepend" the item (so it's on top of the others)
## We need to concat the tmp item file with the existing list, then
## we can push the contents over the existing file
## We use cat -s to squeeze the blank lines
cat -s $FILE_ITEM_OUTPUT >> $FILE_TMP_ITEM
cat -s $FILE_TMP_ITEM > $FILE_ITEM_OUTPUT

# Push items to TMP FEED
cat -s $FILE_ITEM_OUTPUT >> $FILE_TMP_FEED

# Push RSS footer to TMP FEED
cat -s $FILE_RSS_FOOTER >> $FILE_TMP_FEED
echo $FILE_TMP_FEED

# Publish feed
cat -s $FILE_TMP_FEED > $FILE_RSS_OUTPUT

echo "Finished generating feed"

Some key takeaways are:

  1. sed lets you do regex with delimiters that AREN’T / so you can substitute something that shouldn’t actually ever show up in your regex. For me that is ~.
  2. I always forget you can use sed to extract between tokens - which is how I get the CDATA for the RSS: sed -n "/<article>/,/<\/article>/p"
  3. mktemp is really REALLY useful - and I feel is under utilized in shellscripting

The obvious cracks are:

  1. I rely SO much on sed that it’s almost certainly going to break
  2. I don’t have much other flag control to do partial generation - so if I need to do something either starting partway through or not finish the full process, I don’t have that.
  3. Sometimes things can break silently and it will go through, there is no verification or like manual checking along the way before publishing the feed.rss

The final two can easily be managed by writing the feed to a location that isn’t a temp file and I can manually do the cat -s $FILE_TMP_FEED > www/blog/feed.rss myself after I check it over.

But for now I’ll see if I ever have to redo it. I don’t think anyone will actually sub to this so I don’t really need to care that much if I amend the feed.

Where to put the feed URL

I never intended to provide an RSS feed. I doubt anyone but me reads this, and from my previous experience with gemini feed generation was a bit of a headache.

A quick aside: I really only decided thanks to Mastodon. I was thinking during the Twitter meltdown “what if twitter but RSS” (I know super unique idea). But basically like a true “microblog”. And some OSS tools to publish your blog. This got me reading the RSS spec and looking into it more - which then lead me down the using the RSS readers more (in conjunction with gemini, and Cortex podcast talking about using RSS more).

But I’ve decided to just put the RSS feed in the blog index, on my homepage, and that’s it. I don’t need it permanently in the header.

Conclusion

I didn’t have much to share here, it doesn’t make too much sense to write a big post on what can be explained better by just checking out the shell scripts in my git source. The code speaks better than I ever could.

I really, really like shell scripting.