Content Migration From Cockpit SQLite to Markdown

Navigating the world of blogging solo means constantly juggling between creating cool content and figuring out the tech stuff without breaking the ban...

Ditulis Oleh zidan Pada 05 Feb 2024

Navigating the world of blogging solo means constantly juggling between creating cool content and figuring out the tech stuff without breaking the bank. So here's the scoop on why I decided to ditch Cockpit CMS and take a different route. Don't get me wrong, Cockpit CMS was a pretty sweet deal for a while – it was like that sleek, shiny tool that did everything I needed. But as time went by, it started feeling like I was trying to pilot a spaceship just to write and share my thoughts online. Overkill much? Definitely for a one-person show like mine.

The main drag? Keeping up with the whole backend shebang just for blogging felt like wearing a suit to a casual brunch. Plus, the costs – oh, the costs – of hosting and managing this setup were creeping up on me. That's when I knew I needed something simpler, something that felt more like wearing my favorite hoodie – comfortable, straightforward, and, most importantly, cost-effective.

Enter Markdown. It's like the best buddy of content creation – no frills, just you and your words, formatted in a way that doesn't need an engineering degree to understand. The idea of shifting my blog to be powered by Markdown files sitting happily in a GitHub repo felt like hitting the jackpot. It meant I could write, edit, and manage my posts with the simplicity of Git, and even host them using GitHub Pages. Talk about a win-win! This was exactly the kind of setup I needed to keep my blogging vibe alive without the tech overhead.

In this post, I'm diving into the nitty-gritty of how I moved my blog from the clutches of Cockpit CMS into the welcoming arms of Markdown. I'll walk you through the bumps, the eureka moments, and everything in between. This isn't just about cutting costs or simplifying tech – it's about reclaiming the joy of blogging in a way that feels right for me.

Background

Back in the day, when I was scouting for the perfect setup to kickstart my blogging journey, Cockpit CMS seemed like the cool kid on the block. It was all about being lean and mean, without needing a beast of a server to run it. This was a big deal, especially around my neck of the woods in Indonesia, where you could find PHP hosting just about anywhere but Node.js was like searching for a needle in a haystack. And let's not forget, Cockpit made it super easy to tinker with and had this neat feature that let multiple authors chip in – not that I needed it, being the lone wolf writer of my blog.

Fast forward to me actually running my blog solo, and suddenly, the charm of having a CMS felt a bit overkill. I mean, why go through all the hoops of managing a backend, dealing with updates, and all that jazz, when it's just me, my thoughts, and my keyboard? Plus, even though the costs weren't astronomical, why spend extra on something that's supposed to be my creative outlet, right?

That's when I had my "aha" moment with Markdown. Picture this: a super sleek way to write and format your posts without any fuss, all while being easy on the wallet since it's practically free to host on GitHub. It was like finding out your favorite late-night snack spot delivers.

But here's the kicker – getting my content out of Cockpit wasn't as easy as packing up and leaving. The whole setup was sitting on an SQLite database, which is awesome for keeping things light but not so great when you're trying to pull your stuff out for a big move. So, I cooked up a little plan: dump everything into CSV files, do a bit of magic to turn those into JSON, and then, the grand finale, transform all that JSON into sweet, sweet Markdown.

Why the roundabout way, you ask? Well, diving into CSVs with Excel gave me the bird's-eye view I needed to cherry-pick what gets to come along to my new Markdown home. Then, shuffling everything into JSON was like getting my ducks in a row – it made sifting through the data a breeze and set me up perfectly for the final leap into Markdown land.

This whole adventure wasn't just about ditching the extra weight of a CMS. It was about getting back to the basics, focusing on what I love (writing), and keeping things as simple and straightforward as possible. Cockpit CMS was a blast while it lasted, but it's Markdown's time to shine now, paving the way for a more hands-on, no-frills approach to blogging that's all about the content and nothing else.

The Journey

Step 1: Exporting SQLite to CSV

Objective: Extract data from the Cockpit CMS SQLite database and save it as CSV files for a manageable review process.

Process:

I leveraged Node.js and SQLite3 to interface directly with the SQLite database, executing queries to fetch the data table-by-table. Each table's contents were then exported as CSV files, which allowed for an easier visual inspection and manipulation using tools like Excel. This step was crucial for identifying which pieces of content were essential for the migration.

Code Snippet:

const sqlite3 = require('sqlite3').verbose();
const fs = require('fs');
const { stringify } = require('csv-stringify');

let db = new sqlite3.Database('./myblog.sqlite', (err) => {
    if (err) {
        console.error(err.message);
    }
    console.log('Connected to the SQLite database.');
});

db.serialize(() => {
    db.each("SELECT name FROM sqlite_master WHERE type='table'", (err, table) => {
        console.log(`Exporting table: ${table.name}`);
        db.all(`SELECT * FROM ${table.name}`, (err, rows) => {
            if (err) throw err;
            stringify(rows, { header: true }, (err, output) => {
                fs.writeFileSync(`${table.name}.csv`, output);
            });
        });
    });
});

Step 2: Converting CSV to JSON

Objective: Transition from CSV to JSON format to leverage the structured and versatile nature of JSON for data manipulation.

Process:

After exporting to CSV, I used the csv-parse library in Node.js to convert these files into JSON. This step made the data easier to handle programmatically, setting the stage for the final conversion to Markdown. It provided a more structured view of the data, which was especially beneficial for complex content structures.

Code Snippet:


const fs = require('fs');
const parse = require('csv-parse/sync');

const csvToJson = (csvFilePath) => {
    const csvData = fs.readFileSync(csvFilePath);
    const records = parse(csvData, {
        columns: true,
        skip_empty_lines: true,
    });
    return JSON.stringify(records, null, 2);
};

// Example usage
const jsonContent = csvToJson('posts.csv');
fs.writeFileSync('posts.json', jsonContent);

Step 3: Transforming JSON to Markdown

Objective: Convert the structured JSON content into Markdown, a format known for its simplicity and wide support across various platforms and static site generators.

Process:

For the final leap to Markdown, I used a combination of custom scripts and the turndown library to map JSON fields to Markdown syntax. This included formatting titles, headers, body content, and metadata into Markdown files. Each JSON object represented a blog post or page, which was then transformed into a Markdown file with appropriate front matter.

Code Snippet:


const fs = require('fs');
const TurndownService = require('turndown');

const turndownService = new TurndownService();
const posts = JSON.parse(fs.readFileSync('posts.json'));

posts.forEach((post) => {
    const markdown = turndownService.turndown(post.content);
    const frontMatter = `---
title: ${post.title}
date: ${post.date}
tags: [${post.tags.join(', ')}]
---
`;
    fs.writeFileSync(`${post.slug}.md`, frontMatter + markdown);
});

This three-step process, though seemingly complex, facilitated a smooth transition from a database-driven CMS to a static, file-based content management workflow. The choice of tools and libraries aimed to minimize manual intervention and ensure data integrity throughout the migration.

Conclusion and Reflections

So, what's the lowdown on this whole switcheroo from Cockpit to Markdown, you ask? Well, it's been a bit of a rollercoaster, but the kind where you scream from excitement, not fear. The biggest takeaway? Simplification is the name of the game. Trimming the fat off the CMS and getting down to the bare essentials of writing felt liberating. Not gonna lie, it wasn't all smooth sailing – wrestling data from the grips of SQLite into the airy freedom of Markdown had its moments. But, seeing my content breathe in its new Markdown abode? Priceless.

There were hiccups, sure. Deciphering data relationships and ensuring nothing got lost in translation was a bit like deciphering ancient scripts. Would I do things differently? Maybe a tad more planning on the data structure front. But, in the grand scheme, the process was a solid win.

And here's the kicker – this isn't just about my liberation from a CMS. It's a blueprint, a beacon for anyone looking to declutter their digital content life. Whether you're a solo blogger or a small team tired of the backend ball and chain, this workflow could be your ticket to simplicity.

Call to Action

Feeling inspired? Got a stash of content looking for a new lease on life? Roll up your sleeves, grab my scripts, and give them your own twist. Every blog, every content project, has its own flavor, and maybe, just maybe, you'll find a new secret ingredient in this workflow.

Got questions, stuck in a rut, or just want to chat about how awesome (or daunting) the world of Markdown is? Hit me up in the comments or slide into my socials. Sharing is caring, and I'm all ears.

Additional Resources

For the brave souls ready to dive into this adventure, head over to my GitHub repository https://github.com/muhzulzidan/cockpit-sqlite-markdown-converter You'll find all the scripts

maybe give it a star.