I sure fell down a math rabbit hole yesterday in addition to the usual one of inundating myself with fanciful ideas. Clearly I do not need math support in an initial Marked implementation let alone getting all twisted up over not having the latest and greatest most computationally efficient one in KaTeX over MathJax. In fact, math is VERY much a distraction at this point. I will seldom blog about math, and even very basic equations I will want to use other methods to demonstrate, and that's for good reason.
Unsurprisingly my wannabe-wrinkly brain went straight for something shiny. It does seem straightforward enough so I might attempt to incorporate Marked right from the get go running in a node worker. What could go wrong? Well, the biggest footgun with workers is probably that the chrome devtools debugger in particular cannot debug workers. Like, it doesn't even make an attempt to do so, whereas at least node.js will still emit the console API output (I'm referring to console.log and friends) called from workers into the terminal. You just won't even get anything nor any sense that anything happened when you do stuff from workers under a debug session with the Chrome debugger!
With that little rant out of the way, my experience has also shown that if I put in a little bit of extra legwork to make it so that any worker code can also be optionally launched from the main thread, then it can be debugged just fine.
I'm going to incorporate Marked in the simplest way and play with a few plugins so it doesn't ruin my evening.
Here's the first stab at evaluating page logic with what i have so far...
// root blog/ dir contains all blog content
// dirs under here are blog sagas
// loose md files are standalone blog posts.
// navigation and other post specific handling will be done at the post level. Sagas should be viewable and conceptually
// thought of as convenient single page layouts of multiple posts concatenated, not individual posts.
// NOTE I am leaving open for interpretation how to treat nested folders full of markdown files under blog/. It would be
// possible to make saga structure capable of recursion and replicate it in website navigation. However the question is
// more about whether that's genius or stark raving mad.
const dir = 'blog';
const render_pages = {};
const files: {
isFile: boolean;
path: string;
saga?: string;
}[] = fs.readdirSync(dir, { recursive: true, withFileTypes: true }).map(item => {
const isFile = item.isFile();
return {
isFile,
path: path.join(item.parentPath, item.name)
};
});
console.log('files:', files);
const posts = files.filter(f => f.isFile && isPostFile(f.path));
const specialPages = files.filter(f => !f.isFile && isSpecialPage(f.path));
const sagas = files.filter(f => !f.isFile).filter(f => {
const matching_posts = posts.filter(p => p.path.includes(f.path));
matching_posts.forEach(p => p.saga = f.path);
return matching_posts.length > 1;
});
const rest = files.filter(f => !posts.includes(f) && !specialPages.includes(f) && !sagas.includes(f));
console.log('posts, specialPages, sagas:', posts, specialPages, sagas);
console.log('rest:', rest);
// specifically a post is a unit of a blog.
function isPostFile(name: string) {
return /(\.deep)?\.md$/.test(name);
}
function isSpecialPage(path: string) {
return fs.readdirSync(path).some(e => /^index\.html$/.test(e));
}
const independent_posts = posts.filter(p => !sagas.some(s => s.path === p.saga));
console.log('independent_posts:', independent_posts);
const pages = [ ...sagas, ...independent_posts ];
console.log('pages:', pages);
and the output for what i have so far, plus a small number of trivial test files:
[Running: esrun ./scripts/build.ts]
files: [
{ isFile: false, path: 'blog/blog-engine' },
{ isFile: true, path: 'blog/code-as-graphs.md' },
{ isFile: true, path: 'blog/sagas-rant.md' },
{ isFile: false, path: 'blog/test' },
{ isFile: false, path: 'blog/test-ubo-tut' },
{ isFile: true, path: 'blog/blog-engine/1.deep.md' },
{ isFile: true, path: 'blog/blog-engine/2.deep.md' },
{ isFile: true, path: 'blog/blog-engine/3.deep.md' },
{ isFile: true, path: 'blog/blog-engine/4.deep.md' },
{ isFile: true, path: 'blog/blog-engine/5.deep.md' },
{ isFile: true, path: 'blog/blog-engine/6.deep.md' },
{ isFile: true, path: 'blog/blog-engine/7.1.deep.md' },
{ isFile: true, path: 'blog/blog-engine/7.md' },
{ isFile: true, path: 'blog/blog-engine/8.md' },
{ isFile: true, path: 'blog/test/llm-1.draft.md' },
{ isFile: true, path: 'blog/test-ubo-tut/a.md' },
{ isFile: true, path: 'blog/test-ubo-tut/index.html' }
]
posts, specialPages, sagas: [
{ isFile: true, path: 'blog/code-as-graphs.md' },
{ isFile: true, path: 'blog/sagas-rant.md' },
{
isFile: true,
path: 'blog/blog-engine/1.deep.md',
saga: 'blog/blog-engine'
},
{
isFile: true,
path: 'blog/blog-engine/2.deep.md',
saga: 'blog/blog-engine'
},
{
isFile: true,
path: 'blog/blog-engine/3.deep.md',
saga: 'blog/blog-engine'
},
{
isFile: true,
path: 'blog/blog-engine/4.deep.md',
saga: 'blog/blog-engine'
},
{
isFile: true,
path: 'blog/blog-engine/5.deep.md',
saga: 'blog/blog-engine'
},
{
isFile: true,
path: 'blog/blog-engine/6.deep.md',
saga: 'blog/blog-engine'
},
{
isFile: true,
path: 'blog/blog-engine/7.1.deep.md',
saga: 'blog/blog-engine'
},
{
isFile: true,
path: 'blog/blog-engine/7.md',
saga: 'blog/blog-engine'
},
{
isFile: true,
path: 'blog/blog-engine/8.md',
saga: 'blog/blog-engine'
},
{ isFile: true, path: 'blog/test/llm-1.draft.md', saga: 'blog/test' },
{
isFile: true,
path: 'blog/test-ubo-tut/a.md',
saga: 'blog/test-ubo-tut'
}
] [ { isFile: false, path: 'blog/test-ubo-tut' } ] [
{ isFile: false, path: 'blog/blog-engine' },
{ isFile: false, path: 'blog/test' }
]
rest: [ { isFile: true, path: 'blog/test-ubo-tut/index.html' } ]
independent_posts: [
{ isFile: true, path: 'blog/code-as-graphs.md' },
{ isFile: true, path: 'blog/sagas-rant.md' },
{
isFile: true,
path: 'blog/test-ubo-tut/a.md',
saga: 'blog/test-ubo-tut'
}
]
pages: [
{ isFile: false, path: 'blog/blog-engine' },
{ isFile: false, path: 'blog/test' },
{ isFile: true, path: 'blog/code-as-graphs.md' },
{ isFile: true, path: 'blog/sagas-rant.md' },
{
isFile: true,
path: 'blog/test-ubo-tut/a.md',
saga: 'blog/test-ubo-tut'
}
]
As you can see i like to go overboard on debug logging when i'm not dealing with huge test datasets. This code and behavior may seem convoluted at first glance but it does encode the natural way I wanted to perform logic for picking out what I would want to render!
...
How about this for a truly wild idea?
Piggybacking off my earlier concept for a live editing workflow integrated as a lightweight frontend functionality layer to augment the static hosted content, I think that's a beautiful workflow since it's about as far as I could practically take a WYSIWYG approach (going all the way to previewing in the real site...), but it occurs to me that I could go even further by extending such an application to a collection of readers and make live blogging a thing. And then I could do something totally bonkers, like livestreaming a live blog. I was already going to add Giscus but I have now envisioned one better than that by adding a twitch style livechat session to a live blogging session.
That was a very outlandish idea because it means I would need to design a state broadcasting mechanism to be fairly efficient whereas a single user solution would more or less suffice re-sending the entire markdown document periodically.
It can be done though. I also think it would make good sense to have reasonable event boundaries for a live markdown editing situation... basically i should be live updating the paragraph/hunk i'm currently editing in some kind of pre tag, but once my cursor moves far enough away it will get rendered into markdown, that way there will not be as much faff and weirdness from intermediate partially or totally unrenderable intermediate states near where I'm typing.
There are also other insane concepts somewhere around here, i do think that a livestream hack session exploring LLM assisted development on an open source project could be some kind of potentially entertaining activity. Because you have a back and forth cadence of writing a prompt which could benefit from a discussion with a human brain trust, and then we can watch the AI do its thing and all be entertained. If that stops being entertaining it still can serve as a bit of a break to help pace things.
But really though, this train of thought is really ridiculous, the only practical approach is to do a twitch live stream, but I know where the rabbit hole would go: Adding live audio streaming and the special live chat display. Like, it's excessive, it's pointless, it's ridiculous, but by enabling a special case replacement for streaming video via live markdown edit streaming, it really begs for the other components that could support such a thing.
Another slightly related idea: rolling my own analytics. It should be fairly fun to explore the analytics trope. I haven't built any software I'd actually call analytics, but it's firmly in the realm of frontend and I think it would definitely be fun to dive a bit into the very narrow niche of frontend analytics engineering. The notion of crafting something optimized like this (in order not to disrupt the experience of the reader) while gathering data in an effort to deliver a continually improving experience speaks to me. Of course, there is also a salient window in there for dark patterns. It would be good to explore the space a bit first hand to get familiar with the landscape. Keep your friends close and your enemies closer, after all.