Transversal

Template rust application. Rust speed, SPA interactivity.

Backend & Web

Axum

Lets start by the most simpler part, Axum is part of the tokio project, a thin layer of it & hyper, supports tower. Its blazing fast its simple its predictable its great, provides great base and gets out of your way. There are lower level and faster but also more annoying to work with and not as rich ecosystem.

OpenID OAuth 2

The authentication is made though an OpenID implicit PKCE flow, it receives a openid URL a client ID and secret and uses open ID discovery .well-known to discover the server configuration. After receiving a successful redirect, saves user data to the database and starts a session.

Monitoring & Tracing

Axum is part of the tokio ecosystem so has built-in support to the great tracing crate, rich logging, spans, tracing you name it, built in.

Sea Session

A more simpler MVC web application allows to the yester-year of sessions, its like cookies but not as annoying or limited. In your controllers you can simply session.get::<Struct>().await with the glue code (Session Backend) in place to persist arbitrary JSON to DB. Its use to mainly to store auth session, but any other Serializable struct can be persisted.

Frontend & Templating

The main building blocks are UniPoly and beer.css

NO BUILD STEP

This is a quite constraining decision that might be revisited later because there might be a ceiling of functionality that can be achieved without a nodejs build step, but I really wanted to avoid the complexity of managing 2 the projects of different ecosystems and languages. The local DX is already non-ideal (rust compile times) I really rather avoid the extra complexity of having 2 watch process that need to be synchronized.

Beer.Css

There is nothing revolutionary about it per-se whats important is ticks all the boxes I needed:

• Small size
• "batteries included" components
• accessibility
• no build step

It is missing some quality of life utility classes (e.g. margins) and is quite aggressive overriding meaning of normal HTML tags like <i> being now for icons but that characteristic actually helps since when used in raw HTML.

Unipoly

MVC has a bunch of historic limitations which I think have been overcompensated for with a javascript frontend SPA framework dictatorship killing a lot of the richness exists in computer programming.

Unipoly is part of the SSR renaissance and from a family of increasingly popular libraries of HTML enhancement libraries (the most infamous being HTMX) which the goal of bringing richer SPA characteristics to MVC web apps, JS stateful navigation, in place update of certain parts of the page without full refresh, richer form validation, etc.

It works great it really is a great progressive enhancement library has a great foundation and anything I need seems to be a up-* html attribute away no matter how complex the behavior.

Askama

In brief its Jinja, but for rust, parsed and converted at compile time to embedded (in the final binary) rust code. When I was shopping for possible templating solutions I immediately excluded ones that did runtime interpolation/interpretation or anything of the sorts, they just as clunky as templating and still offer a much bigger runtime hit.

Out of the pre-compiled options some were too obscure and/or crazy (see maud) Askama hit the perfect sweet spot of the battle tested Jinja syntax and blazing speed. But not without its issues

Known issues

1. Slow compilation times and no auto refresh

   • This is the main tradeoff of Askama, if your templates become rust code when you compile it it means it will go through the infamously slow rust compiler
   • There is currently no mechanism to auto refresh, it is doable but would be a very complicated bunch of code that needed to be selectively disabled by target

2. The sea-orm Active model <-> serde integration is a little awkward

   • The templates receiving active models to render is a very convenient way to allow per field optional rendering but I got the wrong impression that deserializing into an active model would allow to deserialized missing fields (e.g. ID of relations which make no sense serializing)
   • The JSON value needs to be a valid model which might mean inserting a fake ID during inserts json_value["id"] = json!(UUID::nil().to_string())
   • As per FormValues HTML limitation, everything is a string, even numbers, because JSON has numbers serde refuses to auto convert string into numbers, which is sensible default and can be configured, sadly sea-orm is migration-first you generate entities from DB state, which has limited customization. I settled for something like this in the repository layer:

   json_value["number_prop"] = json!(json_value["number_prop"].as_str().parse::<i32>().unwrap_or_default())
   

FAQ

• How to generate the files from the database?

  sea generate entity -o models/src/generated --with-serde both --serde-skip-deserializing-primary-key --serde-skip-hidden-column
  

I was working in fixing a bug in Mocha, which although renowned testing framework, is starting to show its age (10+ years), so its going through major changes to keep up with the competition. And as with any major changes its normal to have some regressions along the way, I found one and reported it the mocha team was quite agile in confirming my issue, so I dug in the code myself to diagnose the root cause but turns out this bug was much bigger then initially thought.

Being a testing library it necessitates a quite a complex files loading and discovery code, turns out Mocha’s use case was complex enough to reveal a bug in Node JS itself, so this is a story how I managed the communication and reporting and escalation of the same bug to two different major JavaScript ecosystem projects.

ESM vs CJS

For you to understand the bug, you need to understand the difference between an CJS and a ESM module, and especially and how node’s adoption of ESM was… well slow. I'd love to write about the mess which is the story of JavaScript modules, but it would be way too long of a tangent, if you’re curious you can read this article about it, but all you need to know that node.js has two main ways (module systems) to allow files to import each other: import { foo } from “library” (ESM) and const { foo } = require('library') (CJS) node JS is in the process of adding functionality to better support for the “new” ESM way.

One of the main pain points of the module system migration was the fact that although both the systems are conceptually equivalent: “import exposed symbols of some other file”, ESM has extra functionality which made it challenging to make then interoperable, the big one being ESM modules can have await expressions outside of async function body, a.k.a. “top level await”. The node team has dragged its feet with barebones ESM support for years. Only after fierce competition from new runtimes like Deno and Bun node finally added core missing ESM functionality. The important bit for us is: require(ESM)

require(ESM) is the missing piece that the ecosystem needed to start migrating en-masse to ESM, it basically allows you to require() an ESM module as if it were a CJS file helping bridging the gap and ease transition (with caveats), historically if you needed to access a ESM module from CJS (which is the more common use case as libraries tend to migrate faster then applications) you were supposed to use dynamic async imports await import(module) which are very disruptive to any program flow: imagine of instead of having require calls at the top of your file you now had to await import in each of your functions? Because don’t you forget, in CJS you can only await inside an async function! In reality no one ever did that, given the popularity of Typescript, people already had transpilers setup in their projects, which all had functionality to hide node’s lack luster ESM support. Even typescript itself.

Who’s cache is it?

Enough of Node’s quirks, on to the bug: Although convenient, the require(ESM) begs an important question: Whose cache is it? CJS and ESM are historically known for having separate module caches, does that mean require can now write to ESM module cache? Will it check on its cache first? May seem like implementation details but now was a pressing question for me. Since mocha needs to support a lot of node it couldn’t rely anymore on good old ERR_REQUIRE_ESM to fallback to to import(). Mocha’s implementation basically ignored any error thrown by the first require call and trusted node JS to keep the same behaviour if you tried to import a module twice, no matter if require(ESM) or import(ESM). Turns out it was not the case, and I had found a bug to affecting node versions all the way back to LTS which goes like this:

If you tried to require(ESM) a module with had a “top level error” (an error thrown during module evaluation outside the body of a function) the first require call would throw as expected but future calls of import(ESM) would just return an empty object. That broke the mocha implementation, that now couldn’t tell the module had a top level error: the import(ESM) promise was resolved normally hiding the error, when you called mocha it just skipped the said file without registering any tests.

The bug itself it’s not super complex to define and is quite situational. But was very important to more intricate consumers of the ESM API like Mocha. Confident of my code sandbox MRE I opened my first node bug request. Just a quick shout out: tools like code sandbox shine for MREs, allows you to isolate environment differences and pin a specific node version running in a containerized environment. A blessing for creating meaningful bug requests.

The race for the fix

At this time I was confident of my bug and was hoping to be able to claim the laurels of my investigation by merging a bug fix on node’s main branch, but my C skills were not quick enough to be able to setup the project locally and understand node internals and how to fix the bug before the legendary joyeecheung (the original require(esm) implementer) came swooping in with a agile surgical fix. In the end the fix was merged within a week which is quite fast for a project size of node.

I did ended up updating the mocha’s official documentation about the bug while we wait for node js release as a consolation prize from the gracious JoshuaKGoldberg. I hope this inspired you to also post your bug reports to your favorite library and thanks for reading.

The first step of TDD is to write the failing test, but what if the test not even gets detected? That was the start of my journey. After spending a good afternoon after trying to track a bug down to a silly lack of environment variable I noticed that was something very fishy about mocha, it never reported top level import errors, an exception was being thrown during module import but mocha was not reporting on it, just saying “0 tests”, I initially didn’t want to blame it on mocha, I like living on the edge and was running the latest node (24) with native type-stripped ESM typescript files. But although I acknowledged the fact that the problem was somewhat self-inflicted I still believed it was relevant enough to raise an issue in mocha’s repository so they could prepare for when those features reach the masses.

I don’t know about you but personally my biggest struggle with open source has always been the social management aspect of working in public: piles of irrelevant issues, sometimes entitled people, finally you find something to contribute on, open a PR about it and have petty developers nitpick on your variable names. So for me I got quite the rush to have my reported bug not only confirmed but also affecting LTS, having that easy of a score I couldn’t resist, I dug in.

Then I realized that a decade old project might not be as accessible to contribute as one initially though, how do you even write tests on a testing framework? CONTRIBUTING.md files are something you can’t quite value until you need one, you can know conceptually that is supposed to help you understand how to contribute, but I hadn’t realized contributing might also mean working on a bug affecting you too, not a document for select few.

Turns out my bug could be tested using mocha’s “integration” test which is somewhat simpler then the unit tests (turns out is kinda complicated to test a test framework) which mostly means spawning a child process of mocha running a “fixture” test and asserting on std-out, exit code, etc.

I’m not particularly religious about TDD but it goes a long way conserving your sanity when trying to find your way in such a big project, secondly knowing my way around a debugger really turned out invaluable, much better to simply drop breakpoints, know all variable values at any given point and step over from there. Codebase which may I remind you, is mostly ES5 JavaScript so that means: no typescript, prototype based OOP, and node callback hell, the code is older than promises so lets just say the age is starting to show.

And I don't mean to complain, mocha is still wildly used and the contributors are right now working to reboot development and if anything all this tech debt just makes me feel I can make a difference, how many nerds are still around to refactor function.proptotype.instanceMethod = function () {} away? Endless contribution opportunities, and mocha still has 22k stars.

After stepping in, out and over, I finally came across a piece of code that although quite clever, assumes a little much about node, and exposed mocha to all kinds of funny bugs from downstream dependencies, but I’m getting ahead of myself, the code goes a little something like this: (overly simplified of course)

async function loadTestFile(resolvedFilePath) {
    // [...]
    try {
        return require(resolvedFilePath);
    } catch (err) {
        return await import(resolvedFilePath);
    }
}

Its quite the ingenious way to support ESM and CJS modules transparently: tries the path of least resistant first by requiring it, if it fails, try falling back to importing, with the big assumption being that if the error thrown by the require wasn’t related to ESM / CJS integration, the second import will simply throw the same error again. Well turns out I had just struck gold because in fact the import call was not throwing the second time! The same module, with the same top level exception, simply disappeared swallowed by node, which was returning a meaningless empty object instead.

Stay tuned to learn how I reported and almost landed a the fix on node.JS main.