I didn't write any Javascript, and other shortcuts

One slightly unusual thing about csvbase is that, when I wrote it, I didn't write any Javascript.

Most of the Javascript-I-did-not-write falls into a few categories:

I didn't write any frontend form validation. No need - that can all be done with HTML5.

I didn't write a date picker. Again, HTML5.

My pet feature is that you can paste cells straight from MS-Excel into a magic textarea and csvbase will do it's best to figure it out what it is and turn it into a table. Again: no Javascript involved. It's actually a lot easier to infer data types, divine string encoding, detect the cell separator and so on all together on the backend.

The HTML on my pages is generated by templates, as if it were still 2005.

Forms are submitted using multipart/form-encoding, again, as was done by my forefathers, before the advent of XMLHttpRequest.

When a form is "editable" (ie you get some kind of plus button that dynamically adds another form field) I just have that plus button add a query param and use HTTP GET to get a new version of the form with that field added (the minus button does the reverse). Again, HTML5 allows you to give individual buttons a special "formaction" and makes this trick easier than it used to be.

Sticklers will contend that in fact there is a .js file that came with the CSS framework I used. It's true - but I didn't write that file. I don't even read it. It is completely encapsulated as far as I am concerned. So the fact remains: I didn't write any Javascript.

Cars and buses

But why bother? If you can manage it, not writing Javascript will make your life easier and thus save you time.

One way to predict car reliability is by the number of parts. Cars with a higher part count tend to be less reliable, because as a car gets more complicated there are necessarily more ways things can go wrong. If you do have a part, best that it not be a moving one. Interior trim pieces rarely cause trips to the mechanic. If your mechanic mentions your gearbox however, you are about to hear quite bad news. All else being equal: fewer parts are better.

The addition of Javascript to any project invariably brings with it a lot of parts. Almost all of them are of that troublesome moving kind. Three years ago, someone at Microsoft determined that the average Javascript project on Github has just ten direct dependencies --- but 683 indirect dependencies (think: library X requires library Y, requires library Z, ...). Most serious commercial Javascript applications have far, far higher numbers. This could be one reason why NPM, the Javascript package manager, needed venture funding - resolving Javascript build dependencies is presumably at the very edge of the technology available to us now.

Omitting Javascript is not only more reliable but simply easier. The skill set to template HTML is a much, much smaller subset of that required to write even bad React. When I've worked on professional web projects in the past I've usually found both the depth and breadth of the skills used for frontend development to be incredible. For example, did you know that many, even basic, web apps now include an full in-browser event bus? It is one of those things that sounds self-affirmingly bonkers at first but tragically starts to make more sense the more you learn about it.

One of the most important benefits is that not writing Javascript makes your system a damn sight easier to test. Due to the fact that my systems' interest in the the user ends once I've written your HTML down the socket I don't need to model the browser, only an HTML parser. As a result my tests can operate on the HTTP level and because that's quite quick I can make more of my tests "whole-system" tests. As it stands, I have about 360 tests and most of them are testing via an HTTP client. HTTP is a nice level of abstraction to work at because you can say, eg "given a user, when he posts this form, then he gets this sort of HTML".

The runtime of the whole testsuite is 3.5 seconds - without parallelism. That's about 10 milliseconds per test. I should mention that I don't mock out the database. As csvbase is sort-of an ORM so the loss of correctness would not be worth the increase in speed. Some of the queries can get quite complicated.

Use more shortcuts, and less time

The upshot of all of this is saving time. There is an old joke that if any government programme fails, it was because the budget wasn't big enough (and that if it succeeds, well; that only proves the need for even more budget). The analogue of this joke for software projects would certainly refer to time instead of money.

Working software takes a stunning large amount of time to make. How can you make time for your side project?

Nowadays there is a vast and growing "productivity industrial complex" - thousands of podcasters and bloggers advising how to reorganise your life in order to save time, or at least use it more efficiently. To be honest, I never felt kin with them.

Just as money is not really the problem with most government programmes, for software projects, time only feels operative because of issues in other areas. I've always felt that the path to getting something out the bleeding door is less about being supernally efficient and/or "effective" than on finding the right shortcuts that make it possible, in any reasonable amount of time.

For csvbase, not having a Javascript frontend was a shortcut I took in order to get the thing done. It was one of many: csvbase has no background workers; the JSON API uses the same view handlers as the HTML pages, just instead of rendering objects into HTML template they get formatted as JSON; I used the same trick to serve CSV, XLSX and Parquet (nearly) too; I set the HTTP headers carefully so that instead of writing any caching code myself I could leverage the caching layer built into the CDN. Any side project that you can actually complete will be rammed to the gills with these kinds of shortcuts.

This isn't to do down the Facebook Reacts of the world. They have a place, but the occupational hazard of professional programmers is the feeling that if you do something in your day job then you can bring your skills home and use them to do your own project on the side. The problem is that you also bring home the mentality. Your workplace's standard practices are probably an order of magnitude or two more labour intensive than what you can afford given that you are, eg (as in my case), hacking on the project only while your toddler is sleeping.

Was there something in the magic_quotes?

I have always been interested in why so many successful companies/organisations started with LAMP and, specifically: PHP.

Wikipedia, Flickr, Facebook, Slack and Wordpress all used PHP (originally...). What is it about PHP? I don't know PHP well at all but from the outside, it looks to me, an obsessive shortcut-taker, that PHP absolutely conspires to provide shortcuts.

Serious projects build distribution packages (somewhat complex, 5mb), docker containers (secretly very complex, 100s-1000s of megs) or unikernel things (who knows how complex, loadsa megs). I have worked even in smallish companies where it is multiple people's fulltime job to deploy code to production. By contrast, many PHP applications are copied by FTP client straight into prod.

The HTML templating approach I have used is also not common today on serious commercial projects. Typically a GraphQL API is built and then consumed/controlled via frontend Javascript. But PHP goes one further: instead of embedding the templating inside the programming language, it actually embeds the programming language inside the template. They are the same thing. Arguably, PHP is just a templating language.

I mentioned above that package management, depressingly, can be quite a hard problem. PHP now has package manager, but, in the golden era, people simply copied files they wanted to use into their project. (C programming still works the same way - for appearances' sake this 'technique' is called "vendoring").

This is not all to say that I think PHP is some kind of latent ideal language. I didn't write any PHP for csvbase. But I do think there is something to learn from PHP culture. Shortcuts are important. One of the fitness functions for ideas is whether anyone can release a version 1.

That means that there is a scale where deployment-via-FileZilla is appropriate: the scale of a small project. 'Small' is an important scale because while not every project becomes large, every project starts out as small.

Keep in touch

Please do send me an email about this article, especially if you disagreed with it.

I am also on Mastodon: @calpaterson@fosstodon.org.

If you liked this, you might like other things I've written: on the csvbase blog, or on my blog.

Follow new posts via RSS.