SQLite a.k.a. SQLite3

(Specifically, this page is about my role in the sqlite world, not about how to use sqlite or some such.)

sqlite3 is the world's most widely-deployed database engine and is widely believed to contend for first place as the single most widely-deployed piece of software in the world (zlib being the other most likely candidate).

Since 2008 i have had the pleasure of working with sqlite's creator, Richard Hipp, via his Fossil SCM project and, in recent years, in much more limited capacities on other pet projects of his such as the althttpd web server and the pikchr "declarative diagram description" tool (for lack of a better description). (Althttpd is, not coincidentally, the web server which runs this very site.) More than a quarter of my life (as of this writing) has been spent coding in and around that community.

More recently, in late 2021 i was "drafted" into the sqlite team, primarily in an administrative support role, e.g. being upgraded from a moderator of its public support forum to an administrator in order to free Richard up from some of that workload. (Though i'm quite proficient in C, the relational algebra in sqlite's core code is well above my proverbial paygrade, and i've never had any delusions/aspirations of working on the sqlite internals.)

In mid-2022 the project began to investigate the feasibility of using WebAssembly, in combination with with the up-and-coming Origin-Private FileSystem web storage technology, to bring persistent sqlite3 databases to the browsers. (A small number of bleeding edge folks had already been using sqlite3 via WASM for at least 3 years, but without any sort of persistence.) Since WebAssembly is closely associated with JavaScript, and i'm the Fossil SCM's de facto "JavaScript Guy," that new effort fell neatly into my lap.

Our first proof-of-concept was sqlite3 fiddle, an HTML interface for the sqlite3 shell app (which is implemented in C), all run in the browser (as opposed to having the database running on the server). The very first iteration of it took roughly half an hour to get running thanks to the deep voodoo of the Emscripten tools, and about 25 minutes of that time was obtaining, installing, and learning how to invoke Emscripten. The ease with which that could be created convinced us that WASM was definitely something we wanted to play more with.

Going in, none of us on the project had ever investigated WASM, only heard its name. Within only a few weeks of getting our feet wet in with it, we were convinced that having sqlite3 run browser-side, especially once such databases can be made persistent, will open up a floodgate of new opportunities for web app features and catalpult sqlite usage into niches which have historically been off limits to it. And the best part is that the core library requires zero code changes for this: the C code compiles flawlessly to WASM and the friction between its interface and JavaScript is minimal, in particular once one has a bit of JS/WASM "glue code" packed up for reuse.

In October 2022 sqlite.org/wasm went online and in November 2022 sqlite 3.40 became the first release to include the JS/WASM components. The technical goals have been achieved and many docs have been written, but the work is far from "done" (insofar as maintained software is ever "done"). We're now at the stage where we can start to collect and address beta-testing user feedback. It's long been my experience that end users often come up with the best ideas for new features and improvements.

×