checkpoint
This commit is contained in:
Joe Ardent
parent
568352597b
commit
7955a3aa69
|
|
@ -0,0 +1,235 @@
|
|||
+++
|
||||
title = "A One-Part Serialized Mystery, Part 2: The Benchmarks"
|
||||
slug = "one-part-serialized-mystery-part-2"
|
||||
date = "2023-07-09"
|
||||
[taxonomies]
|
||||
tags = ["software", "rnd", "proclamation", "upscm", "rust", "macros"]
|
||||
+++
|
||||
|
||||
# A one-part serial mystery post-hoc prequel
|
||||
|
||||
I [wrote recently](/rnd/one-part-serialized-mystery) about switching the types of the primary keys in
|
||||
the database for an [in-progress web app](https://gitlab.com/nebkor/ww) I'm building. At that time,
|
||||
I'd not yet done any benchmarking, but had reason to believe that using [sortable primary
|
||||
keys](https://github.com/ulid/spec) would yield some possibly-significant gains in performance, in
|
||||
both time and space. I'd also read accounts of regret that databases had not used ULIDs (instead of
|
||||
[UUIDs](https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random))) from the
|
||||
get-go, so I decided it couldn't hurt to switch to them before I had any actual data in my DB.
|
||||
|
||||
And that was correct: it didn't hurt performance, but it also didn't help much either. I've spent a
|
||||
bunch of time now doing comparative benchmarks between ULIDs and UUIDs, and as I explain below, the
|
||||
anticipated space savings did not materialize, and the speed-up is merely augmenting what was
|
||||
already more than fast enough into slightly more faster than that. Of course, of course, and as
|
||||
always, the real treasure was the friends we made along the way etc., etc. So come along on a brief
|
||||
journey of discovery!
|
||||
|
||||
# Bottom Line Up Front
|
||||
|
||||
With sqlite and my final table schema, the size difference and speed differences are negligible,
|
||||
|
||||
TODO MOR STUFFF
|
||||
|
||||
|
||||
However, with my initial database layout and import code, ULIDs resulted in about 5% less space and
|
||||
took only about 2/3rds as much time as when using UUIDs (5.7 vs 9.8 seconds). The same space and
|
||||
time results held whether or not [`without rowid`](https://www.sqlite.org/withoutrowid.html) was
|
||||
specified on table creation, which was counter to expectation, though I now understand why; I'll
|
||||
explain at the end.
|
||||
|
||||
# It's a setup
|
||||
|
||||
My benchmark is pretty simple: starting from an empty database, do the following things:
|
||||
|
||||
1. insert 10,000 randomly chosen movies (title and year of release, from between 1965 and 2023) into
|
||||
the database
|
||||
1. create 1,000 random users[^random-users]
|
||||
1. for each user, randomly select around 100 movies from the 10,000 available and put them on their list of
|
||||
things to watch
|
||||
|
||||
Only that last part is significant, and is where I got my timing information from.
|
||||
|
||||
The table that keeps track of what users want to watch was defined[^not-final-form] like this:
|
||||
|
||||
``` sql
|
||||
create table if not exists witch_watch (
|
||||
id blob not null primary key,
|
||||
witch blob not null, -- "user"
|
||||
watch blob not null, -- "thing to watch"
|
||||
[...]
|
||||
foreign key (witch) references witches (id) on delete cascade on update no action,
|
||||
foreign key (watch) references watches (id) on delete cascade on update no action
|
||||
);
|
||||
[...]
|
||||
create index if not exists ww_witch_dex on witch_watch (witch);
|
||||
create index if not exists ww_watch_dex on witch_watch (watch);
|
||||
```
|
||||
|
||||
The kind of queries I'm trying to optimize with those indices is "what movies does a certain user
|
||||
want to watch?" and "what users want to watch a certain movie?". The IDs are 16-byte blobs; an
|
||||
entire row in the table is less than 100 bytes.
|
||||
|
||||
## A digression on SQLite and performance
|
||||
|
||||
I've mentioned once or twice before that I'm using [SQLite](https://www.sqlite.org/index.html) for
|
||||
this project. Any time I need a database, my first reach is for SQLite:
|
||||
|
||||
* the database is a single file, along with a couple temp files that live alongside it, simplifying
|
||||
management
|
||||
* there's no network involved between the client and the database; a connection to the database is
|
||||
a pointer to an object that lives in the same process as the host program; this means that read
|
||||
queries return data back in just a [few
|
||||
*microseconds*](https://www.youtube.com/watch?v=qPfAQY_RahA)
|
||||
* it scales vertically extremely well; it can handle database sizes of many terabytes
|
||||
* it's one of the most widely-installed pieces of software in the world; there's at least one
|
||||
sqlite database on every smartphone, and there's a robust ecosystem of [useful
|
||||
extensions](https://litestream.io/) and other bits of complimentary code freely available
|
||||
|
||||
And, it's extremely performant. When using the [WAL journal mode](https://www.sqlite.org/wal.html)
|
||||
and the [recommended durability setting](https://www.sqlite.org/pragma.html#pragma_synchronous) for
|
||||
WAL mode, along with all other production-appropriate settings, I got almost 20,000 *writes* per
|
||||
second[^nothing is that slow]. There were multiple concurrent writers, and each write was a transaction that inserted about
|
||||
100 rows at a time. I had [retry
|
||||
logic](https://gitlab.com/nebkor/ww/-/blob/4c44aa12b081c777c82192755ac85d1fe0f5bdca/src/bin/import_users.rs#L143-145)
|
||||
in case a transaction failed due to the DB being locked by another writer, but that never happened:
|
||||
each write was just too fast.
|
||||
|
||||
# Over-indexing on sortability
|
||||
|
||||
The reason I had hoped that ULIDs would help with keeping the sizes of the indexes down was the
|
||||
possibility of using [clustered
|
||||
indexes](https://www.sqlite.org/withoutrowid.html#benefits_of_without_rowid_tables). To paraphrase
|
||||
that link:
|
||||
|
||||
> In an ordinary SQLite table, the PRIMARY KEY is really just a UNIQUE index. The key used to look
|
||||
> up records on disk is the rowid. [...]any other kind of PRIMARY KEYs, including "INT PRIMARY KEY"
|
||||
> are just unique indexes in an ordinary rowid table.
|
||||
>
|
||||
> ...
|
||||
>
|
||||
> Consider querying this table to find the number of occurrences of the word "xsync".:
|
||||
> SELECT cnt FROM wordcount WHERE word='xsync';
|
||||
>
|
||||
> This query first has to search the index B-Tree looking for any entry that contains the matching
|
||||
> value for "word". When an entry is found in the index, the rowid is extracted and used to search
|
||||
> the main table. Then the "cnt" value is read out of the main table and returned. Hence, two
|
||||
> separate binary searches are required to fulfill the request.
|
||||
>
|
||||
> A WITHOUT ROWID table uses a different data design for the equivalent table. [in those tables],
|
||||
> there is only a single B-Tree... Because there is only a single B-Tree, the text of the "word"
|
||||
> column is only stored once in the database. Furthermore, querying the "cnt" value for a specific
|
||||
> "word" only involves a single binary search into the main B-Tree, since the "cnt" value can be
|
||||
> retrieved directly from the record found by that first search and without the need to do a second
|
||||
> binary search on the rowid.
|
||||
>
|
||||
> Thus, in some cases, a WITHOUT ROWID table can use about half the amount of disk space and can
|
||||
> operate nearly twice as fast. Of course, in a real-world schema, there will typically be secondary
|
||||
> indices and/or UNIQUE constraints, and the situation is more complicated. But even then, there can
|
||||
> often be space and performance advantages to using WITHOUT ROWID on tables that have non-integer
|
||||
> or composite PRIMARY KEYs.
|
||||
|
||||
<div class="caption">sorry what was that about secondary indices i didn't quite catch that</div>
|
||||
|
||||
HALF the disk space *and* TWICE as fast?? Yes, sign me up, please!
|
||||
|
||||
## Sorry, the best I can do is all the disk space
|
||||
|
||||
There are some [guidelines](https://www.sqlite.org/withoutrowid.html#when_to_use_without_rowid)
|
||||
about when to use `without rowid`:
|
||||
|
||||
> The WITHOUT ROWID optimization is likely to be helpful for tables that have non-integer or
|
||||
> composite (multi-column) PRIMARY KEYs and that do not store large strings or BLOBs.
|
||||
>
|
||||
> [...]
|
||||
>
|
||||
> WITHOUT ROWID tables work best when individual rows are not too large. A good rule-of-thumb is
|
||||
> that the average size of a single row in a WITHOUT ROWID table should be less than about 1/20th
|
||||
> the size of a database page. That means that rows should not contain more than ... about 200 bytes
|
||||
> each for 4KiB page size.
|
||||
|
||||
As I mentioned, each row in that table was less than 100 bytes, so comfortably within the given
|
||||
heuristic. In order to test this out, all I had to do was change the table creation statement to:
|
||||
|
||||
``` sql
|
||||
create table if not exists witch_watch (
|
||||
id blob not null primary key,
|
||||
witch blob not null, -- "user"
|
||||
watch blob not null, -- "thing to watch"
|
||||
[...]
|
||||
foreign key (witch) references witches (id) on delete cascade on update no action,
|
||||
foreign key (watch) references watches (id) on delete cascade on update no action
|
||||
) without rowid;
|
||||
```
|
||||
|
||||
So I did.
|
||||
|
||||
Imagine my surprise when it took nearly 20% longer to run, and the total size on disk was nearly 5%
|
||||
larger. Using random UUIDs was even slower, so there's still a relative speed win for ULIDs, but it
|
||||
was still an overall loss to go without the rowid. Maybe it was time to think outside the box?
|
||||
|
||||
## Schema pruning
|
||||
|
||||
I had several goals with this whole benchmarking endeavor. One, of course, was to get data on ULIDs
|
||||
vs. UUIDs in terms of performance, at the very least so that I could write about when I publicly
|
||||
said I would. But another, and actually-more-important goal, was to optimize the design of my
|
||||
database and software, especially as it came to size on disk (my most-potentially-scare computing
|
||||
resource; network and CPU are not problems until you get *very* large, and you would have long ago
|
||||
bottlenecked on disk size if you weren't careful).
|
||||
|
||||
So it was Cool and Fine to take advantage of the new capabilities that ULIDs offered if those new
|
||||
capabilities resulted in better resource use. Every table in my original, UUID-based schema had had
|
||||
a `created_at` column, stored as a 64-bit signed offset from the [UNIX
|
||||
epoch](https://en.wikipedia.org/wiki/Unix_time). Because ULIDs encode their creation time, I could
|
||||
remove that column from every table that used ULIDs as their primary key. Doing so dropped the
|
||||
overall DB size by 5-10% compared to UUID-based tables with a `created_at` column.
|
||||
|
||||
But I also realized that for the `watch_quests` table, no explicit
|
||||
|
||||
# At last, I've reached my final form
|
||||
|
||||
In the course of writing this post, I had a minor epiphany, which is that the reason for the
|
||||
regressed performance when using `without rowid` was that the secondary indices needed to point to
|
||||
the entries in the table, using the primary key of the table as the target. So when there was a ULID
|
||||
or UUID primary key, the indexes looked like, eg, this:
|
||||
|
||||
``` text
|
||||
16-byte blob -> 16-byte blob
|
||||
```
|
||||
<div class="caption">left side is, eg, user id, and right side is id of a row in the quests table</div>
|
||||
|
||||
|
||||
|
||||
using implicit rowid with ULIDs:
|
||||
|
||||
``` text
|
||||
*** Indices of table WATCH_QUESTS *********************************************
|
||||
|
||||
Percentage of total database...................... 43.3%
|
||||
Number of entries................................. 199296
|
||||
Average fanout.................................... 106.00
|
||||
```
|
||||
|
||||
``` text
|
||||
$ cargo run --release --bin import_users -- -d ~/movies.db -u 2000 -m 200
|
||||
[...]
|
||||
Added 398119 quests in 20.818506 seconds
|
||||
```
|
||||
<div class="caption">20k writes/second, baby</div>
|
||||
|
||||
size on disk is 75% of previous size (13M vs 17M)
|
||||
|
||||
|
||||
|
||||
----
|
||||
|
||||
[^random-users]: I did the classic "open `/usr/share/dict/words` and randomly select a couple things
|
||||
to stick together" method of username generation, which results in gems like
|
||||
"Hershey_motivations84" and "italicizes_creaminesss54". This is old-skool generative AI.
|
||||
|
||||
[^not-final-form]: The original schema was defined some time ago, and it took me a while to get to
|
||||
the point where I was actually writing code that used it. In the course of doing the benchmarks,
|
||||
and even in the course of writing this post, I've made changes in response to things I learned
|
||||
from the benchmarks and to things I realized by thinking more about it and reading more docs.
|
||||
|
||||
[^nothing is that slow]: old job python 100 reqs/sec fall down
|
||||
|
||||
[an_image]: /images/programmers_creed.jpg "some kinda image idunno"
|
||||
Loading…
Reference in New Issue