127 lines
5.3 KiB
Markdown
127 lines
5.3 KiB
Markdown
# Bottom line up front
|
|
Julids are globally unique, sortable identifiers, that are backwards-compatible with
|
|
[ULIDs](https://github.com/ulid/spec). This crate provides a Rust Julid datatype, as well as a
|
|
loadable extension for SQLite for creating and querying them:
|
|
|
|
``` text
|
|
$ sqlite3
|
|
SQLite version 3.40.1 2022-12-28 14:03:47
|
|
Enter ".help" for usage hints.
|
|
Connected to a transient in-memory database.
|
|
Use ".open FILENAME" to reopen on a persistent database.
|
|
sqlite> .load ./libjulid
|
|
sqlite> select hex(julid_new());
|
|
018998768ACF000060B31DB175E0C5F9
|
|
sqlite> select julid_string(julid_new());
|
|
01H6C7D9CT00009TF3EXXJHX4Y
|
|
sqlite> select julid_seconds(julid_new());
|
|
1690480066.208
|
|
sqlite> select datetime(julid_timestamp(julid_new()), 'auto');
|
|
2023-07-27 17:47:50
|
|
sqlite> select julid_counter(julid_new());
|
|
0
|
|
```
|
|
|
|
Crates.io: https://crates.io/crates/julid-rs
|
|
docs: https://docs.rs/julid-rs/latest/julid/
|
|
|
|
|
|
## A slightly deeper look
|
|
|
|
Julids are ULID-backwards-compatible (that is, all Julids are valid ULIDs, but not all ULIDs are
|
|
Julids) identifiers with the following properties:
|
|
|
|
* they are 128-bits long
|
|
* they are lexicographically sortable
|
|
* they encode their creation time as the number of milliseconds since the [UNIX
|
|
epoch](https://en.wikipedia.org/wiki/Unix_time)
|
|
* IDs created within the same millisecond will still sort in their order of creation, due to the
|
|
presence of a 16-bit monotonic counter, placed immediately after the creation time bits
|
|
|
|
It's that last thing that makes them distinctive. ULIDs have the following structure, from most to
|
|
least-significant bit:
|
|
|
|
|
|
|
|
According to the ULID spec, for ULIDs created in the same millisecond, the least-significant bit
|
|
should be incremented for each new ID. Since that portion of the ULID is random, that means you may
|
|
not be able to increment it without spilling into the timestamp portion. Likewise, it's easy to
|
|
guess a new possibly-valid ULID simply by incrementing an already-known one. And finally, this means
|
|
that sorting will need to read all the way to the end of the ULID for IDs created in the same
|
|
millisecond.
|
|
|
|
To address these shortcomings, Julids (Joe's ULIDs) have the following structure:
|
|
|
|
|
|
|
|
As with ULIDs, the 48 most-significant bits encode the time of creation. Unlike ULIDs, the next 16
|
|
most-significant bits are not random, they're a monotonic counter for IDs created within the same
|
|
millisecond. Since it's only 16 bits, it will saturate after 65,536 IDs intra-millisecond creations,
|
|
after which, IDs in that same millisecond will not have an intrinsic total order (the random bits
|
|
will still be different, so you shouldn't have collisions). My PC, which is no slouch, can only
|
|
generate about 20,000 per millisecond, so hopefully this is not an issue! Because the random bits
|
|
are always fresh, it's not possible to easily guess a valid Julid if you already know one.
|
|
|
|
# SQLite extension
|
|
|
|
The extension, when loaded into SQLite, provides the following functions:
|
|
|
|
* `julid_new()`: create a new Julid and return it as a `blob`
|
|
* `julid_seconds(julid)`: get the number seconds (as a 64-bit float) since the UNIX epoch that this
|
|
julid was created
|
|
* `julid_counter(julid)`: show the value of this julid's monotonic counter
|
|
* `julid_sortable(julid)`: return the 64-bit concatenation of the timestamp and counter
|
|
* `julid_string(julid)`: show the [base-32 Crockford](https://en.wikipedia.org/wiki/Base32)
|
|
encoding of this julid
|
|
|
|
## Building and loading
|
|
|
|
If you want to use it as a SQLite extension:
|
|
|
|
* clone the [repo](https://gitlab.com/nebkor/julid)
|
|
* build it with `cargo build --features plugin` (this builds the SQLite extension)
|
|
* copy the resulting `libjulid.[so|dylib|whatevs]` to some place where you can...
|
|
* load it into SQLite with `.load /path/to/libjulid` as shown at the top
|
|
* party
|
|
|
|
If you, like me, wish to use Julids as primary keys, just create your table like:
|
|
|
|
``` sql
|
|
create table users (
|
|
id blob not null primary key default (julid_new()),
|
|
...
|
|
);
|
|
```
|
|
|
|
and you've got a first-class ticket straight to Julid City, baby!
|
|
|
|
# Rust crate
|
|
|
|
Of course, you can also use it outside of a database; the `Julid` type is publicly exported, and
|
|
you can do like such as:
|
|
|
|
``` rust
|
|
use julid::Julid;
|
|
|
|
fn main() {
|
|
let id = Julid::new();
|
|
dbg!(id.timestamp(), id.counter(), id.sortable(), id.as_string());
|
|
}
|
|
```
|
|
|
|
after adding it to your project's dependencies (eg, `cargo add julid-rs`; note the package name is
|
|
"julid-rs", but the library name as used in your `use` statements is just "julid"). By default, it
|
|
will also include trait implementations for using Julids with
|
|
[SQLx](https://github.com/launchbadge/sqlx), and serializing/deserializing with
|
|
[Serde](https://serde.rs/), via the `sqlx` and `serde` features, respectively. One final default
|
|
optional feature, `chrono`, uses the Chrono crate to return the timestamp as a
|
|
[`DateTime`](https://docs.rs/chrono/latest/chrono/struct.DateTime.html) by adding a `created_at(&self)`
|
|
method to `Julid`.
|
|
|
|
# Thanks
|
|
|
|
This crate wouldn't have been possible without a lot of inspiration (and a little shameless
|
|
stealing) from the [ulid-rs](https://github.com/dylanhart/ulid-rs) crate, as well as the
|
|
[sqlite-loadable-rs](https://github.com/asg017/sqlite-loadable-rs) crate, which made it *extremely*
|
|
easy to write the SQLite extension. Thank you, authors of those crates! Feel free to steal from this
|
|
project!
|