julid-rs/README.md

# Bottom line up front
Julids are globally unique, sortable identifiers, that are backwards-compatible with
[ULIDs](https://github.com/ulid/spec). This crate provides a Rust Julid datatype, as well as a
loadable extension for SQLite for creating and querying them:

``` text
$ sqlite3
SQLite version 3.40.1 2022-12-28 14:03:47
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> .load ./libjulid
sqlite> select hex(julid_new());
018998768ACF000060B31DB175E0C5F9
sqlite> select julid_string(julid_new());
01H6C7D9CT00009TF3EXXJHX4Y
sqlite> select julid_seconds(julid_new());
1690480066.208
sqlite> select datetime(julid_timestamp(julid_new()), 'auto');
2023-07-27 17:47:50
sqlite> select julid_counter(julid_new());
0
```

Crates.io: https://crates.io/crates/julid-rs
docs: https://docs.rs/julid-rs/latest/julid/


## A slightly deeper look

Julids are ULID-backwards-compatible (that is, all Julids are valid ULIDs, but not all ULIDs are
Julids) identifiers with the following properties:

 * they are 128-bits long
 * they are lexicographically sortable
 * they encode their creation time as the number of milliseconds since the [UNIX
   epoch](https://en.wikipedia.org/wiki/Unix_time)
 * IDs created within the same millisecond will still sort in their order of creation, due to the
   presence of a 16-bit monotonic counter, placed immediately after the creation time bits

It's that last thing that makes them distinctive. ULIDs have the following structure, from most to
least-significant bit:

![ULID bit structure](./ulid.svg)

According to the ULID spec, for ULIDs created in the same millisecond, the least-significant bit
should be incremented for each new ID. Since that portion of the ULID is random, that means you may
not be able to increment it without spilling into the timestamp portion. Likewise, it's easy to
guess a new possibly-valid ULID simply by incrementing an already-known one. And finally, this means
that sorting will need to read all the way to the end of the ULID for IDs created in the same
millisecond.

To address these shortcomings, Julids (Joe's ULIDs) have the following structure:

![Julid bit structure](./julid.svg)

As with ULIDs, the 48 most-significant bits encode the time of creation. Unlike ULIDs, the next 16
most-significant bits are not random, they're a monotonic counter for IDs created within the same
millisecond. Since it's only 16 bits, it will saturate after 65,536 IDs intra-millisecond creations,
after which, IDs in that same millisecond will not have an intrinsic total order (the random bits
will still be different, so you shouldn't have collisions). My PC, which is no slouch, can only
generate about 20,000 per millisecond, so hopefully this is not an issue! Because the random bits
are always fresh, it's not possible to easily guess a valid Julid if you already know one.

# SQLite extension

The extension, when loaded into SQLite, provides the following functions:

 * `julid_new()`: create a new Julid and return it as a `blob`
 * `julid_seconds(julid)`: get the number seconds (as a 64-bit float) since the UNIX epoch that this
   julid was created
 * `julid_counter(julid)`: show the value of this julid's monotonic counter
 * `julid_sortable(julid)`: return the 64-bit concatenation of the timestamp and counter
 * `julid_string(julid)`: show the [base-32 Crockford](https://en.wikipedia.org/wiki/Base32)
   encoding of this julid

## Building and loading

If you want to use it as a SQLite extension:

 * clone the [repo](https://gitlab.com/nebkor/julid)
 * build it with `cargo build --features plugin` (this builds the SQLite extension)
 * copy the resulting `libjulid.[so|dylib|whatevs]` to some place where you can...
 * load it into SQLite with `.load /path/to/libjulid` as shown at the top
 * party

If you, like me, wish to use Julids as primary keys, just create your table like:

``` sql
create table users (
  id blob not null primary key default (julid_new()),
  ...
);
```

and you've got a first-class ticket straight to Julid City, baby!

# Rust crate

Of course, you can also use it outside of a database; the `Julid` type is publicly exported, and
you can do like such as:

``` rust
use julid::Julid;

fn main() {
  let id = Julid::new();
  dbg!(id.timestamp(), id.counter(), id.sortable(), id.as_string());
}
```

after adding it to your project's dependencies(eg, `cargo add julid-rs`; note the package name is
"julid-rs", but the library name as used in your `use` statements is just "julid"). By default, it
will also include trait implementations for using Julids with
[SQLx](https://github.com/launchbadge/sqlx), and serializing/deserializing with
[Serde](https://serde.rs/), via the `sqlx` and `serde` features, respectively. One final default
feature, `chrono`, uses the Chrono crate to return the timestamp as a
[`DateTime`](https://docs.rs/chrono/latest/chrono/struct.DateTime.html) by adding a `created_at()`
method to `Julid` instances.

# Thanks

This crate wouldn't have been possible without a lot of inspiration (and a little shameless
stealing) from the [ulid-rs](https://github.com/dylanhart/ulid-rs) crate, as well as the
[sqlite-loadable-rs](https://github.com/asg017/sqlite-loadable-rs) crate, which made it *extremely*
easy to write the SQLite extension. Thank you, authors of those crates! Feel free to steal from this
project! (though I did submit a [pull request](https://github.com/asg017/sqlite-loadable-rs/pull/13)
to the sqlite-loadable-rs project, to add an entrypoint macro that returns
[`SQLITE_OK_LOAD_PERMANENTLY`](https://www.sqlite.org/rescode.html#ok_load_permanently), which would
let me use that instead of manually implementing the entrypoint to return that)