Be part of JetBrains PHPverse 2026 on June 9 – a free online event bringing PHP devs worldwide together.

migrane's avatar

Spark , 64-bit row IDs and modern/distributed databases ... a no-go

Got excited here and bought Spark and come to find out that in fact if one wants to use a distributed database (cockroachdb in our case) ... one cannot really use Spark.

This is more of a database issue and Javascript shortcoming rather than purely a Spark issue. This being said one would hope that future versions of the software will account for this.

Deep dive ...

Modern databases use various methods for generating unique row IDs. In our case CockroachDB is using the UUID data type with the gen_random_uuid() function as the default value, which generates 128-bit values or a smaller SERIAL 64-bit using the unique_rowid() function.

This means that by using the migrations provided with Laravel Spark we end up with a 64-bit SERIAL (read 64-bit INT) for our unique row IDs.

The generated SQL snippet from say the teams table for the "id" field looks like this :

id INT NOT NULL DEFAULT unique_rowid(),

where "INT" is 64-bit

Since I am assuming that Laravel Spark was designed with MySQL in mind where the data type INT is 32-bit this poses the following problem.

When we create a new user with associated team, the record that is being created has a 64-bit unique identifier. When we try to use the kiosk and search for a user and say lookup a user all the Javascript functionality fails because when Javascript (VueJS in this case) tries to read our 64-bit unique identifier it will mangle the actual number.

The search in the kiosk seems to be working fine but when one goes to click further and lookup the info on one of the users, in the search results, the route fails. This is because the destination route contains the record ID and since the record ID is 64-bit , JS/Vue will generate a mangled record ID and hence an invalid destination route.

Take for example the following user record as coming from PHP/Laravel/Spark

[

{
    "id":316466291839008769,
    "name":"testbot",
    "email":"[email protected]",
    "photo_url":"https:\/\/www.gravatar.com\/avatar\/5e644ca1d422bcaa947f93832d1e1cd2.jpg?s=200&d=mm",
    "uses_two_factor_auth":false,
    "two_factor_reset_code":null,
    "current_team_id":316466291903037441,
    "stripe_id":null,
    "current_billing_plan":null,
    "billing_state":null,
    "vat_id":null,
    "trial_ends_at":"2018-01-22 19:10:47",
    "last_read_announcements_at":"2018-01-22 19:10:47",
    "created_at":"2018-01-22 19:10:47",
    "updated_at":"2018-01-22 19:10:47",
    "subscriptions":[
    ],
    "tax_rate":0
}

]

If we take this record and plug it in a JSON parser such as http://json.parser.online.fr/ we will note that the evaluated JS object is

[

{
    "id":316466291839008800,
    "name":"testbot",
    "email":"[email protected]",
    "photo_url":"https://www.gravatar.com/avatar/5e644ca1d422bcaa947f93832d1e1cd2.jpg?s=200&d=mm",
    "uses_two_factor_auth":false,
    "two_factor_reset_code":null,
    "current_team_id":316466291903037440,
    "stripe_id":null,
    "current_billing_plan":null,
    "billing_state":null,
    "vat_id":null,
    "trial_ends_at":"2018-01-22 19:10:47",
    "last_read_announcements_at":"2018-01-22 19:10:47",
    "created_at":"2018-01-22 19:10:47",
    "updated_at":"2018-01-22 19:10:47",
    "subscriptions":{
    },
    "tax_rate":0
}

]

This is very much similar if not identical processing that JS/Vue goes through in the Spark app. Note the mangled ID.

In conclusion a modern database yields 128-bit or 64-bit row ids, JS fails to read these properly and hence makes the app unusable.

I have tried to create some custom migrations to see if we can reduce the size of the id to 32 bit but it seems that we cannot successfully do that in CockroachDB because the unique_rowid() function always generates a 64-bit number.

Looking forward, it seems best if Spark would account for this by leveraging 128-bit UUIDs as it seems that there are libs out there to generate and operate on these 128-bit UUIDs both for PHP as well as JS.

If anyone has any other suggestions I may be able to try please let me know.

Migrane

0 likes
3 replies
Cronix's avatar

AFAIK, it's Laravel itself that doesn't fully support CockroachDB, and since Spark is built with Laravel...

Are you able to use CockroachDB on a vanilla Laravel app successfully (not Spark)? Everything I've read about it says Eloquent/DB doesn't work with CockroachDB very well, but raw queries work.

This is actually the first post I've seen dealing with Spark and CockroachDB, or wanting to use 64+ bit ints. I wouldn't hold my breath waiting for a solution in Laravel, Spark, or the js libraries Spark is using, with such low demand for it. Too many pieces to fix, and not all under Laravels control (such as Vue).

migrane's avatar

Laravel works with the postgres driver and there is also a decent cockroachdb driver https://github.com/nbj/cockroachdb-laravel ... while the driver is not 100% it does a good job and works in most parts we care about ... also php/laravel as well as the php spark code seem to be fine ... its when the data gets pulled from the spark api into VUE that things go awry. This now becomes a matter of dealing with 64-bit integers on the client side within the framework VUE and langue JS

Migrane

Cronix's avatar

One trick I learned is you can convert the 64bit int...to a string. Then it won't get truncated by js. Maybe you can add it to the $casts on the affected models. Not sure how that would work, really, but might be worth trying.

Please or to participate in this conversation.