Geode benchmarks
Results
Updated · from 10 runs on builds 0.5.0, 0.5.0+format, 0.6.0, 0.6.1
Load, save and release
Live Roblox serversEach run loads five fresh profiles, saves each, releases each after a change, loads them again and releases them again, with the steps on one key at least 8 seconds apart. Times are seconds from calling the library until its DataStore request finished.
Load, new profile
Median (the upright tick) and min to max of every sample (the bar). Medians less than one timing step (16.7 ms (1/60 s)) apart are ties.
Table: Time
| Subject | Median | Min | Max | n | Failed | Tie with |
|---|---|---|---|---|---|---|
| Geode | 283 ms | 216 ms | 433 ms | 25 | 0 | Geode, mailbox = true |
| Geode, mailbox = true | 299 ms | 182 ms | 383 ms | 25 | 0 | Geode, ProfileStore |
| ProfileStore | 300 ms | 216 ms | 366 ms | 25 | 0 | Geode, mailbox = true, ProfileService |
| ProfileService | 316 ms | 183 ms | 400 ms | 25 | 0 | ProfileStore |
| DataStore2, Standard saving | 83 ms | 65 ms | 117 ms | 25 | 0 | – |
| Historical build | ||||||
| Geode 0.5, mailbox read after every load | 383 ms | 283 ms | 516 ms | 25 | 0 | – |
Requests per sample, by method. A range means the count varied between samples. 2 methods used.
| Subject | UpdateAsync | GetAsync | n | Counts |
|---|---|---|---|---|
| Geode | 1 | – | 25 | Same in every sample |
| Geode, mailbox = true | 1 | 1 | 25 | Same in every sample |
| ProfileStore | 1 | – | 25 | Same in every sample |
| ProfileService | 1 | – | 25 | Same in every sample |
| DataStore2, Standard saving | – | 1 | 25 | Same in every sample |
| Historical build | ||||
| Geode 0.5, mailbox read after every load | 1 | 1 | 25 | Same in every sample |
Load, stored profile
Median (the upright tick) and min to max of every sample (the bar). Medians less than one timing step (16.7 ms (1/60 s)) apart are ties.
Table: Time
| Subject | Median | Min | Max | n | Failed | Tie with |
|---|---|---|---|---|---|---|
| Geode | 266 ms | 166 ms | 317 ms | 25 | 0 | Geode, mailbox = true, ProfileStore |
| Geode, mailbox = true | 250 ms | 200 ms | 283 ms | 25 | 0 | Geode, ProfileStore |
| ProfileStore | 266 ms | 200 ms | 350 ms | 25 | 0 | Geode, Geode, mailbox = true |
| ProfileService | 283 ms | 199 ms | 350 ms | 25 | 0 | – |
| DataStore2, Standard saving | 33 ms | 16 ms | 83 ms | 25 | 0 | – |
| Historical build | ||||||
| Geode 0.5, mailbox read after every load | 333 ms | 266 ms | 400 ms | 25 | 0 | – |
Requests per sample, by method. A range means the count varied between samples. 2 methods used.
| Subject | UpdateAsync | GetAsync | n | Counts |
|---|---|---|---|---|
| Geode | 1 | – | 25 | Same in every sample |
| Geode, mailbox = true | 1 | 1 | 25 | Same in every sample |
| ProfileStore | 1 | – | 25 | Same in every sample |
| ProfileService | 1 | – | 25 | Same in every sample |
| DataStore2, Standard saving | – | 1 | 25 | Same in every sample |
| Historical build | ||||
| Geode 0.5, mailbox read after every load | 1 | 1 | 25 | Same in every sample |
Save
Median (the upright tick) and min to max of every sample (the bar). Medians less than one timing step (16.7 ms (1/60 s)) apart are ties.
Table: Time
| Subject | Median | Min | Max | n | Failed | Tie with |
|---|---|---|---|---|---|---|
| Geode | 233 ms | 165 ms | 301 ms | 25 | 0 | Geode, mailbox = true |
| Geode, mailbox = true | 233 ms | 166 ms | 317 ms | 25 | 0 | Geode |
| ProfileStore | 250 ms | 133 ms | 300 ms | 25 | 0 | ProfileService |
| ProfileService | 266 ms | 150 ms | 316 ms | 25 | 0 | ProfileStore |
| DataStore2, Standard saving | 300 ms | 199 ms | 416 ms | 25 | 0 | – |
| Historical build | ||||||
| Geode 0.5, mailbox read after every load | 233 ms | 151 ms | 300 ms | 25 | 0 | – |
Requests per sample, by method. A range means the count varied between samples. 1 method used.
| Subject | UpdateAsync | n | Counts |
|---|---|---|---|
| Geode | 1 | 25 | Same in every sample |
| Geode, mailbox = true | 1 | 25 | Same in every sample |
| ProfileStore | 1 | 25 | Same in every sample |
| ProfileService | 1 | 25 | Same in every sample |
| DataStore2, Standard saving | 1 | 25 | Same in every sample |
| Historical build | |||
| Geode 0.5, mailbox read after every load | 1 | 25 | Same in every sample |
Release
Median (the upright tick) and min to max of every sample (the bar). Medians less than one timing step (16.7 ms (1/60 s)) apart are ties.
Table: Time
| Subject | Median | Min | Max | n | Failed | Tie with |
|---|---|---|---|---|---|---|
| Geode | 267 ms | 183 ms | 766 ms | 25 | 0 | Geode, mailbox = true, ProfileStore, ProfileService, DataStore2, Standard saving |
| Geode, mailbox = true | 266 ms | 216 ms | 450 ms | 25 | 0 | Geode, ProfileStore, DataStore2, Standard saving |
| ProfileStore | 266 ms | 183 ms | 350 ms | 25 | 0 | Geode, Geode, mailbox = true, DataStore2, Standard saving |
| ProfileService | 283 ms | 198 ms | 350 ms | 25 | 0 | Geode, DataStore2, Standard saving |
| DataStore2, Standard saving | 267 ms | 200 ms | 350 ms | 25 | 0 | Geode, Geode, mailbox = true, ProfileStore, ProfileService |
| Historical build | ||||||
| Geode 0.5, mailbox read after every load | 250 ms | 183 ms | 400 ms | 25 | 0 | – |
Requests per sample, by method. A range means the count varied between samples. 1 method used.
| Subject | UpdateAsync | n | Counts |
|---|---|---|---|
| Geode | 1 | 25 | Same in every sample |
| Geode, mailbox = true | 1 | 25 | Same in every sample |
| ProfileStore | 1 | 25 | Same in every sample |
| ProfileService | 1 | 25 | Same in every sample |
| DataStore2, Standard saving | 1 | 25 | Same in every sample |
| Historical build | |||
| Geode 0.5, mailbox read after every load | 1 | 25 | Same in every sample |
Server hops
Live Roblox serversTwo Luau Execution tasks at once play the old and the new server. The old server loads a profile, saves 777, waits 8 seconds, then changes it to 888 (999 when it keeps the profile) without saving. Times are seconds from the new server's load call until it has the data.
Old server releases
The old server releases when the new one starts loading, the way a server releases when a player leaves for another one.
| Subject | Saw the final save |
|---|---|
| Geode | 15 of 15 |
| Geode, mailbox = true | 15 of 15 |
| ProfileStore | 15 of 15 |
| ProfileService | 15 of 15 |
| DataStore2, Standard saving | 0 of 15 |
| Historical build | |
| Geode 0.5, mailbox read after every load | 15 of 15 |
Median (the upright tick) and min to max of every sample (the bar). Medians less than one timing step (16.7 ms (1/60 s)) apart are ties.
Table: Time
| Subject | Median | Min | Max | n | Failed | Tie with |
|---|---|---|---|---|---|---|
| Geode | 0.70 s | 0.48 s | 0.88 s | 15 | 0 | – |
| Geode, mailbox = true | 0.68 s | 0.50 s | 0.98 s | 15 | 0 | – |
| ProfileStore | 5.50 s | 0.63 s | 5.63 s | 15 | 0 | – |
| ProfileService | 7.30 s | 0.68 s | 7.42 s | 15 | 0 | – |
| DataStore2, Standard saving | 0.03 s | 0.02 s | 0.05 s | 15 | 0 | – |
| Historical build | ||||||
| Geode 0.5, mailbox read after every load | 4.95 s | 4.55 s | 5.17 s | 15 | 0 | – |
Requests per sample, by method. A range means the count varied between samples. 3 methods used.
| Subject | UpdateAsync | GetAsync | GetAsync past the cache | n | Counts |
|---|---|---|---|---|---|
| Geode | 2 | – | 1 | 15 | Same in every sample |
| Geode, mailbox = true | 2 | 2 | 1 | 15 | Same in every sample |
| ProfileStore | 1 to 2 | – | – | 15 | Varied between samples |
| ProfileService | 1 to 2 | – | – | 15 | Varied between samples |
| DataStore2, Standard saving | – | 1 | – | 15 | Same in every sample |
| Historical build | |||||
| Geode 0.5, mailbox read after every load | 4 | 1 | – | 15 | Same in every sample |
Old server keeps it
The old server never releases on its own. 999 means the library got it to hand the profile over with its last change, 777 means the new server went ahead without it.
| Subject | Value seen |
|---|---|
| Geode | 999 in 5 of 5 |
| Geode, mailbox = true | 999 in 5 of 5 |
| ProfileStore | 999 in 5 of 5 |
| ProfileService | 999 in 5 of 5 |
| DataStore2, Standard saving | 777 in 5 of 5 |
| Historical build | |
| Geode 0.5, mailbox read after every load | 999 in 5 of 5 |
Median (the upright tick) and min to max of every sample (the bar). Medians less than one timing step (16.7 ms (1/60 s)) apart are ties.
Table: Time
| Subject | Median | Min | Max | n | Failed | Tie with |
|---|---|---|---|---|---|---|
| Geode | 0.73 s | 0.65 s | 0.83 s | 5 | 0 | – |
| Geode, mailbox = true | 0.72 s | 0.63 s | 1.05 s | 5 | 0 | – |
| ProfileStore | 5.50 s | 5.43 s | 5.63 s | 5 | 0 | – |
| ProfileService | 21.35 s | 21.28 s | 21.37 s | 5 | 0 | – |
| DataStore2, Standard saving | 0.03 s | 0.02 s | 0.03 s | 5 | 0 | – |
| Historical build | ||||||
| Geode 0.5, mailbox read after every load | 4.98 s | 4.62 s | 5.20 s | 5 | 0 | – |
Requests per sample, by method. A range means the count varied between samples. 3 methods used.
| Subject | UpdateAsync | GetAsync | GetAsync past the cache | n | Counts |
|---|---|---|---|---|---|
| Geode | 2 | – | 1 | 5 | Same in every sample |
| Geode, mailbox = true | 2 | 2 | 1 | 5 | Same in every sample |
| ProfileStore | 2 | – | – | 5 | Same in every sample |
| ProfileService | 4 | – | – | 5 | Same in every sample |
| DataStore2, Standard saving | – | 1 | – | 5 | Same in every sample |
| Historical build | |||||
| Geode 0.5, mailbox read after every load | 4 | 1 | – | 5 | Same in every sample |
Throttling
Live Roblox serversA burst of 160 or 161 UpdateAsync calls on throwaway keys uses up the server's request budget, then the library loads, saves and releases one profile back to back. One sample per run.
Load
Shown as a range only: the shortest and longest sample, no median.
Table: Time
| Subject | Min | Max | n | Failed |
|---|---|---|---|---|
| Geode | 32.71 s | 32.88 s | 5 | 0 |
| Geode, mailbox = true | 33.70 s | 34.10 s | 5 | 0 |
| ProfileStore | 31.20 s | 31.35 s | 5 | 0 |
| ProfileService | 31.17 s | 31.59 s | 5 | 0 |
| DataStore2, Standard saving | 31.04 s | 31.10 s | 5 | 0 |
| Historical build | ||||
| Geode 0.5, mailbox read after every load | 33.79 s | 34.11 s | 5 | 0 |
Requests per sample, by method. A range means the count varied between samples. 2 methods used.
| Subject | UpdateAsync | GetAsync | n | Counts |
|---|---|---|---|---|
| Geode | 1 | – | 5 | Same in every sample |
| Geode, mailbox = true | 1 | 1 | 5 | Same in every sample |
| ProfileStore | 2 | – | 5 | Same in every sample |
| ProfileService | 1 | – | 5 | Same in every sample |
| DataStore2, Standard saving | – | 61 | 5 | Same in every sample |
| Historical build | ||||
| Geode 0.5, mailbox read after every load | 1 | 1 | 5 | Same in every sample |
Save
Shown as a range only: the shortest and longest sample, no median.
Table: Time
| Subject | Min | Max | n | Failed |
|---|---|---|---|---|
| Geode | 1.13 s | 1.32 s | 5 | 0 |
| Geode, mailbox = true | 0.22 s | 1.23 s | 5 | 0 |
| ProfileStore | 0.18 s | 0.28 s | 5 | 0 |
| ProfileService | 0.13 s | 0.35 s | 5 | 0 |
| DataStore2, Standard saving | 1.08 s | 1.28 s | 5 | 0 |
| Historical build | ||||
| Geode 0.5, mailbox read after every load | 0.20 s | 1.23 s | 5 | 0 |
Requests per sample, by method. A range means the count varied between samples. 1 method used.
| Subject | UpdateAsync | n | Counts |
|---|---|---|---|
| Geode | 1 | 5 | Same in every sample |
| Geode, mailbox = true | 1 | 5 | Same in every sample |
| ProfileStore | 1 | 5 | Same in every sample |
| ProfileService | 1 | 5 | Same in every sample |
| DataStore2, Standard saving | 1 | 5 | Same in every sample |
| Historical build | |||
| Geode 0.5, mailbox read after every load | 1 | 5 | Same in every sample |
Release
Shown as a range only: the shortest and longest sample, no median.
Table: Time
| Subject | Min | Max | n | Failed |
|---|---|---|---|---|
| Geode | 0.17 s | 0.40 s | 5 | 0 |
| Geode, mailbox = true | 0.22 s | 0.27 s | 5 | 0 |
| ProfileStore | 0.17 s | 0.28 s | 5 | 0 |
| ProfileService | 6.98 s | 7.28 s | 5 | 0 |
| DataStore2, Standard saving | 0.12 s | 0.20 s | 5 | 0 |
| Historical build | ||||
| Geode 0.5, mailbox read after every load | 0.17 s | 0.28 s | 5 | 0 |
Requests per sample, by method. A range means the count varied between samples. 1 method used.
| Subject | UpdateAsync | n | Counts |
|---|---|---|---|
| Geode | 1 | 5 | Same in every sample |
| Geode, mailbox = true | 1 | 5 | Same in every sample |
| ProfileStore | 1 | 5 | Same in every sample |
| ProfileService | 1 | 5 | Same in every sample |
| DataStore2, Standard saving | 1 | 5 | Same in every sample |
| Historical build | |||
| Geode 0.5, mailbox read after every load | 1 | 5 | Same in every sample |
Geode edit time
Lune, Geode onlyGeode only, in Lune against Geode's test universe on the machine that runs the bench, not on a Roblox server. One loaded profile with the same data as the other suites takes batches of 1000 profile:increment("coins") calls, after one batch that isn't counted. Each edit runs through the profile's transaction pipeline and is recorded for the next save, without a DataStore request. One sample per batch.
Increment
Median (the upright tick) and min to max of every sample (the bar).
Table: Time per edit
| Subject | Median | Min | Max | n |
|---|---|---|---|---|
| Geode | 10.97 µs | 7.89 µs | 11.45 µs | 15 |
Not measured
- Lapis
- Lapis sends every DataStore request from a RunService.PostSimulation handler, and in a Luau Execution task PostSimulation, PreSimulation and Stepped never fired while Heartbeat fired about 62 times a second. Measuring it needs a server that steps physics, such as a live game server.
- Lock takeover
- Taking over a crashed server's lock waits for the lock to time out (Geode 300 s, ProfileStore 630 s, ProfileService and Lapis 30 minutes), longer than a five minute task.
- DataStore2, OrderedBackups
- DataStore2's default OrderedBackups method needs DataStores named per player, which the bench's one allowed DataStore rules out. Only the Standard method ran.
- Memory per loaded profile
- Every run still records it, but the numbers are noise. A script can't make Luau finish a garbage collection cycle, and a cycle that ends between the two heap readings moves the heap by more than twenty profiles take, in either direction. Two runs on the same build gave Geode -0.5 and 4.0 KB per profile, ProfileStore 11.3 and 0.5, DataStore2 10.1 and 21.9, so the order of the libraries flips from one run to the next.
- Edit time across libraries
- Only Geode's edits are timed. A Geode edit runs through its transaction pipeline, while ProfileStore and ProfileService change a plain table and DataStore2 calls :Update, so per-edit times side by side would compare different work.
- Servers with players
- Request budgets grow with the player count, and every task here had none.
Every run
Newest first. The results above come from the runs marked as used.
| Run | Date | Build | Results above | Data |
|---|---|---|---|---|
| 74d6b2a5 | 0.6.1 | Used | JSON for run 74d6b2a5 (opens in a new tab) | |
| 3e27444c | 0.6.1 | Used | JSON for run 3e27444c (opens in a new tab) | |
| 080eceb1 | 0.6.1 | Used | JSON for run 080eceb1 (opens in a new tab) | |
| 3abd0330 | 0.6.1 | Used | JSON for run 3abd0330 (opens in a new tab) | |
| 1faf45fa | 0.6.0 | Used | JSON for run 1faf45fa (opens in a new tab) | |
| 0ba4d5bc | 0.6.0-pre | Not used | JSON for run 0ba4d5bc (opens in a new tab) | |
| 7d89ae32 | 0.6.0-pre | Not used | JSON for run 7d89ae32 (opens in a new tab) | |
| 1b5acb20 | 0.6.0-pre | Not used | JSON for run 1b5acb20 (opens in a new tab) | |
| 0f6881b6 | 0.5.0+format | Used | JSON for run 0f6881b6 (opens in a new tab) | |
| 76954271 | 0.5.0 | Used | JSON for run 76954271 (opens in a new tab) | |
| 3561e8c8 | 0.5.0 | Used | JSON for run 3561e8c8 (opens in a new tab) | |
| 25532752 | 0.5.0 | Used | JSON for run 25532752 (opens in a new tab) | |
| 1d449b46 | 0.5.0 | Used | JSON for run 1d449b46 (opens in a new tab) |
