Interlude: Obsidian vs. 100,000
Interlude: Obsidian vs. 100,000
During the past benchmarks, Obsidian has crushed the competition. So let's raise the bar and lead a 100,000-page army into battle.
Hej,
Today we’ll do a quick look into how Obsidian works with a really large amount of interconnected pages.
If you haven’t read about the Obsidian benchmark yet, you can do this here:
If you haven’t read about the methodology before, you can do this here:
Importing the 100,000 pages was straightforward but took its time. While you could start working with Obsidian after a few minutes, building the complete index took more than two hours.
Here are some statistics:
Pages: 100,000
Paragraphs (Blocks): 853,409
Sentences: 14,555,418
Page Links: 1,873,962
Unique Links: 198,385
Words: 92,738,209
In the video, I show how it's like to work with such a large database afterward (it’s surprisingly well).
And just because we can - here is the animated 100,000 pages graph:
By the way, the explosion at the end is just a nice effect to cover obsidian finally crashing on my computer.
Conclusion
Obsidian is also convincing in this test; even 100,000 highly linked pages are digested by the application with a bit of puffing. The developers should think about caching the backlinks at least in the meantime and not recalculating them every time the active window changes. Unfortunately, the graph loses any usefulness with these amounts of data, since it cannot be worked with reasonably for performance reasons.
If you have any questions or suggestions, please leave a comment.
If you want to support my work, you can do this by becoming a paid member:
Or you can buy me a coffee ☕️. Thank you so much for your attention and participation.
Just FYI, the backlinks themselves aren't being recalculated: it's reading the surrounding text for each of the references and rendering those bits as HTML in the viewer that takes the time. Even though it doesn't put them all in the DOM at once, it does populate an infinite scroller for them. If you don't have literally thousands of backlinks for a single file, backlinks load pretty much instantaneously. (As can be seen in your video, backlinks already load at a rate of hundreds per second!)
So, additional caching wouldn't speed anything up in the more common case, where you might have *dozens* or maybe even hundreds of backlinks, but not literally thousands of them for each note. They load faster than you can scroll through them if you're actually reading any of the text.
Interesting! Can you share the specs of the computer you ran this on?