Gödel's

Share this post

Interlude: Obsidian vs. 100,000

www.goedel.io

Interlude: Obsidian vs. 100,000

During the past benchmarks, Obsidian has crushed the competition. So let's raise the bar and lead a 100,000-page army into battle.

Alexander Rink
Jan 14, 2022
3
8
Share this post

Interlude: Obsidian vs. 100,000

www.goedel.io

Hej,

Today we’ll do a quick look into how Obsidian works with a really large amount of interconnected pages.

If you haven’t read about the Obsidian benchmark yet, you can do this here:

Gödel's
TfT Performance: Obsidian
Hej, And welcome to the Benchmark of Obsidian, made by the authors of Dynalist, based on Markdown files stored in local folders and backed by a big community with hundreds of themes and plugins. If you are new here, you may want to read about the methodology…
Read more
a year ago · 1 like · 8 comments · Alexander Rink

If you haven’t read about the methodology before, you can do this here:

Gödel's
TfT Performance: Methodology
Introduction If you want to get realistic results from performance tests, you must create natural conditions. If you're going to compare different applications, you have to use the same data. I've chosen synthetic test data over a copy of one of my personal knowledge graphs to ensure this…
Read more
a year ago · 5 likes · Alexander Rink

Importing the 100,000 pages was straightforward but took its time. While you could start working with Obsidian after a few minutes, building the complete index took more than two hours.

Here are some statistics:

  • Pages: 100,000

  • Paragraphs (Blocks): 853,409

  • Sentences: 14,555,418

  • Page Links: 1,873,962

  • Unique Links: 198,385

  • Words: 92,738,209

In the video, I show how it's like to work with such a large database afterward (it’s surprisingly well).

And just because we can - here is the animated 100,000 pages graph:

By the way, the explosion at the end is just a nice effect to cover obsidian finally crashing on my computer.

Conclusion

Obsidian is also convincing in this test; even 100,000 highly linked pages are digested by the application with a bit of puffing. The developers should think about caching the backlinks at least in the meantime and not recalculating them every time the active window changes. Unfortunately, the graph loses any usefulness with these amounts of data, since it cannot be worked with reasonably for performance reasons.


If you have any questions or suggestions, please leave a comment.

If you want to support my work, you can do this by becoming a paid member:

Or you can buy me a coffee ☕️. Thank you so much for your attention and participation.

8
Share this post

Interlude: Obsidian vs. 100,000

www.goedel.io
8 Comments
PJ
Jan 14, 2022

Just FYI, the backlinks themselves aren't being recalculated: it's reading the surrounding text for each of the references and rendering those bits as HTML in the viewer that takes the time. Even though it doesn't put them all in the DOM at once, it does populate an infinite scroller for them. If you don't have literally thousands of backlinks for a single file, backlinks load pretty much instantaneously. (As can be seen in your video, backlinks already load at a rate of hundreds per second!)

So, additional caching wouldn't speed anything up in the more common case, where you might have *dozens* or maybe even hundreds of backlinks, but not literally thousands of them for each note. They load faster than you can scroll through them if you're actually reading any of the text.

Expand full comment
ReplyGift a subscriptionCollapse
4 replies by Alexander Rink and others
Stephan Ango
Jan 14, 2022

Interesting! Can you share the specs of the computer you ran this on?

Expand full comment
ReplyGift a subscriptionCollapse
1 reply by Alexander Rink
6 more comments…
TopNewCommunity

No posts

Ready for more?

© 2023 Alexander Rink
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing