Obsidian does have support for block references though, it just doesn't create them by default (i.e. always). So I wonder if there is a way to sort of simulate the burden that the other apps are dealing with by modifying your .md test files *just* for Obsidian to include block references on every, er, "paragraph" I guess it'd be. You know what I mean? 🤔
Obsidian actually does internally track every "block" in every file, along with list items and tasks and headings and links and whatnot. All are indexed live as you update files. You can see this if you type `[[some note#^^` and you will get suggestions for which paragraph you want to link to.
That's true. But is that the same thing as what Roam, etc. are doing? There does appear to be a difference in Obsidian between being able to find some piece of text with e.g. a header or paragraph link search, vs. the actual block ID that gets assigned if you complete the link. I'm not confident there is a performance difference, but there is at least some kind of functional difference. So that's what I'm curious about testing, though I'm unsure whether it would be worth the effort of adding the block ref creator in the tools used to generate these files.
I don't know what you mean by block ref creator, but Obsidian will accept any unique-within-the-file alphanumeric string as a block ID, you can literally generate them at random or just use ^1, ^2, ^3 as far as I know.
As I said, though, Obsidian does actually index every block *and* heading in every file, in the sense that it has in-memory data for "a markdown block of this type exists at this position/length within this file, and (optionally) has this ID". Thus, if you link to a paragraph, list item, or other block with an ID, it jumps directly there without any searching, and can also embed (transclude) the block in another note.
Heading links are also indexed, which is why Obsidian can know (again, without reading or searching the file) what range of characters in the file corresponds to "the section under the heading of the given name". These are some of the things that make it so fast, e.g. for hover-previewing links to such sections or blocks -- it doesn't have to parse and render all the markdown in the file, just the referenced part.
There are some small aspects of what you're saying that don't quite align with my understanding. However broadly it seems you are correct. It is just a bit hard to believe that Obsidian and e.g. Roam, Logseq (local as well), etc. are truly doing the same work, and Obsidian is just *that much* faster. Part of the reason I say this is that Obsidian is not considered a "true" outliner, even with the outliner plugin(s).
Whether this is simply a misinformed opinion, or based more on semantic argument (Obsidian is not a *strict* outliner, therefore not a "real" one), I don't know. But it has been strongly suggested that those who favor outliners miss certain features in Obsidian that "real" outliners have, and my understanding was that often centered around block-based functions. Are there not real differences between how Obsidian and Roam actually work?
That said I'm not as knowledgeable as you seem to be about the inner workings/behind the scenes. I'm more going off of the comments of others previously, and comparisons others have done between e.g. Obsidian and Roam. I'm an Obsidian user, I couldn't deal with the restrictions of outliners, so I'm definitely not advocating for Roam or any other tool over Obsidian. 😄 Just wanting to make sure that the benchmark results for Obs are "fair" and *truly* comparable. If they are, it's very impressive indeed.
I have 20,000 notes with lots of attachments and have found that renaming the top level folder breaks obsidian. It also cannot use the knowledge graph without just breaking. I have to force quit and then it has to re-index everything.
Obsidian does have support for block references though, it just doesn't create them by default (i.e. always). So I wonder if there is a way to sort of simulate the burden that the other apps are dealing with by modifying your .md test files *just* for Obsidian to include block references on every, er, "paragraph" I guess it'd be. You know what I mean? 🤔
Obsidian actually does internally track every "block" in every file, along with list items and tasks and headings and links and whatnot. All are indexed live as you update files. You can see this if you type `[[some note#^^` and you will get suggestions for which paragraph you want to link to.
That's true. But is that the same thing as what Roam, etc. are doing? There does appear to be a difference in Obsidian between being able to find some piece of text with e.g. a header or paragraph link search, vs. the actual block ID that gets assigned if you complete the link. I'm not confident there is a performance difference, but there is at least some kind of functional difference. So that's what I'm curious about testing, though I'm unsure whether it would be worth the effort of adding the block ref creator in the tools used to generate these files.
I don't know what you mean by block ref creator, but Obsidian will accept any unique-within-the-file alphanumeric string as a block ID, you can literally generate them at random or just use ^1, ^2, ^3 as far as I know.
As I said, though, Obsidian does actually index every block *and* heading in every file, in the sense that it has in-memory data for "a markdown block of this type exists at this position/length within this file, and (optionally) has this ID". Thus, if you link to a paragraph, list item, or other block with an ID, it jumps directly there without any searching, and can also embed (transclude) the block in another note.
Heading links are also indexed, which is why Obsidian can know (again, without reading or searching the file) what range of characters in the file corresponds to "the section under the heading of the given name". These are some of the things that make it so fast, e.g. for hover-previewing links to such sections or blocks -- it doesn't have to parse and render all the markdown in the file, just the referenced part.
There are some small aspects of what you're saying that don't quite align with my understanding. However broadly it seems you are correct. It is just a bit hard to believe that Obsidian and e.g. Roam, Logseq (local as well), etc. are truly doing the same work, and Obsidian is just *that much* faster. Part of the reason I say this is that Obsidian is not considered a "true" outliner, even with the outliner plugin(s).
Whether this is simply a misinformed opinion, or based more on semantic argument (Obsidian is not a *strict* outliner, therefore not a "real" one), I don't know. But it has been strongly suggested that those who favor outliners miss certain features in Obsidian that "real" outliners have, and my understanding was that often centered around block-based functions. Are there not real differences between how Obsidian and Roam actually work?
That said I'm not as knowledgeable as you seem to be about the inner workings/behind the scenes. I'm more going off of the comments of others previously, and comparisons others have done between e.g. Obsidian and Roam. I'm an Obsidian user, I couldn't deal with the restrictions of outliners, so I'm definitely not advocating for Roam or any other tool over Obsidian. 😄 Just wanting to make sure that the benchmark results for Obs are "fair" and *truly* comparable. If they are, it's very impressive indeed.
I have 20,000 notes with lots of attachments and have found that renaming the top level folder breaks obsidian. It also cannot use the knowledge graph without just breaking. I have to force quit and then it has to re-index everything.
I'm already an adoring user of Obsidian, but reading this and the above comments made me even happier. Thanks!