Proposing development of a Fibery <-> Obsidian sync plugin (independent, open source)

Ideas & Features Integrations
1

I’m putting this here, though it could arguably belong in #api-programming or somewhere else. I have talked before about the possibility of developing an Obsidian plugin to sync Fibery with Obsidian (not just a one-time import), and now I think I want to see if we can really make this happen. At this stage it’s just an idea and a bit of funding, I’m not a developer and don’t have a coder selected for this yet, so definitely seeking developer and funder support, as well as just ideas and feedback. I will expand this post with more based on discussion below, but I wanted to hopefully get the conversation started and not procrastinate it any longer. :smile: So to begin with, here are some summaries of ideas and features, etc.

:bulb: So why do this, you ask? I think there are actually a number of compelling reasons!

First and foremost it could be a form of automated backup and “offline support”, at least for Rich Text/markdown content. It also gives access to a wide variety of experimental Obsidian plugins and features, many of which will likely never make it into Fibery natively, but which could be used on the text data non-destructively, augmenting Fibery’s capabilities. The thing to keep in mind, of course, is that a basic initial implementation would probably be restricted to simple markdown .md content. However in the future it may be possible to sync via CSV and access even more capability!

An Incomplete List of Whys

  • Automated partial offline backup via .md export and maintaining links (better than current bulk .md export?)
    • Because Obsidian’s native format is primarily .md, once synced into Obsidian, any number of other tools could operate on the same data, which could then get synced back to Fibery
  • Immediate access to mobile text editing support (e.g. Obsidian mobile, but other mobile .md editors too)
  • Fibery users could to some degree take advantage the huge array of Obsidian plugins to operate on (some) of their Fibery data
    • E.g. Obsidian’s Canvas view is superior to Fibery’s Whiteboards at present, so I could just create a Canvas in Obsidian that references .md files that translate into Fibery docs/entities
    • Get the benefit of unique, likely never-to-be-supported tools like Strange New Worlds
    • Or in the case of the many import sources Obsidian can handle that Fibery doesn’t yet support, Obsidian could be a “bridge” of sorts
  • Custom visual themed viewing of Fibery text content, for those who like fancy themes :smile:
  • Additional “publishing” options for Fibery content without having to do manual exports, fiddle with Fibery API, etc. (there are numerous turnkey Obsidian publishing options with custom theming, etc.)
  • Fibery team can focus on their core market, while still gaining interest and userbase from broader PKM market, etc.
  • Etc. (more to come)

Implementation

This will likely be an Obsidian plugin using Fibery API (assuming this is possible). I would rather not rely on e.g. Fibery’s long-term support of their “Filesystem Sync” system that they experimented with, although that could be another valid route to making this happen, if they are willing to commit to finalize and offer long-term support for it. I assume the Fibery API has long-term support already, even if the Obsidian plugin has to be updated from time to time to maintain compatibility.

Currently I have no developer in mind, but any devs in the community are invited to chime-in with their thoughts, proposed rates, etc.! @Matt_Blais is this something you’d be capable of and interested in working on? Obviously if you’re not already familiar with Obsidian’s API that’d take some ramp-up, but I hear it’s pretty friendly.

Proposed Features, etc.

  • Automated, scheduled, and manual syncing of all Rich Text data from Docs and (ideally) Entities in Databases
    • Option to only sync one way
  • Maintain compatible file/page/entity names bi-directionally
  • Mechanisms for handling in-line linking (“References”) that are as compatible as possible with both Obsidian and Fibery, e.g. plugin may be “Fibery aware” and allow dynamic search+link of Fibery pages/content
  • Sensible handling of tool-specific functions, e.g. Fibery “Link text to Entity” may become text link to URL in Obsidian/markdown, vs. Entity Reference (# link) as simple [[Page Name]] link in Obsidian, etc. (details to be determined)
  • For unsupported data types
    • In Obsidian it would sync as a .md file with contents being a fully qualified URL to the target content in Fibery, e.g. . | Fibery
    • In Fibery the same except to Obsidian URL, e.g. Canvas in Obsidian would show up this way in Fibery: obsidian://open?vault=VaultName=FolderName%2FFileName.canvas
  • More to come

Funding

I have about $1000 USD to contribute to development right now. I may have more in the future. I’m hoping others will be willing to donate to this as well. The resulting code will be open source forever, license to be determined but we can decide collectively.

I might use OpenCollective by default, but I am happy to have Fibery team be the funds custodian if preferred (and they are willing), or to make some other arrangement if enough people want to contribute to the plugin effort and have a group preference for some other funding/custodial platform.

Let me know what you think! Are you interested in using this? What are your use cases?

P.S. I’m also considering posting this on the Obsidian forums, but Fibery awareness there is fairly low vs. e.g. Notion. That said there is a lot of interest from people in Notion-like functionality, so I do think if that conceptual connection could be made (i.e. "Hey Obsidian folks, Fibery is Notion-like but better, what if we made a sync plugin with it?), it could gain some interest. But I think for now there may be more core interest here, we’ll see.

2

An Obsidian + Fibery combo would be the best of both worlds for me. I would be willing to contribute a few thousand dollars for a well done integration.

Features I would be looking for:

  • Automatic syncing after a change from either end.
  • Fibery DB type = tag in Obsidian
  • Entity fields/columns in Fibery = frontmatter/inline dataview fields in Obsidian
  • If I tag a page in Obsidian the Fibery entitity fields are automatically added to the page and I can choose whether they are frontmatter or inline by default
  • Create a new DB/type in Fibery when a new tag is used in Obsidian.
  • Add new field/column in Fibery type if I add a new field in Obsidian and propagate the field to all existing Obsidian pages with that tag.

There’s more I’m sure, but I’d like to see what others have to say and think more deeply about how it might work.

4 Likes
3

Thanks for thinking of me @Oshyan! It does sound like an interesting project, though I’m not familiar with Obsidian. But unfortunately I don’t currently have the time available to to justice to a project like this :worried:

2 Likes
4

Fantastic, thank you! Would you be interested in a progressive contribution based on a phased project, milestones, etc? E.g. Phase 1 is basic .md content sync, no databases, no Dataview, etc. Phase 2 is basic database sync. etc., and each phase you could contribute some portion of your total? Or are you only interested if it is as extensive an integration as you outline above (i.e. all features in your bullet list and more)?

Some great ideas here, getting into some aspects I hadn’t considered more deeply yet. Love it!

No worries Matt! Just thought I’d ask. :grin: If you happen to know any other devs with Fibery experience who might have interest let me know.

I’ll look over the Fibery Partners page and do some more research, but perhaps @Dimitri_S might have interest? I’m also going to work on how to present this idea to the Obsidian community. Hopefully with the relative ease of Fibery API, an experienced Obsidian dev could do just fine with it, even without prior Fibery experience. My main concern would be the data model and translating concepts/data from one to the other, which I think requires some understanding of Fibery’s way of doing things vs. Obsidian.

But this is a promising start!

5

Chiming in.
@Oshyan sounds like an awesome idea. Do you have any clients interested in that use case directly ?
I’m concerned with open source license with amount of funding provided.
We made toggl integration free and open source but there are plans to extend it with paid tier.
I’d consider similar model for this one instead of free forever for any extension of basic functionality.

1 Like
6

I would be willing to progressively contribute. The milestones would need to be discussed and agreed upon beforehand.

1 Like
7

Hi, that’s an interesting idea. I’m also both a user of Obsidian (perso) and Fibery (pro).

I’m still wondering what are the first use cases you’d like to address with such a plugin.

If that can be helpful, I can share what I did at the moment (similar to, but not quite what you are describing).

I don’t have too much time to code in a plugin at the moment (+ javascript is not my cup of tea) but I’d be happy to provide some help if a developer has questions about these 2 pet projects.

1 Like
8

@Oshyan wanted to check if you are aware of current limitation of fibery sync api. It’s one way only meaning that ANY content changes made in Fibery directly to obsidian data will be lost.
The following should be preserved:

  • New fields/formulas added to existing data entries from Obsidian
  • Relationships in fibery
  • Views and ranks in them in fibery

I do remember @mdubakov mentioned that one day - we will get two ways syncing support. Not sure when is this one day coming though.

1 Like
9

Just to clarify - it is possible to create custom apps that will ‘push’ data to external sources, but this manifests as external actions available in automations.
There is no automatic mechanism for always updating the external data when data is changed in Fibery, AFAIK

4 Likes
10

There is something used for Jira. I don’t believe it’s using external actions but some-kind of triggers instead.
External actions can be a potential solution combined with rules to update certain things in Obsidian but it’s quite a clunky solution TBH.

3 Likes
11

I do not have anyone on my side besides me who is interested at this time. However it may well be something that I would recommend to people if it became available, strengthening the use cases for either Obsidian or Fibery respectively (both of which I recommend but both of which also have concerns come up from users due to various limitations that would be partly or wholly solved by syncing them).

Funding is not wholly determined at this point, we’re still gathering a sense of who will contribute and how much. Until a formal effort is launched, e.g. on OpenCollective, it’s hard to say how much might be available. But to be fair it will likely not be a huge amount.

What are your specific concerns about the licensing?

Yes, absolutely!

For me the minimum useful capability would be sync of Rich Text/Document content (including from Entities). For me this provides backup of a reasonable amount of my content, though far from all, and also allows me to do fast, local, offline search, local LLM AI operations for privacy/cost savings, etc.

That said the initial milestone/MVP of this is open to discussion and should be planned by all interested parties who would use the plugin, IMO.

Thanks, that’s good to know! That 2nd tool seems particularly interesting just for my own use as well.

My assumption was that it could be a simple periodic sync like most others Fibery currently has, but do it bi-directionally. Is that possible given current options?

12

Maybe there is a misunderstanding here. Let me elaborate on our current business model.
Taking for example the Toggl integration.
It’s code is open source and even hosted by Reify Academy. Can be used by anyone free of charge. However, that is basic version of toggl integration only.
For advanced version with extra features we will be charging monthly subscription fee AND source code won’t be published.

For custom integration for our clients we use similar model - however as client is essentially sponsoring initial stage of development they are given X months to use the premium version for free.
For custom integration open source version extracted from premium.
A client(in case of open collective - clients) receives support and pro-rated usage of premium version NOT the source code for integration.
How does that sound ?

2 Likes
13

I understand your approach, yes; it’s a fairly common one in my experience and can certainly work well. But I wanted to understand your objection to/concern with a fully open source approach (all features open/free). Are you just saying that you feel some ongoing revenue is needed to sustain development and make it a long-term viable project? And in that case that you suggest this “hybrid” open/premium model as a way to do that?

14

Spot on. The main cost of software is not in development, maintenance overtime adds up to a lot more.

OSS projects in general are notoriously bad in getting any level of funding, even popular ones. One famous recent example was a log4j bug.

So yes I’m thinking about what happens after the initial version.
Either we will continue to extend it with growing interests from clients willing to use it
OR
Community can continue extending it given it becomes popular.
Both of those things could happen in theory which would mean win-win for both parties.

The last thing I want is to create something and abandon it cause it takes too much time and energy to maintain for free.

I did find this interesting example of paid plugin in obsidian. Very similar idea.

3 Likes
15

Yes, just as I thought. I would say I’m open to the possibility of OS + premium (paid) features, especially if an interested dev is inclined to go this route. However a majority of successful Obsidian plugins are not commercialized this way, so I would be concerned about compatibility with the general Obsidian community ethos. These concerns - longevity/maintainability and reception by the respective communities - need to be balanced.

It’s my hope as well that a more direct integration between Obsidian and a true database tool like Fibery would perhaps start to make that tool the de facto choice for those who want additional DB capability. Currently I think this is mostly Notion due, I think, to its higher profile and ease of use. I think Fibery has a stronger case for being a good Obsidian companion, even without a direct sync, but especially so if that can be developed. So it would be my hope that a smaller group of enthusiastic users kick-start the dev of this, which then helps prove its value to Obsidian people, which gains more users, and thus helps create long-term support for the project through dev contributions, ongoing voluntary funding, etc.

Note that Open Collective specifically has ongoing (e.g. monthly) contribution options precisely to try to address the kind of long-term maintenance concerns you have. E.g. someone can pledge $1200, but as $100/mo for a year rather than in a lump sum.

1 Like
16

i wrote a workflow in n8n to sync obsidian files from Google Drive to Fibery, extracting YAML and putting all in the corresponding fields, and the whole document as MD in a rich text field.

its not a two way sync, but a one way from Obsidian to Fibery. it probably works with other file storage systems as well

the javascript that picks apart the Markdown can be adapted to work on a button inside fibery too i guess? it looks like this

var mdData = $json.data;

// Split the markdown content into sections
const sections = mdData.split(/(?:\n---\n|> \[!Meta\]\n)/);

// Initialize variables for extracted data
const extractedData = {};

// Check if a frontmatter section exists
if (sections.length > 1) {
    // Extract data from frontmatter section
    const frontmatterLines = sections[0].split('\n');
    frontmatterLines.forEach(line => {
        const match = line.match(/^\s*([^:]+):\s*(.*)$/);
        if (match) {
            const key = match[1].trim();
            const value = match[2].trim();
            extractedData[key] = value;
        }
    });
}

// Extract data from body section
const bodyLines = sections[sections.length > 2 ? 2 : 1].split('\n');
bodyLines.forEach(line => {
    const match = line.match(/^\s*([^:]+)::\s*(.*)$/);
    if (match) {
        const key = match[1].trim();
        const value = match[2].trim();
        extractedData[key] = value;
    }
});

// Extract data from meta section
if (sections.length > 2) {
    const metaLines = sections[sections.length > 2 ? 3 : 2].split('\n');
    metaLines.forEach(line => {
        const match = line.match(/^\s*([^:]+):\s*(.*)$/);
        if (match) {
            const key = match[1].trim();
            const value = match[2].trim();
            extractedData[key] = value;
        }
    });
}


return(extractedData);

the file is a bit more complex than it would need if data is kept the same throughout the vault. but mine are a mix of frontmatter YAML and “postmatter” callouts

3 Likes
17

Hi Oshyan,

I previously used Obsidian and I had one of my teams develop custom plugins for me. It was a love/hate relationship for me and while ultimately it was a good learning experience but a significant drain on my brain (and I’m recovering from a brain injury, lol).

I got to a point of mastery - though not mastery of all the programming.

I’ve found that Fibery is so much more intuitive and amazing, and while the are some shortcomings, I do miss the development community.

What is it you would want to use both for versus just Fibery? Is it a privacy thing? I know the majority of Obsidian users are highly focused on that.

If there is a development community, or if not and one can be started, I will volunteer time, money, and resources. I’ve only just start bringing over some team members, but right off the bat this tool is illuminating for me unlike any other I’ve used and I have used everything.

1 Like
18

for me some reasons to use obsidian:

  • local and offline (this really improves speed and also mobile use. nothing is more of a kill than having no access to a note when wanting to add an idea. one of the main reasons i will quit the clickup journey, as i lost hours of work with a lack of reliable offline access ((editing documents messing stuff up)))
  • keyboard speed (its unbelieavable fast to edit documents with just keyboard never to click anywhere. )
  • linking (having the ability to create links to documents with cmd+L … this should work in fibery too, but as its in the browser cmd+L activates the brwoser bar)
  • task management. its so easy to create inline tasks and have these accumulate at other places.
  • document first; dataview later (obsidian has all the related information in the document. there is no framework in how i need to lay out the info on the page. it can be structured or unstructured. no sidebars and hidden options - all in plain sight. but when i need a table view, i have the option to get it all with dataview later on. this is perfect to work with my thought process)

resons to use fibery (over obsidian):

  • online, team capable (obsidian is not usable for teams at all. because of the unstructured way of things and the easyness of editing and destroying anything)
  • structured data (as much as i dont like to be in a corset, in a team its important that everything follows rules

so using them both can give me an offline experience that is fast. i can setup my templates and go about my day as I did the past three years. then it syncs to fibery and those informations that need to be in the team pop up at the right place in the right format.
of course - a fibery offline mode and mobile mode would solve a lot of these hickups and would probably lead me to adopt fibery a lot more. currently its still clickup (becuase of their mobile tool and team sharing capabilities)

1 Like
19

Something to add here:

As a very new plugin for Obsidian just came about there is yet another reason to really dig into the sync options:

khoj uses LLM or chatGPT to query and talk to your whole second brain. start a conversation with your vault is mindblowing. it would be amazing if fibery would also allow to access the whole information accross all DB and documents to write new documents, forms, pitches etc. but for now khoj is a way of doing this. still very early dev but really amazing already

2 Likes
20

I’m pretty confident that it will at some point be possible to chat with AI in Fibery where the information available to the AI includes the content of the workspace.
It’s a natural progression from semantic search I guess.

3 Likes