Yesterday I published relatively long article that shows how Fibery can fix many problems in product company. Your feedback is really welcome, as usual!
6 Likes
Great article.
Love the humility and self deprecation, as usual.
A few forgivable grammatical errors, as usual.
Amazing and laudable vision. Thank you for your persistence.
3 Likes
@mdubakov This article and the one about augmenting organizational intelligence is why i love Fibery!
Let me preface this reply by saying that i have a PhD in computational organizational sociology. I have been wrestling with the problems of enterprise knowledge production and organization for almost 30 years now (fuck, i’m old), both theoretically as a researcher and practically as consultant and data scientist. Currently, i am part of a consortium/network of small companies that have this article as our common framework.
Fibery is pretty close to being a “platform of platforms” for knowledge organizations, what with Chat, Transclusions/bi-directional links, and GraphQL (easier interoperability!) already in dev. I already have some posts about binding whiteboard card-connections to DB entity-relations. I’m hoping that is also in dev.
Fibery Graph DB?
The big thing i think Fibery is lacking is a view into, or reflection of, workspace data as a graph DB. I want the content of a Fibery workspace to be queryable as a graph. Data in Fibery already function well as ordinary DBs: tabular data in a relational database. But some types of problems are more easily dealt with if the data could be expressed as a graph, both in terms of visualization and tractability. The explosive growth of graph DBs and related toolchains in recent years is proof of that. I have mostly worked with property graphs and Cypher, but recently i’ve been getting into RDF ontologies.
I am convinced that a graph approach to data and information will be necessary for augmenting organizational intelligence.
GraphQL is actually not a graph query language, but an API. At least it’s not a graph query language yet. If i want to find strongly connected subgraphs in a Wiki-like DB where entities are linked to each other (either through many-to-many relations or mentions), the entities need to be backed by a graph DB.
I’m wondering if you have thought about Fibery Graph DB and if that is part of Fibery’s future. I’d be happy to pay more for our licenses to get everything connected in a “supergraph”. I see four options here (not a DB engineer, so i could be way off the mark here).
-
Fibery reflects the existing DBs in a graph DB (with two-way sync) and lets the graph DB be queryable, much like the GraphQL API allows querying the existing DBs. This duplicates the data and requires constant traffic between the two backends.
-
Fibery inserts middleware that transcodes graph queries to queries in the query language you are using for your backend. This does not duplicate data, but will drive computational load.
-
Fibery switches the DB backend to a graph DB; the old tabular DBs are replaced with queries that return tabular views. Each space gets a schema editor. I have no idea how much dev work this takes.
-
Fibery makes it easy for the users to integrate a workspace to their external graph DB, with two-way sync. This duplicates the data and requires constant traffic between the two backends.
Number 3 is, i think, the best option in the long run. Any data tables can be implemented with a property graph. My Contacts DB in Fibery can be returned as a view where each row is a Contact node and the columns are properties. Of course, this would be wasteful if my Contacts didn’t have links and relations to other entities. However, the entire point of Fibery is to get everything connected…
4 Likes
We prototyped Fibery on Graph database in 2017 and it was not good for our cases. It is extremely hard to create a performant solution for more than 10K entities if you need Types (and in Fibery everything is around Types). It is close to impossible to have Views like Board, etc with just GraphDB. We decided to went with Postgres and for our main cases it was a good idea, we are happy with the solution.
We are working on GraphQL API that will simplify querying, but this is not a Graph DB for sure.
I want to hear your desired cases, what you need to achieve in Fibery but can’t? Since we are speaking about solution now (Graph DB) and not about problems. Maybe these problems can be solved differently.
We might add Graph DB for some specific scenarios, like support bi-directional weak links better (Option 1 in your list I suppose), but we should dig into problems before that to understand the use cases.
5 Likes
It is very interesting!
May you please enumerate some examples of the problems and their graph solutions please 
Just to grasp how the problem complexity folds by graph solution and try to map that solution on Fibery if possible.
We have a Database concept as a container of structured information which can be changed during runtime. So, you can look on entities like (entity-id, field, value) which is kinda similar to limited RDF EAV triplet representation (subject, attribute, value) 
but with an ability to change structure in the scope of the current Database but not in the scope of single entity.
This is an implementation detail. That’s why we provide Entity Query API to query and change entities and Entity Schema API to query and change schema. There is no foreign keys concepts and it is possible to navigate through the relations by using collection or reference fields.
3 Likes
Pattern matching. Assume a list of Discussions that link to each other. These links are unorganized, because people link Discussions they feel are related, but each individual only participates in small sample of Discussions. I want to find Discussions that are not linked but should be. In a graph DB i can query “find all Discussions A and B where A and B are not linked to each other and where the number of Discussions that both link to is at least 3”. Pattern matching is extremely useful for identifying clusters and structures in a graph.
A similar type of query is finding maximally connected subgraphs. “Find all sets of Discussions larger than 4 where each node is connected to each other node in the set”. The real fun begins when you start chaining such queries.
A slightly more involved example:
I have a bunch of unconnected Discussion nodes in my graph. I extract keywords and add them to the graph, linking Discussions to their Keywords. Then i can link Discussions to each other based on shared Keywords. I can also link the keywords to each other with weighted relations calculated from how many times each pair of keywords co-occur in a given text. Now I can analyze clusters of Keywords and label clusters with Topics and Domains and publish these lists, so people can find interesting Discussions they might have missed. Even more interesting are the Discussions that exhibit weakly linked Keywords, because it is likely that they are bridges between Domains and between Topics.
Then, I add the participants to the graph, link them to their Discussions, and to each other with weighted relations determined by the number of Discussions shared by any two participants. I can now link Keywords to Participants. This allow me to pattern match participants to keywords and suggest Discussions to Participants based on similarity. Even more interesting is that i can invite Participants that are far apart to Discussions that need new perspectives. I can matchmake meetings between Participants based on similarity or dissimilarity depending on what we need.
@mdubakov @Andray_Shotkin This is important to me, since we are trying to build a co-creation community. EDIT: given the collaborative prod dev vision of Fibery, this should be important to you as well.
3 Likes
Betweenness. This is a centrality measure that quantifies the number of times a node occurs along the shortest path between any two other nodes in the graph. If you think about a standard org chart as a graph, the CEO will have the highest betweenness. If you map all the backend systems of the Fibery platform and how they connect to each other as a graph, the systems with the highest betweenness are the most critical for keeping Fibery running. In a corporate wiki, the pages with the highest betweenness are good starting points.
Another point is that property graphs have natural polymorphism. A node can have any number of labels (= Types) and any properties it needs to act like any one Type when queried. This completely eliminates the need for one-to-one relations: any entity of type A can be an entity of type B simply by labelling the node as such (and add the properties it needs to act as B entity). Or, if you do not like labels, link an entity node to Type node. An added bonus, that I really like, is that the Type ontology is easily mutable without breaking any old queries.
@mdubakov @Andray_Shotkin Now that i think about it, i’m surprised that you found it almost impossible to implement Board views with graphs. If you define “Open”, “In progress”, and “Done” as State nodes, then you can just link each of your Tasks nodes to one of the State nodes. A query can then return your Tasks for each State as three columns of data. And you can do the same thing with single- or multi-select fields, or relations to other entities.
3 Likes
Since the Fibery team knows Postgres, I assume you know about AgensGraph?
EDIT: Sorry for hijacking this thread!
2 Likes
It’s great to see this article and get an increasingly clearer idea of where you are trying to go with Fibery. The goals make a lot of sense, to be sure.
I think one of the key insights and realizations for me is that implementing chat in Fibery actually does totally make sense (whereas until recently I felt it would be a big distraction and waste of dev resources). I suppose I’d have to say I am probably better able to appreciate this now after having spent a year or so using Slack in a business context, as well as participating in a variety of Discord servers. Prior to that my real time discussion (i.e. chat) use was quite limited, for the most part just to people I knew (WhatsApp/Messenger/SMS/etc). And I now see that indeed people do use chat a lot to (try to) get work done. From my perspective it is often tremendously inefficient, and prioritizes immediacy and low friction over all else, to the detriment of many other factors. But… there is no denying that people work this way, and that to some degree they want to.
And yet Slack (and all other current chat systems as well) is a horrible tool for collective sense making, and an even worse one for long-term knowledge organization and maintenance. Zulip does a bit better job, at least with its particular take on “threading”, etc. But again they are both inferior to collaborative documents (e.g. GDocs, Fibery), entity-based comment threads, etc. in many respects, to say nothing of the value of structured data that is also attached to those Entities. It’s just that they (Slack and the like) take far less discipline to use them well, so even though the results are not as good, not as structured, it is “easy” and “quick” and the clarity of the downsides only comes in hindsight, for the most part. And so likely we cannot fight this tendency directly.
So what is needed is a way to connect “chat” to “real work” in far more direct ways. Not just “Create task from this Slack message” (as we already have now), but e.g. “transclude these 3 chat messages into this Entity” or “instantiate chat on this entity” or “count the votes on this chat message in a formula on this entity to derive a score”, etc, etc.
Interestingly there are other companies who are further along on this path in at least having integrated chat, but so far no one appears to be doing well at figuring out how to do deeper integration like this. ClickUp has surprisingly rich chat built-into it already for instance (and I hope you can follow their example and make Fibery chat messages very “rich” and capable!), in fact I think they have had it for some 2 years now! But when I have used ClickUp in teams, they never used the chat. Why is this? It seems to me the first step along the road of the needed unification is just to get the feature adopted. I think you know that moving people off of Slack is going to be a tough sell, so hopefully you have now (or will eventually have) some good ideas for making that happen. 
It is probably the toughest nut to crack, but getting people to use an in-app chat is likely the one integration of functionality that would have the biggest impact on adoption of any centralized work management tool. So I’m glad you also see that, and now wonder about its priority, etc… (while also acknowledging the difficulty of prioritizing the nearly unlimited backlog, and that I also just posted my own lengthy argument for prioritizing Whiteboards instead of other things 
)
I also want to point out what I think is an important nearer-term capability that will arguably be vital to achieving this long-term vision:
Assuming these “properties” could include “type”, this is critical to “sense-making” in my view. Here’s a key example within the “Product Companies” niche that is often referenced in Fibery context.
Take the now-familiar Fibery superpower of linking some random text to some entity and thus creating a “reference”. Hooray, we know this text is related to this entity. Now in your recent “How Fibery uses Fibery” video, you talk about how this process can let you quickly create an understanding of the less immediately quantitative information about product needs, etc. Very important! The problem is not all information is expressing the same “sentiment”, e.g. some may be in support of something, while other information may be opposed to it. How do we deal with this? Well, in current Fibery model I guess we’d have to an entirely new “Insight” Type with a Field you mark “support” vs. “oppose” and only link to one or the other, depending on need, etc. Then link those to your “Feature” or whatever, and calculate support vs. oppose via formula. It probably works but it’s a bit hacky.
What if instead you just had a “type” for the relationship? When you create the reference, you indicate whether it supports or opposes the entity being referenced. Better still, while in the current model you can only really either loosely evaluate the references by reading them all, or inaccurately evaluate them quantitatively by simply counting them, with relationship properties (assuming you can have multiple), you could also have a “strength” or “importance” value along with support/oppose value. Now we can evaluate the strength or importance of each reference as we go, and then on the other end - the referenced entity - it is trivial (and also far more accurate) to then quantify the data and get a numeric “correctness” or “level of support” value for that entity.
Now we bridge a huge (IMO) gap in the whole concept of insight generation, etc. because we can actually codify and act on more of the understanding and information we actually do already have in our heads while we work across broad swaths of source data. Yes, it will still not be perfect. One evaluator of claims may have a different judgement of “strength/importance” than another (hey, you can formally express that in relationships too, if you need to!), etc. But it’s a big improvement over what we have.
This is really just my take on something I read today, which I immediately connected back to Fibery and this recent article (after having some similar thoughts while originally reading Michael’s view):
Of course it gets quite complex, and it’s a question of UI/UX as well as back-end, performance, etc. Do we want to have a Relationship with a Type and Strength as well as Author? Yes, OK, and maybe we assign different “authority” to different authors, such that we can calculate the value of their data (e.g. their weighting of importance). We can filter by who created what relations, allow relations from multiple people between the same objects with different Property values, etc. Do we even want to have Properties on Properties? (probably not for a while, if at all?)
Rob Haisfield (author of the above article on data structures to facilitate decentralized knowledge synthesis) goes further, incorporating block relationships, and more:
My current thinking is that the most valuable set of primitives are basically: typed concepts, typed relationships, expressive user-generated data structures, and rules. I want to be able to write in blocks with inline data structures, arbitrary relationships between blocks, block referencing, inline querying, and inline coding.
Q- What are powerful primitives for a user of a decentralized knowledge graph
And some further depth on related topics, though more disconnected from Fibery’s central (current) use cases. The whole ScalingSynthesis.com site is quite interesting for this “second brain for teams” concept area:
3 Likes
I agree that Chat in Fibery might be a transformational feature. But:
Good chat is VERY hard to build and VERY hard to convince people to use instead of their favorite app (Slack, Telegram, etc). For example, ClickUp chats organization is totally awful. There is no good way to build good navigation for Chats in ClickUp and I believe this is the number 1 reason of failure.
You have to have a very decent mobile app on top of that.
I think it will take half of current Fibery team and no less than 6-8 months to build a good enough chat, so we are reluctant to move into this direction so far, since this is very risky. We have to build a more solid ground and polish Fibery before jumping into this novel and huge area.
A year ago I was quite sceptical about chat in Fibery, now I am convinced we should have it eventually.
2 Likes
This problem is very interesting and we are thinking about it. So far we found a very easy, but quite powerful solution… emojis. In nearest future we will allow to enhance relations with emojis, add filter, etc. So every team will be able to encode information in relations. Imagine, for a support you will just use 
and for oppose 
It is easy to get and with filtering it will be possible to aggregate number of opposing/supporting requests.
We may go further and add more META to the relations, it is relatively easy to do, but so far we want to see how emojis will work out, maybe they will solve most of the problems
1 Like
I’m sure it is. However I think the harder problem is not building it or making it “good”, it’s:
So the question is: why? Is it simple inertia/habit? Do you yet have any ideas how Fibery chat can possibly win against Slack, etc?
I agree, 100%! But… doesn’t it also seem like a fairly easy problem to solve? Or at least improve on…
OK, yes, with that long of a time estimate I have to agree with you 
. Still it also seems very important. 
I am curious to see how and when this ends up happening…
You are probably right, though I pity whoever is trying to do real work on mobile apps (even chatting). 
So… is this some kind of acknowledgement that Fibery will have to have a mobile app one day? Maybe just for chat?
Huh. What an interesting idea. It seems quite honestly silly at first, but… maybe it’s brilliant. As long as the team has agreement on what each emoji means (e.g. if you start to use ones beyond the obvious ), it could certainly have some notable value.
However… it is a clunky solution for the need for richer relationship data like what Chris was suggesting in his Relationship properties topic, specifically the ability to specify a quantity for a relationship. I was working on a project to create Recipe and bakery production management in Fibery 
for a little while, but the existing solutions were quite unsatisfying for anyone but a data nerd, in my view. 
This was to be for a baker, hardly a techie person. If they could simply choose a quantity for a relationship of ingredient to recipe, that would solve a lot of problems and avoid a ton of otherwise duplicate data and workarounds.
Now… there are possible workarounds with an emoji-based system. If you can encode multiple emoji in a Relationship, for example, and you can run formulae on them and derive the identity of the emoji (and not just a simple count), then you could do e.g. 
to encode “19” as a quantity on a relationship. It is workable but a bit clunky perhaps. I’m curious if this is one way you envision your relationship-emoji-properties being used.
2 Likes
Absolutely, at some point Fibery will have it.
We are thinking about emoji for References, not typed Relations. In your case you really need Fields for Relations. I am not sure whether/when we will go for it. Most likely not this year… First we have to nail permissions and this is the task for our core team for this year.
2 Likes
Ah, OK. Thanks for the clarification. I guess that emoji for references (which are a type of relationship, really 
) is easier to implement then, and would indeed support the basic “insight generation from free text” use case that Fibery is pursuing right now.
Came across this interesting research paper looking at the challenges and potential solutions/ideas for effective co-creation of documents/text. Lots of little gems in here (may be something Fibery team already has in their research library for the future ).
