Merging synced data to existing types and avoiding "type proliferation"?

Not sure if this belongs in #ideas-features or not, but probably, given how it turned out after I wrote it. :grinning_face_with_smiling_eyes:

So, it’s clear that one of Fibery’s greatest strengths is in its ability to replace multiple systems, bring data from various places into one location and interconnect it, etc. But the more I do that (now with Discourse and Airtable connected, and plans to add more), the more “type proliferation” I have. And the thing that is most challenging with this is that in many cases there are overlapping intents for some component of the data, most often it is the “contact”/“user” Type. While this may be by far the most common case, I’m sure there are other examples.

So I’m wondering if there is a general solution to avoiding having a bunch of types that all store and represent similar or overlapping data. Obviously the email address is often shared, and you can auto-match on that to connect Entities together. But that doesn’t solve the “Type Proliferation”, and is also far less convenient than seeing all data in a single view, on a single Type. We don’t have a way of moving Fields between Types, and even if we did that would probably break the syncing, which I’d very likely want to avoid. What I’d really love is the option to just pick an existing Type and setup syncing of a single “type” (e.g. Contacts, Companies, etc.) from an external tool into the existing Type in Fibery.

In comparison in a system like Airtable I can import a CSV, match on Email, and “update” or “enrich” the existing rows/entities/values/fields with the new data (and create any missing entities as well). Of course I recognize things are different in Fibery, and it has its own CSV import as well. But the integration functionality has so much value, I think the kind of capability I’m describing is really needed. Anyone else have similar concerns/needs? Is there an existing way to even partly address this that I’m missing? (besides just linking the same “kind” of Type together via rules)

Can’t answer the first question, but I’m pretty sure that the only ‘fix’ at the moment is to define auto-relations that match on e.g. email address. It would allow you to see that there is related data in other apps, but is far from ideal, as you say.

It sounds like you’re after a solution where, e.g. a contact present in multiple apps exists as only one entity, shared between the places where it’s used.
Fwiw, I think with the current permissions model, it might be challenging. For example, if someone has permissions for the Airtable sync app but not for the Discourse sync app, where does the shared contact ‘live’ and what permissions should be applied?

1 Like

Yes, that’s exactly what I’ve been doing. It’s helpful, but not that helpful. :smile: It is unfortunately a far cry from the kind of “pulling it all together” that I’d like, and that I think Fibery has some promise to allow.

Well, more or less, yes, sort of, mostly. :smile: Any fields that had the same/common data would be shared, yes. E.g. Name, Email, etc. If necessary you might have to do a little jiggery pokery to make e.g. a system that uses a single “name” field compatible with one that uses first/last names (separate fields). And you might need to set one source as “canonical”, where the rest can’t write to it and only use it for matching, or just don’t write to it at all. But really the point is a field-level data sync, I think…

Mm, that’s a good point and, to my surprise, raises the already-rejected idea of field-level permissions, e.g. Customize Fields via Rules. Hmm.

So I don’t know. But the promise of it seems, to me, to make the challenges of it worth thinking seriously about, and potentially solving. If Fibery wants to truly become the unifier of data, and thus the central “sense maker”, the place where all data comes to become “sensible” and derive insight from (as seems to be Fibery’s overall ambition), then I think this will have to be a part of it to truly achieve that.

1 Like

Indeed.
And things that are really worth doing are seldom easy.

1 Like

Indeed!

One thought I had this morning that would at least help, and possibly work around the permissions issue a bit (or at least avoid having to do per-field permissions) is to work this kind of thing into the hopefully-forthcoming “Tabs on entities”/field groupings stuff. For example you could have one “Contact” Type, with 5 tabs, each one being synced with a different Integration, and each tab having its own permissions perhaps. Something like that. Just thinking out loud here…

1 Like

I’m not sure I actually defined the problem that well (or clearly) here, originally. Basically as Fibery has an increasing number of (great!) integrations, you end up with numerous systems which might have the concept of a “Contact” or “Person”, e.g. Hubspot, Fibery itself, Intercom, etc. It sucks to have all of them have totally different contact Types when a lot of the basic information is shared. Obviously there are potential issues with sync and what integration overrides the others (or do they just keep overwriting each other? :smile:), but the real desire here is to have a “single source of truth”, which might ultimately require Bi-directional integrations/sync. In any case I just wanted to clarify this since I have an increasing need for this kind of capability, as I am raising elsewhere in my posts today.

1 Like