Bi-directional integrations/sync

@Chr1sG it is not about migrating or replicating other tools in any way.

It is about having a single workspace for users where they can use all of the strengths of Fibery and use and update some legacy data in one place (the Fibery workspace). We are talking about data that will never be migrated, but may be used inside Fibery.

The Integrations API is so close! Just adding an update to it will allow devs to do with it whatever their use case requires. Please don’t dismiss this so easily - it will be a huge feature and I think it is worth discussing.

1 Like

@Chr1sG I’m happy to take the discussion offline if you want to drop me an email, but I really think extending the Integrations API is the right play here.

Maybe we misunderstand each other.
I get that this request is about using and updating data in Fibery (without migrating) but the point is that Fibery is not currently aiming to serve the users who have this very broad need.

@Chr1sG all we are trying to do is what the Fibery home page promises:

All under “one roof”:

  1. New flexible workflows; and
  2. Legacy data that needs to be maintained.

I think the intention of the ‘under one roof’ is to imply that you don’t need to maintain multiple tools and that ‘everything’* lives in Fibery, not that Fibery aggregates all your data in existing/legacy tools.
*Everything = everything a product manager might need.
If there are product management processes that are not currently served well in Fibery, then we will try to add features so that they are.

Anyways, I’m merely trying to set expectations for users in the community, and it’s just not likely that we will devote significant effort to improvements to the API.

I think we touch upon Fibery’s long term product goals and strategies, and a tension where Fibery does not yet have a platform or process for users to have overview, insight or organized and structural input in such high level goals and strategies, or at least high potential or high risk feature development.

For that reason I see users limit their feedback to more cosmetic changes, because these are more likely taken into consideration. This is a common dynamic in growing businesses where resources are limited and taking up on a big feature can actually be a vital mistake.

I think the least resource intensive step here is that a dedicated space and processes are created (not just a forum category) to make better use of senior/experienced Fibery users that may colleborate together to improve collective intelligence of the Fibery developer/user ecosystem. This will keep such users engaged and contribute as part of the collective intelligence of the Fibery development ecosystem.

The quesion is who is willing to initiate this, since at least one or a few people need to feel called to put effort in kickstarting a Fibery community ecosystem that does not drain, but actually supports the core team in insight and work.

3 Likes

For those who wish to have fibery → calendar sync, I’ve created a tool which can serve a fibery calendar as an ICS file!

Now you can use Fibery as the source for your calendar data!

3 Likes

Hi Fibery folks! Just wanted to see where things are at on this/any potential timeline on a release of either two-way hubspot (see below) or gcal. Thanks!

I think Yuri’s response touches on a very good point and direction.

The very latest developments on often (even purposedly) very opinionated closed-source platforms, applications and services is going extremely fast. The competition is obviously huge.

But given the fact that proprietary monopolies come with their own, mostly greed- and subscription-related issues; as seen with the adobe suite, autodesk/maya etc. and (thankfully) matured OSS applications that manage to fill in the gap of economic problems, not only to compete, but often surpass their closed-source competition… one should be aware that even in this specific “nocode” “niche”… a lot is happening on the OSS side. And it is happening fast. It simply fills or compensates for classic/typical issues and problems with many proprietary products.

Priorities and preference might always differ and remain subjective: But I truly think that Fibery’s main focus inevitably has, and should be focusing on native integrations. They have to be feature-complete and bi-directional to provide the full benefit of integrating with other tools. And as Yuri suggests where that simply is not possible, full effort should be focused on documenting, elaborating or even collaborating with advanced community users to potentially leverage some contributions or “extensions” of existing integrations or functionality.

Some serious thought or pilots should also be put in any functionality, decoupling or possibility to leverage advanced community users or creating tech ecosystems/functionality.

With Fibery’s newest Linear integration I’ll try to make a very current and practical example that touches on some of the newest, hyped platforms in a structurally rather challenging and complex field:

  • Github is extremely legacy and in many things just “work” but from user pov feels like abandoned tech-debt, basically. Compared to Gitlab they have unpaywalled “Milestones” for at least some structure and “Project Management”. But it’s all just legacy, tacked ontop… the structure, interface and clicking back and forth all over is basically un-usable.

  • Gitlab has the advantage of having a more modern codebase alleviating some of these issues. They started having some basic model views as well, no idea since when exactly. However they know about GH’s limitations, and reap/price the remaining influx/demand “generously”. So Milestones or more structued, PM functionality are rather expensive and paywalled.

  • Let’s look at Linear now, the newest and very popular direction and platform. It leverages the demand and lack of usability, structural tools that both GH+GL have. There is obviously a demand. I mean it’s 2024 after all. Who wouldn’t want to have some sort of at least basic “Task + Subtask” relation and structure to at least have a chance to organize and overview without constantly clicking over 3 functions, across 5 screens about perceived 27 times per single task… to do anything. It’s actually surprising to see twhat lackluster state of software and services is in our supposedly “modern” and “advanced” civilizations. The reasons for those massive problems, are very obvious, but another topic.

So let’s look at not only the Integration, API, Webhook challenge… but at a little “structural, practical and functional comparison”:

  • GH has milestones + issues. But basically unusable in practice and interface. (Strong focus on labels in any application btw, is imho also strong legacy symptom in desperate attempts to deal with these legacy deficiencies. Not only in SW-dev, but many archiving or searching applications)
  • GL will give you those fancy, little milestones + issues to somewhat structure/plan. But they will come at a price. IIRC they are called “Epics” in GL-speak.
  • With such limited competition, pricing and functionality it is only obvious that another service arrives to cover such obvious needs. Linear comes into play to alleviate those very specific, structural and organizational problems.

I hope you are sitting tight. Because Linear comes not only with Issues + Sub-Issues, but with a whopping number of other containers and structures like: Projects, Milestones, Cycles (for recurring/incomplete issues) aaaand… Initiatives! Which are meant to drive efforts across several projects, maybe even departments or organmizations. Who knows.

Don’t get me wrong; I am very excited and love the practical, modern direction of what Linear is trying to improve or solve. But it is indeed strongly opinionated. And afaik they do so willingly, by purpose to streamline problems more legacy services have.

All of this initial comparisons and brief tests, evaluations should be taken with a grain of salt: As it’s only been a couple days. Let’s breakdown some structural and process insights (AFAIK):

  • GH + Linear have Milestones. I might be wrong but so far, but they are actually not the same, fully decoupled. Probably due to API/Legacy limitations. Linear simply skipped GH’s only structural mechanims/container… entirely. Afaik GH vs. Linear Milestones are not synced at all. Basically they are different things. This decision or circumstance to me makes somewhat sense, yes. But down the line it actually leads to problems when trying to somehow sync or manage both to a third platform or service, or obviously: Fibery. Because now you have two kinds of Milestones in two Workspaces, API-Setups or Integrations. But they are ultimately… completely different things.

  • I’d argue that despite very fast and modern interface Linear’s Initiatives, and even Projects… will also be less used in terms of practicability, speed and needs. Cycles, recurring, daisy-chained issues surely are probably the most interesting for many, even moderate teams and companies.

  • Personally I argue that in practice simply using “Issues + Sub-Issues” is the fastest, most efficient way. This seems to be confirmed in usage/preference by public usecases and company/team insights. Issues translate well to fibery, at least in current Integration state. As unlike most other container-types synced to Fibery, they have a property called “Original URL”. Which is basically a requirement for single-click access to Linear entities, from within Fibery’s one-way sync. I’d argue they are also essential to reliably remove leftovers and archived orphans from within Fibery’s Linear View… as Linear’s “Trashed” state does not seem to be enough to exclude all of Linear’s internal Entity/Archiving states reliably.

  • Admittedly within all these tests and evaluations I didn’t have a chance to look at all low-level API and schema situations. However I suspect that Linear implemented Issues + Sub-Issues via their linking functionality and schema. Therefore aside from linked state or relation both belong to the same data-type. Might be a “user problem”, but from initial tests that prevents any sensible way of native self-relations/grouping in Fibery. As the integration already syncs or delivers them in self-related state.

*** On a (yet another) sidenote I would love to be able to leverage Fibery’s self-relations for visual clarity, separation or possibly even leverage content display. For instance: The “Line” size in board view seems great in it’s condensed form. Without self-relation, duplicate/separate entity display the name/title text gets drowned. Self-relations could be used to have a separate size for a separate title/name box that allows full text, i.e. a separate “small” or “medium” display size. While all other icons or fields could be kept in a condensed, line display size. Generally a wide content or view setting would be great to further leverage line-size. As despite collapsing all unneeded columns will not expand the cards’ widths enough. (Even if there is more than enough room)

Anyways.

  • As indicated somewhere somewhat hidden comments were not included in Linear’s integration sync. Understandably in terms of scope. But they are rather essential to potentially keep up to date. Especially with comments (amongst others) lack of bi-directional sync becomes rather obvious. In General faster, recurring sync intervals and/or manual sync automations or button-actions for editor/user roles also becomes a necessity. Some things can be done via One-Way sync leveraging Linear’s “Original URL” to click and edit/write. But for users in Fibery view it would mean waiting for up to an hour to verify/notice any changes or additions.

  • Ultimately the last, and probably most essential missing piece in this GIT/Dev context would be to include required PR/MR, commits, Open vs. Closed Code-Diff Review states and discussions. Those are surely the hardest to implement and depend mostly on Target system’s API exposure most before all. However I am pretty sure (IIRC) that at least in Gitlab terms, those should be exposed.

So now to the conclusion after this whole situation and analysis. What does one do to potentially extend, fix or implement missing requirements from a admin/user-perspective?

Afaik there are:

  • Webhooks (?)
  • Potentially modern/new “Synced” Actions? wip/backlog?
  • API/V1
  • afaik rather obscure, undocumented/unofficial API/V1/Sync
  • API/V2?

Most of which are not exactly/fully, or sometimes not not at all officially documented in practical, bi-directional CRUD context.

Initial questions that immediately arise:

  • What makes more sense, or is possibly “best-practice”? Implement/Sync towards integrated DB’s, in separate DB’s within workspace? Or other options?
  • Is it possible to extend/fix existing, official integrations.
  • How to handle and implement (if hopefully possible) missing functionality via v1/v1sync/v2 or webhook API’s. I also believe the case of external Webhooks linked incoming to Fibery is not well or not at all documented.

What is the suggestions. Or what’s even realistically possible in first place?
To get back to Yuri’s impulse; What else could be done to include, leverage and enable advanced community users or responsible company roles to improve, or possibly even practically fix that sort of problems.

I might have misunderstood, or there might be some confusion on my end in terms of recent blog posts or recent history, future plans. Maybe the conclusions are actually similar. Imho focus should remain or most probably be emphasized on lower levels and tech-users. It must be extremely hard and some of the most difficult challenges to decide and weigh decisions, directions also in light of bare economical necessities in such complex and challenging context. Even just from external user perspective it can already be daunting when delving deeper into involved, functional structures and dependencies.

However the high-level intent or proposed solution in terms of ideology, direction, opinionated streamline towards financially promising fields or departments imho simply does not exist or work out efficiently, or not at all. After all there are many good, valid and probably even necessities for separation of concerns between users and more technically responsible roles: After all it’s always tech that has to clean up and fix the “mess” and problems that are usually taking place on corporate, inter-personal, political and process-, tool-related levels.

I truly LOVE Fibery and I believe it truly has 80%+ percent for it’s (possibly?) original vision. I absolutely appreciate and respect the team’s exceptional vision and extremely challenging work. I hope my often dry, process-oriented depiction of experiences and impressions do never detract from that fact.

An interesting, interjected ideological/structural example is rather exceptional, visionary Django/Wagtail CMS, for instance Unlike many CMS and especially Wordpress. They clearly decouple and strictly follow the belief… that Tech, vs. Designers vs. Authors/Editors/Users should absolutely be decoupled. Down to the very foundation of all interfaces/ux and even their application/code architecture. And from a technical pov: for good reason.

Not only from a “tech pov”; In actuality it is for the best for all parties and roles involved. At first glance it might seem tempting or desirable for an editor to be able to change some typo, style or banner color. However a designer with branding or visual aspects unknown to an editor might disagree or have very valid reasons to not want such a hap-hazard change. If the editor considers or wishes such change - he should actually verify and comm with the designer. Similarly both of those roles and desires might have effects or issues trickling down to the cms/backend admin. Things might lead to problems or challenges to be resolved/implemented, unknown to the editor, and even to the designer/frontend. If they want or need something on a structural level, they should verify and contact with the backend/admin.

→ I argue this separation of concerns is very much valid in any complex or structural application. Enable tech and admins to have the tools and means, to resolve that kind of implementation and issue on a lower level. It would ultimately benefit everyone, including Fibery’s workload once more of those initial tools are further developped. After all, imho Fibery already is extremely close to more of those tools and possibilities. → Help us to help you, and any sort of department or field, and their specific needs.

It has been 3 years or more since this feature suggestion. What can we do to improve exchange, collaboration and docs in that domain? What is the feedback on the practical API/Extension/Fixing side of things. Is there a way to decouple or leverage that problem in some kind of tools, structure or work/exchange groups?

I’d even reduce other responsibilities to free up part-time or internal means if that were needed, desired. If those impressions and challenges somehow resonate. I and most probably other more tech-related users would surely chime in and spend some time and effort, to somehow alleviate or even fix that kind of issues in a streamlined or sensible form.

1 Like

I had some ideas about how to set up a two-way sync using the existing integration API and some potential improvements. It’s not automatic like the Jira sync, but personally that’s actually desirable for me in some use cases. https://community.fibery.io/t/custom-integration-manual-two-way-sync/7464