Questions about the integration API webhook endpoints

I am currently working on a personal development of an integration between Todoist and Fibery.
I was able to implement synchronizing Todoist information to Fibery by referring to the document and sample repository.

However, I have not been able to find a specific example using the integration API webhook endpoint shown below, nor have I been able to find similar implementations in other projects.

https://api.fibery.io/apps.html#post-api-v1-synchronizer-webhooks
POST /api/v1/synchronizer/webhooks
DELETE /api/v1/synchronizer/webhooks
POST /api/v1/synchronizer/webhooks/verify
POST /api/v1/synchronizer/webhooks/transform

Question 1
First of all, is it correct that these endpoints are used to update the corresponding entities on the Fibery side in response to webhooks coming in from external systems?

Question 2
If I understand the above correctly, I can set a specific webhook callback URL on the Todoist side, but which endpoint URL should I set for this?
Is there a dedicated endpoint for receiving webhooks, similar to https://oauth-svc.fibery.io/callback in oauth?

Question 3
Instead of answering these questions, it would be totally fine if you could just publish a sample that actually uses the above endpoints.

Thank you very much for providing the integrations API!

1 Like

Hello, @oooooreona

I will try to answer your questions:

Question 1:

Yes, this is correct

Question 2:
When you create new integration type the new url for incoming messages is created as well, so each integration has its own unique url.

It works well for systems like GitHub where you have API to register webhooks on custom urls but it’s hard to use it with systems like Todoist which expect you to have only one Public url for all notifications.

But there is a trick you can try to use:
You will need to implement the public endpoint by yourself in your connector and proxy payload to registered webhooks.
We do the same for our Intercom integration so I’ll try to describe the idea briefly here:

POST /api/v1/synchronizer/webhooks

This endpoint is required to register webhook for specific Fibery types. As one of parameters it also passes url (docs miss it for some reasons). We save all the info in a database (e.g. MongoDB) and return our webhook object

POST /api/v1/synchronizer/webhooks/verify

here you check that event is valid and indeed came from todoist. Should return object with webhook’s id if it’s valid and can be trusted

POST /api/v1/synchronizer/webhooks/transform
map event’s payload to data that needs to be sent to Fibery

DELETE /api/v1/synchronizer/webhooks

is required to delete webhook for specific Fibery types. Here we remove the webhook we created from our MongoDB

And here comes the trick:
We implement one more endpoint:

POST /api/v1/synchronizer/webhooks/income

We make it publicly available and use it as endpoint for all incoming messages from external system.
When we receive a message we look in our MongoDB for webhooks that should be triggered for this message and send the payload to the urls of those webhooks…

I know it’s hard without examples but I hope I was able to explain the main idea…

Question 3
I’d be happy to give you an example but unfortunately we don’t have them open-sourced yet… But I will be glad to help you with your Todoist connector if you are still going to implement it )

1 Like

Thank you so much for the details!
I will try to develop it more based on what you have taught me.

I would like to share my project with you once I have developed it a bit more and it has taken shape.

Again, thank you very much.

2 Likes

It has been quite some time since my last post, and although I finished all the development work by the end of February, I started working for a new company, so I could not make time to write this article.

I have implemented what I am currently able to do in the development of the Todoist integration application, and I would like to ask you to check the code and ask some questions.

I know you are busy, so I will wait patiently and would appreciate it if you could check it when you have some time.

I would be very happy if this app I developed could be of use to someone in the future!

Project link

Questions about features I want to implement

About Bi-directional sync

The bi-directional sync of JIRA released in the beta version was great.

It would be great if you could release the actual implementation code as OSS in the near future!

This is not a question, just a request.

About Entity update with incoming webhook

I tried to implement this based on your previous answer, but I was not able to implement it to the best of my ability.

I would like to wait for actual samples from Intercom app and others to be released.

Also, I actually tried Intercom’s integration and it was still a great experience.

As I’m sure you know, I have seen requests on GitHub for near realtime sync based on Webhook to be implemented, so it would be great to see a detailed real-world example published.

I think many people would be happy to see this feature enabled in the new JIRA integration as well.

This is also not a question, just a request.

Q. About the partial update of Entity.

This is a question and a request for partial entity override in the integration API.

The Todoist API handles completed tasks differently than normal tasks, and Tasks and Completed Tasks are retrieved from different API endpoints.

In addition, Tasks retrieved from the Completed Task endpoint omit some properties.

As shown in the image below, for example, “task creation date” and “multiline text description” are properties that are omitted in the Completed Task endpoint.


On the Fibery side, if you separate the Task and Completed Task DBs, the task will be moved to a different DB when it is completed, so if it is mentions in a text somewhere, it will no longer exist.

Screenshot 2023-03-16 at 23.55.47

Therefore, I implemented to display both Tasks and Completed Tasks on one Task DB.

However, if the Fibery data is updated with the information obtained from the new Completed Task, the missing properties from the already existing task will disappear.

Screenshot 2023-03-16 at 23.59.07

To avoid this, if there is a synchronization method called “Partial Overwrite”, I can update only the properties that already exist, such as “Completion Date” and “Status”, etc. Is it possible to implement this at this time?

By the way, I found a description of delta synchronization in the document of the Integration api, and thought this might be the feature I was looking for, so I tried it. However, it seems that this one does not “partially overwrite” but “completely replace”.

The documentation didn’t give any details about delta synchronization, so I would also like to ask you to check if my understanding is correct.

Specification of delta synchronization:

  • Only one of “SET” or “REMOVE” can be specified.
  • If SET is specified, the target Entity will be overwritten with all newly passed information. (Missing properties for the Schema definition are overwritten with null.)
  • If REMOVE is specified, the target Entity is deleted.
  • Entities that are not included in the items array will remain in the Fibery side.

If the above is correct, I would like to have another option like “UPDATE”, “UPSERT”.

I think it would be possible to implement this at this time by starting up a database on my application side and keeping a copy of the data, but I don’t want to do that for maintenance and security reasons.

I would be very happy if you could publish an actual example here as well!

That’s all I have to ask.

Requests for Integration API

Creating views from the Integration app

This is just a suggestion, but I thought it would be nice to be able to generate views that the developer recommends while developing the integration.

Here is a concrete example from the integration I created this time.

In Todoist, not only for tasks as mentioned above, but also for projects, it is necessary to hit different end points for active projects and archived projects.

However, on the other hand, when retrieving a Task, it is normally retrieved with active status, even though the Task is in the Archived project.

Frame 1 (11)

In the integration I created this time, I omitted the getting the Archived Project.

All Todoist tasks always belong to a parent project, but since Archived Projects are not retrieved, they are displayed in Fibery as uncompleted tasks with no parent.

Therefore, in order to match the Todoist display, the “Project is Empty” filter must be applied after the view is created.

I think it would be troublesome for users of the integration feature to go through this kind of detailed setup each time they use it, so I thought it would be a good idea to provide a function to generate a sample view.

In particular, since the integration feature is likely to be used by users of the original app, it would be a memorable experience if a view similar to the original app is generated on Fibery at the same time they start using the integration feature.

I have come up with several ideas for this matter that can be implemented at this time.

  1. Have the user provide the Fibery api key and domain name when setting up the integration, and generate the view via the Fibery API.

    Receiving Fibery’s API key doesn’t seem like a good security idea.

  2. Provide JavaScript that can Create View by copying and pasting it into Action Button’s script and executing it.

    I don’t think it’s a good idea to have no-code users to do the black box work, because if this becomes the norm, there is a risk that someone with malicious intent could exploit it.

So, I know it’s not a priority, but it would be nice if there was a way to do create view safely with integration api.

About Formula field

I have implemted a db for date and week within this app.

I thought it would be nice to be able to add pictograms to “today” and “this week” to make them stand out in the view that is shown to the user.

I can provide that field on the integration app side, but it will not be reflected until synchronization occurs, so it would be nice if the integration app could be given the ability to generate Fibery’s formula field.



Sorry again for the very long post.

I would be very happy if you could answer briefly when you have time.

Now that GPT-4 has been released, it may be easier to develop integration features.

2 Likes

oh, that’s super cool

I will try to take a look at it by the end of the week. Great job :+1:

2 Likes