Backup entire workspace?

It would be great to have a way to back up or create a checkpoint not just of a single app, but all apps and all their data in my account.

I am paranoid that someone might delete a type/field/app by mistake, resulting in unrecoverable data loss.

7 Likes

HI, we had a similar discussion with @antoniokov the other day and I promised to send some material over. The actual idea was to give the Workspace some kind of AWS Credentials for a Customer Owned S3 bucket. The Whole Workspace will be copied over there including metadata as JSON/YAML what or similar as archive. In an unexpected event, anyone can recover the data as long they are well-structured.

The inspiration was coming from backup, a Github Backup tool:

I think the original feature request relates more to ‘undelete’ (which is WIP according to fibery twitter) rather than disaster recovery, which seems closer to your proposal (also important of course).

@Chr1sG Even though the original request is about the “undelete”, the whole-workspace backup feature is important for the teams relying on the know how captured in fibery.

@Holger_Winkelmann Is there any progress on that?

1 Like

Sorry to resurrect an old thread here,
I’m interested in both un-delete and disaster recovery. I’ve been recommending Fibery to clients but through gritted teeth as we’ve had trouble with restoring deleted data.

In our experience, doing un-delete is achievable with the (now released) audit and trash logs.
However, un-deleting an entire database or large portion of a large connected DB still has issues.
Specifically, restoring more than 200 linked entities from both trash log and audit log can result in errors ( discovered when we needed to restore ~800 from being wiped by an integration )

Of course there is ‘workspace settings’>export all data which is okay for manual DR backups. What we really need is:

  • Trigger backups on a user-defined schedule
  • Store the backups in S3 / gdrive and let us delete/retain them
  • Export any other files or information beyond .csv and md (?)

Ideally, if things were deleted/lost, we could run ‘full export’ of current space, restore the entire workspace to past backup point, use the recent ‘full export’ and .csv import to update any data in db’s missed in past backup point

Seeing “CSV import 2.0” feature being tested is very encouraging for per-DB restore from ‘full export’!

Anyone else doing disaster recovery planning with Fibery?

2 Likes

Un-deleting large amounts of data (that had originally been added into the workspace over a period of time) is quite challenging, for similar reasons to why trying to merge multiple documents is challenging.
Imagine you have a database with a history like this:

Created db
Added some fields
Added entities A and B
Added some relations
Deleted B
Changed some relations (from one-to-one to one-to-many)
Added entities C and D
Added some more fields
Deleted entities A and C
Added entity E
Added more fields

…and now you realise that you need to recover entity B.
What should happen? What values should B take for all its fields?

Now imagine that you have deleted hundreds of entities(!)

Of course, if you realise that you need to restore something immediately after it has been deleted, the challenges aren’t so great, but what if a little time has passed and you/your colleagues have made changes in the mean time? Designing an undelete process to be sufficiently robust is not a trivial one.

Fibery does maintain regular backups of user data for internal purposes, but it is not very scalable to maintain snapshots of every workspace at the various time intervals that would accommodate a restoration process.

It is theoretically possible that Fibery could offer the option for users to do a full workspace backup (databases, files, documents, in a non-human readable format (blob) which users could then choose to store in their own data centres, but so far, the work to develop this functionality (and the functionality to restore from a blob) would be significant, and would take resources away from other areas.
At the moment, it’s just not high enough priority.