# Entity Migration Script: Hierarchical Data Transfer Between Databases

**URL:** <https://community.fibery.io/t/entity-migration-script-hierarchical-data-transfer-between-databases/5899>\
**Category:** Fibery Showcase\
**Created:** [February 23, 2024, 4:18pm UTC](https://community.fibery.io/t/entity-migration-script-hierarchical-data-transfer-between-databases/5899 "2024-02-23T16:18:34Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Yuri\_BC](https://sea2.discourse-cdn.com/flex020/user_avatar/community.fibery.io/yuri_bc/32/8803_2.png) [@Yuri\_BC](https://community.fibery.io/u/Yuri_BC)\
**Post date:** [February 23, 2024, 4:18pm UTC](https://community.fibery.io/t/entity-migration-script-hierarchical-data-transfer-between-databases/5899/1 "2024-02-23T16:18:34Z")

</div>

## Purpose

This script automates the migration of entities between two Fibery databases, ensuring the preservation of key attributes (name, description), hierarchical relationships (parent-child), and specific relationships (tags, parent projects). It supports structured data transfer with depth control to maintain organizational integrity.

## Use cases:

- Project Management Reorganization: Migrates projects, tasks, and sub-tasks to new workspace, preserving hierarchy and details.
- Content Management Update: Moves articles, drafts, and edits to an optimized database, maintaining editorial workflow and organization.
- Team Reassignment: Reorganizes team projects and tasks, keeping project associations and categorizations intact.
- CRM Update: Transfers customer records and interactions to a new CRM structure, preserving customer hierarchies and segmentation.

## How to adapt the script to your needs

1. **Identify Source and Target Databases** : Determine the databases you’re migrating from and to.
2. **Update Database Constants** : In the script’s initial constants section, replace `SOURCE_DB` with your source database name and `TARGET_DB` with your target database name.
3. **Match Field Names** : Adjust the field name constants (`SOURCE_NAME_FIELD`, `TARGET_NAME_FIELD`, etc.) to correspond with the field names in your source and target databases.
4. **Parent Project and Tags Handling** : Note that the script assumes ‘Parent Project’ and ‘Tags’ already exist in the target database. It will link the migrated entities to these existing items without creating new ones.

## The script

```auto
// Configuration constants for source and target databases
const SOURCE_DB = 'Space A/Database X';
const TARGET_DB = 'Space B/Database Y';

// Configuration constants for field names
const SOURCE_NAME_FIELD = 'Name';
const TARGET_NAME_FIELD = 'Name';
const SOURCE_DESCRIPTION_FIELD = 'Description';
const TARGET_DESCRIPTION_FIELD = 'Description';
const SOURCE_SUB_PAGES_FIELD = 'Sub Pages';
const TARGET_SUB_PAGES_FIELD = 'Sub Pages';
const SOURCE_PARENT_PAGE_FIELD = 'Parent Page';
const TARGET_PARENT_PAGE_FIELD = 'Parent Page';
const SOURCE_PARENT_PROJECT_FIELD = 'Parent Project'; 
const TARGET_PARENT_PROJECT_FIELD = 'Parent Project';
const SOURCE_TAGS_FIELD = 'Tags'; 
const TARGET_TAGS_FIELD = 'Tags';

const MAX_DEPTH = 5; // Maximum depth for conversion

const fibery = context.getService('fibery');

// Function to fetch document content
async function fetchDocumentContent(documentSecret) {
    if (!documentSecret) {
        return null;
    }
    return await fibery.getDocumentContent(documentSecret, 'md');
}

// Function to convert entities from the source database to the target database, with depth tracking
async function convertEntity(entityId, parentPageId = null, currentDepth = 1) {
    if (currentDepth > MAX_DEPTH) {
        return; // Stop recursion if maximum depth is reached
    }
    try {
        const entity = await fibery.getEntityById(SOURCE_DB, entityId, [
            SOURCE_NAME_FIELD, SOURCE_DESCRIPTION_FIELD, SOURCE_SUB_PAGES_FIELD, SOURCE_PARENT_PROJECT_FIELD, SOURCE_TAGS_FIELD
        ]);
        let descriptionContent = null;
        if (entity[SOURCE_DESCRIPTION_FIELD] && entity[SOURCE_DESCRIPTION_FIELD].Secret) {
            descriptionContent = await fetchDocumentContent(entity[SOURCE_DESCRIPTION_FIELD].Secret);
        }

        const newEntityData = { 
            [TARGET_NAME_FIELD]: entity[SOURCE_NAME_FIELD],
            [TARGET_PARENT_PROJECT_FIELD]: entity[SOURCE_PARENT_PROJECT_FIELD] ? entity[SOURCE_PARENT_PROJECT_FIELD].Id : null,
        };
        const newEntity = await fibery.createEntity(TARGET_DB, newEntityData);

        // Set the 'Description' content for the new entity
        if (descriptionContent) {
            await fibery.setDocumentContent(newEntity[TARGET_DESCRIPTION_FIELD].Secret, descriptionContent, 'md');
        }

        // Link the new entity to its parent, if applicable
        if (parentPageId) {
            await fibery.updateEntity(TARGET_DB, newEntity.id, { [TARGET_PARENT_PAGE_FIELD]: parentPageId });
        }

        // Handling Tags through addCollectionItem for each tag
        if (entity[SOURCE_TAGS_FIELD] && entity[SOURCE_TAGS_FIELD].length > 0) {
            for (const tag of entity[SOURCE_TAGS_FIELD]) {
                await fibery.addCollectionItem(TARGET_DB, newEntity.id, TARGET_TAGS_FIELD, tag.Id);
            }
        }

        // Recursively convert and link sub-entities, increasing depth
        if (entity[SOURCE_SUB_PAGES_FIELD] && entity[SOURCE_SUB_PAGES_FIELD].length > 0) {
            for (const subEntity of entity[SOURCE_SUB_PAGES_FIELD]) {
                await convertEntity(subEntity.Id, newEntity.id, currentDepth + 1);
            }
        }
    } catch (error) {
        console.error('Error converting entity:', error);
    }
}

// Main function to initiate the conversion process based on context entities
async function convertEntities() {
    try {
        for (const currentEntity of args.currentEntities) {
            await convertEntity(currentEntity.id); // Start with depth 1
        }
    } catch (error) {
        console.error('Error in converting entities:', error);
    }
}

// Execute the conversion process
await convertEntities();

```

## Explanation

1. **Define Configuration Constants** :

- Specifies source and target databases, field names, and maximum conversion depth, allowing easy adaptation of the script for different migrations.

1. **Fetch Document Content (`fetchDocumentContent`)**:

- Uses `fibery.getDocumentContent(documentSecret, 'md')` to retrieve the content of document fields based on their secret. This operation is crucial for copying the ‘Description’ field content from source to target entities.

1. **Convert Entity (`convertEntity`)**:

- **Fetch Entity Data** : Retrieves the current entity’s data from the source database using `fibery.getEntityById(SOURCE_DB, entityId, fields)`, including name, description, sub-entity relationships, and any specified relationship fields like ‘Parent Project’ and ‘Tags’.
- **Create New Entity** : Creates an entity in the target database with the same ‘Name’ (and potentially other simple fields) using `fibery.createEntity(TARGET_DB, newEntityData)`.
- **Set Description Content** : Updates the ‘Description’ of the newly created entity with the content fetched from the source entity using `fibery.setDocumentContent(newEntity[TARGET_DESCRIPTION_FIELD].Secret, descriptionContent, 'md')`.
- **Handle Parent-Child Relationships** : For hierarchical structures, recursively calls itself to convert and link sub-entities, managing depth to avoid exceeding `MAX_DEPTH`.
- **Update Relationship Fields** : Specifically for ‘Parent Project’, directly sets the relationship by ID. For ‘Tags’ (a many-to-many relationship), iterates over each tag and adds it to the new entity using `fibery.addCollectionItem(TARGET_DB, newEntity.id, TARGET_TAGS_FIELD, tag.Id)`.

1. **Initiate Conversion Process (`convertEntities`)**:

- Iterates over entities provided in `args.currentEntities` and starts their conversion by calling `convertEntity` for each, marking the entry point for the script’s execution.

1. **Execute Conversion Process** :

- The final step where `await convertEntities()` is called to begin the migration based on the entities and relationships defined by the configuration constants.

### Summary of Methods Used:

- **`fibery.getEntityById`** : Fetches detailed information about an entity, including specified fields.
- **`fibery.createEntity`** : Creates a new entity in the target database with specified field values.
- **`fibery.setDocumentContent`** : Sets the content of a document-type field for a newly created entity.
- **`fibery.addCollectionItem`** : Adds items to a collection-type field, used for managing many-to-many relationships like ‘Tags’.
- **`fibery.updateEntity`** : Updates simple fields of an entity, used for setting single-select relationships not covered by `addCollectionItem`.

# If you want to remove fields from this example script

For example, if you want to remove the ‘Tags’ field conversion form the script because your databases don’t have that field:

1. **Remove Constants** :

- Delete the `SOURCE_TAGS_FIELD` and `TARGET_TAGS_FIELD` constants from the initial constants section of the script.
- Example:

javascriptCopy code

```auto
// Remove these lines
const SOURCE_TAGS_FIELD = 'Tags';
const TARGET_TAGS_FIELD = 'Tags';

```

1. **Remove Handling Code** :

- Within the `convertEntity` function, delete the code block that handles the ‘Tags’ field, specifically the loop that adds tags to the target entity using `fibery.addCollectionItem`.
- Example:

javascriptCopy code

```auto
// Delete this code block
if (entity[SOURCE_TAGS_FIELD] && entity[SOURCE_TAGS_FIELD].length > 0) {
    for (const tag of entity[SOURCE_TAGS_FIELD]) {
        await fibery.addCollectionItem(TARGET_DB, newEntity.id, TARGET_TAGS_FIELD, tag.Id);
    }
}

```

---

<div class="post-metadata">

**Author:** ![Chr1sG](https://sea2.discourse-cdn.com/flex020/user_avatar/community.fibery.io/chr1sg/32/3941_2.png) [@Chr1sG](https://community.fibery.io/u/Chr1sG)\
**Post date:** [February 23, 2024, 4:34pm UTC](https://community.fibery.io/t/entity-migration-script-hierarchical-data-transfer-between-databases/5899/2 "2024-02-23T16:34:06Z")

</div>

It’s just for recursive 1:n self-relations, right?

FYI, you can achieve thing manually by duplicating the entities using copy+paste and then duplicating the relationships using copy+paste.

![firefox_JwoEm8hj3u](https://us1.discourse-cdn.com/flex020/uploads/fibery/original/2X/e/e72d25bce0cfdfa95708cf5f162fc841291fe6f7.gif)

---

<div class="post-metadata">

**Author:** ![Yuri\_BC](https://sea2.discourse-cdn.com/flex020/user_avatar/community.fibery.io/yuri_bc/32/8803_2.png) [@Yuri\_BC](https://community.fibery.io/u/Yuri_BC)\
**Post date:** [February 23, 2024, 4:45pm UTC](https://community.fibery.io/t/entity-migration-script-hierarchical-data-transfer-between-databases/5899/3 "2024-02-23T16:45:50Z")

</div>

> [@Chr1sG](#):
>
> It’s just for recursive 1:n self-relations, right?

Yes, it’s for recursive one-to-many self-relations.

> [@Chr1sG](#):
>
> FYI, you can achieve thing manually by duplicating the entities using copy+paste and then duplicating the relationships using copy+paste.

Yes you are right about copy paste in a table, thank you for mentioning that.  
I think the script is still useful to automate this process for larger datasets or more complex entity structures.

---

<div class="post-metadata">

**Author:** ![Yuri\_BC](https://sea2.discourse-cdn.com/flex020/user_avatar/community.fibery.io/yuri_bc/32/8803_2.png) [@Yuri\_BC](https://community.fibery.io/u/Yuri_BC)\
**Post date:** [February 27, 2024, 11:57am UTC](https://community.fibery.io/t/entity-migration-script-hierarchical-data-transfer-between-databases/5899/4 "2024-02-27T11:57:36Z")

</div>

@Chr1sG The copy pasting using table views does not work for rich text fields, right?

---

<div class="post-metadata">

**Author:** ![Chr1sG](https://sea2.discourse-cdn.com/flex020/user_avatar/community.fibery.io/chr1sg/32/3941_2.png) [@Chr1sG](https://community.fibery.io/u/Chr1sG)\
**Post date:** [February 27, 2024, 12:02pm UTC](https://community.fibery.io/t/entity-migration-script-hierarchical-data-transfer-between-databases/5899/5 "2024-02-27T12:02:28Z")

</div>

> [@Yuri\_BC](#):
>
> The copy pasting using table views does not work for rich text fields, right?

Correct. The rich text snippets are read-only