[IN DEV] Easy import of markdown docs to wiki

[Michel_Daviot]

@Michel_Daviot

IE, how to I replace ![Untitled](/Untitled.png) in my original markdown file ?

I found the answer by peeking at the network view when doing it by hand, it is simply
![Untitled](/api/files/ba511eea-f8bb-447e-9fda-b8c5fddfaa22)

(with “fibery/secret”: “ba511eea-f8bb-447e-9fda-b8c5fddfaa22” in the answer to the upload request)

[Michel_Daviot]

I’ve made great progress on one area : I can now with this script (it’s scala but you should be able to port it easily to python or javascript) update all the entities with markdown + images, in a chosen field.

I’m now struggling with creating documents and folders from the API. I have not found documentation here and based on these posts I’m worried if this is possible.

Maybe @Oleg or @Chr1sG you could clarifiy the status about the API for managing documents and folders ?

[Oleg]

Hi, @Michel_Daviot

Please find my js function to create/update docs

    const getDocumentsBatchCommand = (command) => async ({client, format, args}) => {
        logger.info(`[Fibery API]: Collab doc command: ${command} with ${args.length} ${format} docs`);
        return client.post(`api/documents/commands`, {
            searchParams: {
                ratelimit: 'on',
                format,
            },
            json: {
                command,
                args,
            },
            headers: await getAuth(),
        });
    };
    const updateDocuments = getDocumentsBatchCommand('create-or-update-documents');
    const appendDocuments = getDocumentsBatchCommand('create-or-append-documents');

where format is one of md, json, html or text

and args is array of elements which looks like

{
            secret: `XX-YY-ZZZ-it is just a doc secret`,
            content:`Hello`
 }

[Oleg]

Please find below an example of creating folder

    const createFolder = async ({client, app, name, parent = null, publicId = null}) => {
        const operationId = uuid.v4();
        const folderId = uuid.v4();
        const result = await client.post(`api/views/json-rpc`, {
            json: {
                jsonrpc: `2.0`,
                method: `create-folders`,
                params: {
                    values: [
                        {
                            'fibery/Parent Folder': parent,
                            'fibery/id': folderId,
                            'fibery/name': name,
                            'fibery/container-app': {'fibery/id': app.id},
                            'fibery/public-id': publicId,
                        },
                    ],
                },
                id: operationId,
            },
            headers: await getAuth(),
        });
        if (result.error) {
            throw new Error(result.error.message);
        }
        return _.get(result, `result[0]`);
    };

In my code call looks like

const folder = await fibery.createFolder({
    app: app,
    name: folderName,
    parent: {'fibery/id': getFiberyId(step)},
});

[Michel_Daviot]

I’ve added some documentation to the script. Also here is how I export from notion :

Thanks @Oleg I’ll try those to upload the rest of my data !

[Oleg]

@Michel_Daviot

Nice, I believe we will add support for integration apps in future to allow the import of documents and folders. It is partly implemented, but I am waiting for fibery core implementation of sub-documents.

[Michel_Daviot]

@Oleg

Please find my js function to create/update docs

I guess I did not make my request clear enough, I want to create a document in a folder (or directly in a space), how can I specify the document name and location ?

[Michel_Daviot]

Never mind, I manged to create a folder and I will be able to guess how to create a document by peeking at the developper tools while doing it from the front-end. It was a good idea to use the same API for front-end and automations

I’ll finish this next week, thanks again for your help !

[Michel_Daviot]

I’ve updated the scala script, it is now also able to import all mardown files into fibery documents (inside folders).

As documented in the script, I was surprised that the secretId is generated on the client side (what would happen if a secret is reused ?).

Note for future users : it is not perfect, for instance links to notion pages are note translated to internal fibery links. But it saves some time when importing hundreds of documents with attached images !

[Chr1sG]

I guess you’ve seen this topic:

Mapping links in imported content into Fibery links/references

I am about to import a bunch of content into Fibery from several other tools I use, within some of which I have more than 1000 files/pages/docs. I link between files/docs/pages quite a lot in many of my workflows and processes. In some of these tools I also get “backlinks”/references, similar to Fibery, which I would ideally like to maintain in some way. Since I plan to import basically all of this content into Fibery, and so any page that another would link to should also be in Fibery somewher…

I reckon you could use the same logic to create working links within Fibery to replace the Notion links.

[Michel_Daviot]

I had not seen it, thanks ! We had not that many internal links to migrate so we updated them by hand, but the automation is a good reference.

[Jose_Quesada]

Thanks Michel_Daviot, this script is really promising. For us to move to fibery a massive md import is needed, keeping the links between documents. This doesn’t seem to be easy (or supported by this script).

Is Fibery going in this direction with a native import? Many teams are stuck with data from other systems, might facilitate migration.

[Chr1sG]

@Jose_Quesada
What platform do you hope to import from?

[Jose_Quesada]

This is plain text files on my hard drive. Well, and images.

[Chr1sG]

@Jose_Quesada

a massive md import is needed, keeping the links between documents.

@Jose_Quesada

This is plain text files on my hard drive.

How are they linked? Is it via some proprietary syntax?

[Jose_Quesada]

Using double bracket syntax [[title of the markdown file without the .md extension]]

[kalman]

Good news, Everyone! This is Kalman, PM at Fibery. We’ve just started work on the markdown import feature and can’t wait to see where it takes us.

I’ve been going through the topic, and some seriously awesome use cases and ideas are out there. If you have more to share, now’s the time to let it rip!

Big thanks to all of you for your insights. Please keep them coming!

thanks,
kalman

[mdubakov]

First version of markdown import is here

May 25, 2023 / ✔️ Search: done items are greyed, filter by space for all Views, new Settings, Import markdown files

Mark Done entities grey in search result All entities in the final state (such as closed bugs and completed tasks) are dimmed and marked with a icon in search results. [Screenshot 2023-05-18 at 9.47.46 AM] Filter by Space for all Views (including Docs) Now you can filter views in search results by space. This feature will help you find documents, whiteboards, and other views more quickly. [Screenshot 2023-05-25 at 4.00.13 PM (1)] …

[dukevannori]

this is awesome!! thanks so much for bringing this into fibery!

will the markdown (soon/at all) support

• [[links]] to other files?

• YAML frontmatter? (translate to columns in database)

• automatic translation of - tasks to “task”-items in a database

• (and would that also be possible to run through already imported files?)

this would make this into a perfect companion to obsidian and finally a replacement for it in my workflow.

i am really excited right now!

[kalman]

@dukevannori

thanks so much for bringing this into fibery!

Wow, we’re so glad you’re excited about the recent releases on this topic.
As for your questions, the good news is that all the features you’ve mentioned are on our radar and have been added to our backlog but we don’t have a specific ETA for them just yet.