Published entity cannot be read by external apps (e.g. AI, data analysis, marketing apps)

Currently if I share an entity in which the description contains an article, that text cannot be read by external apps.

How to test
I tested with my own published pages as well as pages from the Fibery online manual.
For example, the URL:
https://the.fibery.io/@public/User_Guide/Guide/Three-reasons-why-you-need-Fibery-2

Results of external apps:

ChatGPT, plugin Link Reader:
“I’m sorry, but I was unable to retrieve the content from the provided URL. This could be due to the website’s settings preventing external tools from accessing its content.”

https://www.tomedes.com/summary-tool
“THE REQUESTED PAGE IS IN AN UNRECOGNISABLE FORMAT”

Please help test this issue so we better understand the cause and context when this appears.

Importance

The issue of external applications being unable to access and analyze data from Fibery’s published or shared pages extends beyond a mere inconvenience. It represents a critical bug that:

  • poses a significant threat to the platform’s functionality, user experience, and future growth.
  • obstructs interoperability, an essential feature in our increasingly interconnected digital world,
  • hampers the integration of AI tools, which are becoming indispensable for data analysis and automation tasks.
  • it curtails the potential for collective intelligence, a potent paradigm that harnesses the collaborative power of multiple users or systems.

Addressing this bug is not merely about rectifying a single problem, but rather about unleashing the full potential of Fibery in the evolving digital landscape.

A (tedious) workaround is to create a PDF of the entity and attach it as file, then share the link to the PDF to the AI app. But this needs to be fixed soon, or Fibery cannot function with external AI apps that read content.

[Edit] This only works if you first publish the entity that contains the file, then open the public link in another tab, then right-click the attachment. This attachment link now has another URL which is accessible by external apps.

There could be several reasons why the content from the fibery.io page is not accessible to AI models like ChatGPT. Here are a few possibilities:

  1. Dynamic Content: If the webpage content is dynamically loaded using JavaScript, it might not be accessible when the page is initially loaded. AI models typically access the static HTML content of a page, and if the content is loaded or altered after the initial page load (for example, through AJAX requests), the AI might not be able to see it.
  2. Robots.txt or Meta Tags: Websites can use a file called robots.txt or meta tags to control what parts of their site can be accessed and indexed by bots and crawlers. If fibery.io has settings that prevent bots from accessing certain pages, AI models would not be able to access the content on those pages.
  3. Website Structure: If the website uses a complex or unusual structure, AI models might have difficulty parsing the content. This could be due to things like nested frames, complex CSS, or unconventional HTML tags.
  4. Privacy Settings: Some websites have privacy settings or require login credentials to access certain pages. If the fibery.io page requires a login, an AI model would not be able to access it.
  5. Technical Issues: There could be other technical issues at play, such as network errors, server issues, or compatibility problems between the website and the AI’s web scraping tools.

Please note that these are general possibilities and may not specifically apply to fibery.io. The exact reason would require a more detailed technical analysis.

1 Like

This is a significant concern for several reasons:

  • Limitation on Collective Intelligence: Collective intelligence refers to the shared or group intelligence that emerges from the collaboration and competition of many individuals. In the context of Fibery, it could mean the ability for multiple users or external applications to analyze and work on the same data. If external applications can’t access the data, it limits the potential for collective intelligence.
  • Impediment to AI Integration: AI tools can provide valuable insights, automate tasks, and enhance user experience. If AI tools can’t access Fibery’s data, it limits the potential for AI integration, which could be a significant disadvantage in today’s AI-driven world.
  • Barrier to Interoperability: In today’s digital world, interoperability - the ability of different systems and applications to work together - is crucial. The reported issue could act as a barrier to interoperability, limiting Fibery’s ability to work seamlessly with other tools and systems.
  • Potential Impact on User Experience: If users want to use external tools to analyze or work with their data on Fibery and they can’t, it could lead to a negative user experience. This could potentially affect Fibery’s user retention and growth.
  • Impact on Fibery’s Development Potential: The ability for external applications to interact with Fibery’s data could open up new possibilities for features and improvements. The current issue could limit Fibery’s potential for further development.
  • Marketing and Visibility Challenges : This issue significantly hampers Fibery’s visibility and marketing. Many apps that scan, compare, and promote platforms can’t properly access Fibery’s data, keeping it off their radar. This lack of visibility could lead to reduced website traffic and fewer potential customers. Resolving this issue is crucial for Fibery’s market presence and user acquisition.

Is this you or ChatGPT talking? :stuck_out_tongue_winking_eye:

“Is this you or ChatGPT talking? :stuck_out_tongue_winking_eye: …Both since posts that I generate with the help of ChatGPT4, often requires hours of long conversations and research (with plugins) around the subject, to make it as well researched, well motivated and as objective as possible. It requires my experience to make it ‘more intelligent’ and include relevant context.

This topic is of great importance, in my view for Fibery’s growth optimization (see motivation above)
and for our projects which focus on facilitating collective intelligence in (open source) collaboration networks. The internal Fibery AI is useful but more advanced research and analysis requires external AI apps.

If this poses serious challenges, a workaround would be for now to use the Fibery API to create a front end public website using an external app. Upgrading the existing publishing feature is of course better.

I like to invite the Fibery team to share their ideas and plans about this issue.