Killer search function that is a NATURAL fit for Fibery

Please read through this and give a vote if you agree with this brilliant idea (brilliant - but not my idea) for a search that would be a massive boost for Fibery. Images of before and proposed after included.

I have figured out one of the main things I’m struggling with regarding Fibery, which I love but which I want to be better: search.

Here’s a small sample of my search results for “fibery” in my workspace:

I have absolutely no idea how to find what I’m looking for in Fibery. In fact, I find myself making Index pages to manually track things. Search is much too basic and ineffective.

This is likely exacerbated by me using it as the workspace owner, so I see everything, but still.

Why is this search ineffective and user-unfriendly, and more importantly, how could it go from unhelpful to amazing?!??!

  1. First, as you can see in the image above there is a good size vertical scrollbar showing that there are many more entities to go through.
  2. More importantly, I have virtually no context. I might TRY to use the UUID to determine what’s recent and what’s not, but I’ve found that it only works within one table.
  3. There’s no date - I don’t know what’s old and what’s new.
  4. I don’t know what’s from where, unless I go through each one individually. If I go one-by-one I can see the table it’s from (badge and subheading) and that’s it.
  5. I can only filter by one item (e.g. table or view, etc) or from one space, so in order to accomplish my current goal, I have to repeat a certain process six or eight times.
  6. Or, I can try to emulate YVettte (Ninja fibery workaround specialist) and build a hack. But alas, in order to actually use search, I have to try to figure out scripting myself - and my brain just can’t do it.

To me, this is a massive problem with search, which I have come to the conclusion is by far Fibery’s biggest shortcoming by a million miles.

To show why it’s so ineffective, let me tell you about a program called X1. I have no affiliation with X1 but I used it for a while. It’s kind of a power search application. Also and somewhat coincidentally, I have been obsessively looking for (and trying) the ideal search app going on a few years now - prior to AI - and while you can do a ton with AI - you need to have a solid foundation first.

While X1 is not really very good in 2023 (it doesn’t integrate well with a multitude of external cloud apps or ipass which others do), it was a super helpful search when I used it. I think it has morphed into a legal forensics search tool, that’s how good it was. I especially had to use it a few times when it was a life-saver: once for a lawsuit where I had to go through tons of data, another time for a very large enterprise customer who had a lawsuit and I needed to find data on his behalf. Both times it was private and confidential so I had to do it myself. I also used it routinely for everyday use.

It was lightning fast - it let you hone down your search in ways intuitive to you (the user) and considering how you’d set up your files, documents, and database.

What’s interesting is that X1 is laid out just like a Fibery Grid, with one main UI adjustment/difference which is a box/filter on every column header. I can envision Fibery looking like the image below, with these search capabilities listed from most important to less important:

  1. You can filter on each column in a combined manner across multiple columns. So, you could select 4 Types from 5 databases from a 2-year period, etc.
  2. You can save your searches (illustrated in the left panel) and one-click get to them.
  3. Indexing would make it lighting fast and be done on a regular basis. (Once a night was more than enough for me when using X1).
  4. Search results would be updated as you type.
  5. You could include or exclude columns/filters in a way each user likes - maybe you don’t want to see so many columns. Also, you may want a command popup so the view with that would be more minimal, but I used a search tool (forget which one) that did that across apps yet retained the advanced filtering and saved searches, I’ll see if I can track that down.
    6. You could have multiple different views. For me, Grid is most important, but you could have lists and others too.
    7. In the event you do incorporate YAML sometime, it would work with that.

(Trying to pare down features to make this doable per Oshyan.)

You could also have global search options that work for all searches and themselves be customized (but more as a one-time thing than ongoing).

X1 was by far the most effective search I’d ever used. It helped me accomplish things much faster, including on occasions where I had to function more like a forensic tech than anything else.

And it seems to “fit” with the Fibery looks & feel and philosophy.

In the above example I show a filter dropdown on Type column, a column I created. Each of the columns would have a filter, including the Database Badge and Created By (avatar) which I left out of my screenshot. This would let you include/exclude, find or ignore to your heart’s content, and again, when a program used this method (X1), it became invaluable to me.

First and foremost, I 100% agree that Search needs some improvement and that it is currently one of Fibery’s key problems. I definitely think some others are ahead of it for broad adoption, general UI/UX stuff, arguably bi-directional sync, and other things. But Search is definitely in the top 5 for me, at least.

So on to your proposal. This is some cool imagining! But it is really a pretty massive collection of not-necessarily-related or interdependent features, from Dynamic Filtering, to Saved Searches, to changes/improvements in Search Indexing, to entirely changing the Search UI to basically be a View. Any one of those could make a good feature request, but I think if you’re hoping to get support for this single post as a “Feature Request”, you may have a hard time. This topic might do better in another Category for more general ideation and discussion rather than as a voted set of specific features/ideas for a new Search UI.

That said I do particularly love the idea of having Fibery Search optionally be a sort of temporary (or not so temporary, e.g. for Saved Searches) “View”, because what you’re showing is basically just visualizing the Search Results in a Table/Grid type of view. If search results as they are now can be sort of “piped” easily to a Grid (or other View?), that could go a long way toward making this possible and solving some of the technical issues. The UI/UX around all of this could still be challenging, but the potential for Grid improvements to affect Search Results functions is exciting. For example Grid does not currently support dynamic Filtering per-column but hopefully one day it will! And if/when it does, that could then immediately be available in Search. I’d love that.

From my external position, not knowing much of how Fibery works in the back-end, it does seem quite promising and not that hard to basically work on a way of temporarily instantiating Views that search/filter on the entire Workspace. That said I know there are already limits on what can be included in each View, so e.g. including all ~50+ DBs that many people (like me) have in a single View could be a performance issue on its own (for a full Grid UI; obviously existing Search already supports all DBs). Search also only indexes certain fields, however I’d personally be happy just getting a more powerful search UI, even with only the currently indexed fields.

There is also quite a lot of prior discussion on Search and how to improve it, including some more specific Feature Request topics that cover some of your ideas to some degree, and have existing votes:

Isn’t YAML basically just “fields for Markdown” (some/all of which are hidden, depending on the UI of the reading app)? Why are people suddenly talking about wanting to support YAML in Fibery when it’s already a full-blown Database application that goes far beyond what YAML does (in some respects)? Maybe I’m not familiar enough with YAML to understand. But again this seems tangential to the topic here…

3 Likes

I completely agree about the highest priority being Search improvement, as well as loss of context. Both issues are in my view the result of a navigation architecture that is promising but still lacks the basic needs. I think that Fibery understandibly tries to create a competitive edge with some special features but leaving behind some basic features that put new users off. Voted!

2 Likes

You mean it’s not as easy as making a photoshop screenshot?!? Lol. I see your point Oshyan and I also realized that the feature request area works a bit different. I’m going to go through those search posts you provided (thanks) and either defer to those - or lobby all those users to come over here!!! Heh.

I ran across YAML’s extensive use when I was working with FiberyObsidian. It’s basically designed for cross-app, platform, os, type (etc) interoperability. Your point about it being in a database so why bother is understood. YAML enables more communication and comprehension across different data types and systems, which is much more important for some than others.

The chief reason I bring it up is two-fold (1) I have a process that makes database + flat file interoperability very beneficial and I’m planning to pursue it more and (2) I am still thinking about that Obsidian/Fibery integration we discussed, and my massive workload of late is finally clearing up. (By the way, on that subject, I had mentioned N8N previously. Two interesting things: first, there’s a user here who does a lot of N8N-driven Fibery and Obsidian stuff, I saw him active in the N8N forum, and (2) N8N is so good that we standardized on it and transferred our pipedream work there.

YAML is sort of like turning everything (a file, a doc, dabase xyz entity, app database 123 entity, etc.) into the equivalent of a text file, kind of).

After reading your reply, I realized that perhaps there is a way to emulate what I’m looking to accomplish above with search, because truly that X1 search was wildly effective. I realized that - for example - I could take 10 example searches and then create 10 different views with certain filters in place. Then perhaps something that would do a popup to enter criteria. Maybe to achieve the results it’s possible to make a Fibery hack a la Yvette.

Since I’m still recovering from a brain injury, I find it really difficult to do these things so I’m going to ask my team to do it. Thanks!

2 Likes

Thanks Yuri!

Yes, it seems like a great method to potentially use for something like development of a Fibery <-> Obsidian sync plugin (independent, open source) (Obsidian new native “properties” also store to YAML in the .md). But it’s more of an interop/sync thing than something to support in Search explicitly IMO. I.e. you translate some other system’s fields/properties into Fibery’s in the sync, and then in Fibery it’s just regular Search on Fields.

I would still really prefer to see a “native” approach, probably as an Obsidian Plugin since I think that’s where the challenging stuff would have to be done, locally, and then just connect via the Fibery API. But I don’t have dev skills or resources, and if someone creates an N8N approach that I could use, well, I’ll probably use it. :smile:

Yep, absolutely. You can have a whole Space that is just “Search” if you want, and then just create a bunch of Views that can be re-used as Searches with sort of “default” parameters, sort them into Folders, etc.

Thanks for the feedback and suggestions.
Unfortunately, I think the UI proposed by @Robert_B is not practically viable.
Imagine you have a couple of dozen databases, each with a handful of unique fields (which definitely is not unusual amongst the users I know of). This implies that the number of possible columns could start to reach three figures. The horizontal scrolling needed to be able to meaningfully adjust filter settings per column would quickly become painful, and who knows what the potential performance issues might be.
I expect that the existing search-related challenges would be replaced by a different set of challenges.

And if you want to check it out for yourself, try creating a grid view with half a dozen databases, and with all possible fields enabled. I’m pretty sure you will become disabused with the idea :wink:

1 Like

I can imagine that it’s not handy to search in all databases and all fields. That’s also not needed since a lot of databases does not contain information that you are looking for.

If we can have some kind of ‘search view’, that would be awesome.

More or like a ‘predefined search’ option. So that you can decide:

  • In which database you want to search
  • And if possible, also in which fields of that database you want to search

It can be a solution for tons of use cases.

Example

  • We’ve built the Wiki of our workspace inside the workspace of the client.
  • Currently there is no simple way to search for wiki articles only → you can only use normal search function and narrow down databases but that are a lot of clicks and it looks difficult for non tech-savvy users, especially when you have >50+ databases.

What if you can create a search view in the left menu? Basically you set all filters for a user and within those filters they can use the search functionality :smirk:

Yes that is also awesome. I have multiple use cases where that becomes handy.

But sometimes you don’t want to overwhelm a user with all information in a grid → they just want to search and only see/use the outcome of that search. So that’s why I also like the idea of a search view :nerd_face:

2 Likes

Hi Chris! You rained on my parade :frowning:

One of the important things about X1 was the indexing. So I agree that what I’m describing above - particularly given how Fibery works (which I don’t know or understand) might seem completely impractical. However, what if it were solely based on the capabilities of the indexing and it did not require real-time indexing?

I realize that while I mention those two use-cases I had above, I actually had to do something similar perhaps 7-10 times. This was much more of a “find a needle in a haystack” then use a tool like fibery, clickup, or coda.

I tried to do this in a few ways over multiple projects:

  1. Windows search - by far the most awful.
  2. X1.
  3. A few advanced search products that focused on searching on your computer, in other apps, etc.
  4. Export to Excel.
  5. An Outlook plugin.
  6. Obsidian and plugins.

Now, they were all impractical but became practical (except windows it seems to be perpetually awful) until a full index was done, then, only new things get indexed, and the index I found was more than enough when it was daily (e.g. overnight). So the first time, it could take hours (days in one case where I had to index a bunch of network storage). But after that first time it would not be any problem, could run real-time indexing after or not, user choice.

Also, there were tons of options on how that index was built. If I could simply “uncheck” a bunch of database - and limit to “Name” field only that itself would be a huge plus.

Maybe reconsider that this concept need not be impossible - particular leaning towards it over time. I had forgotten about X1 until i brough this up and it was really incredibly helpful. :slight_smile:

The difference in what you’re describing (file system search) vs. Fibery is that the operating system (Windows) defines a single set of file meta data that any given file might have, i.e. the number and names of data “columns” (fields, metadata) is not only fixed but common to every single file you search/index. In contrast in Fibery every Database can be dramatically different, with totally different Fields, but also simply different Names for the same “type” of field (e.g. you might have two “Last Name” fields in two different Databases in Fibery, but the name of one is “Last Name” and the name of the other could be “Surname”. Technically the same intent or purpose for those fields, but because they have different names they can’t be associated together, and thus require separate columns on a table/grid view. In most cases your field names will not correspond for any databases, so the more databases you include in a search (and want to filter/sort the fields of in a search result), the more columns you have, the worse performance gets, etc. Again this is completely unlike the local Windows (and Mac) file system where the columns are fixed. At most a tool like X1 might add a little bit of additional, custom data like indexing file contents for specific types, e.g. PDF and DOC, but there are so few of these exceptions.

Having said all that I still think there could be some significant improvements to Fibery Search! Sort by Date vs. Name would be great, among many other things already mentioned here and in other topics.

2 Likes

Indeed, sadly.

did you mean the name of one is “Last Name”

1 Like

Indeed, of course I did :laughing: Fixed.

Thanks Oshyan, you’ve poured on my parade!!! Lol.

I get it - so it’s like even if X1 could index 100 types of files - they were all the same (1) name, (2) meta, (3) contents, (4) extension.

Whereas with Fibery it could be more like 100x 100x 100x 100x and those four things could be forty things and they had no standard so they could all be totally different from one to the next.

Okay I get it now. With that in mind I’m going to go back to the drawing board - now that I understand it conceptually - and take another crack at something that could actually be doable.

Thanks!

2 Likes