[DONE] Geospatial fields

I agree with @Oshyan that this seems outside the area of focus for fibery at the moment. However, I am wondering if adding geospatial data type is going to that big of a leap from other data types. I don’t think right now it is necessary to provide any geospatial analysis/query capabilities. But if we at least allow people to collect the information, they would be able to either do the analysis elsewhere or if there is a plugin environment in the future, someone could build those capabilities.

If it possible to add the geometry data types as well as an interface to add/draw these and display them (leaflet seems the most natural fit), then I think that would be plenty for most users.

I am also hoping that along with the data types and display in fibery, a map view is also added to vizydrop as well to make their dash boarding capability more complete.

I’m glad I’m not the only one. @mdubakov if I can help brainstorm ideas or contribute on this, let me know. This is something nobody else is paying attention to, but I think if it is done right can be another differentiator for fibery.

2 Likes

Before I respond more specifically, I’ll just reiterate I am generally in favor of both a geospatial data type/field (lat/long at a minimum) as well as a Map view type for Vizydrop.

I’m sure most of us understand this, but still I did want to say that a collection of differentiators does not necessarily make a compelling and cohesive product for any given market segment in itself. Having a feature no other product in your market has is not necessarily an indicator of beneficial innovation. It is at least as likely to be due to “implementing something that the core market for your product doesn’t actually need” as it is to “getting there first with a key new feature”. So having something be identified as a differentiator is interesting, but it needs to harmonize with other features and overall goals of the application, the markets they are targeting, etc.

In this case I am not really seeing how geospatial-oriented features would help Fibery capture significant market and thus revenue given its current feature set and general orientation as a tool. That said, this is not an outright denial of that possibility. Rather, I am curious to perhaps hear more about what market(s) people think are A: only partially served by Fibery’s existing capabilities and B: would be significantly better served (and likely to adopt Fibery) if geospatial tools were added.

1 Like

I fully agree with the sentiment here and concede being different is probably not the best reason to do something. I don’t want to prolong the discussion, but thought I should put my request in context:

One of the problems that I’ve seen with almost all work management software and services (at least the ones I’ve tried like todoist, Monday.com, wrike, coda, etc.) that are currently available is that they are primarily geared towards software development, marketing, communication, etc. which often do not have a basis in the real/physical world. That is what makes them very difficult to use for anyone working in fields whose work involves something physical (AEC, real estate, asset and facility management, heritage conservation, governments, etc.). At some point we will need to record information about locations and physical features and that is when you are forced to use a different tool. Examples of this are location of all ongoing projects, available listings, outstanding work orders, boundary of properties, location of all assets you need to manage (e.g. light poles, benches, streets).

I know ESRI’s complete dominance of the geospatial field discourages most companies from even considering entering this area and there are specialized software for all the fields I discussed above. But one of the selling points for fibery is that you can build your own tool and grow your process without having to spend huge amounts on software/system acquisition up front. However, without a way of attaching some of the records to the real world, the tools I am making would be incomplete. I think the investment to support storage and representation of geospatial data is no longer as daunting as it was before with standard libraries and universal database support. I do agree that most companies at this point will not be able to compete with the likes of ESRI on geospatial analysis, so we will have to live with using those tools when the need arises. But for basic level of data, it would be nice to have an all in one tool.

One final thought: if you are considering adding this as a feature, please consider going the full way and adding all geometry types. Monday.com supports single points (for some reason, insist on geocoding rather than also allowing just inputting/importing coordinates) but that will quickly become a barrier if you need to associate multiple points or a boundary.

I really appreciate the interest in this topic from everyone. If anyone has ideas on other systems that would work well as a workaround for now, I would be very interested to hear about them.

2 Likes

Thanks for the additional detail! So what if we shift, then, to thinking of creative potential ways to address some or all of this need. Here are a couple.

1: This is all based on the idea that there is some need for integration between project/work management and location-correlated data and tools. As you pointed out, for generating and editing the location-specific data, a wide range of dominant tools exist. So it makes me wonder, would it be adequate for your needs and/or a good number of the needs of this kind that you’re aware of, for the data to be added into Fibery and viewable (or “visualizable”, e.g. a map) but not necessarily editable? For example if you could simply upload a .kml created in another tool and Fibery would have appropriate fields to hold that data, and basic map visualizations to display it.

I ask because I’ve worked on a tool that incorporated GIS functions before, and as you probably know well, in order to implement these features well it requires a lot of domain knowledge and attention to detail. From accuracy issues to varying map projections, etc. If one can use a library like GDAL for some of the heavy lifting that’s great, but that in itself is a very “heavy” library to implement even a small part of.

Anyway, in summary, generation and editing of geospatial info beyond simple lat/long is complex. So perhaps simply integrating in some ways with existing tools would be good enough? This could either be through data import/export (aforementioned .kml), or through actual API connections, assuming ESRI or one of the other dominant players has such a thing (I’d imagine they do). This approach would seem to connect more readily with how Fibery is orienting itself, with integrations coming fairly early on (coming soon, in fact), and the recent automations, etc. helping to serve those needs as well.

2: Another thought might be, if Fibery team can identify geospatial as a “premium” profit center, i.e. they can charge more for those features (ESRI’s business would indicate potentially yes), then they could charge an add-on fee or higher overall fee for the “geospatial edition” or whatever, and thus potentially justify the higher complexity of that development and fund it directly through those higher proportional revenues.

1 Like

I would like to revive this topic.

I am using Fibery as a GIS and Geo-ICT Consultant. First, to organize my own work, which also involves many non-geospatial workflows. For this, Fibery is a great tool.

Secondly, I am using it for clients. I invite people to on my Fibery workspace to collaborate on specific projects, or as partners for multiple projects. And the non-geospatial features are all very useful.

But for all of this, having the ability to add geodata (especially vector data: (multi-) points, lines, polygons) would be helpful. I wish to describe a few usecases:

  • For a Sales CRM it would be very helpful to add the location of the client’s office and plot this on a map. As a starting entrepreneur, I plan to network a lot and having an overview of where my (potential) clients are could help me make decisions.
  • Event planning: having a map of an event could be useful. For example, I have ideas to make tools to organize sporting events. Fibery could fulfil this use case if you can make a map of where everything around the event is (volunteers, first aid, start, finish, etc).
  • Asset management: knowing where things are that you manage is useful, it’s hard to manage things that you lose :grinning_face_with_smiling_eyes:

Those are my main use cases, in that order.

Some ideas on how to do this:

Using GeoJSON for a geospatial field and Leaflet/OpenLayers/MapLibre would work for my use cases. A puzzle is on how to do the visualization. And it would be nice if you can add layers from other sources to the map (WMS/WFS/WMTS/etc). I could elaborate on this if anyone is interested.

That is all for now. Curious what you all think (and I will read the previous posts and will respond to that as well)

1 Like

While geo functions are interesting, I continue to feel they’re pretty niche as far as Fibery’s market is concerned. Which is a little ironic since I work in real estate so I’m one of the people that might benefit a little from it. :wink: But I think that may be also partly what informs my opinion that it should be low/no priority for the Fibery team. None of the use-cases you describe seem that compelling to me given the convenience of other options we already have. I’ll go case-by-case with your examples to explore a bit and then get to my proposed alternative(s) as far as Fibery functionality is concerned.

  • Sales CRM: Does the map truly serve a functional purpose here that helps you “network a lot” with your clients? For seeing their (individual) locations on a map you can just right-click on a selected address and do a Google search and have a map extremely quickly. Of course if you really do want to see all their locations on a map together, it would help to have a map that could pull location info from Fibery. But for this specific situation it feels like there are probably workarounds via Integromat/Zapier or another integration that would do pretty well and not require a whole new type of data, new types of views (maps), etc. in Fibery.
  • Event planning: I feel like this one is simple. Obviously you could do it outside Fibery, but I get that there may be value in having the Entity specifically associated with a position/location on a map of the event. However… I am a bit skeptical that actual geodata is at all necessary (or would even be useful) for this, depending on the size of the event. What I would do is just get a static image of a map of the area (could even be hand-drawn and scanned!), however large/small, and use that as a background of a Whiteboard and then you can easily add your Entity links, position them (or arrows to them) to show location, etc. What is much less obvious to me is any particular value from having the specific geocoordinates of these positions, and what you would do with that info in Fibery.
  • Asset management: again, why is the map - the specific geo coords - useful in itself beyond just having an address or location name or whatever that tells you where the asset is? You could, for example, have 15 business locations, each one a Fibery Entity, and each Asset has a relationship with the Business Entity Type, so then you get a list of what Assets are at each Location. I don’t really see a major value in having a map associated with this, nor even specific geodata (e.g. lat/long, etc.), but I’m interested to hear if there are use-cases or situations I’m not thinking of.

The common theme in my above approach is to take the simplest route, to focus on the end result capabilities (finding something on a map of an event, knowing which assets are in which location, etc.), rather than to be looking at it as “all of these things have position information associated and therefore could be used in a geodata-aware system”. Perhaps there are other examples where a more clear and critical business case can be made to have it integrated in Fibery, or maybe I’ve missed or misunderstood some of the aspects in the examples, so feel free to clarify. :slight_smile:

That said, the one major piece of functionality that does suggest itself for most of these (and other) scenarios is the ability to embed arbitrary remote web pages into Fibery, like some other systems can do. This would solve at least some of the need here, I think, especially in combination with e.g. Integromat-driven integration to a mapping system from Fibery data. You could then have data in Fibery, push it out to a remote mapping service, and display the results within Fibery still. And the upcoming Block system would seem to be an ideal way to implement an iFrame, too, rather than having to paste arbitrary code into a page, or from the / menu, potentially making it difficult to edit, maintain, resize, etc. The Blocks seem to harmonize very well with the idea of an iFrame and the tools you’d want to manipulate it…

Bottom line, I’d like to see some of this discussion revolve around consideration of what kinds of more broadly useful/applicable functionality might help serve some of the core needs of this feature request (or at least an 80/20 view of what very few features would have the highest benefit/impact, like e.g. KML support as I mentioned above). It seems unlikely that dedicated geo fields will be added, or at least if they are, then unlikely that any full mapping functionality will be included (due to complexity, the “weight” of libraries to do it, API access and request limit complications, etc.). So if you take that assumption, then what creative workarounds can you imagine, that might suggest new features that the broader core use-cases of Fibery would also benefit from? (iFrame being my particular example)

Some related discussion is also in this thread:

4 Likes

I would certainly want Fibery to focus on the core aspects that are useful for most clients, not on niche applications. I can find some workarounds (when I do I will share them), interesting to find out what is possible now.

A quick response to the use cases: of course I am a geo-specialist and like all other people from my field, we tend to have a strong inclination to use maps whenever we can :grinning_face_with_smiling_eyes:

The idea to embed remote web pages would certainly be helpful. Will think of what more broadly useful/applicable functionality might do the trick. I already have some ideas for this, in line with what you share.

I will respond to the rest of your post, I wanted to respond quickly from now on. Will look for some time to ponder your questions and response and experiment with Fibery :wink:

1 Like

The way I have been trying to get around this issue is by building a small mapping tool where you can create a map (including adding some popups and controlling basic styling) and the URL-encode it so that you can save it in a URL field in fibery. It is based on leaflet and the leaflet-geoman library with a few additional tools.

I’ve got some information and details on how to use the tool below in case it is of interest:

Toolbar Explanation

Editor Window

You can also add some data and style the features by using the edit feature data tool (pencil icon) and then clicking individual features:

I’ve tried to hack together various ways of making the URL as compact as possible, but there is still a limit to how much data can be transmitted this way. The URL is still very long and ugly to store in a URL field in fibery, so I go through an additional step of running it through url shortner.

You can try it if you want at QuickMap and can fork the repo (GitHub - cannibalflea/quickmap) if you would like to play around and improve it (e.g. move the centre/zoom to your area of primary interest). I have to say this was a couple of hours of work here and there and it is probably one of the most poorly-written pieces of code on the internet. I am hoping to clean it up in the future and add some other tools. I also need to add some of the images above to the readme to make using it easier.

I was initially going to save the GeoJSON in fibery in a rich-text field and maybe just pass an ID to the tool and have it query fibery for GeoJSON data. But the text fields can’t properly format the JSON and there is no validation, etc. not to mention how ugly it looks. The rich-text fields also are more particular in terms of access through the API, so I thought it makes more sense to do it this way for now.

This approach only allows you to visualize content on a map but doesn’t really help with any type of spatial analysis. It is also possible to build a small app with its own backend database so that you can save content and link it to fibery entities so that you are not storing any spatial data in fibery. This also allows you to do some spatial analysis (e.g. using turf.js) as well as consume other layers, etc. However, this again takes you away from a single tool to manage your data. It also means hosting some things and managing the various pieces. I don’t know if the fibery team or community have other suggestions to make this easier.

As @Oshyan said, I am looking forward to possibility of being able to build custom blocks and extensions for fibery. However, one of the things we may need to have natively in fibery is a JSON data/field type so that other extensions can be used on top of it.

4 Likes

Wow, bad code or not, this is fairly impressive and cool for a few hours of work. I realize Leaflet, etc. provides a lot of functionality, but still.

Thinking along here, let’s say you do the work of creating a little app with its own DB, as you say, and make it reusable for any project with similar baseline data (i.e. using common data definitions, types, etc. for geo type work). If you could link this data to Fibery via the API, and then embed the editor/app in Fibery interactively, would it really be a problem then that it was not “integrated” per se? That it is not, as you put it, “a single tool to manage your data”. If so, what are the primary issues/challenges of it that would ideally be solved?

I’m actually unclear on the plans for making custom blocks and other extensions possible (would love to hear clarification from @mdubakov ), but I think a JSON field type is an interesting and possibly very good feature request. Consider breaking it out into its own feature request topic.

1 Like

Have you tried pressing the Alt key when adding a new field…?

(completely undocumented and unsupported, i.e. here be dragons)

6 Likes

:exploding_head: Well now… that’s very interesting indeed.

1 Like

Wow, was this already 2 weeks ago? :blush:

@cannibalflea - Wow that is really impressive!! :white_heart: Lovely!! And will use it and let you know when I have found a nice application for it!

Having a JSON field would be useful, I agree with your suggestion as I see many applications (as @Oshyan mentioned it’s good to have more broadly useful/appicable functionality and I think this would work. I think in combination with other functionalities (like embeddable maps as Oshyan mentions) this could be pretty powerful… will think more about this! :slightly_smiling_face: I think we really are getting somewhere… thanks guys for your smartness I love it!

@Chr1sG - Ooooeh I will try pressing this Alt key, wonder what will happen… secret functions? Easter eggs? :thinking::grinning_face_with_smiling_eyes:

1 Like

In progress

https://the.fibery.io/@public/Public_Roadmap/Roadmap_Item/Field-Location-154

1 Like

This is exciting :partying_face: I won’t repeat why I think this is an important feature! And I know it is ungrateful to say anything critical when this was totally a no-go previously, but I feel very strongly that the proposed implementation plan is short-sighted.

If I read the feature description correctly, the intent is (1) to limit this to point features and (2) the point features are actually just going to be generated using address reverse lookup through one of the existing geocoding APIs. I feel like this is really a missed opportunity. While this approach will get you to an address and a map view pretty quickly it will really limit what you can accomplish. I think it would be much better to actually spend the effort to develop a geometry data type/field type that can be used for a variety of features, including multiple points in a single field.

If I understood finery’s stack properly, it is built on top of Postgres which offers one of the most solid implementation of geometry types natively and spatial functions through PostGIS. There are also very mature geospatial libraries that provide all the necessary spatial functions that most users will need. So all the ingredients are there to really develop something amazing. And as I said before, fibery doesn’t need to offer spatial analysis or complicated cartography options. The real feature here is having geometry be a fundamental data type.

I know this is a bigger ask and based on what had been said before I know the community doesn’t think there is value here. However, I can tell you that there is currently no geospatial tool that offers the data management capabilities that fibery has and that even the most advanced tools that ESRI offers through AGO, either limit you in how you can structure your non-spatial data or require an amazing amount of coding to get a fraction of the functionality that fibery offers. By the same token none of the data management tools seem to take spatial/geometry data seriously.

One final point: if you are going to go down the root of just supporting one point per field, I would strongly encourage you to at least allow direct lat/long entry without geocoding them again. Most geocoding APIs seem to take coordinates and apply the same nearest POI search and then return a coordinate along with metadata associated with the location. This is not a big issue in urban areas due to the density of addresses/POIs. However, you can end up with huge errors in less developed areas as the nearest POI/address point might be quite a ways away from your original coordinate. So you may input exact coordinates into the system (say in a csv) but when you export the same data your coordinates would be totally different. This is how Monday.com’s implementation works and it is really frustrating!

Apologies for the long message. I totally support whatever direction you take but felt I should again make a pitch just in case :blush:

2 Likes

This feedback is really helpful, thanks!

  1. We consider Geometry as a separate Field Type. We did discuss it, but so far we don’t see useful cases for our domain. We might consider it in future, but we need to understand the use cases and demand.

  2. Point field flaw you mentioned is indeed a problem, we will think how to solve it. Technically it is just UI issue, since Location Field can store just coordinates or augment it with geolocation meta data. We will think how to support both cases it should not be hard.

4 Likes

I’m in the same boat as @cannibalflea. I’m happy to see ANY geospatial abilities coming to Fibery, and I also hope to have simple “manual” point and geometry abilities at some point in the future as it will fill the huge gap between platforms like ESRI (hardcore cartographic analysis stuff) and all the other platforms of your competitors (like Monday and Clickup with their overly simplistic yet strangely complicated and poor UI/UX mapping systems that allow only the most basic address stuff).

My use case is admittedly not for software product management, but if I was in that space I could DEFINITELY see a use case for tracking sales, customer interaction metrics (such as number of tickets per category) per region with existing defined boundaries (like a country) and even manual geometric boundaries one could draw out themselves/provide lat long bounding points, and entities with a lat long within that boundary can be related as such if desired.

For my actually use personally, I’d use it to visually track my companies facility locations, property boundaries as they change due to add ons, customer job site locations and certain metrics for each, blah blah more non-primary Fibery use case purposes blah blah.

Note that this will be possible, since Location field in Fibery will be complex and with metadata, so you will be able to define at least lookups like Location.Country or Location.City and this have some filter to rely on in reports and views.

1 Like

That is seriously so awesome, there is so much potential with that ability, let alone all the other possibilities. Sky is the limit so it seems, very nice!

The most important thing is that when the user supplies a lat/long coordinate (in the location field textbox or via the map view), a lot of care must be taken if it is going to be parsed through a geocoding endpoint (e.g. google, bing, etc.) since when a coordinate is geocoded like a street address, that is usually when the a nearest point-of-interest returns and the location error is introduced.

You can see this in Google’s reverse geocoding example where the REST call was made with a lat/long pair (40.714224, -73.961452) but the “results” have different coordinates (geometry object). So if you were to take the (first) result returned, you have already introduced an error into the underlying data. The level of error is not significant in a dense areas with lots of results, but can become quite significant in less “interesting” areas.

Note: using the location_type=GEOMETRIC_CENTER parameter seems to reduce the error but I haven’t done that much testing to confirm this.

Apologies if I’m again repeating myself, but this can really make life hard for users who import point coordinates in, but when they export it out to use it with a geospatial tool (like QGIS or ArcPro), they will have completely different geometries.

We are working on Location field. It will be possible to store plain coordinates as well like this:

3 Likes