Yes, I think there is a really big missed opportunity in smart folders and lists by limiting grouping and hierarchies to only relation types (see also Group by fields in hierarchical views). Expanding grouping to other field types would really make for really interesting ways of viewing data.
This would be in addition to supporting grouping in fibery table views which seems to be in high demand (and a standard function in other tools, even something as horrible as SharePoint 
)