Use a "User Group" for notify action in automations

I want add a user group as the addressee for a notification via automation. These are people who just need to be “informed” of a change, so they are not related or assigned to the entity that was edited. It should be a user group since the specific user(s) can change (this user group is from our Role database), and those in the relevant roles are the ones that need to be notified.

I am struggling with creating a successful formula since the Assignees field is a collection. Is there a way to accomplish this?

You can just reference this entity using an entity query, e.g.
Roles.Filter(Name = "Dev team").Members

(assumes you want to notify the “Dev team” and assumes that the field name for the relation between Role and Users is Members)

I tried this, but it gives me the error “Can’t access field Assignees on collection field. Use aggregation function on collections instead.”

Sorry, my bad. Try this:
Roles.Filter(Name = "Dev team").Sort().First().Assignees

It now returns with this error
Can't access collection field here

Is the role:user relationship 1:m or m:m ?

m:m. A user can have multiple roles, and a role can have multiple users.

OK, I think we need to solve this from a different direction.
I suggest creating a formula in the User database that concatenates the Role names, e.g. Roles.Join(Name,"")
Then in the automation, you can do this:
Users.Filter(Find(RoleNames,"Dev Team")>0)

This will work provided there is no risk of overlap in role naming. In other words, if you have Roles called ‘Engineer’ and ‘Dev Engineer’ then it won’t work when looking for only Engineer (it will find assignees from both roles).

If you want to be more careful to avoid this risk, you can use delimiters, e.g.
the formula could be
Roles.Join("<"+ Name + ">","")
and in the automation:
Users.Filter(Find(RoleNames,"<Dev Team>")>0)