Answer to my original question
No, ācall groupsā do not limit who can call who, they are meant to work together with pickup groups to control which extensions can grab calls ringing in nearby extensions.
Ok, but letās make them do it anyway! 
Go to Admin / Config Edit, select extensions_custom.conf and paste this bit of custom dialplan:
[macro-dialout-one-predial-hook]
;this is called from /etc/asterisk/extensions_additional.conf, check other available vars there
exten => s,1,Noop(Entering user defined context macro-dialout-one-predial-hook in extensions_custom.conf)
exten => s,n,Set(grp_src=${PJSIP_ENDPOINT(${CALLERID(num)},named_call_group)})
exten => s,n,Set(grp_dest=${PJSIP_ENDPOINT(${CALLERID(dnid)},named_call_group)})
exten => s,n,Set(grp_src=${STRREPLACE(grp_src," ",)})
exten => s,n,Set(grp_dest=${STRREPLACE(grp_dest," ",)})
exten => s,n,Verbose(CUT: ${CUT(grp_src,\,,1)} ${CUT(grp_src,\,,2)} ${CUT(grp_src,\,,3)})
exten => s,n,GotoIf($[${ISNULL(${CUT(grp_src,\,,1)})}]?blocked)
exten => s,n,Set(modifiedString=${STRREPLACE(grp_dest,${CUT(grp_src,\,,1)},)})
exten => s,n,GotoIf($[${LEN(${grp_dest})} != ${LEN(${modifiedString})}]?substring_found:substring_not_found_1)
exten => s,n(substring_found),NoOp(src group FOUND in dest group list: CALL ALLOWED)
exten => s,n,MacroExit
exten => s,n(substring_not_found_1),NoOp(src group 1 <${CUT(grp_src,\,,1)}> NOT found in dest group list)
exten => s,n,GotoIf($[${ISNULL(${CUT(grp_src,\,,2)})}]?blocked)
exten => s,n,Set(modifiedString=${STRREPLACE(grp_dest,${CUT(grp_src,\,,2)},)})
exten => s,n,GotoIf($[${LEN(${grp_dest})} != ${LEN(${modifiedString})}]?substring_found:substring_not_found_2)
exten => s,n,MacroExit
exten => s,n(substring_not_found_2),NoOp(src group 2 <${CUT(grp_src,\,,2)}> NOT found in dest group list)
exten => s,n,GotoIf($[${ISNULL(${CUT(grp_src,\,,3)})}]?blocked)
exten => s,n,Set(modifiedString=${STRREPLACE(grp_dest,${CUT(grp_src,\,,3)},)})
exten => s,n,GotoIf($[${LEN(${grp_dest})} != ${LEN(${modifiedString})}]?substring_found:substring_not_found_3)
exten => s,n(substring_not_found_3),NoOp(src group 3 <${CUT(grp_src,\,,3)}> NOT found in dest group list)
exten => s,n(blocked),NoOp(No more src groups to search for: CALL BLOCKED)
exten => s,n,Playback(silence/1&cannot-complete-as-dialed&check-number-dial-again,noanswer)
exten => s,n,Hangup()
After pasting, press Save and then the red 
"Apply config**.
Caveats
- this is not proper security, it can be circumvented by clever users. Itās just a basic first-line of nudging people to call the right extensionsā¦
- for simplicity, this only checks the first 3 groups listed as Call Groups in the calling Extension settings.
- you will get funny behaviors if the group names include each other as a substring, so avoid that. The string comparison is simplistic so a group called Department will also match a group called AllDepartments, for example.
- my skills writing dialplan suck, there are surely many better ways to do this
Explainers
- this is a generic solution, suitable for non-technical users. Apart from copy-pasting a bit of code, the rest of your experience is to just set call groups in the UI, and then it should ājust workā.
- Here is a rigorous description of what this does:
Calls will only go through if one of the first three Call groups in the calling extension matches any of the Call groups in the called extension.
- The default is to block: when there is no match, or there are no defined groups in one of the two extensions, the call gets blocked. So youāll need to go in every extension and set call groups, or use bulk update.
Technical Explainers
- I used the existing extension field Call Groups but you might prefer to use something else, like a custom field that only exists in the dialplan, or some other field. You have to think how this affects the call pickup functions, if you use them. My feature has nothing to do with the basic FreePBX feature - I am basically redefining it to my convenience.
- the dialplan functions have no easy way to search for a substring in a string, so I used a trick with STRREPLACE (replacing the searched for string with an empty string) followed by a LEN comparison (if there was a replace, the string is now shorter than the original), so I know there was a match. Has some quirky edge cases, but works for the most part.
- I didnāt test group names with spaces or punctuation or funny characters. Just go for simple group names separated by commas. Mine are
ao, conv, rs
- when testing this or troubleshooting, I recommend running a console with
tail -f /var/log/asterisk/full | grep predial to see all the messages.
Thanks to everybody that helped 
. I am open to suggestions for improvement, obviously.
I guess using proper programming languages gives me bad habitsā¦