One might think that mom would be relieved to get some time off once in a while. Is she hostile to the idea of them spending time with you? It wont be long until where they go and with whom they spend time isnt up to her anyway.
Absent the ability to work something out informally, then of course these things are matters of state law, and states have wildly different laws concerning divorce and family matters.
Even if a divorce specialist were to read your post, there would be no hope of getting any sort of an answer to your question, absent knowing what state you are in, if the question is whether you have any potential visitation rights.
https://www.nolo.com/legal-encyclopedia/grandparent-caretaker-visitation-rights-29548.html
Approximately 20 states have "restrictive" visitation statutes, meaning that generally, only grandparents can get a court order for visitationand only if the child's parents are divorcing or if one or both parents have died. In a restrictive state, even divorced parents who agree about preventing grandparent visitation have the right to keep the grandparents away.
However, like Washington, many states have more permissive visitation laws that allow courts to consider a visitation request even without the death of a parent or the dissolution of the family, so long as visitation serves the child's best interest. Others allow caretaking adults besides grandparents to petition for visitation. Still, these states usually require that the caretaker have a history of living in the home with the child for a certain period of time in order to be able to file the request.
Maybe you live in one of the restrictive states. Maybe you dont.