Jesse Williams' Ex-Wife Asks Court to Deny His Request to Change Custody Agreement

Jesse Williams' ex-wife's asking the court not to make any changes to their custody arrangement ... saying giving Jesse more time with the kids just means they'll see their nanny more.

The actor's ex, Aryn Drake-Lee, filed a response to Jesse's motion asking for more time with their children Wednesday ... saying Williams is out of town more than he's in town, so a babysitter will primarily be interacting with their kids if they stay with him more.

Drake-Lee claims their kids stayed with a nanny for much of this past spring while Williams was off working somewhere ... and says the kids missed more school during this period than any other time in their lives.

ADL also claims her ex and his lawyers are constantly threatening to take her to court if she doesn't fall in line with his demands.

Aryn's asking to keep the existing custody agreement intact and deny Jesse's effort to switch to a 50/50 split.

For his part, Jesse claims Aryn repeatedly interferes with his visitation and communication with the kids ... interrupting FaceTime calls and blocking the kids from attending activities he signs them up for.

Jesse says Aryn's trying to cause drama and fracture his relationship with their kids ... regardless of their children's own happiness. So, this is clearly a contentious custody case.

We've reached out to Jesse about Aryn's response ... so far, no word back.