You are able to modify the configuration by using the actionAdminControllerSetMedia hook.
Inside the JS file you added with this hook, you need to create a global object containing your custom configuration:
window.defaultTinyMceConfig = {
[...]
}
and as you can see in the tinymce core file it will load it instead of the default core configuration.
Keep in mind that the config is not completely replaced, it’s an Object.assign, this means that if you want to remove a certain configuration, you’ll have to add it inside your own config, otherwise it will be kept.