Denial of due process is wrong in any circumstance. Once it's denied it in one area, why not deny in another?
Just look at the family courts for an extant example of this.
Regarding illegal entry into the country, I struggle. Yes, I believe in asylum as something that is good.
No, I do not believe that taking gross advantage of a good thing is something we should tolerate.
The answer? Beats me. Not my wheelhouse.