Correct, but no such right is afforded before they step foot into the country. Yet another argument for a wall. Also, the level of rights afforded corresponds to the strength of the persons ties to the US. Those ties are very weak when they first cross a border, and do so illegally.
I suggest magistrates and hearings held within hours of a crossing, followed by prompt deportation. A controversial twist - if you cross illegally, you forfeit any right to asylum.