not sure if I am holding the word THEM under a microscope too closely or not.

Article 2 Section 3 states

he may, on extraordinary Occasions, convene both Houses, or either of them, and in Case of Disagreement between them, with Respect to the Time of Adjournment, he may adjourn them to such Time as he shall think proper;

Scholars are saying the powers are linked to timing of adjournment only and that seem correct…….sorta

If POTUS calls them to special session and they cannot agree on the subject matter and wish to adjourn, it must be read as POTUS being included in the word them.

If POTUS can call only 1 house to special session and a disagreement between them as to when to adjourn takes place, POTUS is the other side of the disagreement

https://usconstitution.net/xconst_A2Sec3.html