I agree, iterating factors is literally the exact thing that we're supposed to be circumnavigating.
Can you give me your thoughts on T=X? What exactly is the big square, and what exactly is the small square?
I've also noted that the -E side (F), values seem to have (|E|, N) and (D,X) swapped? There may be insights in the grid if mirrored into negative space, or rotated into imaginary space.
Column 0 is actually very simple. (0,0) contains all of the perfect squares once, while (1+,0) has them again a second time. Also every (1+,0) coord is filled. Why do perfect squares perfectly fill a column, but imperfect squares don't?