The chief justice appointed everyone on the FISA court. It would make him look bad if they got big questions wrong.
It would effectively mean that jurists he appointed failed in hugely consequential ways – that they permitted civil liberties abuses that they ought to have stopped. If SCOTUS reached such a decision, it would certainly be understandable if the result and its implications made Roberts uncomfortable. Depending on the scale of the abuses, it might even make him fear for his legacy.
He won't (and can't) be forced to recuse himself.
The funny thing is all these judges in the FISA system were appointed right before Obama launched his illegal FISA spy ring on Trump and thousands of Americans right before the FISA court warrant to spy on Trump was given.
DiGenova states first-hand knowledge that FISA Court Judge Rudolph Contreras did not recuse himself – but was rather forcibly recused from the Michael Flynn case by either U.S. Supreme Court Chief Justice John Roberts or the aggregate FISA court.
The only two significant things that happened between the initial Mike Flynn plea hearing and the recusal of Judge Contreras was:
#1) The stories about anti-Trump FBI Agent Peter Strzok and his involvement with Fusion-GPS and Christopher Steele; and
#2) FBI Director Chris Wray appearing before the House Judicial Committee and hearing Representative Jim Jordan demand to see the 2016 FISA application.
In fact, Judge Contreras was recused only a few hours after that House Judicial Committee hearing.
https://theconservativetreehouse.com/2018/01/07/wow-interesting-interview-with-joe-digenova-links-doj-nsd-john-carlin-with-fisa-court-judge-rudolph-contreras/
https://www.puppetstringnews.com/blog/fisa-judges-that-gave-obama-fisa-warrant-on-trump-appointed-under-obama
https://www.theatlantic.com/politics/archive/2013/07/john-roberts-should-recuse-himself-from-challenges-to-fisa-court-decisions/277825/