I wrote custom Nagios NRPE checks that output perfdata, which I then graph with PNP4Nagios. These allowed me to track changes over time and see what actually worked and what did not.
After all of the testing, I found that Delta AFC1212DE case fans over the 1070 cards, and Delta TFC1212DE case fans over the 1080TI cards, give the above results. You can see when I finalized the changes in the graphs – the temps went down and power draw (performance) went up. I am not running any AC or swamp cooler in the garage (returning both), and using only four each of the two aforementioned fans is keeping things well under dangerous temperatures. It’s loud, but out in the garage, that isn’t an issue.
I also added this fan and door vents that seem to do far more than either the AC or swamp cooler, and make the garage considerably more comfortable. They’re sending insulation to put on the inside of the door that should arrive this week, as well.
This is another graph I made to show the trends:
And after some more time for things to stabilize: