Going to have to do some clean-up.
Raised beds after vacation…
McKinney Falls State Park
We finally made it out to McKinney Falls State Park this week – arriving last Sunday and staying until this morning. It’s consistently booked well in advance, probably due to the proximity to Austin, and we’ve never been able to arrange a trip there at a time that worked for us. We decided to spend a week there, arriving the day most people leave and leaving the day most people arrive, in order to work around the booking issues. Still, the park was surprisingly busy for a Sunday-Friday stay.
I didn’t get a great photo of our site, but snapped this one of the rig in place. When we arrived they wanted to put is in a pull through site due to the size of the rig (most check-in people seem to ask the length, and upon hearing 40′, assume I won’t be able to back it), but they gave us a couple alternate sites to choose from if it wasn’t free yet (we arrived slightly before checkout times were over). We ended up skipping over the pull through site because someone was parked in it (we found someone wasn’t even in it, just using it for parking their DRW truck (rude!)) and it worked out great, because the first alternate was a back in site that was easy enough to get into and was much more private. We didn’t have any neighbors on the door side of the trailer, and for the majority of the stay, the site on the street side was empty. There were no neighbors behind us, instead, there was a trail through the brush that led to the Onion Creek trail. When we go again, we’re going to ask for this site, which was #17. We were able to book another stay for Kelly and her friend, Mandy, in September for a girl’s trip.
Parked at the site.
Map of the park
The creek running through the park. There are several hiking trails which you can use to get to it.
Kelly at the Rock Shelter – a natural structure of overhanging rock that comes out about 15′, and is probably 75′ long.
Another view of the rock shelter.
Interesting enough, the roof of that is seemingly covered with clusters of spiders. I wonder how many people notice.
Kelly at the upper falls.
Maggie really wanted to play fetch with this drift wood she found…
The terrain and a smooth portion of the lower falls.
The trip turned out great and, being only a half hour away, McKinney Falls State Park is definitely a place we’ll frequently return to.
Goodbye CarPC, Hello Alpine X110-SRV
This is unfortunately an example of “chasing good money after bad”. I’ve had CarPCs before, and they really work great in the right vehicle. Those vehicles, I now realize, are older ones that don’t rely on the head unit for the level of integration today’s cars do. In addition to losing things like HVAC visibility (not a huge deal), some of the steering wheel controls (Joycon wouldn’t recognize them), and various other small features, the biggest problem I ran into was the $249 Axxess GMOS-MOST-01 harness you have to use with any aftermarket head unit on these trucks kept dying. I was on my second one (first one replaced under warranty), and the third has began having issues with not coming on (have to cycle the ignition a few times) or spontaneously not producing sound. It’s out of warranty now, and needed replacement. To replace it means pulling apart and reassembling part of the dash. Some of the other issues I had were with Windows itself, like poor bluetooth reliability for hands-free calling even with BlueSoleil, the slow and/or non-existent development of applications like Centrafuse or CoPilot (and no support to lower music volume when NAV speaks), and overheating in 100+ degree weather. CarPCs have their place, but this truck isn’t one of them.
To install the CarPC, I had to cut away some of the interior of the dashboard; portions of it that were used to mount the factory radio components. Also I had sold off some of the factory components to make some of my money back on the CarPC install. Whatever I ended up replacing it with, I knew I’d have to deal with that mess during the install. And there aren’t really any options out there, all rely on the Axxess harness I’ve been having so many issues with. That is except for the Alpine Restyle series. They make highly integrated vehicle-specific factory replacement aftermarket units, but charge an arm and a leg for them.
I’ve had my eye on the Alpine X110-SRV for a while now, but they’re asking way too much at the retail asking price of $4,000. Even their special offer pricing of $2,499 seems extremely steep for what it is, but with the recent overheating issues and the inability to easily return to stock (I’d have to find components, go to a dealer, have them do the install because it would need to be programmed to the vehicle and I don’t have the equipment, etc) I finally broke down and bought it at the special offer price from Crutchfield. Note: I suspect they’ve permanently dropped the price to this, because it wasn’t selling at full MSRP. Or if not, they should consider it.
Anyway, I spent several hours today removing the CarPC and all the cables I ran all over for it, cleaning up some of the ham radio and dashcam cables, and installing the Alpine system. I love it so far, and the NAV has a commercial truck routing mode that I believe will work well for us with the RV. It comes with free HERE map updates, (it better for that price), and I updated the maps to Q1 2017 after the install was done. Both the install and the upgrade process was fairly painless. Of course, that’s partially attributable to the extensive experience gained from messing with the CarPC so much, and the ham installs. I can pull apart the interior pretty quickly now.
Due to the massive 10″ screen, and the controls it adds, I lost the mounting location for my Kenwood TM-D710G head, so I used some tape used for attaching molding to vehicles to stick it to the top of the new head unit. I also used that tape to keep a few components in place during the install, but I was surprisingly impressed with how well everything fit in the dash even after the modifications I had done.
I lose the ability to use CHRIP for programming my ham radios, and to use APRSISCE/32 to visualize APRS locations, but that’s about it. It even has OBDII and can provide additional gauges, engine codes, and such.
Temperatures under control
I wrote custom Nagios NRPE checks that output perfdata, which I then graph with PNP4Nagios. These allowed me to track changes over time and see what actually worked and what did not.
After all of the testing, I found that Delta AFC1212DE case fans over the 1070 cards, and Delta TFC1212DE case fans over the 1080TI cards, give the above results. You can see when I finalized the changes in the graphs – the temps went down and power draw (performance) went up. I am not running any AC or swamp cooler in the garage (returning both), and using only four each of the two aforementioned fans is keeping things well under dangerous temperatures. It’s loud, but out in the garage, that isn’t an issue.
I also added this fan and door vents that seem to do far more than either the AC or swamp cooler, and make the garage considerably more comfortable. They’re sending insulation to put on the inside of the door that should arrive this week, as well.
Indoor vent to the attic fan. It’s hardwired into the power source for our garage door opener, and has a thermostat that activates it at 90F.
Also had to cut holes in the garage door for these vents.
This is another graph I made to show the trends:
And after some more time for things to stabilize:
Mining Rig Update
We’ve been struggling with cooling issues for the last week. The two rigs with the GTX 1070 cards are running warm, close to their thermal caps, but are stable and happy working that way. The GTX 1080TI cards, however, were throttling themselves and even turning themselves off because they were simply running too hot out there. I tried adding case fans in all the locations allowed by the case, as well as using the Tripplite SRCOOL7KRM air conditioner, a swamp cooler to cool the room, running without lids, and various other things. Now I am going to return the $750 air conditioner and $600 swamp cooler, since neither made any difference.
The cards are quite crowded and prone to overheating.
I was having so much trouble getting temperatures under control, I stood up a Nagios instance on my VMware Environment and made custom NRPE checks with PNP4Nagios graphs to visualize temperatures as I made changes. Eventually, I found that sitting Delta 120MM 200CFM case fans directly on top of the cards, blowing air down between them, seems to work effectively, for a total of $20 per fan and two fans per machine:
No Delta fans over the GPUs, Delta fans around 13:30, reboot around 14:10, and up with a 200w power limit at 14:20. The more power they draw, the faster they operate. There is a performance/watt ratio that you look for, and I’m currently running the cards in miner3 at a 200w limit. They dynamically adjust based on load, using that figure as an upper limit.
The fans won’t fit inside the case, and there was no good way to mount them, so I took out a section of each case lid with an angle grinder and a cutting disc.
The Delta fans are 38mm thick, which is a tiny bit too tall for the case lid to close. So each 4U server needs about 6U for effective cooling.
The rest of the rack has now found its place on the back wall of the garage, near the entrance door to our house.
The spacing on the racking is odd because of the AC unit I initially installed. The Chenbro rails are a huge pain to move, but I’ll probably space them more evenly at some point. There were also a few 1U servers in there I’ve since pulled.
I’m using blanks to keep the exhaust air from being pulled through the intakes of the servers. It does make a small (1C) difference on the graphs.
The HP DL180 G6 runs ESXi, and hosts my OpenBSD router/firewall, domain controller, file server, a backup server for a remote location, my development environment, and all the framework used to manage the miners.
I bought this 24 port SMC POE switch off Ebay, new in box, for about $70 shipped. The model isn’t listed anywhere, and it doesn’t have updates available, but for home lab purposes works well. It looks to use the generic Broadcom firmware, with the same management interface as Dell PowerConnect and some Netgear switches. I’m actually fairly confident that this is identical to the PowerConnect 5424 internally, and the aforementioned firmware would work on this.
Cable management isn’t the best, but Velcro ties pull everything out of the way nicely. The managed PDU allows for me to monitor voltage and current remotely, and I have Nagios alerting me when power limits fall below or exceed certain thresholds.
The 240V/30A drop to the rack comes in through the ceiling, as does one of the Ethernet connections back to the house (this one leading to a TP-Link EAP-330 wireless AP). To comply with fire code, I apparently need to fill the rest of the hole with expanding foam.
Anyway, now that the machines are running cool, they’ve been put to work mining and we’re receiving ETH, ZEC, and LBRY deposits daily.
Edit – ReRacked
I reracked everything with 4U of spacing between nodes, and the switch on the back of the rack. That provides space for one more miner, which is probably about the safe limit with cooling and electricity here, assuming I upgrade to all 1080TIs (which I’ll do before I add a fifth node).
Ignore the askew thermometer – I noticed it right after taking these and fixed it.
Cabling is a lot easier to manage when it all just needs to go up.
Even 4U spacing leaves plenty of room to get in and work on these things without removing them from the rack or using the shoddy but overpriced rails.
Mining Rig Updates
In the interest of a quiet and cool environment inside the house, I ended up ditching the open frame chassis and moving to rackmount servers in the garage. The progress on those has been slow, because of parts issues and dealing with the heat in there.
I ordered a low end Tripp-Lite SR2000 rack off Ebay, it was “open box, but in perfect condition”, but arrived missing a side panel. I saved about $400 on it, so for now I’m living with out. I threw a large piece of wood over the side that faces the wall, leaving the side with the panel exposed. I also ordered a rackmount AC unit by TrippLite, but it was damaged during shipping (arrived with a loose faceplate) and isn’t cooling well (possibly my fault, as I didn’t wait long enough for the coolant to settle after it being shipped on its side, despite plenty of “This Side Up!” labels plastered on the box). It does blow cold air, but even just the back pressure of the optional exhaust vent is enough to cause it to overheat and shut the compressor off. TrippLite service is awesome, though, as they’re sending a new one immediately and it will be here next week. I found a old monitored TrippLite vertical 208/240v 30A PDU on Ebay and that completes the rack.
It’s on casters, so I push it up against the wall when I don’t need to open the door all the way.
As for the machines, I’ve opted to do four machines, two with 4x GTX 1070 cards and two with 4x GTX 1080TI cards. The systems with the 1070s have been going strong for a few days, making about $30 a day between the two of them. The 1080TI machines should make a good bit more, and hopefully the market will improve to get the numbers where I really want them. Still, I’m looking at $60-$80 day, depending on the market, at this point with all these systems in place, and that makes for a ROI in five months or so (not including the rack), which is considerably shorter than most investments.
I’ve bought and returned several components over the past few weeks, and have finally come to a conclusion as to which parts are best for one of these rigs. If I were to do it all over again, I’d standardize on:
- Chenbro RM41300-FS81 chassis
- Corsair HX1200 power supply
- MSI Z170A XPOWER Titanium motherboard
- Celeron G3930 processor
- 2x 4GB Crucial DDR4-2133 ram
- Samsung 32GB M.2 SATA ssd
- 4x EVGA GeForce GTX 1080TI SC2 video cards
- Lots of velcro straps to keep cables out of the way
Mining Rig (Open Frame)
I abandoned open frames for my deployment (more on that later) but did end up building my open frame design for Larkin. I have the parts to build three more, so they might be published on Ebay soon. I think it turned out rather decent, though there are a few things I’d change the next time around.
Mining Rig Progress
We’re still waiting on parts to come in, and disappointingly, twelve of the cards we ordered (six for Larkin, six for me) from Dell are backordered until August. We’re currently trying to decide whether to cancel the orders or hold out, since we can’t seem to find any alternatives.
I was able to put a small rig together with three GPUs to perform testing on, and have created a github repo to manage the configuration scripts I’m using.
https://github.com/jshupe/ubuntu-nvidia-miner
We’re seeing 30MH/s from each card with ETH and 400 Sol/s with ZEC, while drawing around 348 from the wall, and squeeze out another 25 Sol/s from each card for ZEC if we let the system pull 435w.
The machine itself is just put together on a test bench, but seems to be stable and has facilitated all the software tested necessary to streamline the operation.
Home Office, Part 2
I posted the other day that I ordered 2x 27″ 4K monitors, new mounts/stands, a (new to me) Steelcase Leap V2 desk chair, and a standing desk. Most of the items came in, but the standing desk was lost by UPS, and Amazon wouldn’t replace it for the same price (it was an “open box warehouse deal”). I ended up deciding not to go with the standing desk, since that will save near a thousand dollars, and I’ve already spent enough money with the recent investment in cryptocurrency.
The new monitor array – dual 4K 27″s right in front of me, and a 40″ 4K above, is a little overwhelming, even for someone who has been using a 40″ flanked by 21″ 1080P monitors for the past couple years. One of my complaints about using the 40″ as a primary was that it was too large to use as a primary work area: moving from one corner of the screen to another, when working on the same task (imagine, for instance, working with a single maximized window) is a problem. The 27″ is a much better size as for this, but requires display scaling for comfort which wasn’t required on the 40″. Fortunately, with the stands that bring the monitors to me, I’m comfortably using them at only 110%.
The stands are by far the most impressive component of my new setup. I’m able to move my monitors around to how I want to sit, and primarily find myself leaning back into my chair, for once, which has never been the case with my prior configurations. I use DisplayFusion to split my left and top 4K monitors into four areas, and use my left monitor exclusively for email (two email clients), HipChat, and Spotify; the top monitor currently keeps an eye on the security cameras and has a browser open to GDAX for following the coin markets. I also sent my XMPP and Skype buddy lists up there, which still leaves space free for another couple windows to remain visible all the time. The last monitor, the lower right, is not split and is what I bring close to myself using the stands. I use it for actual “work”, while the others serve the purpose of monitoring various things as described above, and a place to throw the odd video or piece of documentation from time to time.
The stands allow for almost infinite adjustability of the monitors across the desk.
The new chair is also fantastic so far. It came in near perfect condition, and certainly doesn’t look as if it is four years old. I doubt it has been used much, or if it has, it has worn extremely well. There are a few ripples in the back of the seat, as is the nature of leather, but aside from that everything looks brand new. The adjustability is tremendous, comfort is great, and an odd thing that stuck out to me was the ease and smoothness in witch it rotates, rolls, and generally moves in comparison to my old chair. The jury is still out on whether it will alleviate my back pain, as my back actually hurts more since using it. I currently attribute that to having better posture in this chair, which I’m not used to.
Overall, I’m a little disappointed in not having a standing desk, but not enough to shell out $1,000 for one (especially with my recent investment into cryptocurrency). But the monitors and monitor stands are great, the chair seems like a big step up, and I’m so far very impressed with the setup I was able to put together with the items Amazon/UPS actually delivered.
