FreeNAS Xeon D Build Journal

While my existing FreeNAS server works well and has not had a single problem, there are some things I wish I had done differently. Thus I present to you version two of my FreeNAS server.

* Do not use this case. Get the SuperChassis 743TQ-865B-SQ instead.


Fans
Let us start with the elephant in the room and the reason for the above recommendation. While both the 743T-665B and 743TQ-865B-SQ come equipped with a Super Quiet Power Supply, albeit different wattages and 80+ ratings, only the 743TQ-865B-SQ is a “Whisper Quiet System Chassis (28dB)”.

 

Part Number Rated Current (A) Fan Speed (RPM) Air Flow (CFM) Static Pressure (in.H20) Noise (dBA)
FAN-0104L4 0.13 2,800 32.9 0.14 24.0
FAN-0103L4 0.15 2,400 40.0 0.12 25.0
FAN-0074L4 0.58 5,000 68.3 0.53 45.0

The case I used (743T-665B) comes with four FAN-0074L4 installed in the middle. While not as loud as some servers, they still create quite a loud droning noise. This is mostly due to their high operating speed. The lowest I saw them was at 3,200 rpm. This is consistent with their maximum speed and the Standard fan speed setting in the SuperMicro BIOS.

Unfortunately, the X10SDV-TLN4F does not have an Optimal setting like my MBD-X10SLM-F-O. Perhaps this would have allowed me to use the original fans. The case I recommend instead(743TQ-865B-SQ) comes equipped with two FAN-0104L4 and one FAN-0103L4. These fans reach as low as 1,700 and 1,800 rpm respectively. Due to the fact that I am running thirteen drives in my server I elected to install all four of the middle fans.

case fan

Hard Drive Trays
My biggest complaint about the Rosewill RSV-L4411 4U Rackmount Server Chassis is with the drive trays. While plastic, the trays are decently solid. The latching mechanism, not so much. But for home use they were still serviceable. The problem comes when you want to purchase additional trays, be it because you broke one or simply want spares. The only way to get additional trays for the case is to purchase an entire cage and then use the four trays that it comes with. The SuperMicro trays are available to be purchased individually. Unfortunately, you do need to decide if you want to mount a 3.5″ or 2.5″ drive as the trays are different, unlike the Rosewill. The 2.5″ is not the easiest tray to mount drives in as there are no holes to allow access to the drive screws.

ssd tray

Case
Barring the previously mentioned fan debacle, the SuperChassis 743T-665B is a great case. It is a tower case that can be converted to a 4U rackmount. While it is slightly deeper, it is actually narrower and shorter than the Corsair Obsidian 650D I use for my desktop. It is quite heavy, especially when fully loaded with drives.

case side open

The 743 comes equipped with 8 hot swap bays that are covered with a locking door. I needed more bays so I added the Mobile Rack CSE-M35T-1. While I was told this was the proper addition for this case, the manual does not list the 743. Whether this was just a printing oversight or not, the case will work with a few caveats. As you can see, there are blue tabs holding in each of the 5.25″ bays. They are what lock the bay into the case and prevent it from sliding around.

You will need to remove the blue tab bracket from one of the 5.25″ bays and install it on the Mobile Rack. This may fix the second problem I had, but I did not realize the tabs needed to be transferred until after everything was assembled and I decided not to unhook and redo all of my cabling.

cage tabs

The tabs shown in this picture prevent the Mobile Rack from being inserted. The rear tabs are designed to automatically retract as you slide something into the bay but the front tabs must be pressed in. This may not be an issue if you transfer the blue locking tab over. If not, you may be able to push the tabs in by hand from the back side.

The clearance is tight enough that I did not find this possible. Instead, I took two pieces of card stock and placed them in the bay, then sliding the Mobile Rack in, which pressed the tabs in and allowed for installation. Once everything is inserted, simply pull the card stock out.

card stock

As has been typical with this build, I did not realize my clearance issues until I had already gone through the above machinations to install the Mobile Rack. The Mobile Rack comes with a fan on the rear. That fan protrudes far enough that it conflicts with the middle case fan.

fan clearance

I chose to remove the fan on the Mobile Rack. It is held in with a few clips that you squeeze to loosen. Then just unplug the standard fan connector. However, you must change two jumpers that are located behind the grill where the fan was mounted. This will notify the Mobile Rack that there is no fan present and therefore it should not alarm.

mobile rack rear

Power Supply
The 743 uses a custom power supply size. This unfortunately precludes me from using the same CORSAIR HX series HX650 Power Supply from my previous build. The HX650 is 80+ GOLD rated and operated primarily in fanless mode due to the low idle power usage of the server. This is actually the reason I chose the SuperChassis 743T-665B instead of the SuperChassis 743TQ-865B-SQ.

The 743T-665B includes a 665W 80+ Bronze PSU vs the 865W 80+ PSU in the 743TQ-865B-SQ. I just failed to realize that the 743T-665B was only Quiet rated for the PSU, not the case fans. I now have the efficient PSU and quiet case fans but it would have been cheaper and easier to purchase the 743TQ-865B-SQ originally.

CPU/Motherboard/Memory
While the Intel Xeon E3-1220V3 Server Processor in my original build worked well, lack of reading the full specifications caused me to miss that it did not include hyperthreading. In building the new server I could have gone with an E3 v3 processor that had hyperthreading but that would only have given me a small boost in CPU performance and left me with all of the other constraints of the E3 v3 platform.

I considered moving to the E5 v3 platform as that would have allowed me to use more memory but decided the power usage and cost was not worth it. Intel recently changed the E3 memory limit from the 32GB it was in v3 to 64GB in v5. This would give me more memory to run my VMs but E3 v5 were almost impossible for me to get my hands on at the time of ordering. Finally I settled on the X10SDV-TLN4F.

d1540

This provided me with low power usage and a high memory limit of 128GB. It also includes 10GB copper connectivity. Sadly, the Mini-ITX form factor only provides me with one PCIe slot. That and the high cost are the two main negatives I have with the board. Unfortunately, the back plate only contains two USB ports. In order to plug my UPS in I had to relocate one of the two USB drives I use for FreeNAS to the front of the case. The motherboard also includes only three fan headers. They are four pin but not enough to power all of the fans in the case. The center header is taken by the CPU cooler fan. As I am not using a 1U case with high airflow I elected for the version of the board with the integrated cooler.

Despite its small size it is fairly quiet. The top fan header connects to the SilverStone SST-CPF04 which in turn is connected to the center case fans. Therefore they operate as a single unit, all spinning at the same speed. The final fan header is connected to the rear case fan. Even though the fan headers are on the far side of the motherboard from the rear of the case, the Mini-ITX size means that the rear fan is still able to connect.

ddr4 memory

While the motherboard supports 128GB of DDR4 ram, I elected to only populate it with 64GB. However, I did use two of the 32GB Samsung MEM-DR432L-SL01-ER21 instead of four 16GB sticks. While slightly more costly, this will allow me to easily upgrade to 128GB once the price of DDR4 comes down.

Host Bus Adapter
I am using an M1015 along with a pair of 1m 30AWG Internal Mini SAS Breakout Cables again. They provide the connectivity to the eight main drive bays. I am using the motherboard SATA ports for the five bays in the Mobile Rack.

final wiring

Build/Testing
When testing the case airflow I used DBAN to stress the CPU and drives at the same time. DBAN 2.2.8(the most current version on the UBCD) would not see the M1015, only reporting the five drives attached to the motherboard. I had to get 2.3.0, the latest version of DBAN directly from the website. Here are the power figures I measured during testing. All except the last figure were with thirteen hard drives connected. Idle figures were checked at the DBAN configuration screen.

  • 154W – Original fan configuration idling
  • 221W – Original fan configuration zeroing drives
  • 146W – Quiet case fans, no rear fan idling
  • 211W – Quiet case fans, no rear fan zeroing drives
  • 148W – Quiet fan configuration idling
  • 212W – Quiet fan configuration zeroing drives
  • 64W – Quiet fan configuration idling with two SSDs connected

The quiet fans take longer to bring the temperature down but will eventually end up at the same idle temperature as the original case fans. They do however allow the system to reach a few degrees higher when under load. This does not appear to have any adverse affect on the server as there still remains headroom in the fan speed.

7 thoughts on “FreeNAS Xeon D Build Journal

  1. thanks for your detail on your build. I was planning on getting the 743T case until I ran into your review on the 743TQ case. I have found that the 743TQ is not that quiet according to some posts over the net.

    On the PSU, is the sizing a Supermicro design? It kind of turns me off since if that is the case, then this puts a limitation for future upgrades on the PSUs.

    How is your FreeNAS install doing? I have two servers with it (the cases are U-NAS). They are pretty nice but the only downside I have is their effectiveness to cool.

    • I am glad you enjoyed the post. I tried to detail all of the issues I ran into when building the machine.

      The only difference between what I built and the 743TQ is that the 743TQ has a higher rated power supply. I am not sure what posts you have read but the machine is no louder than a regular desktop. The only time the case becomes noisy is when the server is under load. But even that is quieter than the original 743T fan configuration.

      Yes, this is a SuperMicro designed PSU sizing. I have not investigated the exact dimensions of it. It may be possible to adapt a regular PSU to fit. What upgrades were you considering?

      The FreeNAS install is doing great. I have some more posts about my experience that I am working on. This case cools very well. The fans ramp up when Plex starts loading the CPU but then quickly bring the temperature(and noise) back down when the transcodes finish.

  2. Got a question?
    I have this case but with the X10SL7-F mobo. On this case, how did you connect the activity HDDs lights at the backplane? I can’t seem to figure it out and tech support at Supermicro has not responded.

    Would love to see more pictures on how you did your set up.

    Thanks!

  3. Hi! Thanks for writing this down. It was nice reading it and seeing that you put as much work into part selection than I did and arrived at a very similar system and the same conclusions. It would have helped a lot if had found this article earlier. I have an X10SL7-F with Xeon E3 1220 and the 743TQ case, which I was lucky to find for only 130 Euros in perfect condition on an auction platform (the case; bought the board in 2014). Took me some patience to get a good deal, but it was worth waiting. I also put a mobile rack in it, the 2,5″ variant the M28SACB, and removed its 15K(!) RPM fan. Additionally a sharkoon 6×2,5″ bay. In hindsight I could have ordered the OEM version of the supermicro mobile rack without fan and saved a few bucks. These are attached to the onboard controller and an AOC-S2308L-L8i, both flashed to IT mode. For 10G network i added dual SFP+ Intel card.
    The system is quiet enough that I only notice it in the night, when it’s very silent. My bed is four meters away from it and the noise is totally fine.
    Power consumption is around 85W in idle with 3 3,5″ and 6 2,5″ disks and 8 SSDs. My goal was low noise and low power consumption with 10G speeds from the network to disks and I’m very satisfied with what i have now.

    • I want to add one more thing. I can’t remember if i tinkered with the IPMI fan speed thresholds, but I have the same fans and they go as low as 1100 RPM. (My lower thresholds show 300/500/700)

      • I haven’t felt the need to change the fan settings. The only time I really notice them is when the server is loaded.

        I did have the temperature alarm go off recently when the server was fully loaded so I need to take it apart and clean everything.

Leave a Reply to Rob Fern Cancel reply

Your email address will not be published. Required fields are marked *