Setting up a home server

[XavierMace]

I'm using Crucial MX100's for my ZIL's right now, no complaints.

 

[mfenn]

Well, I am not familiar with that particular metric when it comes to drives. I guess I will scrap those drives from the build then, or at least from the ZIL/L2ARC portion. Would the Agility 3's be sufficient in Raid 0 for the OS do you think?

Don't RAID0 an OS drive, that's just asking for avoidable issues. Honestly 60GB is more than plenty for a Linux server OS (hell an 8GB SD card is enough).

The other 2 drives that I have are Toshiba Q Series HDTS225XZSTA. Would these suffice for L2ARC and ZIL partitions?

Those drives would work, but are super pricy. I would strongly consider the Intel SSD Pro 2500 240GB at $135. You pay about a $35 premium over a drive like an MX100, but you get high, consistent IO performance. The MX100 is consistent too, and would save you some money, but has lower steady state performance. You really can't go wrong either way.

 

[EtherealDragon]

Don't RAID0 an OS drive, that's just asking for avoidable issues. Honestly 60GB is more than plenty for a Linux server OS (hell an 8GB SD card is enough).



Those drives would work, but are super pricy. I would strongly consider the Intel SSD Pro 2500 240GB at $135. You pay about a $35 premium over a drive like an MX100, but you get high, consistent IO performance. The MX100 is consistent too, and would save you some money, but has lower steady state performance. You really can't go wrong either way.

The Toshiba drives that I linked are a zero cost, because I already own 2. One of them is in my gaming laptop, but it makes no sense for an SSD in that one, still takes forever to boot with an SSD. The twin to that drive is the one that is in my PS3. That doesn't need to be in there, as I want a larger drive in there, and have a 1TB 2.5" drive to replace it. Just a matter of backing up the PS3 and replacing it, but that is an issue since the dumb thing won't recognize any external drives I have.

 

[mfenn]

The Toshiba drives that I linked are a zero cost, because I already own 2. One of them is in my gaming laptop, but it makes no sense for an SSD in that one, still takes forever to boot with an SSD. The twin to that drive is the one that is in my PS3. That doesn't need to be in there, as I want a larger drive in there, and have a 1TB 2.5" drive to replace it. Just a matter of backing up the PS3 and replacing it, but that is an issue since the dumb thing won't recognize any external drives I have.

Ah OK. Well, they'll work perfectly fine considering the cost!

 

[EtherealDragon]

The

Here is the 22U HP server rack I picked up. It is on casters, so mobility is great should there ever be a need to move it. I am not in a HUGE rush, but a future project will be to cover the bottom space, likely with a thin sheet of plywood, or something like that, then to incorporate some type of filter media to keep the equipment inside clean.

Here is the pretty much completed hard drive caddy. It has the two OCZ Agility 60 gig's on the bottom. One will be the OS, and the other one I am thinking I will keep as a copy of the OS drive, NOT IN RAID 1, just as a copy that I can move to should anything happen to the main drive. The middle shelf is dual Toshiba Q Series HDTS225XZSTA - 256 Gig drives. These will both have an 8 gig partitions, mirrored for ZIL. Will have the remainder of one of the drives for L2ARC. I am not quite sure what to do with the other remaining ~240 gig partition, but I am sure I will figure out something.

Here is the fully trimmed hard drive caddy inside of the server. There is PLENTY of room for it inside of the server. It will be a bit tight with drives on the top shelf, but it will be manageable.
NOTE - The Raid card still makes me nervous with the bend radius of the cable. I may end up finding a new home for this one and pick up one that has the SFF connectors on the end of the card facing INTO the chassis rather than facing up, but I have to look into this further. I honestly haven't done anything with any of the hot swap drives to see if the the cable bend down as pictured would have any effect on the connectivity of the drives.

Here are the CPU heat sinks. They don't have fans on them, but there are just about 2U tall. they go from the CPU to just under the plastic cover that keeps airflow within the CPU lane. There are 3x 80 mm PWM fans which you can see in the lower left portion on the frame.

Here is the memory that the server came with. I will be looking into increasing this amount of memory, as ZFS recommends 1 Gig of RAM for each TB of storage space. I am not sure if that is for TOTAL capacity, or USED capacity, but I figure it would be best to just max out what I can in the beginning so I don't have to take if offline if it isn't needed. Plus I will need some RAM for the OS, and any virtual machines running. For now, I am thinking of doubling to 48 Gigs, for under $180 shipped. Not quite ready to pull the trigger on the memory yet, but I have the funds.

Memory:
Nanya - NT4GC72B4NA1NL-CG
Speed - DDR3-1333 PC3-10600 667MHz (1.5ns @ CL = 9)
Organization - 512Mx72
Power - 1.5V
Contacts - Gold

I have Ubuntu server 14.04 installed at this point, but not much else. I have only run a few commands to update the server, but I still don't know a whole lot about what I am doing. I have LOTS of reading left to do.

apt-get update
apt-get upgrade

I can't recall the other few commands I have run so far, but I also installed and updated ebox. My understanding is that it allows for web-base administration of the Ubuntu server, but perhaps I am wrong. For the life of me I can't figure out how to get it running, but again, more reading to do.

 

[Carson Dyle]

The nice thing about picking up used server equipment that's a generation or two old is that it can be had for relatively little money. The bad thing is that newer gear is typically much more energy efficient for the same amount of computing power.

See if you can find a rackmount UPS. They seem to be fairly common on Craigslist, although you may have to purchase new batteries. It will make better use of that cabinet space than will UPSes sitting on shelves.

 

[mfenn]

You can have more than 1 L2ARC device, so you can use the rest of your second SSD for that.

Regarding memory, ZFS doesn't actually need 1GB of RAM per 1TB of used capacity, it's more like 1GB of RAM per 1TB of active working set*. Unless you're using dedup, then you need a lot more (a lot more), but you won't be doing dedup.

I wouldn't sweat the SAS cabling too much, I've done horrible horrible things to SAS cables in my time, and know that they're quite tough.

 

[EtherealDragon]

I have made some significant progress with the server so far this week.

DONE list:

• Verified that all 12 of the hot swap bays on the backplane work. Haven't moved data to drives in each slot yet, but verified that the drives are at least seen by the OS in each slot.
• Flashed IBM M1015 to run in IT mode with no BIOS, for quick boot, verified that it sees all 8 of the drives connected to it in JBOD.
• Fresh install of Ubuntu Server 14.04.1 LTS Trusty Tahr
• openssh-server package installed and updated
• verified able to ssh to server from local network machine via putty
• Stumbled my way through getting ISPConfig 3 web admin panel setup(https://www.howtoforge.com/perfect...p-mysql-pureftpd-bind-dovecot-ispconfig-3-p3)
• ISPConfig web admin panel up and working, able to reach locally and from internet, changed outside port on router so it isn't using default port, but it is easy enough to disable if I need to, won't be enable for outside normally, just wanted to test
• Squirrelmail setup and able to log in

TO DO list

• Get front bezel and get it installed with filter to keep dust and junk out of chassis
• Original install of OS is sitting on sdb, need to figure out how to WIPE that drive, or delete the install from the command line. Need to then make copy of OS residing on sda onto sdb
• Get squirrelmail WORKING. Not getting emails yet. Not sure what is up here, have to do more reading. I WAS getting bounce backs, but I am no longer getting those from my Gmail, but nothing is coming into squirrelmail web interface. Oh well, not imperative, but I want to get it working eventually. Not sure if I should leave this on the host, or put it on a guest machine.
• Set up ZFS
• Figure out how to divide up drives into Vdev's, single raidZ2 with 6 drives, or two raidZ's with 3 drives each.
• Figure out how to create a virtual host, will have Plex Media Server running in the first one, with access only to the media directory, not the whole zpool. Will be tinkering with a Minecraft server on the other. Not sure what else I will look into for guest machines.
• Get HP server rack into the house and in the basement where it will be living.
• Get rack mount rails for the chassis, and install it in the rack
• Get a battery backup for the server to be able to gracefully power down during power loss. (rack mount, most likely an APC unit)

I have been TRYING to do research to get a front bezel for the server, but it has been a PITA. If anyone HAS one, and can enlighten me on the details of how exactly it mounts and whatnot, I would appreciate it. I sent emails to just about every seller on ebay with them, asking if they come with keys, not a SINGLE one knew, they all punted to Supermicro. Other pages I tried to find info (picture below), were TOTALLY fudged, would show completely different part numbers, so no what to know what is ACTUALLY being ordered. I even went as far as to send a detailed email to Supermicro directly explaining I am trying to get more info on part MCP-210-82601-0B to find out if it comes WITH keys, or what, how it mounts, etc. I got a single sentence response saying that "only a bezel itself and nothing else." I sent a response asking if there is a part number for the mounting bolt and keys for the lock, was told this has no mounting bolt, and that keys are generic. SOOoooo... can anyone tell me how the bezel attaches to the chassis (if additional parts are needed), and where I can find a set of keys for it?? I would be most appreciative.

Got the new SAS breakout cables installed, they are PERFECT! The sheathing pulled up pretty easily from the SFF-8087 connector, so the bend radius isn't an issue. One of the short sets of cables I had the sheath heat shrink at the connector was pretty much fused to the connector, so it didn't bend very well at ALL. There is also PLENTY of clearance for the last set of hard drives on the on the top of the drive sled, even with the SAS breakout cables. Power cables are all set as well for 2 additional drives if/when they get installed.

The cable management on the back plane needs some work. The SATA cables are good, but the power cables are a PITA, and I want to try and get them tucked more out of the way, so not to disrupt airflow, although, I guess with the back plane being there it probably doesn't much matter.

Another few shows of the drive sled and cable management .I also have a free PCI Express slot between the IPMI card and the IBM M1015 card. The ONLY thing I didn't take into account is the possibility of adding in a network card for future expansion. There ARE two 1 gig ports build onto the Motherboard, which I can (down the road, when I get a switch that supports it) team them up for a 2 gig link. Not totally urgent, just something to think about.

I messed with the order of the cables on the motherboard connectors until I got the drive arrangement that I wanted, so that sda is boot, sdb will be boot backup, sdc and sdd will be LOG and L2ARC drives. The good news is that all 8 of the hot swap drives in place are all seen as well.

 

[Carson Dyle]

Does your server look like this? What makes you think there's a front bezel for it, and what exactly would you need it for, if there is one?

** Edit **

Ok, I guess there is a bezel. Yes, it appears to have a lock. Looks like there are four tabs that fit into the protruding aluminum sides by the handles. My guess would be that the two on the left are stationary, while the two on the right are spring-loaded and released by sliding the red latch. The lock would block the latch from sliding.

The server looks a lot better without the bezel, IMO. I can't tell you how many bezels from rackmount computers we just left in the box when installing systems in data centers. They're little more than a pain in the ass when it comes to dealing with the hardware.

http://www.amazon.com/Supermicro-MCP-210-82601-0B-Front-Bezel-Chassis/dp/B001VWSM2Y

 

[EtherealDragon]

Yes, that is the chassis, it is a SC826TQ-R800LPB. And that is the bezel as well, model MCP-210-82601-0B according to Supermicro's website. This will be racked in my house in the basement, so I want the bezel on there so that I can have some filter media between it and the chassis (hopefully) to hopefully keep all of the fans and heat sinks free from dust and getting clogged. Open it up once a month or every few weeks to clean it out, and hopefully all will be well.

I didn't even NOTICE those little notches on the sides of the bezel. My thought was that it used the 2 larger holes which face forward which are directly below the handle loops (above the power button on the left side).

Thank you for the response!!

 

[mfenn]

I messed with the order of the cables on the motherboard connectors until I got the drive arrangement that I wanted, so that sda is boot, sdb will be boot backup, sdc and sdd will be LOG and L2ARC drives. The good news is that all 8 of the hot swap drives in place are all seen as well.

You shouldn't rely on the /dev/sdX names staying constant because they're strictly ordered by the time that Linux saw them. Especially on a server, hot-swapping drives is guaranteed to change the names.

Instead, you should use filesystem labels or UUIDs in /etc/fstab (the Ubuntu installer should do this automatically), and the symlinks in /dev/disk/by-path for ZFS.

<putty image>

You can easily copy and past from Putty by just selecting the text. Anything that you select is automatically put into the clipboard for pasting elsewhere.

I want the bezel on there so that I can have some filter media between it and the chassis (hopefully) to hopefully keep all of the fans and heat sinks free from dust and getting clogged.

I don't think I would do this. That chassis is already fairly restrictive in terms of airflow, and adding a bezel makes it even more so. Then if you add a filter on top of that, you're looking at potential cooling issues.

 

[EtherealDragon]

I was working on the Zpool last night for ENTIRELY TOO LONG. I was up till nearly 02:30, so I only got about 3.5 hours of sleep.... JOY....

My issue was that the pool would create without issue, I could get the directories working, and visible from a computer on the network, but I couldn't connect. I am still working that issue, but I am not worried about it at this point. Just to test out the setup, I rebooted the server, and uh-oh, NO ZPOOL.... Worked on that all of last night, and a little bit this morning. I found myself continually rebooting to see if this was the one, so I created the zpool maybe 2 dozen times. The repetition is nice if for nothing else just becoming more familiar with the commands and the outputs. After hours and hours of searching, I found a SINGLE thread that had the answer, https://github.com/zfsonlinux/zfs/issues/2927 . All I had to do was log into root, and issue "echo manual >/etc/init/zpool-import.override". Issued that, rebooted, and BOOM, everything worked. Right now I don't have quotas setup or anything like that (yet), but I have 2 directories I have created, 1 obviously for backup up important data from other machines, and the other will be media back ups, photos, videos, movies, TV shows, music, and anything else the Plex server will read. Please will have access (hopefully) to the entire /mnt/Storage/Media folder.

Today I will be working on connecting to the shared Media folder and mapping a network drive from my desktop machine. (As long as I can remote to the Desktop, not currently at home). Call it a lunch project. If nothing else, I can pull the disks and insert them into the server and copy the files that way if need be (if that will work). I will also be working on the sensors to see what I can do to monitor temps. I could probably safely remove all the fans and run a pure silent set up since I have 3 sensors reporting -48* C...... [/badjoke]. I did end up ordering the front bezel but it hasn't processed yet almost a week later, may just end up canceling it, not sure what happened there...


As far as the server goes, I am not sure if I am going to keep this setup, or format the boot drive and start over, again. I thought that the web admin panel would be more useful than it is, so not sure if I will want that, or just do a fresh install, update, then set up ZFS and the zpool. I was thinking that I would have been able to CREATE the virtual guest form the ISPConfig web admin panel, but it appears you can only manage existing guests from there. Oh well. I am digging the command Line. It isn't foreign as I am in Cisco and Juniper router command lines all day at work, just different commands. Sometimes Unix jumper servers too, nothing other than creating router configs with vi, changing permissions, searching, and copying files though with those.

ziggidy@Ziggidy-VH:~$ sudo zpool status
  pool: Storage
 state: ONLINE
  scan: none requested
config:

        NAME                                            STATE     READ WRITE CKSUM
        Storage                                         ONLINE       0     0     0
          raidz2-0                                      ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E1304884    ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E1344552    ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E1344610    ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E1324763    ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E1380342    ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4EFUL6XA9    ONLINE       0     0     0
        logs
          ata-TOSHIBA_THNSNH256GBST_83IS10JLTE8Y-part1  ONLINE       0     0     0
          ata-TOSHIBA_THNSNH256GBST_83JS106NTE8Y-part1  ONLINE       0     0     0
        cache
          ata-TOSHIBA_THNSNH256GBST_83IS10JLTE8Y-part2  ONLINE       0     0     0
          ata-TOSHIBA_THNSNH256GBST_83JS106NTE8Y-part2  ONLINE       0     0     0

errors: No known data errors
ziggidy@Ziggidy-VH:~$
ziggidy@Ziggidy-VH:~$
ziggidy@Ziggidy-VH:~$ sudo zfs list
NAME              USED  AVAIL  REFER  MOUNTPOINT
Storage          1.80M  14.3T   272K  /Storage
Storage/Backups   272K  14.3T   272K  /mnt/Storage/Backups
Storage/Media     272K  14.3T   272K  /mnt/Storage/Media
ziggidy@Ziggidy-VH:~$
ziggidy@Ziggidy-VH:~$
ziggidy@Ziggidy-VH:~$ sensors
coretemp-isa-0000
Adapter: ISA adapter
Core 0:       +29.0°C  (high = +85.0°C, crit = +95.0°C)
Core 1:       +29.0°C  (high = +85.0°C, crit = +95.0°C)
Core 9:       +29.0°C  (high = +85.0°C, crit = +95.0°C)
Core 10:      +26.0°C  (high = +85.0°C, crit = +95.0°C)

coretemp-isa-0001
Adapter: ISA adapter
Core 0:       +23.0°C  (high = +85.0°C, crit = +95.0°C)
Core 1:       +25.0°C  (high = +85.0°C, crit = +95.0°C)
Core 9:       +24.0°C  (high = +85.0°C, crit = +95.0°C)
Core 10:      +30.0°C  (high = +85.0°C, crit = +95.0°C)

w83627hf-isa-0a00
Adapter: ISA adapter
in0:          +3.84 V  (min =  +0.00 V, max =  +4.08 V)
in1:          +3.86 V  (min =  +0.00 V, max =  +4.08 V)
in2:          +3.84 V  (min =  +2.82 V, max =  +3.79 V)  ALARM
in3:          +3.06 V  (min =  +4.08 V, max =  +4.08 V)  ALARM
in4:          +3.09 V  (min =  +4.08 V, max =  +4.08 V)  ALARM
in5:          +3.14 V  (min =  +4.08 V, max =  +4.08 V)  ALARM
in6:          +3.15 V  (min =  +4.08 V, max =  +4.08 V)  ALARM
in7:          +3.31 V  (min =  +4.08 V, max =  +4.08 V)  ALARM
in8:          +3.25 V  (min =  +4.08 V, max =  +4.08 V)  ALARM
fan1:           0 RPM  (min =    0 RPM, div = 8)
fan2:           0 RPM  (min =    0 RPM, div = 8)
fan3:           0 RPM  (min =    0 RPM, div = 8)
temp1:        -48.0°C  (high =  -1.0°C, hyst =  -1.0°C)  sensor = thermistor
temp2:        -48.0°C  (high = +80.0°C, hyst = +75.0°C)  sensor = thermistor
temp3:        -48.0°C  (high = +80.0°C, hyst = +75.0°C)  sensor = thermistor
cpu0_vid:    +1.419 V
beep_enable: enabled

ziggidy@Ziggidy-VH:~$

 

[EtherealDragon]

Throwing in an additional quiestion about the virtual guests....

Would it be better to set up a folder inside of the zpool at /mnt/Storage/Virtual/<hostname> for each virtual host running on the server, or would it be better to have a separate drive, say a 500 Gig drive for the virtual guests, or would they need to be on the OS drive itself?? This part I am not too sure of, but I am leaning towards a pair of drives similar to the OS drives, where they are separate drives, and master drive backs up to the back up drive.

I may need to work on the OS drive as well to expand the partition, as I think I used only 50% for the OS install, as I wasn't sure if the virtual guests would be using that drive as well or not...

 

[aigomorla]

i honestly think a dual socket system is a waste on freebsd unless it was virtualized.
You have all those workable threads that will just sit there... unless again it was virtualized..

but wow you did get one hell of a deal on that system...

Also note have you done all the research required for the IBM1015 card?
You know to set it in IT mode, and also that on some occasions will you not be able to pull SMART info off them on freebsd?

 

[ViviTheMage]

Yeah, i'd run vmware ESXi on that guy, then run whatever I want and attach those raids as data stores, using the SSD's are C drives for boot of the VPS's.

 

[EtherealDragon]

As for FreeBSD, I scrapped that idea, and am running Ubuntu Server 14.04.1 as the host OS currently.

With the M1015, I was able to get it to IT mode I think about a week ago, so I am good there. I haven't quite gotten as far as looking for SMART info yet, not sure if there is any workaround, a header on the back plane or anything like that which would be able to provide that info, or if the IMPI card is able to look at that at all (to be honest, I haven't done ANY research on IPMI cards, so I really don't know what it is for, or what it is capable of.

I really haven't looked much at ESXi, not sure if it can run as a virtual guest, but maybe down the road if I do look at it, I can try it out virtually to see what it is.

 

[ViviTheMage]

IPMI is just a way to get into your box if you lose network. Out of band management if you will.

http://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface

I would run ESXi off a USB drive or the SSD, then load up your OS root/c drives on the SSD's then attach the raid arrays as data stores. So you can do really simple backups of your OS, play with different OS's, snapshot, etc with ESXi. With a box with that much horsepower, you might as well. You are wasting it away by just running ubu + file sharing IMO.

 

[EtherealDragon]

Thank you for the info on IPMI, not sure if I will need any of that, as this is running in a basement in a house. My current out of band management is rotating my office chair to the close where this is and opening the door, LOL.

I do agree that the current set up is a giant waste, but that is only because I am taking baby steps. I have plans down the road to find out how to get virtual guest systems up and running on this Ubuntu box.

As for ESXi, that is a really good idea, I can just unplug the current OS drive, and run ESXi from USB to test it out and see what it is and how I like it. Would that be considered the host OS, or would there be another OS that sits on top of it NON-virtualized? Would have to look to see how that would work, as I have had suggestions to NOT run a ZFS setup as a virtual guest OS, but on the host OS. How difficult is it to create a guest system within ESXi?

 

[ViviTheMage]

So you'd install ESXi on a jump drive, SSD, whatever. That would be your host OS. Then you can load up whatever ISO you want via the ESXi interface, as many as you want (within cpu, memory, disk limitations). It's a wonder that ESXi is free for what it offers!

Here is a easy as pie way of loading it to a USB: https://pubs.vmware.com/vsphere-50/...UID-33C3E7D5-20D0-4F84-B2E3-5CD33D32EAA8.html

Once in, you just point ISO's to it and specify sizes you want. My advice is thin provision when it comes to loading your OS's up.

 

[mfenn]

Yeah, i'd run vmware ESXi on that guy, then run whatever I want and attach those raids as data stores, using the SSD's are C drives for boot of the VPS's.

I would definitely not run ESXi on a machine that is meant to be a storage server. You either have to give up the performance and flexibility of ZFS to go with hardware RAID or try to virtualize ZFS and lose the performance and reliability features built into ZFS.

 

[mfenn]

I am digging the command Line. It isn't foreign as I am in Cisco and Juniper router command lines all day at work, just different commands. Sometimes Unix jumper servers too, nothing other than creating router configs with vi, changing permissions, searching, and copying files though with those.

Being comfortable with the command line and a text-mode editor puts you head and shoulders above most people who are learning to administer a Linux system. I would honestly recommend learning to do things the command line way so that you understand how they work rather than relying on a web form that does "magic" for you.

ziggidy@Ziggidy-VH:~$ sudo zpool status
  pool: Storage
 state: ONLINE
  scan: none requested
config:

        NAME                                            STATE     READ WRITE CKSUM
        Storage                                         ONLINE       0     0     0
          raidz2-0                                      ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E1304884    ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E1344552    ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E1344610    ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E1324763    ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E1380342    ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4EFUL6XA9    ONLINE       0     0     0
        logs
          ata-TOSHIBA_THNSNH256GBST_83IS10JLTE8Y-part1  ONLINE       0     0     0
          ata-TOSHIBA_THNSNH256GBST_83JS106NTE8Y-part1  ONLINE       0     0     0
        cache
          ata-TOSHIBA_THNSNH256GBST_83IS10JLTE8Y-part2  ONLINE       0     0     0
          ata-TOSHIBA_THNSNH256GBST_83JS106NTE8Y-part2  ONLINE       0     0     0

This zpool looks good except that the log (ZIL) is not mirrored like it should be. You can fix this online by detaching the log devices and reattaching them as a mirror.

zpool remove Storage ata-TOSHIBA_THNSNH256GBST_83IS10JLTE8Y-part1
zpool remove Storage ata-TOSHIBA_THNSNH256GBST_83JS106NTE8Y-part1
zpool add Storage log mirror ata-TOSHIBA_THNSNH256GBST_83IS10JLTE8Y-part1  ata-TOSHIBA_THNSNH256GBST_83JS106NTE8Y-part1

 

[EtherealDragon]

Thank you for the heads up on that. Totally forgot. I had to alter the commands from zfs to zpool, but it worked like a champ!!

ziggidy@Ziggidy-VH:~$ sudo zpool remove Storage ata-TOSHIBA_THNSNH256GBST_83IS10JLTE8Y-part1
ziggidy@Ziggidy-VH:~$ sudo zpool remove Storage ata-TOSHIBA_THNSNH256GBST_83JS106NTE8Y-part1
ziggidy@Ziggidy-VH:~$ sudo zpool add Storage log mirror ata-TOSHIBA_THNSNH256GBST_83IS10JLTE8Y-part1  ata-TOSHIBA_THNSNH256GBST_83JS106NTE8Y-part1
ziggidy@Ziggidy-VH:~$
ziggidy@Ziggidy-VH:~$
ziggidy@Ziggidy-VH:~$ sudo zpool status
  pool: Storage
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(5) for details.
  scan: none requested
config:

        NAME                                              STATE     READ WRITE CKSUM
        Storage                                           ONLINE       0     0     0
          raidz2-0                                        ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E1304884      ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E1344552      ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E1344610      ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E1324763      ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E1380342      ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4EFUL6XA9      ONLINE       0     0     0
        logs
          mirror-1                                        ONLINE       0     0     0
            ata-TOSHIBA_THNSNH256GBST_83IS10JLTE8Y-part1  ONLINE       0     0     0
            ata-TOSHIBA_THNSNH256GBST_83JS106NTE8Y-part1  ONLINE       0     0     0
        cache
          ata-TOSHIBA_THNSNH256GBST_83IS10JLTE8Y-part2    ONLINE       0     0     0
          ata-TOSHIBA_THNSNH256GBST_83JS106NTE8Y-part2    ONLINE       0     0     0

errors: No known data errors
ziggidy@Ziggidy-VH:~$

 

[ViviTheMage]

I would definitely not run ESXi on a machine that is meant to be a storage server. You either have to give up the performance and flexibility of ZFS to go with hardware RAID or try to virtualize ZFS and lose the performance and reliability features built into ZFS.

Personal preference I guess. If it was mine, at home, i'd want to be able to run more then just ZFS. Performance wouldn't take that big of a hit.

Web portals are nice, but command line is where it's at!

 

[dave_the_nerd]

I would definitely not run ESXi on a machine that is meant to be a storage server. You either have to give up the performance and flexibility of ZFS to go with hardware RAID or try to virtualize ZFS and lose the performance and reliability features built into ZFS.

That's not really true - if the hardware is enterprisey enough, it'll support PCI passthrough - just drop in a SATA card, pass it through the the VM, and it'll address the drives directly as though it was a hardware box, then create iSCSI LUNs on the ZFS server and pass them back to ESX as datastores.

Now, that said, it's not recommended since you can end up with chicken/egg problems if you put the SAN server datastore on a LUN it manages. And the FreeNAS guys will tell you (gleefully) that it's not supported and you're an idiot for doing it. But as long as you're careful with that, it's fine for test environments or home labs.

OTOH, this is another reason my home server runs FreeBSD and not just FreeNAS - I can configure ZFS and NAS stuff, and then run a hypervisor on the same box.

If running ESX were a real priority I guess I'd need a second box. Or I could install that in its own VM, but... meh.

Edit: OP is using Linux? Well, that's fine. Just install ESX in a KVM VM.

 

[mfenn]

Thank you for the heads up on that. Totally forgot. I had to alter the commands from zfs to zpool, but it worked like a champ!!

Sorry about that, total brain fart on my part!