Help ! With Linux Mint 18.3

[biodoc]

Need first part of the log file to see if you got CUDA error like brownstone.

Things to try:

restart FAHClient:

sudo /etc/init.d/FAHClient restart

does that solve the problem?

Install CUDA and OpenCL support:

If you have 387 nvidia drivers:

sudo apt-get install nvidia-opencl-icd-387

sudo apt-get install libcuda1-387

 

[Markfw]

Dang that was it. If you add this right after you do the drivers in the Brony thread, it works ! system number 3 up, and now to see the ppd. It was 1.3m ppd. When it gets stable, I will reply again

 

[Markfw]

@biodoc, there were a few lines missing from your instructions that were in the Brony guide. But they are missing these 2 lines. I would love to take their list, add this, and nake a comprehensive list after a few more systems. But this is number 3 and I might now be at 12.6 million ppd from about 11.3 starting. Same hardware, just linux, and I have only done 3 of 10 boxes

More points for us ! Might have a chance if some others can join/optimize/add hardware.

 

[biodoc]

@biodoc, there were a few lines missing from your instructions that were in the Brony guide. But they are missing these 2 lines. I would love to take their list, add this, and nake a comprehensive list after a few more systems. But this is number 3 and I might now be at 12.6 million ppd from about 11.3 starting. Same hardware, just linux, and I have only done 3 of 10 boxes

More points for us ! Might have a chance if some others can join/optimize/add hardware.

Go ahead and make changes to the guide. You are the one with experience now!!

 

[biodoc]

So is the ppd of the 1070Ti similar to the 1080 or is it too early to tell?

 

[Markfw]

So is the ppd of the 1070Ti similar to the 1080 or is it too early to tell?

The 1070TIs are running right close to 800k ppd, and the 1080's are going 970k ppd. Quite the difference. I might have to get 1080's next time, I only paid $450 each for the 1070ti's and the 1080s can be had for $500. If course my 1080's are EVGA FTW3's factory overclocked, and the 1070TIs are cheap Zotac ones, so that may also be a factor.

Whats funny is, under Windows my 1080TI's are only doing 900k... I can't wait until I get them under Linux !

The next boxes to go are first, dual 1070's. Then a Threadripper with dual 1080TI's

So then I will have 2 980TI, 2 1070, 2 1070TI and 2 1080 and 2 1080TI all for comparison to windows.

 

[Howdy]

@Markfw You need to OC your 1070Ti's.........although down a few cuda's they are quite the performers compared to a 1080........from what I see on all the review sites.

 

[StefanR5R]

My dual-1080Ti hosts with 4.4 GHz CPU and Linux do ~1.1 M per card.
My triple-1080Ti host with 2.8 GHz CPU and Win 7 does ~0.87 M per card.

I had difficulties to install Mint on 1080Ti hosts, due to the initial nouveau driver going belly up all the time. @Markfw, I wish you luck with your 1080Tis... I resorted to clone the Mint installation of one successfully installed host (with nvidia-driver and all the other packages already installed) for the other two which had more trouble getting past nouveau. Then edited /etc/hostname, and the network config via GUI (I use fixed IP addresses) on those cloned hosts.

 

[biodoc]

My dual-1080Ti hosts with 4.4 GHz CPU and Linux do ~1.1 M per card.
My triple-1080Ti host with 2.8 GHz CPU and Win 7 does ~0.87 M per card.

I had difficulties to install Mint on 1080Ti hosts, due to the initial nouveau driver going belly up all the time. @Markfw, I wish you luck with your 1080Tis... I resorted to clone the Mint installation of one successfully installed host (with nvidia-driver and all the other packages already installed) for the other two which had more trouble getting past nouveau. Then edited /etc/hostname, and the network config via GUI (I use fixed IP addresses) on those cloned hosts.

Stefan, with the 1080Ti and problems with the nouveau driver, I think the solution might have been to edit the Grub kernel options and replace "quiet splash" with "nomodeset". There's more here under the section of solving freezes.

Hold down the right shift key during boot to get to the grub menu.
Press e to edit
replace quiet splash with nomodeset
exit and resume boot

@Markfw , As Stefan mentions, you may have the black screen/freeze problem when installing linux mint on your 1080Ti systems so we'll have to work through this.

 

[Markfw]

OK, the below is the short abridged version compiled from hiigaran@home.org, TennesseeTony , stephanR5R and biodoc. They are the ones that really created this. (several reboots are required)

First, disable secure boot. The easiest way I found was to delete the keys in key management.[/I][/B][/I][/B][/I][/B][/I][/B]
[B][I][B][I][B][I][B][I]
Next, after booting with a linux mint USB or CD, install it. Use an empty disk, or one you don't care if its wiped clean.

Then open a terminal window and do the below commands

wget http://launchpadlibrarian.net/109052632/python-support_1.0.15_all.deb

sudo dpkg -i python-support_1.0.15_all.deb

sudo add-apt-repository ppa:graphics-drivers/ppa

sudo apt-get update

sudo apt-get install nvidia-387 (or if it appears on the linix home page drivers list, you can do it from there.

sudo apt-get install nvidia-opencl-icd-387

sudo apt-get install libcuda1-387

sudo apt-get -y install python-gnome2 mesa-common-dev freeglut3-dev nvidia-settings gedit

Install fah client and console using these links:
Folding at home client (background service)  https://folding.stanford.edu/releases/public/release/fahclient/debian-testing-64bit/v7.4/fahclient_7.4.4_amd64.deb
Folding at home control panel (user interface) https://folding.stanford.edu/releases/public/release/fahcontrol/debian-testing-64bit/v7.4/fahcontrol_7.4.4-1_all.deb

cd /var/lib/fahclient

sudo wget http://fah-web.stanford.edu/file-releases/public/GPUs.txt

Then fire up fahcontrol and add the GPU client(will show under education in the menu), and maybe also the remote access by adding ,192.168.1.1-192.168.1.255 to the two IP places

sudo nvidia-xconfig -a --cool-bits=28 --allow-empty-initial-configuration

cd ~

gedit fancontrols

and add and then save (for 2 video cards)

nvidia-settings -a [gpu:0]/GPUFanControlState=1
nvidia-settings -a [fan:0]/GPUTargetFanSpeed=100
nvidia-settings -a [gpu:1]/GPUFanControlState=1
nvidia-settings -a [fan:1]/GPUTargetFanSpeed=100

Edit. A couple of notes. If you use the linix drivers link, you may not have to do the 2 lines below that. If you do use the link, and then it (sort of) errors out saying they are installed, don't worry. Also, reboot whenever in doubt or whenever you see an error.

Using this exact text, I just installed and configured a system in one hour, the only error ? after updating with GPUS.txt, I forgot to stop and restart the client. I just rebooted, as I forget the command to do that. So at least for one box, this script is 100%

 

[Markfw]

Odd, under windows 7, these 2 1070s were getting 1.3 million ppd. Under linux, this is the first time the ppd is the same. Well, maybe I have not given it enough time.

 

[Markfw]

Well, after more research, they were doing anywhere from 580k to 640k and now they are up to 680-690k each, so probably 200-300k total better.

 

[Markfw]

New problem. With 1080ti, I get this:
[06.432831] nouveau 0000:65:00.0: fifo: SCHED_ERROR 08 []

I googled and found this:
We want to boot with the nomodeset boot parameter to force the system to boot the the fall back graphic's driver:
http://ubuntuforums.org/showthread.php?t=1613132

Once booted then install the graphics driver:
Code:
sudo apt update
sudo apt full-upgrade
sudo ubuntu-drivers autoinstall"


Well, how do I do that ? with this Linux mint 18.3 USB stick ? I can't add the code until after I install something that will boot.

 

[Markfw]

Edit: I figured it out. While you see the "auto boot in 10,9," etc, press F6 then choose the compatibility or safe mode(I forget what it said).

Also, even if your primary disk is the one you want to install on, it will default to possibly the wrong one. It scared my, but I continued, and the next screen gave me a choice of what disk to wipe clean. ( I didn't want to have to remove the nvme SSD, but wanted the win 10 install with a license to remain in case of problems)

BTW, doing the threadripper with dual 1080TI's and it was getting 1,850,000 ppd at that time with 2 GPU clients.

 

[Markfw]

OK, its installed, but now will not boot into linux. How to boot in safe mode ? (I tried the F6, does not work)

Edit after research all I find is hold down the left shift (does not work) or repeatedly press <esc> does not work. But those were for ubuntu.

 

[Markfw]

Well, this may not be good, but.... So I chose F11 at boot for boot loder choice. I picked what I thought was the nvme drive(windows 10). When it booted, it gave me a linux menu, and I chose recovery mode, and play around with the options, and somehow got it to boot. Now trying to install video drivers.

So, I might have lost my windows 10 good install, in case this goes badly.

 

[Markfw]

OK, so its running. Not sure what all is going on, but it appears that the nvme drive may be safe. But the weird part is that I have to select the nvme drive in the boot menu, to get it to boot, then chose the cinnamon mint, then it works. When I goot, its says 300 some gig free in the filesystem, so it can't me the nvme drive, and I get an error in the devices on the filesystem that says this drive is unmountable.

Regardless, its running, and was at 1.85 million ppd and is now at 2.45 million ppd and rising (no CPU) So I added at least another 600k to my farm !

 

[StefanR5R]

Sounds like the boot loader (grub) was put onto the NVME drive, but the actual Linux system went onto the SATA drive as intended. The BIOS boots the bot loader, and that one boots the kernel + initrd (initial RAM disk with a minimal system) either from another partition on the NVME drive or from the SATA disk, and then finally the real Linux root filesystem gest mounted from the SATA disk.

You can have a quick look what's there with
$ cat /proc/partitions

A popular GUI on Linux to inspect and edit partitions and filesystems is "gparted". It may not be installed by default but can be loaded quickly from Software Manager.

 

[Markfw]

Sounds like the boot loader (grub) was put onto the NVME drive, but the actual Linux system went onto the SATA drive as intended. The BIOS boots the bot loader, and that one boots the kernel + initrd (initial RAM disk with a minimal system) either from another partition on the NVME drive or from the SATA disk, and then finally the real Linux root filesystem gest mounted from the SATA disk.

You can have a quick look what's there with
$ cat /proc/partitions

A popular GUI on Linux to inspect and edit partitions and filesystems is "gparted". It may not be installed by default but can be loaded quickly from Software Manager.

So do you think my win10 boot(nvme) will never work again if I remove the sata drive with linux ?

 

[StefanR5R]

So do you think my win10 boot(nvme) will never work again if I remove the sata drive with linux ?

The Win 10 installation may still be there. In the best case, the grub bootloader menu even offers it as an alternative boot.

Hold down the right shift key during boot to get to the grub menu.

 

[Markfw]

The Win 10 installation may still be there. In the best case, the grub bootloader menu even offers it as an alternative boot.

After spending all I did on the win 10 licenses, I didn't want to loose all these activated installs, even though after hardware upgrades I may have had a problem using them. And if I ever have to reboot linux, it may be problematic also if its so far in the future that I forget how to get them to boot.

Bottom line ? If you are converting to Linux, remove any NVME drives and add a regular drive first.

 

[biodoc]

Try this to fix the problem.

sudo update-grub

Hopefully the next time you boot, you will see the normal grub menu with the option of booting windows or linux.

 

[TennesseeTony]

The WinTen license is tied to your hardware. If you ever need to reinstall it, activation is instant and automatic. Not sure if it ties to the unique code of the CPU, or the motherboard somehow, but after more than a few reinstalls (new hard drives, mostly), I can verify activation isn't an issue.

Pretty sure though, GRUB has simply modified the boot order somewhat, but the previous installation is there and ready to roll, once you figure out how to access it (as already mentioned).

 

[biodoc]

After spending all I did on the win 10 licenses, I didn't want to loose all these activated installs, even though after hardware upgrades I may have had a problem using them. And if I ever have to reboot linux, it may be problematic also if its so far in the future that I forget how to get them to boot.

Bottom line ? If you are converting to Linux, remove any NVME drives and add a regular drive first.

Grub has an OS-prober function that scans for bootable operating systems. To run it:

sudo update-grub

The output will be written to /boot/grub/grub.cfg

open up the grub.cfg file using:

Xed /boot/grub/grub.cfg

and search for Windows 10. You should see something like this:

### BEGIN /etc/grub.d/30_os-prober_proxy ###
menuentry "Windows 10 (loader) (on /dev/sda1)" --class windows --class os $menuentry_id_option 'osprober-chain-BC503597503558FA' {
    insmod part_msdos
    insmod ntfs
    set root='hd0,msdos1'
    if [ x$feature_platform_search_hint = xy ]; then
      search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1  BC503597503558FA
    else
      search --no-floppy --fs-uuid --set=root BC503597503558FA
    fi
    parttool ${root} hidden-
    drivemap -s (hd0) ${root}
    chainloader +1
}
### END /etc/grub.d/30_os-prober_proxy ###

If a section like above is in the grub.cfg file then the next time you boot, you will see a list of boot options including linux and Windows.

If the above doesn't work, then you will need a windows 10 repair disk or installation disk to "fix" the windows 10 boot loader. Instructions for this process are at this link. If you do that, then grub is lost and so is your ability to boot into linux.

 

[Markfw]

OK, new twist... The latest install on my second threadripper box was perfect when I went to bed last night, and it rebooted sometime in the night, and now if just has a mouse cursor, no other screen. So until I know what to do, its back to running win 10.