Linux gurus: please help me automate this process

[DrMrLordX]

Hi! I am having problems automating my Ethereum mining process on Linux. Here's what I do manually:

1). Log in automatically (no password required at boot; yes, poor security I know)
2). Run a script (./setclocks.sh) in my home directory to configure clockspeeds for all the GPUs on the machine (there are four).
3). Open four terminal windows via LXTerminal (open home directory, hit f4 four times).
4). Send two terminal windows to Desktop 2 (optional, but it looks better this way)
5). Set all four terminals to root via sudo su - root. I enter the password by hand when prompted.
6). Activate ethminer once in each terminal, once for each card, with a line like:

ethminer -F http://us-east1.nanopool.org:8888/0xaddress/machinename -G --cl-local-work 256 --cl-global-work 16384 --opencl-device 1 --farm-recheck 100

. . . and that's pretty much it.

Automating this process has been damn near impossible. I've been using .profile to start everything, which works, and I have discovered that if I use sleep to delay activation of scripts by a minute or two, the desktop environment will load fully before the mining process begins (if I do not do this, sometimes the desktop will not load at all). I have placed the process of initiating a single card via ethminer in a script and then executing those scripts in individual terminal windows via xterm -e command, and I have also discovered that if I put & at the end of a command (in this case, opening the terminal) that the script won't "hang up" on one miner and will go ahead and execute everything else in the script.

I have been having two main problems: one, the terminals often appear and then nothing happens inside them (that I can tell) and two, using

echo password | sudo -S su - root

does not work anywhere near as well as I would like inside a newly-spawned terminal window. sudo -S seems to be having problems getting input from stdin when I do things like that. I have sort-of bypassed that problem by using sudo to execute the main script, which does work via sudo -S most of the time.

So right now, my scripts just produce four blank terminal windows in which nothing happens. On those rare occasions when I can put things together the right way and get commands to execute in the terminal windows, sometimes I get problems from the system acting as if the proper environment variables haven't been set (for mining with 2g cards; one of mine is an old Pitcairn) despite setting these variables in .profile for both my main login AND root and setting them again in the scripts that are meant to run when the terminal window is first opened.

I recognize that it would be easier to automate the process if I would just log in as root and then use -t 4 instead of spawning four different instances of the miner. But there are some major advantages to running the cards independently, and I am wary of flat-out logging in as root.

 

[fackamato]

Start them as root directly in rc.local or your distro's equivalent at boot, each in its own screen session?

 

[Fallen Kell]

Start them as root directly in rc.local or your distro's equivalent at boot, each in its own screen session?

Would essentially be my suggestion as well.

 

[jhu]

If you were running Windows, I would suggest Autohotkey. If you're not familiar with it, it basically allows scripting of keyboard and mouse inputs including on-screen image analysis to find certain windows, text input boxes, etc. I have had *cough* friends *cough* automate actions in MMORPGs with this program.

Unfortunately for Linux, I don't know what the equivalent is. A quick Google search brings up Autokey, but I've not tried it before.

 

[mv2devnull]

Why do you have to run as root? Is it the access to the opencl-device? Is there no way to give an another account access to them?

A job that should start at boot and run in the background is usually called "service" or "daemon", and one should be able to create a configuration for one. Which "init-system" does your distro use? (The rc.local is a "last resort" way to spawn such a service.) Note though that services use no GUI nor console. They should redirect the output to files.

 

[Fallen Kell]

Why do you have to run as root? Is it the access to the opencl-device? Is there no way to give an another account access to them?

A job that should start at boot and run in the background is usually called "service" or "daemon", and one should be able to create a configuration for one. Which "init-system" does your distro use? (The rc.local is a "last resort" way to spawn such a service.) Note though that services use no GUI nor console. They should redirect the output to files.

For the most part I agree with you, but if the process does need a terminal (which one can make an assumption based on the way OP describes how he opens it), using screen to give it a terminal will work and allow him to later connect to it and view the progress once he logs in at some later point in time.

 

[DrMrLordX]

Hi! Thanks for your responses.

Access to --opencl-device appears to require root access. I *think* I can mine with -t from my main account, though anything else is going to require root. And yes I do want each session in its own terminal so I can monitor progress later either by hotplugging a monitor or . . . something. Output to a file would produce massive logs and eat up space on the dinky little 32Gb flash drive used for boot/storage in no time.

I have attempted to start them as root (via sudo) in .profile which produces some undesirable behavior. The current problem is that I get blank terminal windows in which nothing is happening. I have also had problems with certain key environment variables not being set for anything in the terminals . . . solving that problem by including export commands to the scripts executed in the terminals (xterm -e script.sh) seems to have produced the blank terminals. Maybe I need the & character at the end of each export command? I can't see why I would but . . .

Also, I have to be careful starting anything in .profile that involves GPU compute functions, since it can prevent the desktop manager from loading in the first place, giving me a blank screen with a flashing cursor. The mining still happens in the background, but until something terminates that process, the desktop never loads. I think I can overcome that by using sleep for a minute or two. I will deal with that once I can figure out why my xterms are coming in blank like that.

 

[jhu]

Hi! Thanks for your responses.

Access to --opencl-device appears to require root access. I *think* I can mine with -t from my main account, though anything else is going to require root. And yes I do want each session in its own terminal so I can monitor progress later either by hotplugging a monitor or . . . something. Output to a file would produce massive logs and eat up space on the dinky little 32Gb flash drive used for boot/storage in no time.

I have attempted to start them as root (via sudo) in .profile which produces some undesirable behavior. The current problem is that I get blank terminal windows in which nothing is happening. I have also had problems with certain key environment variables not being set for anything in the terminals . . . solving that problem by including export commands to the scripts executed in the terminals (xterm -e script.sh) seems to have produced the blank terminals. Maybe I need the & character at the end of each export command? I can't see why I would but . . .

Also, I have to be careful starting anything in .profile that involves GPU compute functions, since it can prevent the desktop manager from loading in the first place, giving me a blank screen with a flashing cursor. The mining still happens in the background, but until something terminates that process, the desktop never loads. I think I can overcome that by using sleep for a minute or two. I will deal with that once I can figure out why my xterms are coming in blank like that.

Give this a try: replicating Autohotkey functionality in Linux

 

[Fallen Kell]

I have attempted to start them as root (via sudo) in .profile which produces some undesirable behavior. The current problem is that I get blank terminal windows in which nothing is happening. I have also had problems with certain key environment variables not being set for anything in the terminals .

Sorry that is a bad idea to open it in ".profile". That will get called every time you log into the system as that user, which means it can be called multiple times at the same time (you can log in more than once via ssh, telnet, remote desktop, etc., etc., etc.).

Also, I have to be careful starting anything in .profile that involves GPU compute functions, since it can prevent the desktop manager from loading in the first place, giving me a blank screen with a flashing cursor. The mining still happens in the background, but until something terminates that process, the desktop never loads. I think I can overcome that by using sleep for a minute or two. I will deal with that once I can figure out why my xterms are coming in blank like that.

Correct, the ".profile" happens before some of your other settings are loaded, such as your shell settings for your path and environment. I suspect you are actually running the process as part of your login has started your Xserver and desktop session. At this point in your login, everything needs to finish in order for it to continue the rest of the login and desktop session processes. It sits and waits (just like you noted) if you do what you did. Basically it is waiting for whatever process you started to finish before the rest of the process continues loading up your desktop environment.

This is why we recommended starting these processes at computer boot/startup, not during the login process. We also recommended using the "screen" program which will create its own pseudo terminal with hooks to allow you to attach/detach it from an active terminal, thus, allowing any program running in that "screen" to be accessed later to check on the output.

Without knowing more about exactly what flavor and version of linux you are running, we can't give you any more specifics than we have. You need to figure out how to make your own custom program startup script/service for your type of linux using screen to run your program's startup line. Preferably, you should create two of these, one for each instance that you want to run. You see, some types of linux use the tried and trusted SysV sysinit (derived from System 5 Unix, and used in most Unix, and BSD derivatives for the better part of 30 years, until only very recently). Some use the very new Systemd (latest RHEL 7/CentOS 7/Scientific Linux 7/Oracle linux and Fedora use this). Ubuntu used Upstart until very recently (I think it uses Systemd now as well). FreeBSD/NetBSD and others used runit. Still others use OpenRC..... You see, there are too many ways that your particular OS might be configured for us to give you a blanket how-to. You really need to figure out what your system actually uses and read up a little on how to create your own startup service for the particular init system your linux distribution uses.

 

[DrMrLordX]

Sorry that is a bad idea to open it in ".profile". That will get called every time you log into the system as that user, which means it can be called multiple times at the same time (you can log in more than once via ssh, telnet, remote desktop, etc., etc., etc.).

Yeah I noticed that.

Correct, the ".profile" happens before some of your other settings are loaded, such as your shell settings for your path and environment. I suspect you are actually running the process as part of your login has started your Xserver and desktop session. At this point in your login, everything needs to finish in order for it to continue the rest of the login and desktop session processes. It sits and waits (just like you noted) if you do what you did. Basically it is waiting for whatever process you started to finish before the rest of the process continues loading up your desktop environment.

Yup! Sadly .profile appears to be the only place where I can get Lubuntu 15.10 (or earlier/later versions) to accept shell commands at startup.

This is why we recommended starting these processes at computer boot/startup, not during the login process.

I've never been able to get rc.local to work; granted, I haven't gone to a lot of trouble to make it work, since I found that .profile was easy to use. Generally.

We also recommended using the "screen" program which will create its own pseudo terminal with hooks to allow you to attach/detach it from an active terminal, thus, allowing any program running in that "screen" to be accessed later to check on the output.

Yeah I'll have to look into that. sounds more versatile than using xterm -e or what have you.

As mentioned above, it's Lubuntu 15.10, and it's all systemd now.

Currently I have a bad hack using .profile that's functional, though I'll have to revise it eventually (screen sounds interesting). Guess it's time to do some reading on startup scripts via systemd.

I did figure out why I was having problems getting environment variables to apply after executing anything via sudo: I needed to use the -E switch when using the sudo command.

 

[zokudu]

Systemd unit files are super simple to setup.

https://www.freedesktop.org/software/systemd/man/systemd.unit.html

It may take some tweaking but if you made 4 unit files to start the miners along the lines of:

[Unit]
Description=EthminerGPU1
Requires=setclocks.service

[Service] 
ExecStart=screen ethminer -F http://us-east1.nanopool.org:888 /0xaddress/machinename -G --cl-local-work 256 --cl-global-work 16384 --opencl-device 1 --farm-recheck 100

[Install] 
WantedBy=multi-user.target

and one unit file to run the set clocks script:

[Unit]
Description=SetClocks

[Service] 
ExecStart=./setclocks.sh  

[Install] 
WantedBy=multi-user.target

This may take some tweaking to get it 100% correct, like you will have to input the direct path to the .sh script and I don't know if the ethminer binary is installed globally but they are not hard to write and edit.

 

[imort]

Automating this process has been damn near impossible.

Hey

Did you tried to put your command to the cron and run them with @reboot key?
They should run just with your 'root' user after that and you also can run them with & to run them at background.

It can be easier to do with a script.
You can create a script named say 'launch_miner.sh' consisting:

#!/bin/bash
ethminer -F http://us-east1.nanopool.org:8888/0xaddress/machinename -G --cl-local-work 256 --cl-global-work 16384 --opencl-device 1 --farm-recheck 100

And then run it with cron using following record in the crontab file:

@reboot launch_miner.sh&

You can add three instances with the another parameters and launch them the same way. Should work so far I know.

Take a look here too, there are some examples about cron jobs.

 

[Fallen Kell]

Hey

Did you tried to put your command to the cron and run them with @reboot key?
They should run just with your 'root' user after that and you also can run them with & to run them at background.

Cron is inherently the wrong tool to use for something to run at startup of your computer. Cron should be used for repeated scheduled tasks (i.e. update virus definitions every day, run a log cleanup every month, etc).

 

[DrMrLordX]

Yeah I got the impression that cron was the wrong way to go, but I do appreciate the suggestions. From everyone.