Boot from iSCSI / LUN

[EddyG]

Tnx. For the update. I added this to the above installation text.

[Stepdes]

Code: Select all

sudo apt-get install initramfs-tools

Code: Select all

cd /boot
sudo update-initramfs -v -k `uname -r` -c > /home/pi/initramfs.txt

I get stuck here with this errors:
/usr/share/initramfs-tools/scripts/init-top/ORDER ignored: not executable
/usr/share/initramfs-tools/scripts/local-premount/ORDER ignored: not executable
/usr/share/initramfs-tools/scripts/local-bottom/ORDER ignored: not executable
/usr/share/initramfs-tools/scripts/local-top/ORDER ignored: not executable
/usr/share/initramfs-tools/scripts/init-bottom/ORDER ignored: not executable

What i'm doing wrong?

Regards,
Stephan

[EddyG]

Are those files present at that location? What is the content of that 'ORDER' file?
I suppose those are normal text files, explaining something.
I do not have them (anymore).
In that dir '/usr/share/initramfs-tools/scripts/*' are script files that are executed during the process.
Text files cannot be executed.
It states that these files are ignored, so I suppose that the total process went well.
Look into the file '/home/pi/initramfs.txt', and see if there are any errors mentioned in that file.

[Stepdes]

Look into the file '/home/pi/initramfs.txt', and see if there are any errors mentioned in that file.

Ok, i looked at the file and there where no errors, so i continued, every thing went fine just until i rebooted, now i can't ssh into the system anymore.
Got a "connection refused"

Stephan

[EddyG]

Is there a file 'ssh' present in /boot ?
Did you enable ssh in raspi-config ?
When you created the 'system' and copied it from sd-card to lun, was ssh possible then.
Are IP-address and ssh port the same?

[kolossi]

Hi EddyG

I'm not actually using Domoticz, but I had to sign up on this site to say thank you so much for this guide

It's worked great on a Pi 4B (actually 4 of them!) with 2019-09-26 raspbian / 4.19.75-v7l+ kernel. iScsi targets are on a Synology DS415+.

If anyone's into ansible (I can't offer support on that) I've created a version of the process as ansible tasks - https://github.com/Kolossi/ansible-role ... siroot.yml

It's part of an ansible role I'm creating to turn pi's with stock flashed raspbian cards into a Kubernetes cluster. Still a work in progress, if you watch the repo you'll find when I'm done!

But again, many thanks EddyG for all the work you put in.

Kolossi

[EddyG]

Tnx, sorry for the late response (I was on a holiday )
Nice work on the ansible project.

[EddyG]

Just did an update to version Linux raspberrybuster 4.19.97-v7+ #1294 SMP Thu Jan 30 13:15:58 GMT 2020 armv7l GNU/Linux guided by the first post Step 12, and everything went fine.
I added some steps to Step 12 on the first post.

[EddyG]

Just got my Raspberry Pi 4B 4Gb (Rev.1.2) and it just works fine with the LUN.
If you copy a LUN used for a Raspberry Pi 3B+ or other, then you have to boot to the SD-card because the 'initrd.img-4.19.yy-xxx' file has to be re-created.

Code: Select all

sudo /boot/switch_boot_setup.sh org
sudo reboot

and after the boot re-create the 'initrd.img-4.19.yy-xxx' file with.

Code: Select all

cd /boot
sudo update-initramfs -v -k `uname -r` -c > /home/pi/initramfs.txt

Because the Pi 4B has an other processor. Instead of '-v7+' at the end of the initramfs file you will have '-v7l+' at the end of the file.
Before you reboot you will have to edit config.txt.lun.
At the end of the file 'config.txt.lun' you will have to change then filename to the newly created initramfs file.

Code: Select all

initramfs initrd.img-4.19.yy-v7l+ followkernel

Now you can reboot to the lun with.

Code: Select all

sudo /boot/switch_boot_setup.sh lun
sudo reboot

[dobber]

Great guide!
I was using berryboot for my old rpi3b+, but latest release is broken. At least for rpi4 and iscsi.
This guide made my day! Everything back up running

Edit: One small issue, that probably has nothing to do with your scripts or guide. When connected to lun I can not use sudo reboot. Ssh stops and vnc also. I can still ping the system. But the system does not reboot. When on sd/org I can use reboot just fine. Anyone else experiencing this? I'm on latest buster with kernel 5.10.11-v7l+

[EddyG]

Never experienced that a reboot would not work, both on pi 3B+ or 4B and upgraded the kernel several times.
Perhaps put an extra 'sync' command in the script or just before the reboot.
Or use the -f parameter with the reboot command.

[dobber]

When I connected a monitor I noticed that iscsi was disconnected during the shutdown process. After that the binary shutdown was no longer available, so the system halted. "Failed to execute shutdown binary" and some BTRFS errors.
I do not know if if it is due to me using btrfs file system or something elese. However I solved it by adding 2 lines to .bashrc:

alias sudo='sudo '
alias reboot='sudo mount /dev/mmcblk0p2 /media/pi/rootfs/ && sudo mount --bind /media/pi/rootfs/usr /usr && sudo shutdown -r now'

I temporarily mount the sdcard's /usr as the current /usr. Thereby making it possible to reach the shutdown executable even after iscsi has been disconnected.

adding -f switch did nothing to improve.

[EddyG]

Nice solution, but it should not be, because I am able to reboot without it.
In my system the reboot and shutdown are symlinks to /bin/systemctl. ( Buster Linux 5.4.83-v7l+ )
I am using BTRFS too, so that should not cause the problem.

[EddyG]

I installed Debian 64 bits bullseye on my Pi 4 B and LUN and it works fine too.
The small difference is that Debian installs the Pi files in /boot/firmware etc...
Also the upgrade procedure from Linux 5.10.0-6-arm64 aarch64 GNU/Linux Debian GNU/Linux 11 (bullseye) to the 5.10.0-7 works fine too
Domoticz seems to run a little bit faster.

[mcmikev]

Hi,

I noticed that when I sync from SD to LUN that old dirs and files remain on the LUN.

I remove old domoticz dirs on the SD and then sync to lun but the dirs remain on the lun.
Never did check if this was the case before, just noticed it !!

[mcmikev]

Figured it out.

In the sync script (/root/scripts) there is a line:

Code: Select all

PARAM="-a --update --verbose --delete"

when I changed that line into

Code: Select all

PARAM="-avh  --update --verbose --delete"

then the files and dirs are deleted on LUN when run from SD.

When started from LUN and file is editted for sync to SD then it also is in sync

[EddyG]

I use this parameter for rsync both sides.

Code: Select all

PARAM="--force -rltWDEHXAgoptx --verbose --delete"
 rsync $PARAM --exclude-from $EXCLUDEFILE $SOURCE $MOUNTPOINT >> $LOGFILE

It might be the --delete parameter.

The exclude file contain this, and that works fine for me. Adapt for your own system.

Spoiler: show

- *~
- *.swp
- *.tmp
- *.temp
- *.bak
- *.pid
- *.sock
- *.db-shm
- *.db-wal
- /boot/
- /dev/
- /etc/fstab
- /etc/cron.d/sync*
- /etc/logrotate.d/sync*
- /lost\+found/*
- /media/
- /mnt/
- /proc/
- /root/scripts/sync*
- /root/scripts/system*
- /run/
- /sys/
- /tmp/
- /var/lock/
- /var/log/
- /var/run/
- /var/tmp/
- /var/swap
- /var/log/sync*
- /var/lib/samba/private/

[mcmikev]

Hi EddyG,

Even with your settings there is still a problem in the sync.

I see in the "org" other dirs and files even after sync then in the "lun" environment.

Weird , but the sync seems not to do a 1:1 sync with delete.

I have your settings in the PARAM

[EddyG]

I have those scripts with the rsync running on 4 Raspberry's and 4 LUNs
rsync is critical if you use or don't use as the last character a slash '/' Could that be your problem?
This is part of the documentation I kept for when I get confused.

Code: Select all

# -a, --archive               Archive mode equals -rlptgoD (no -H,-A,-X)
# -x, --one-file-system       Don't cross filesystem boundaries
# -H, --hard-links            Preserve hard links
# -A, --acls                  preserve ACLs (implies -p)
# -X, --xattrs                preserve extended attributes
# -W, --whole-file            Copy whole files, no incremental checks
# -D, --devices --specials    Preserve devices (super-user only) +files
# -u, --update                update only (don't overwrite newer files)
# -v, --verbose               Increase verbosity
# -n, --dry-run               Perform a trial run with no changes made
# --delete                Delete files that don't exist on the sending side
# --force                 Force deletion of directories even if not empty

# rsync -avz foo:src/bar/ /data/tmp

# a trailing slash on the source changes this behavior to transfer all files from the directory src/bar
# on the machine foo into the /data/tmp/.
# A trailing / on a source name means "copy the contents of this directory".
# Without a trailing slash it means "copy the directory".
# This difference becomes particularly important when using the --delete option.

[mcmikev]

Hi,

I am trying to setup a new RPI4b but I don't get the iscsi mounted as /dev/sda.

It shows only this in dmesg after the login command (first few steps)
scsi host0: iSCSI Initiator over TCP/IP

normally you would see sda or sdb as new drive/partition , but with bullseye that seems to have changed.
Do you know if there must be something done different with OS Bullseye?