Page 1 of 1

sd card or usb storage?

Posted: Friday 08 September 2023 23:08
by tiga
i am replacing my old raspberry 2b with a new raspberry 4 and want to set it up new.

but is it better to run it on a sd card or usb storage?
or is there no difference in performance reliability?

i cant seem to find the answor.

thanks

Re: sd card or usb storage?

Posted: Saturday 09 September 2023 9:18
by Kedi
Go for a SSD-drive, the best possible solution.
For my master domoticz I use Samsung drives (250Gb) and for my test system I use a cheap Intenso 250 Gb drive

Re: sd card or usb storage?

Posted: Saturday 09 September 2023 14:47
by lost
tiga wrote: Friday 08 September 2023 23:08 is it better to run it on a sd card or usb storage?
or is there no difference in performance reliability?
In performance, yes, especially using USB3 (better BW+full-duplex) on PI4.
Now, you can use uSD cards in A2 standard (currently Applicative standard exists in A1/A2 flavors, A2 have better IO/sec figures). Those are designed for in smartphone/tablet use case and, being designed for such 24/7 always on use case I have no issue with them (using good brands) since years.

Better IMO if you want to keep something compact & performance OK for Domoticz use case.

Re: sd card or usb storage?

Posted: Saturday 09 September 2023 15:13
by kiddigital
Both options are fine. Most of the time domoticz does not require huge disk storage and/or performance.

Good quality SD card (and a working backup procedure) is sufficient.

Switching from a Pi 2b to a 4th gen Pi, powerconsumption and heat are more likely to give problems.

Make sure you can cool your Pi or maybe underclock it so it does not run on maximum speed.

Good stable powersupply with at least 3 or 4 amps or more depending on connected peripherals.

Re: sd card or usb storage?

Posted: Saturday 09 September 2023 16:06
by lost
kiddigital wrote: Saturday 09 September 2023 15:13 maybe underclock it so it does not run on maximum speed.
Default lower speed setting can also be minimized under default (core min was 400MHz on my system), so power governor can slow things even more when possible and mean temperature drops. Only reduced max from default 1200 (on my PI3B) to 1100, but 1.2GHz was almost never reached thus the min was clearly the improvement here.

In the [all] section at the end of /boot/config.txt I have this setting changes for my PI3B:

Code: Select all

# Underclock PI3B max arm=1200/core=400
# Monitor :  watch -n1 vcgencmd measure_clock [arm|core]
# https://www.raspberrypi.com/documentation/computers/config_txt.html
arm_freq=1100
core_freq=350
arm_freq_min=400
core_freq_min=200
When I reinstalled from debian 10 32 bit to debian 11 64 bit, my system temperature increased by ~4/5°C, thus these settings that allowed to get back to previous figures.
Don't know if reason is the version change or going 64bit. Most probably the last hypothesis, even if surprising...

That's just 50MHz min and 100MHz drop on core freq, but effect on mean temperature is there with no noticeable perf change.

For SD, on the other side, interface can be overclocked: PI3B setting was quite conservative (50MHz, max is 100). I use 83MHz since years, same place for tuning this:

Code: Select all

dtparam=sd_overclock=83
Lots of "knobs" on FS/virtual mem management side as well to increase reactivity & SD use efficiency (commit delay defaulting at 5s etc).

My systems runs with a battery backup (thus a commit delay set to 2mn/120s) so I have this added in my /etc/sysctl.conf:

Code: Select all

vm.dirty_ratio=60
vm.dirty_expire_centisecs=120000
These settings can also be changed using sysfs for test (but then not restored after a reboot). This allows much more caching & write merge: With this domoticz http UI feels more reactive.

Re: sd card or usb storage?

Posted: Saturday 09 September 2023 21:13
by tiga
thank you all for the reactions!

Re: sd card or usb storage?

Posted: Sunday 10 September 2023 7:55
by Kedi
lost wrote: Saturday 09 September 2023 16:06

Code: Select all

vm.dirty_ratio=60
vm.dirty_expire_centisecs=120000
Isn't that 1 zero to much? 120000 centiseconds equal 1200 seconds equals 20 minutes.

B.t.w. I did buy a RPi 4B for performance, so just buy a good fan if it gets to hot.

Re: sd card or usb storage?

Posted: Sunday 10 September 2023 10:01
by lost
Kedi wrote: Sunday 10 September 2023 7:55 Isn't that 1 zero to much? 120000 centiseconds equal 1200 seconds equals 20 minutes.
Should be centisec according to setting name... but on a live system system with default setting (looks that's 3s, not 5 as I said previously) that should show a few seconds:

Code: Select all

$ cat /proc/sys/vm/dirty_expire_centisecs 
3000
The PI3B that hosts domoticz with boot settings shows hereupper value:

Code: Select all

$ cat /proc/sys/vm/dirty_expire_centisecs 
120000
Must agree that's a bit confusing but IMO that's 120s (2mn). Anyway, that's a max so if ratio is reached due to heavy writes commit will occur sooner. This setting clearly allows to merge much more writes to constantly appended log files and much simpler to setup a log2ram.

Here is the doc for virtual management tunables (unmodified for long, so maybe that's now millisecs but variable naming used in many systems remained?):
https://docs.kernel.org/admin-guide/sysctl/vm.html

Re: sd card or usb storage?

Posted: Sunday 10 September 2023 10:18
by Kedi
Do you really think that everybody is just copying the 'centi' and its explanation about 1/100th of a second?
Even Microsoft does this: https://learn.microsoft.com/en-us/azure ... stem-cache
Or Oracle: https://support.oracle.com/knowledge/Or ... 973_1.html
Or could it be that the default 3000 is just 30 seconds?

Edit: from 9 month back: https://lonesysadmin.net/2013/12/22/bet ... rty_ratio/

Edit2:
You could put the command

Code: Select all

cat /proc/vmstat | egrep "nr_dirty "
in a loop and see how it adds up and jumps to zero again.
I my case it looks like the default 3000 is 30 seconds.

Re: sd card or usb storage?

Posted: Monday 11 September 2023 9:54
by lost
Kedi wrote: Sunday 10 September 2023 10:18 You could put the command

Code: Select all

cat /proc/vmstat | egrep "nr_dirty "
in a loop and see how it adds up and jumps to zero again.
I my case it looks like the default 3000 is 30 seconds.
You're right: Really had in mind for long this was a few seconds to limit losses in case of sudden power fail (or any kernel crash in an uncontrolled way that may not allow some VMM caches flush) for anything that gets cached there even before making it's way to FS and their sync/commit.

And ext4 default commit time is 5s for instance, so the same order of magnitude for VMM caches expire time was in my mind making sense: I was wrong.

Looks I got lured for many years in configuring this 10x the figures I expected. Did not observed side effect, luckily. Thanks for pointing this out!