sd card or usb storage?

Topics (not sure which fora)
when not sure where to post, post here and mods will move it to right forum.

Moderators: leecollings, remb0

Post Reply
tiga
Posts: 156
Joined: Friday 27 May 2016 20:15
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: nederland
Contact:

sd card or usb storage?

Post by tiga »

i am replacing my old raspberry 2b with a new raspberry 4 and want to set it up new.

but is it better to run it on a sd card or usb storage?
or is there no difference in performance reliability?

i cant seem to find the answor.

thanks
Kedi
Posts: 536
Joined: Monday 20 March 2023 14:41
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Somewhere in NL
Contact:

Re: sd card or usb storage?

Post by Kedi »

Go for a SSD-drive, the best possible solution.
For my master domoticz I use Samsung drives (250Gb) and for my test system I use a cheap Intenso 250 Gb drive
Logic will get you from A to B. Imagination will take you everywhere.
lost
Posts: 616
Joined: Thursday 10 November 2016 9:30
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: sd card or usb storage?

Post by lost »

tiga wrote: Friday 08 September 2023 23:08 is it better to run it on a sd card or usb storage?
or is there no difference in performance reliability?
In performance, yes, especially using USB3 (better BW+full-duplex) on PI4.
Now, you can use uSD cards in A2 standard (currently Applicative standard exists in A1/A2 flavors, A2 have better IO/sec figures). Those are designed for in smartphone/tablet use case and, being designed for such 24/7 always on use case I have no issue with them (using good brands) since years.

Better IMO if you want to keep something compact & performance OK for Domoticz use case.
User avatar
kiddigital
Posts: 435
Joined: Thursday 10 August 2017 6:52
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Netherlands
Contact:

Re: sd card or usb storage?

Post by kiddigital »

Both options are fine. Most of the time domoticz does not require huge disk storage and/or performance.

Good quality SD card (and a working backup procedure) is sufficient.

Switching from a Pi 2b to a 4th gen Pi, powerconsumption and heat are more likely to give problems.

Make sure you can cool your Pi or maybe underclock it so it does not run on maximum speed.

Good stable powersupply with at least 3 or 4 amps or more depending on connected peripherals.
One RPi with Domoticz, RFX433e, aeon labs z-wave plus stick GEN5, ha-bridge 5.4.0 for Alexa, Philips Hue Bridge, Pimoroni Automation Hat
One RPi with Pi foundation standard touch screen to display Dashticz
lost
Posts: 616
Joined: Thursday 10 November 2016 9:30
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: sd card or usb storage?

Post by lost »

kiddigital wrote: Saturday 09 September 2023 15:13 maybe underclock it so it does not run on maximum speed.
Default lower speed setting can also be minimized under default (core min was 400MHz on my system), so power governor can slow things even more when possible and mean temperature drops. Only reduced max from default 1200 (on my PI3B) to 1100, but 1.2GHz was almost never reached thus the min was clearly the improvement here.

In the [all] section at the end of /boot/config.txt I have this setting changes for my PI3B:

Code: Select all

# Underclock PI3B max arm=1200/core=400
# Monitor :  watch -n1 vcgencmd measure_clock [arm|core]
# https://www.raspberrypi.com/documentation/computers/config_txt.html
arm_freq=1100
core_freq=350
arm_freq_min=400
core_freq_min=200
When I reinstalled from debian 10 32 bit to debian 11 64 bit, my system temperature increased by ~4/5°C, thus these settings that allowed to get back to previous figures.
Don't know if reason is the version change or going 64bit. Most probably the last hypothesis, even if surprising...

That's just 50MHz min and 100MHz drop on core freq, but effect on mean temperature is there with no noticeable perf change.

For SD, on the other side, interface can be overclocked: PI3B setting was quite conservative (50MHz, max is 100). I use 83MHz since years, same place for tuning this:

Code: Select all

dtparam=sd_overclock=83
Lots of "knobs" on FS/virtual mem management side as well to increase reactivity & SD use efficiency (commit delay defaulting at 5s etc).

My systems runs with a battery backup (thus a commit delay set to 2mn/120s) so I have this added in my /etc/sysctl.conf:

Code: Select all

vm.dirty_ratio=60
vm.dirty_expire_centisecs=120000
These settings can also be changed using sysfs for test (but then not restored after a reboot). This allows much more caching & write merge: With this domoticz http UI feels more reactive.
tiga
Posts: 156
Joined: Friday 27 May 2016 20:15
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: nederland
Contact:

Re: sd card or usb storage?

Post by tiga »

thank you all for the reactions!
Kedi
Posts: 536
Joined: Monday 20 March 2023 14:41
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Somewhere in NL
Contact:

Re: sd card or usb storage?

Post by Kedi »

lost wrote: Saturday 09 September 2023 16:06

Code: Select all

vm.dirty_ratio=60
vm.dirty_expire_centisecs=120000
Isn't that 1 zero to much? 120000 centiseconds equal 1200 seconds equals 20 minutes.

B.t.w. I did buy a RPi 4B for performance, so just buy a good fan if it gets to hot.
Logic will get you from A to B. Imagination will take you everywhere.
lost
Posts: 616
Joined: Thursday 10 November 2016 9:30
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: sd card or usb storage?

Post by lost »

Kedi wrote: Sunday 10 September 2023 7:55 Isn't that 1 zero to much? 120000 centiseconds equal 1200 seconds equals 20 minutes.
Should be centisec according to setting name... but on a live system system with default setting (looks that's 3s, not 5 as I said previously) that should show a few seconds:

Code: Select all

$ cat /proc/sys/vm/dirty_expire_centisecs 
3000
The PI3B that hosts domoticz with boot settings shows hereupper value:

Code: Select all

$ cat /proc/sys/vm/dirty_expire_centisecs 
120000
Must agree that's a bit confusing but IMO that's 120s (2mn). Anyway, that's a max so if ratio is reached due to heavy writes commit will occur sooner. This setting clearly allows to merge much more writes to constantly appended log files and much simpler to setup a log2ram.

Here is the doc for virtual management tunables (unmodified for long, so maybe that's now millisecs but variable naming used in many systems remained?):
https://docs.kernel.org/admin-guide/sysctl/vm.html
Kedi
Posts: 536
Joined: Monday 20 March 2023 14:41
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Somewhere in NL
Contact:

Re: sd card or usb storage?

Post by Kedi »

Do you really think that everybody is just copying the 'centi' and its explanation about 1/100th of a second?
Even Microsoft does this: https://learn.microsoft.com/en-us/azure ... stem-cache
Or Oracle: https://support.oracle.com/knowledge/Or ... 973_1.html
Or could it be that the default 3000 is just 30 seconds?

Edit: from 9 month back: https://lonesysadmin.net/2013/12/22/bet ... rty_ratio/

Edit2:
You could put the command

Code: Select all

cat /proc/vmstat | egrep "nr_dirty "
in a loop and see how it adds up and jumps to zero again.
I my case it looks like the default 3000 is 30 seconds.
Logic will get you from A to B. Imagination will take you everywhere.
lost
Posts: 616
Joined: Thursday 10 November 2016 9:30
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: sd card or usb storage?

Post by lost »

Kedi wrote: Sunday 10 September 2023 10:18 You could put the command

Code: Select all

cat /proc/vmstat | egrep "nr_dirty "
in a loop and see how it adds up and jumps to zero again.
I my case it looks like the default 3000 is 30 seconds.
You're right: Really had in mind for long this was a few seconds to limit losses in case of sudden power fail (or any kernel crash in an uncontrolled way that may not allow some VMM caches flush) for anything that gets cached there even before making it's way to FS and their sync/commit.

And ext4 default commit time is 5s for instance, so the same order of magnitude for VMM caches expire time was in my mind making sense: I was wrong.

Looks I got lured for many years in configuring this 10x the figures I expected. Did not observed side effect, luckily. Thanks for pointing this out!
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest