zfs list

shows USED and AVAILABLE after parity/redundancy are removed

zpool list

shows ALLOCATED and FREE including space occupied by parity/redundancy

• use ZFS's cache is postgresql dataset exceeds RAM, use postgres's cache if dataset fits in RAM
  • disable postgres cache, enable ZFS cache
  • enable postgres cache, disable ZFS cache

• atime=off – disable access time recording for files when they are read
• compression=lz4 – turn on compression
• recordsize=8k – set record size to postgres's default write block size 8k
• logbias=throughput –
• primarycache=metadata – ARC only cache metadata (dnode entries), not page data itself. DO NOT disable if PostgreSQL's cache is disabled!
• quota=500G

• disable full_page_writes
• disable PostgreSQL TOAST compression
• dedicate 15%-25% physical ram to ARC, and 50% to shared_buffers

# ZFS tuning. arc min at 2GB, max at 8GB. ashift to 12 for 4k sectors
# settings are in 1024*1024, not sure if need be
vfs.zfs.min_auto_ashift="12"
vfs.zfs.arc_min="2147483648"
# real memory  = 6442450944 (6144 MB)
# avail memory = 5916946432 (5642 MB)
# Setting limit to 2/3 of available memory
vfs.zfs.arc_max="3944630954"

Create ZFS raid6 zpool named storage with da devices

zpool create storage raidz2 da0 da1 da2 da3 da4 da5 da6

Create ZFS dataset media

zfs create storage/media

Create ZFS dataset data with lz4 compression, acl mode passthrough, acl inherit, and visible snapdir

zfs create -o aclmode=passthrough -o aclinherit=passthrough -o snapdir=visible -o compression=lz4 storage/data

Create folders in storage/media and storage/data, set proper owner:group

mkdir /storage/media/4k/
mkdir /storage/media/anime/
mkdir /storage/media/anime_movies/
mkdir /storage/media/documentary/
mkdir /storage/media/international/
mkdir /storage/media/japanese/
mkdir /storage/media/movies/
mkdir /storage/media/mp3/
mkdir /storage/media/russian_movies/
mkdir /storage/media/russian_shows/
mkdir /storage/media/shows/

mkdir /storage/data/smb/

chown -R nobody:nobody /storage/media
chown -R nobody:nobody /storage/data

<datatables>

</datatables>

ZFS Tuning Guide[(https://wiki.freebsd.org/ZFSTuningGuide)] FreeBSD Enterprise 1 PB Storage[(https://vermaden.wordpress.com/2019/06/19/freebsd-enterprise-1-pb-storage/)]