MDADM Raid

Setting up a softwar raid

Intro

Machine with following disks and specs

DeviceModelSizehoursstartupslabek
/dev/sdaST1000LM049-2GH1931.511872165raiddisk4
/dev/sdbHitachi HDS723021.82157241096
/dev/sdcST1000LM024 HN-M931.515390894raiddisk3
/dev/sddST1000LM049-2GH1931.511610513raiddisk2
/dev/sdeSPCC Solid State59.6311010161
/dev/sdfST1000LM024 HN-M931.514462899raiddisk1
/dev/sddST1000LM048-2E71931.513915134raiddisk5

Set up disks

sudo wipefs -a /dev/sdX
sudo mdadm --zero-superblock /dev/disk/by-partlabel/raiddisk1

Then run parted on each disk

sudo parted /dev/sdX
(parted) mklabel gpt
(parted) mkpart primary ext4 0% 100%
(parted) name 1 raiddisk
(parted) set 1 raid on
(parted) print
(parted) quit

Gould have done it with fdisk:

sudo fdisk /dev/sda

Then interactively:

  • g Not GPT
  • n New partition
  • t Change partition type to Linux RAID (29)
  • w write changes

Create the array:

sudo mdadm --create --verbose /dev/md0 --level=5 --raid-devices=4 /dev/disk/by-partlabel/raiddisk1 /dev/disk/by-partlabel/raiddisk2 /dev/disk/by-partlabel/raiddisk3 /dev/disk/by-partlabel/raiddisk4

Watch the progress (took about 8 hours)

watch cat /proc/mdstat

Make FS and mount it

sudo mkfs.ext4 /dev/md0
sudo mkdir /mnt/myraid

Edit fstab:

echo '/dev/md0    /mnt/myraid    ext4    defaults    0    0' | sudo tee -a /etc/fstab

Add a disk

sudo mdadm --add /dev/md0 /dev/disk/by-partlabel/raiddisk5
sudo mdadm --detail /dev/md0

The disk was added as a spare. Now we can grow the array:

mdadm --grow --raid-devices=5 /dev/md0
sudo e2fsck -f /dev/md0
sudo resize2fs /dev/md0


sudo mdadm --detail --scan | sudo tee -a /etc/mdadm/mdadm.conf

Issues

At some point, I restarted the nas, and the array went missing.

cat /proc/mdstat showed that the array was inactive, so did mdadm --detail /dev/md0

When doing mdadm --manage /dev/md0 --run, I got “cannot start dirty degraded array”. After this command, mdadm --detail /dev/md0 would tell me that some disks were removed.

/etc/mdadm/mdadm.conf did not contain an entry for the array, so I tried to do mdadm --examine --scan >> /etc/mdadm/mdadm.conf (as suggested here

Doing echo "clean" > /sys/block/md0/md/array_state as suggested e.g. here did not work as I got cat invalid arguments.

I finally followd this advice

Stoped the array

mdadm -S /dev/md0

The force-reassemble:

mdadm —asssmble —forec /dev/md0 /dev/sda1 /dev

Mount via sshfs fuse

griessbaum@10.0.0.4:/home/griessbaum/    /home/griessbaum/raccoon/       fuse.sshfs   noauto,delay_connect,_netdev,user,IdentityFile=/home/griessbaum/.ssh/id_rsa     0 0

Create NFS export

edit `/etc/exports’

/path/to/export/ 10.0.0.0/24(rw,async,subtree_check,crossmnt)

restart:

exportfs -ra

Mount on the client via:

apt install nfs-common

edit /etc/fstab

10.0.0.4:/home/griessbaum/ /home/griessbaum/raccoon_nfs/ nfs user,rw,noauto 0 0

Consider ansync; ChatGPT says: “the sync mount option in an NFS mount can significantly impact write performance. When you use the sync option, it forces synchronous writes, meaning that the NFS server will not acknowledge a write request until the data has been physically written to the disk. This behavior ensures data consistency but can result in slower write performance.

If you are experiencing poor write performance and can tolerate some level of data inconsistency in case of server crashes or failures, you might consider using the async option instead. The async option allows the NFS server to acknowledge write requests as soon as they are received, without waiting for the data to be written to the disk. This can improve write performance but introduces a slight risk of data loss in case of a server crash.”

Add samba

sudo apt-get install samba
sudo nano /etc/samba/smb.conf

[Movies]
        path = /home/griessbaum/raid/movies
        writable = no
        gues ok = no
        read only = yes
sudo smbpasswd -a your_username
sudo systemctl restart smbd

On the windows client:

\\your_linux_server_ip\ShareName

Monitor health using MDADMMON

sudo apt install nginx
sudo nano /etc/mdadm/mdadm.conf
MONITOR pid file=/var/run/mdadm/mdadmmon.pid

sudo nano /etc/nginx/sites-available/mdadmmon server { listen 8080; server_name _; #matche any host name.

location / {
    root /usr/share/mdadm/;
    index index.html;
}

}

sudo ln -s /etc/nginx/sites-available/mdadmmon /etc/nginx/sites-enabled/

sudo ufw allow 8080 sudo ufw reload