Fedora MD RAID check WTF

Today, out of the blue my box decided to do the RAID check on my MD devices. I can’t remember seeing it before while I was running Gentoo, but now with Fedora things feel somewhat different. Fedora does automate quite a few things out of the box – the things I have omitted in my previous Gentoo experience.

What have caught my attention was both high load on machine (out of the blue) and:

# cat /proc/mdstat
 Personalities : [raid1] [raid0]
 md126 : active raid1 sdc6[1] sda6[0]
 308793280 blocks [2/2] [UU]
 [========>............] check = 40.7% (125758464/308793280) finish=43.7min speed=69702K/sec

which lead me to a nearby Google outlet where I immediately borrowed some wisdom on a somewhat related subject: disks and S.M.A.R.T.:

# smartctl --health /dev/sdc
smartctl 5.41 2011-06-09 r3365 [x86_64-linux-] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

SMART overall-health self-assessment test result: PASSED
Please note the following marginal Attributes:
190 Airflow_Temperature_Cel 0x0022   053   040   045    Old_age   Always   In_the_past 47 (2 51 47 25)

so after enjoying rather interesting feature (smartctl that is) I have also checked around and found out that in some configurations it’s an “automatic behavior”. Which lead me to further discoveries this time from Ubuntu-land and ended up in glorious discovery of “magic device” in my posession:

# cat /etc/cron.d/raid-check
# Run system wide raid-check once a week on Sunday at 1am by default
0 1 * * Sun root /usr/sbin/raid-check

…back to sorting out the rest of my Gentoo -> Fedora migration…

My saga of exodus to Fedora

After successfully installing Fedora Core 15 on my home box I am moving all my stuff from Gentoo to Fedora. I’m still questioning my move, but lately I have less and less time to dedicate to proper maintenance of Gentoo, not to mention that at work I run RedHat servers so I’m much more familiar with RedHat insides than Gentoo’s at the moment (wasn’t like that a year or two ago). I still think Gentoo is a brilliant distro and taught me a lot about inner workings of things; I didn’t want to go Ubuntu (I really does piss me off how it gets in my way all the time) so Fedora was more like a happy medium between Gentoo and Ubuntu and it would provide some learning grounds for my office use of RedHat. Prior to my home move I moved my office machine with no problems whatsoever, but then my office setup was not as elaborate as the one at home. I have already tried migrating to Fedora at home once and failed thanks to LiveCD’s. This time I’ve got a system that works and doesn’t show too many signs of instability. Another thing is to keep Gentoo around in a VM just in case I have to fall back to it for some apps/functions.

Plan of actions:

  • add/migrate all filesystems from Gentoo to F15
  • make F15 boot from a sandwich I ran in Gentoo: RAW->MD-(raid1)->LVM
  • enable SELinux
  • depending on success: migrate Gentoo into a VM

Couple of interesting hurdles/glitches:

  • Lightspark is way more unstable in fedora vs Gentoo
  • my NVidia sound keep throwing some odd messages in F15 (not in Gentoo)
  • I have to deal with sytemd startup (quite a bit of learning here)
    • MD RAID and LVM issues
  • I have to deal with SELinux
    • NFS issues
    • ReiserFS issues

So here’s the story so far…

Installing flash I ended up with 32bit Adobe crap bolted via nsplugin-wrapper (lightspark turned out to be quite unstable in Fedora). But at least it works…

To know where my problems start you’d have to know where am I coming from. So on my Gentoo box I’ve been using ReiserFS for quite some time now due to effectiveness of it on systems with lots of small files. I also have a NAS where most of my stuff lives (or is synced to) mounted over NFS.

After installing F15 on a fresh new partition[s] and making sure install is functional it was the time for migration of the systems. Couple of problems I ran into on the first go:

  • SELinux wouldn’t let me use ReiserFS partitions as they don’t support Extended Attributes
  • As soon as I plug-in Gentoo entries into /etc/fstab all goes to hell and systemd rebels against me

First one was easy – SElinux operates in “permissive” mode now and I slowly collect it’s reports and combine fixes either into a policy or fix contexts etc. on-disk. Very tedious task. Couple of really useful tips:

from Dan Walsh:

If he had the te files from the previous run, he could use audit2allow to add rules to the te file.

# audit2allow >> myexim.te << /var/log/audit/audit.log,

I haven’t realized that every time I ran “audit2allow -M” it was leaving .te file for me in root’s home directory. So instead of generating gazillion tiny policies I ended up buffing up one generated by audit2allow further and further. Cuts down on clutter and keeps things neat.

Fedora SELinux pages have quite a bit of info too and were quite helpful in understanding what I’m dealing with so far.

My NFS shares from NAS mounted ok as root, but when I got to user access I discovered that I can’t view anything on NAS. Which naturally pissed me off. Then it dawned on me that NAS has groups out-of-sync with my workstation so I have to create nas-like group on my machine and get myself into it. So after simply adding;


to /etc/group and re-logging in things started to look bit better.

Next punch was delivered by systemd – it clearly gets ahead of itself and tries to mount MD RAID/LVM volumes prior to their initialization. So my first attempt was to get mounts into systemd-like form by crafting things like that:

$ cat /etc/systemd/system/mnt-gentoo.mount
#  This file is part of systemd.
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2 of the License, or
#  (at your option) any later version.

Description=Gentoo root

# Options=bind

which looks and feels as an abomination to me. So it took me some time and effort and my last (but not least hackish) attempt looks like this: we initialize all MD/LVM devices from boot string via dracut – truly a strike of evil genius:

title Fedora (
        root (hd0,0)
        kernel /vmlinuz- ro root=/dev/mapper/vg_gamer-rootfs rd_luks=0 rd.dm=0 LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYTABLE=us nodmraid nouveau.modeset=0 rdblacklist=nouveau rd.md.uuid=61111111:11111111:11111111:dd6f447f rd.md.uuid=21111111:11111111:11111111:3ac2a021 rd.md.uuid=f1111111:11111111:11111111:96e591a2 rd.md.uuid=f1111111:11111111:11111111:bb65be89 rd.lvm.vg=rvg rd.lvm.vg=vg_gamer rd.lvm.vg=backup
        initrd /initramfs-

If that doesn’t look evil – I don’t know what does…

Fedora Core 15 install (I hate LiveCD’s)

I’ve been using Fedora on my workstation at the office for quite a while and was pretty happy with it but when it came to install on home machine I was never able to complete setup for one reason or another.

What really surprised me is how flaky LiveCD’s were when it comes to my system and setup. I had installer crash on me in random places and overall quality of installed system being below what I have on my workstation at the office. After multiple failed attempts and aggravation caused by Fedora’s inistance on renumbering of my MD devices which causes annoying inconvenience booting Gentoo back, I have arrived at a setup option that looks workable: Install from DVD. Not liveCD! Fedora didn’t make it an apparent choice – it’s burried in their download screens under “alternative media”. Installing F14 I used that option which explains why I never had problems with that install. Now after using F15 Installer DVD I think it should’ve been the default option and liveCD should be a fall-back.

Here’s the difference – liveCD already has a pre-installed version of the OS that you may or may not agree with and apparently some settings there didn’t quite agree with my system design. Now that I went through full install and was able to customize system at install time (not to mention net install option where you install up-to-date system) things look much brighter.