All posts by daemon

Fedora MD RAID check WTF

Today, out of the blue my box decided to do the RAID check on my MD devices. I can’t remember seeing it before while I was running Gentoo, but now with Fedora things feel somewhat different. Fedora does automate quite a few things out of the box – the things I have omitted in my previous Gentoo experience.

What have caught my attention was both high load on machine (out of the blue) and:

# cat /proc/mdstat
 Personalities : [raid1] [raid0]
 md126 : active raid1 sdc6[1] sda6[0]
 308793280 blocks [2/2] [UU]
 [========>............] check = 40.7% (125758464/308793280) finish=43.7min speed=69702K/sec

which lead me to a nearby Google outlet where I immediately borrowed some wisdom on a somewhat related subject: disks and S.M.A.R.T.:

# smartctl --health /dev/sdc
smartctl 5.41 2011-06-09 r3365 [x86_64-linux-2.6.38.8-32.fc15.x86_64] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
Please note the following marginal Attributes:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
190 Airflow_Temperature_Cel 0x0022   053   040   045    Old_age   Always   In_the_past 47 (2 51 47 25)

so after enjoying rather interesting feature (smartctl that is) I have also checked around and found out that in some configurations it’s an “automatic behavior”. Which lead me to further discoveries this time from Ubuntu-land and ended up in glorious discovery of “magic device” in my posession:

# cat /etc/cron.d/raid-check
# Run system wide raid-check once a week on Sunday at 1am by default
0 1 * * Sun root /usr/sbin/raid-check

…back to sorting out the rest of my Gentoo -> Fedora migration…

My saga of exodus to Fedora

After successfully installing Fedora Core 15 on my home box I am moving all my stuff from Gentoo to Fedora. I’m still questioning my move, but lately I have less and less time to dedicate to proper maintenance of Gentoo, not to mention that at work I run RedHat servers so I’m much more familiar with RedHat insides than Gentoo’s at the moment (wasn’t like that a year or two ago). I still think Gentoo is a brilliant distro and taught me a lot about inner workings of things; I didn’t want to go Ubuntu (I really does piss me off how it gets in my way all the time) so Fedora was more like a happy medium between Gentoo and Ubuntu and it would provide some learning grounds for my office use of RedHat. Prior to my home move I moved my office machine with no problems whatsoever, but then my office setup was not as elaborate as the one at home. I have already tried migrating to Fedora at home once and failed thanks to LiveCD’s. This time I’ve got a system that works and doesn’t show too many signs of instability. Another thing is to keep Gentoo around in a VM just in case I have to fall back to it for some apps/functions.

Plan of actions:

  • add/migrate all filesystems from Gentoo to F15
  • make F15 boot from a sandwich I ran in Gentoo: RAW->MD-(raid1)->LVM
  • enable SELinux
  • depending on success: migrate Gentoo into a VM

Couple of interesting hurdles/glitches:

  • Lightspark is way more unstable in fedora vs Gentoo
  • my NVidia sound keep throwing some odd messages in F15 (not in Gentoo)
  • I have to deal with sytemd startup (quite a bit of learning here)
    • MD RAID and LVM issues
  • I have to deal with SELinux
    • NFS issues
    • ReiserFS issues

So here’s the story so far…

Installing flash I ended up with 32bit Adobe crap bolted via nsplugin-wrapper (lightspark turned out to be quite unstable in Fedora). But at least it works…

To know where my problems start you’d have to know where am I coming from. So on my Gentoo box I’ve been using ReiserFS for quite some time now due to effectiveness of it on systems with lots of small files. I also have a NAS where most of my stuff lives (or is synced to) mounted over NFS.

After installing F15 on a fresh new partition[s] and making sure install is functional it was the time for migration of the systems. Couple of problems I ran into on the first go:

  • SELinux wouldn’t let me use ReiserFS partitions as they don’t support Extended Attributes
  • As soon as I plug-in Gentoo entries into /etc/fstab all goes to hell and systemd rebels against me

First one was easy – SElinux operates in “permissive” mode now and I slowly collect it’s reports and combine fixes either into a policy or fix contexts etc. on-disk. Very tedious task. Couple of really useful tips:

from Dan Walsh:

If he had the te files from the previous run, he could use audit2allow to add rules to the te file.

# audit2allow >> myexim.te << /var/log/audit/audit.log,

I haven’t realized that every time I ran “audit2allow -M” it was leaving .te file for me in root’s home directory. So instead of generating gazillion tiny policies I ended up buffing up one generated by audit2allow further and further. Cuts down on clutter and keeps things neat.

Fedora SELinux pages have quite a bit of info too and were quite helpful in understanding what I’m dealing with so far.

My NFS shares from NAS mounted ok as root, but when I got to user access I discovered that I can’t view anything on NAS. Which naturally pissed me off. Then it dawned on me that NAS has groups out-of-sync with my workstation so I have to create nas-like group on my machine and get myself into it. So after simply adding;

nas_users:x:102:me,myself,myfamily

to /etc/group and re-logging in things started to look bit better.

Next punch was delivered by systemd – it clearly gets ahead of itself and tries to mount MD RAID/LVM volumes prior to their initialization. So my first attempt was to get mounts into systemd-like form by crafting things like that:

$ cat /etc/systemd/system/mnt-gentoo.mount
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2 of the License, or
#  (at your option) any later version.

[Unit]
Description=Gentoo root
After=dev-md124.device
Requires=dev-md124.device

[Mount]
What=/dev/md124
Where=/mnt/gentoo
Type=ext3
# Options=bind

which looks and feels as an abomination to me. So it took me some time and effort and my last (but not least hackish) attempt looks like this: we initialize all MD/LVM devices from boot string via dracut – truly a strike of evil genius:

title Fedora (2.6.40.3-0.fc15.x86_64)
        root (hd0,0)
        kernel /vmlinuz-2.6.40.3-0.fc15.x86_64 ro root=/dev/mapper/vg_gamer-rootfs rd_luks=0 rd.dm=0 LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYTABLE=us nodmraid nouveau.modeset=0 rdblacklist=nouveau rd.md.uuid=61111111:11111111:11111111:dd6f447f rd.md.uuid=21111111:11111111:11111111:3ac2a021 rd.md.uuid=f1111111:11111111:11111111:96e591a2 rd.md.uuid=f1111111:11111111:11111111:bb65be89 rd.lvm.vg=rvg rd.lvm.vg=vg_gamer rd.lvm.vg=backup
        initrd /initramfs-2.6.40.3-0.fc15.x86_64.img

If that doesn’t look evil – I don’t know what does…

Customizing Samsung Galaxy S II (I9100) under Linux (pt. 2)

After some time with LiteningROM I did notice that my idle (stand-by) battery use decreased (good thing) – allowing me to get 2d of moderate use out of the phone. However I did notice that LiteningROM was memory hungry and it also looked like any interactivity caused immense power drain in the system.

Today I re-flashed with new Cognition 1.0.8

First observation: memory footprint does seem to be smaller (I couldn’t get “used RAM” under 400M with LiteningROM and with new CognitionROM I get into sub-300M zone. nice)

Now, I wait and see whether power consumption goes down to the levels of LiteningROM or below and then decide which one I’m going to stick with until CyanogenROM produces stable image for SGS2.

Fedora Core 15 install (I hate LiveCD’s)

I’ve been using Fedora on my workstation at the office for quite a while and was pretty happy with it but when it came to install on home machine I was never able to complete setup for one reason or another.

What really surprised me is how flaky LiveCD’s were when it comes to my system and setup. I had installer crash on me in random places and overall quality of installed system being below what I have on my workstation at the office. After multiple failed attempts and aggravation caused by Fedora’s inistance on renumbering of my MD devices which causes annoying inconvenience booting Gentoo back, I have arrived at a setup option that looks workable: Install from DVD. Not liveCD! Fedora didn’t make it an apparent choice – it’s burried in their download screens under “alternative media”. Installing F14 I used that option which explains why I never had problems with that install. Now after using F15 Installer DVD I think it should’ve been the default option and liveCD should be a fall-back.

Here’s the difference – liveCD already has a pre-installed version of the OS that you may or may not agree with and apparently some settings there didn’t quite agree with my system design. Now that I went through full install and was able to customize system at install time (not to mention net install option where you install up-to-date system) things look much brighter.

 

SpeakOut, Rogers and my beautiful SGS2

Looks like Data usage on SpeakOut may be an issue (or may be not..). I started gathering some info on configuring appropriate APN.

Picked up some wisdom from howardforums :

Name: Rogers
APN: goam.com
Proxy: <not set>
Port: <not set>
Username: wapuser1
Password: wap
Server: <not set>
MMSC: <not set>
MMS proxy: <not set>
MMS port: <not set>
MCC: 302
MNC: 720
Authentication type: None
APN type: <not set>

and it looks like that would limit use down to Mozilla and other browsers that have proxy settings.

# Run Fennec, and in address bar type: about:config. This will take you to the advanced configuration panel.
# Now you need to find proxy entries, to simplify that you can filter it out by taping proxy in the field below address bar.
# Modify network.proxy.http and network.proxy.http_port to your proxy server and port. (Proxy 10.128.1.69, Port 80)
# You can also modify network.proxy.ssl and network.proxy.ssl_port to be able to access https addresses. (Proxy 10.128.1.69, Port 9203)
# Set network.proxy.type value to 1.

So fun is not over just yet…. It also looks like above IP is the same IP that pops from default Rogers APN when you reset device. I’ll have to get “browsing” activated to tinker with it a bit more…

Wireless shenanigans with android (on SGS2)

While I was busy beating the dust out of my new SGS2 and tinkering with ROM’s, I came across quite a few “hidden features” that were not known to me.

*#*#4636#*#*

above will give you “test mode” in which you can run things like “ping test” etc. Nice for testing your data connection without actually connecting anywhere 🙂

*#9900#

that looks like debugging screen in which you can manipulate logs etc. for reporting, and “Disable dormancy” for better battery life.

*#1234#

above will produce phone information – very handy for reporting and looking up issues etc.

 

Customizing Samsung Galaxy S II (I9100) under Linux

I’ve got SGS2 and it’s damn cool. But that’s not enough. I really have to pimp it up/down to arrive at a device that I can call mine. Looks like Cyanogen mod that claims to be fully open source will be my goal.

So this is where I’m going to keep track of my progress and document the procedure.

Some links and tools to start:

To get phone info needed for proper images etc. you’d have to dial:

*#1234#

in “phone” app.

Follow [ROM + Guide]Official i9100 KE7/KE8/KF1/KF3/KF4/KF2/KF3 download and Root Guide and get your stock ROM and Kernel if you’re as paranoid as me.

Now the rooting…

  • Get the CF-Root. I used CF-Root-SGS2_XW_XEU_KE7-v4.0-CWM4
  • extract zip, extract tar inside and get zImage file
  • Shut down the phone
  • Make sure USB is not plugged in
  • boot phone in download mode ( “Vol down”+”HOME”+”Power”)
  • plug USB back in
  • In Heimdall use above zImage to flash it over to SGS2
  • phone reboots
  • on boot you’re presented with ugly yellow warning sign – click “Vol down” to bypass it

Immediate actions:

  1. Backup!
    1. boot device into “Recovery mode”: “Vol up” + “Home” + “Power”
    2. follow the menu

Fix-ups:

Disable Fast Dormancy:

  1. Dial

    *#9900#

  2. Press Disable Fast Dormancy
  3. Press Exit

VillainROM

VillainROM is available for SGS2 now so I had to try it. Now that I was comfortable with Heimdall operations following VillainROM Flashing Instructions was really straight-forward:

  • CF-Root – I’ve got that already
  • upload VillainROM .zip file to phone’s SD card
  • disonnect the phone from PC
  • shut down the phone
  • boot into recovery mode (“Vol up”+”Home”+”Power”)
  • from ClockWorkMod launch backup
  • once backup is done – “wipe data/factory reset” and “wipe cache” (I suspect that’s the step that killed ALL data on the phone, including photos etc.)
  • “Install zip from SD” – locate VillainROM .zip file on SD and proceed
  • reboot phone into VillainROM (takes a while on first boot)
  • from booted VillainROM – install VRTweaks
  • Launch VRTweaks (it requires internet connection)
  • from VRTweaks install “Symlink fix”
  • reboot
  • mount freshly rebooted phone from PC
  • copy CF-Root .zip to the phone
  • launch CWM
  • from CWM flash kernel using CF-Root .zip file from above
  • reboot (automatic)
  • download MODEM file
  • reboot into “Download mode” (“Vol Down”+”Home”+”Power”)
  • flash modem file onto the phone
  • reboot (automatic)

Now that VillainROM is there: backup! Launch CWM app and initiate backup from there – you can give backup custom name from here vs “Recovery mode” where it’s predefined timestamp.

I got fed up with “Yellow Warning sign” on boot so I followed the steps to get rid of it:

  • find out which stock kernel is your phone running
  • download stock ROM and MODEM
  • extract Stock ROM (first .zip then .tar – you’ll end up with bunch of .img files and zImage – you will need zImage)
  • make sure your VRTweaks installed
  • from VRTweaks run SymlinkFix and reboot into Download Mode (“Vol Down”+”Home”+”Power”)
  • in Download Mode flash above extracted kernel with Heimdall
  • reboot
  • extract CF-Root .zip/.tar file until you get to zImage
  • upload CF-Root’s zImage to the phone
  • from CWM app flash CF-Root zImage

Now it’s a good time for another backup – we’re so far ahead – it would be a shame to lose things now.

After all that I had VillainROM running on my phone. VillainROM however turned out to be not as funky as I hoped it will be. Issues I have discovered:

  • for some strange reason it wakes up camera on certain operations (which stock ROM doesn’t do)
  • there is no Russian Keyboard layout
  • browser doesn’t respond to pinch-zoom actions and “hangs” in some odd state

Constant clicking from the camera made me re-think my strategy and I decided to fall back to Stock Firmware (and discover that on-board data files are gone, d’oh! It’s not an issue if I had backed it up onto PC, which I didn’t)

Cognition S2

Now that I’ve gone through all the trouble above flashing new images is a piece of cake. Download ROM and flash it via ClockWorkMod’s Recovery Mode:

  • download ROM
  • copy it over to the phone
  • reboot phone into Recovery Mode (“Vol Up”+”Home”+”Power”)
  • select “install zip from sdcard”
  • locate .zip
  • flash

note that as a result CWM Recovery Mode version has been bumped (to 4.0.0.5) and volume keys are used for moving around and “Power” is used for selection vs 4.0.0.2’s “Home”.

Cognition looked OK, but statusbar icons were constantly changing colors, some of the original Widgets on Samsung that I’ve got used to are gone.

LiteningROM 1.5

For flashing this with Heimdall you need to extract the .RAR, then extract the tar.md5 (it’s just a tar), then use resulting

  • factoryfs.img
  • modem.bin
  • zImage

and plug them into appropriate fields in Heimdall. Looks like supplied kernel brought back Yellow Warning Sign.

After flashing phone did make that clicking sound it made with VillainROM but only on boot so I can possibly live with that.

Meteorological attack. Second front – Workforce

After dealing with Cloud vs Users case it’s time to take a look behind the scenes and uncover what else is impacted by Clouds and what the impacts are. We have already established that for users move to the cloud means parting with their data, it surely means the same thing to business entities, after all they are too users. Let’s get beyond that. If you’ve ever read “No Logo” by Naomi Klein you are familiar with the chapter “No Jobs” that follows immediately after “No Choice”. It looks like the order of chapters is not coincidental. The process of dissociation of corporation with manufacturing process and workforce is like the mushroom cloud – both stunning and horrifying.

It’s not required to read No Logo to understand simple principles at work. Principles and mechanics employed by corporations in their search of “brand identity” and “brand experience”. As usual it comes with some collateral damage:

No Choice

…Dazzled by the array of consumer choices we may at first fail to notice the tremendous consolidation taking place in the boardrooms of the entertainment, media and retail industries. Advertising floods us with the kaleidoscopic soothing images of United Streets of Diversity and Microsoft’s  wide-open “Where do you want to go today?” enticements. But in the pages of the business section the world goes monochromatic and doors slam shut from all sides: every other story – whether the announcement of a new buyout, an untimely bankruptcy, a collossal merger – points directly to a loss of meaningful choices…

So how does it translate to IT? It’s an attempt to wipe out diversity by “streamlining business practises” and “bringing them closer to the base”. Say, institution has been priding itself in it’s uniqueness in catering to a very specific customer base and generating quite a loyal following. Institution that stood out and can’t be matched by others only because… of it’s unique business practises.  However in the boardrooms this must’ve looked annoying or out of place because the decision has been made to “consolidate”, “streamline” and “merge”. In other words – all the products that are used by competition “as-is” and “out-of-the-box” are to be applied to this institution as well essentially wiping it’s uniqueness. The only logical conclusion would be that institution is being moved in the “branding” direction where product is essentially the same and service about the same as the rest but what is sold is “brand” and “experience”, not the product itself (not surprisingly since it’s the same product):

The difference between products and brands is fundamental. A product is something that is made in a factory; a brand is something that is bought by a customer

No Jobs

…corporations should not expend their finite resources on factories that will demand physical upkeep, on machines that will corrode or on employees who will certainly age and die. Instead, they should concentrate those resources in the virtual brick and mortar used to build their brands; that is, on sponsorship, packaging, expansion and advertising…

So after “No choice” invariably comes “No Jobs”. Discussed institution in this case is not exception. There is a clear sense of direction in cleansing institution of any traces of IT department outsourcing most critical applications and systems. Institution doesn’t want to burden itself with infrastructure or workforce, it needs to concentrate on what’s important – building image. Resulting in exploded marketing departments (or just expanded budgets outsourcing that activity someplace else) and reduced funding for manufacturing and R&D.

Above principles no longer apply strictly to corporations and other businesses. Now they are applied to governments and government institutions as well as education.

Some naive people assume that government is there to serve people or that education should be accessible by people and serve people’s needs. Only in case of ongoing “branding” government serves businesses believing in “trickle down effect” that has never been proved to work and education is serving business needs of companies and government. Application of business rules in government and education sectors has devastating effects: hollow and emasculated they can’t serve people anymore and have to abide by business rules serving only what business demands. Which for education means that you can’t produce any more “free thinkers” or offer “non-marketable” programs because there’s no business need for them. So instead of government shaping the economy and busineses we have businesses serving themselves with hollow government watching the carnage from afar unable to do anything.

Current hysteria about financial crisis provides fertile ground for those seeking excuses to enact “touch measures”, “trim the fat” and “streamline operations” at expense of workers, taxpayers and customers. It’s a “disaster capitalism” at work: create or wait for a crisis and then while people are dazed and confused implement everything you’ve dreamed about bypassing all the normal processes excusing yourself by extraordinary situation at hand and repeating “ad nauseum”: “In a critical time like this we have to act fast.”

What is that magic bullet that can kill that undying beast of IT department? It’s all on the frontpages of magazines – “Cloud”. Single word that spells emasculation of IT departments everywhere and narrowing of choices for consumers as well. For a government that seeks to hollow itself out it’s a prime destination.

What was previously known as “outsourcing” and became lame and unpopular over time is now called “cloud computing” and is shoved down everybody’s throat using every possible excuse.

Favourite argument of cloud-defenders is that “computing” is “new electricity” and “cloud provider” is new equivalent of “power company” with companies paying for computational power like they do for electricity. However it’s not enough for company to move infrastructure. Once it has made that step – why not make the second step and go to SAAS instead? Running VM’s on the cloud is not sexy, plus it creates tons of problems with security VPNs and it doesn’t resolve the “problem” of having IT staff. Once everything is hosted by SAAS provider you have no worries. The only insulation required is the contract. Cloud computing started as an idea of running VMs on the remote infrastructure yet still managed by a company staff, but with time term got overloaded with much more meaning making it impossible to differentiate one proposition from another and  creating new common ground for understanding. Now cloud computing equals outsourcing.

“Cloud” is what powers the transition from “unique organization” and “self-sufficient organization” to “No Choice” and “No Jobs”. Incantation that has a viral effect essentially wiping off any living cell in it’s path. “Cloud” takes all that annoyance of managing IT and removes it from institution. What’s interesting – it removes it in “unknown” direction. From that point on Institution is free of workforce and infrastructure, while whoever runs the “Cloud” is bound by limited contractual agreements and operating most likely in un-unionized environment and is free to expand and contract at will having only “temp” staff in it’s employment catering to demands of clients. So for a heavily unionized institution it’s a blessing – you move your IT services outside and your “IT Crowd” is no longer a unionized headache but rather “workforce on-demand”. That workforce doesn’t have to reside in the same country either, opening up brand new frontiers of exploration (or exploitation?).

Here’s the thing – one has to answer simple questions to realize the depth of it. “What drives organization?” – “Maximizing profit”. “What drives employees of organization?” – “Making sure organization profit margins are high, so that their jobs are secure”. Now, in the relationship “Could provider” – “Cloud client” both are driven by above principles. However their goals are orthogonal. So what is the difference between organization’s own employee and “cloud” employees? – The motivation that drives them. In case of “own employee” – his interest is for his organization to prosper or at least not to go under. In case of “cloud employee” at best his interest is to serve his employer which is orthogonal to goals of “cloud client” – he needs to maximize profit of “cloud provider” by minimizing impact “cloud client” has on provider’s resources which in turn means – spending less cycles serving the client. This entire equation is then translated into Contracts, SLA’s and Change Requests which all are then monetized at the expense of the client. Will cloud employee be interested to offer money-saving scheme to client organization, if it doesn’t maximize profit of his employer? Will client’s employee be inclined to do the same?

To be fair – it is reasonable to accept Cloud limitations and impact in organizations that didn’t have IT to begin with and are too small to own their own IT shop. However it is very hard to come up with justification for organization to drop it’s IT department and “move to the cloud” remaining at mercy of provider and contract lawyers. So while it is not in organization’s best interest to part with it’s IT for million reasons, it does make sense for the executives to move in that direction, especially if their IT department is unionized. Move to the cloud removes all the barriers and part of their job that has to do with people. It’s hard to tell person “shut up and do it” but it’s much easier with vendor, especially when vendor is de-personified and is located across the ocean…

Cloud is truly unifying and transcending entity. It’s a Borg. “You will be assimilated. Resistance is futile”.

P.S.

Quotes above come from “No Logo 10th anniversary edition”.

Open sources

I am biased. You should know it by now. You should also know who influences my biases. The sources of my biases start with my parents who taught me to be responsible and independant. My mom’s care for us, her lifestyle rubbed off on me one way or another. My dad’s persistence and dedication to principles and his words: “invest your time and money in people first of all” are the things that guided my decisions. And then I grew up. Didn’t change my attitude but I have sharpened it’s edges with works of Machiavelli, Sun Tsu, Naomi Klein, Neitzche, Richard Stallman, Slavoj Zizek and multitude of other people. Now I know where I stand (well, at least today I do) and what I stand for. I have learned to aim high to shoot farther, lose form to fight chaos and uncertainty, see bigger picture hiding behind small events, understand the philosophy that propels corporations, and to stand alone.