boot RHEL7.2 on ceph RBD

experiment: Environment: Ceph : Monitor + OSD on the same node, SSD:HDD=1:4 Ceph HW configuration : QuantaGrid D51PH-1ULH x 3 , 12 x ST6000NM0034, 3 x INTEL SSDSC2BA20, Dual port intel 82599ES 10-Gigabit NIC client : QuantaPlex T41S-2U, intel 82599ES 10-Gigabit NIC Network : Quanta Fibre switch

boot RHEL7.2 on ceph RBD

R/W speed

Next: boot on RBD through PXE

arguments hiding from ps command

http://netsplit.com/hiding-arguments-from-ps

setuid in c

#include unistd.h
#include sys/types.h
 int main () {
  int real = getuid();
  int euid = geteuid();
  printf("The REAL UID =: %d\n", real);
  printf("The EFFECTIVE UID =: %d\n", euid);
  printf("setuid%d\n",setuid(euid));
  system("cat /etc/shadow");
}
chmod u+s binary

encode binary in script

base64 -d <-- base64 encoded content
EOF

disk fast clone on rhel7

create partition and format them first
mkdir /dev/shm/source /dev/shm/destination
mount /dev/src_rootfs /dev/shm/source
mount /dev/src_boot_part /dev/shm/source/boot
mount /dev/src_rootfs /dev/shm/destination
mkdir /dev/shm/destination/boot
mount /dev/dst_rootfs /dev/shm/destination/
mount /dev/dst_boot_part /dev/shm/destination/boot

cd /dev/shm/source

tar cf - * | ( cd /dev/shm/destination/; tar xfp -)

update fstab and grub.cfg

mount -o bind /proc /dev/shm/destination/proc

mount -o bind /sys /dev/shm/destination/sys

mount -o bind /dev /dev/shm/destination/dev

chroot /dev/shm/destination grub2-install /dev/dst_dsk (Please note, you must specify the whole disk instead of partition)

touch /dev/shm/destination/.autorelabel (run this only when you desire to enable selinux)

ceph-ansible stuck at "collect admin and bootstrap keys"

ceph 1.3.2 + ansible 2.1.1.0
root cause: mon was not properly launched
resolution: simply create a empty file, sysvinit to /var/lib/ceph/mon/{cluster}-{host}/sysvinit and then manually restart ceph mon service

nfs on tmpfs

add option fsid=1 in exports file

rhel7 boot option tips

static ip
format=ip:method:gw:netmask:hostname:ifname:none
ip=IP_ADDR::192.168.0.1:255.255.255.0:hostname:eth0:none

leave ifname to empty if you cannot predict the IFname. add net.ifnames=0 to boot option to disable device name inquiry from BIOS. (biosdevname=0)

inst.stage=nfs:nfsvers=3:nfs:192.168.0.1:/yum/rhel7.2
inst.repo=nfs:nfsvers=3:nfs:192.168.0.1:/yum/rhel7.2
inst.ks=http://192.168.0.1/yum/rhel7.2


pxelinux.0 4.0.4 supports image fetching through http.


inst.stage2=http://10.5.15.10/repo/rhel7.5_ceph3.0 text inst.sshd console=tty0 console=ttyS1,115200n8 vlan=vlan1214:enp61s0f0 ksdevice=vlan1214 ip=10.102.14.53::10.102.14.1:255.255.255.0::vlan1214:none

extract centos7/rhel7 initramfs

# /usr/lib/dracut/skipcpio /boot/initramfs-3.10.0-327.18.2.el7.x86_64.img | gunzip -c | cpio -dcim
# /usr/lib/dracut/skipcpio /boot/initramfs-3.10.0-327.18.2.el7.x86_64.img | zcat | cpio -idmvc
extract initrd.img resided in isolinux folder.
xz -dc initrd.img|cpio -idmv

http://www.constantstream.com/2016/06/16/extract-initramfs-on-rhelcentos-7/

repack:
xz -z -9 -C crc32 -F xz 1
-C for --check
-F for --format

find . |cpio -c -o |xz -z -9 -C crc32 -F xz > /dev/shm/test.img

network interface name tweaking

apply following options for installation process.
net.ifnames=0 biosdevname=0

process bonding to CPU

list process which are run on specific processor.

ps -ax -o pid,cpuid,args,user|awk '$2 == 3{print $0}'

ps -ax -o pid,cpuid,args,user|awk '$3 == "/usr/bin/ceph-osd" {print $0}'

ref: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-cpuset.html
https://codywu2010.wordpress.com/2015/09/27/cpuset-by-example/

ses raw command example

sg_ses -c -p dm /dev/sgxx -d 00,00,00,00,0f,00,00,00,00,00,00,00,00,00,00,00

for sending download microcode mode 0x0f

sg_ses -p 0x2 -c -d 00,00,00,00,00,00,00,00,80,80,40,00,80,80,40,00,80,80,40,20 /dev/bsg/0:0:89:0  -vvv
sg_senddiag -pf -r 2,0,0,c,0,0,0,0,0,0,0,0,80,80,40,0 -vvv /dev/sg0
sg_ses  --index=_23,00 --set=2:1:1=1 /dev/sg0 -vv (in this format, _23 refers to type code 0x17 - Array device slot. for the set option, it refers to byte 2 bit 1 right shift 1-1 bit)


ex:
data was 80 80 42 00
set=2:1:3=1
changed to 80 80 40 80

trick: no need to append page header in raw format. to check header type code. sg_ses -e , to check supported diag command : sg_ses -ee

scsi_wait_scan not found

kernel 3.10.x已移除scsi_wait_scan module但預設的系統卻會包含此module的probe

進而造成kernel panic issue

simply remove scsi_wait_scan module probe from init to get it fixed. 

build iso image

 mkisofs  -P "Kido Yen" -V "Redhat 6.x/7.x boot disk" -o ../file.iso -b isolinux.bin -c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -R -J -v -T ./


with efi support
mkisofs -b isolinux/isolinux.bin -c boot.cat -no-emul-boot  -boot-load-size 4 -boot-info-table -eltorito-alt-boot -e images/efiboot.img -no-emul-boot -R -J 

grub tips

grub install for specific drive without BIOS drive name mapping grub --device-map=/dev/null
grub> device (hd0) /dev/sda

CentOS/RHEL rootfs on tmpfs

1: remember to disable selinux first
2: update mount process in /usr/share/dracut/modules.d/95rootfs-block/mount-root.sh

            mkdir /mytmpfs
            mount -t "$rootfs" -o "$rflags","$rootopts" \
                "${root#block:}" "/mytmpfs" 2>&1 | vinfo
            mount -t tmpfs -o size=100% none "$NEWROOT"
            cp -rfa /mytmpfs/* "$NEWROOT"
            sed -i '/\/boot\ /d' "$NEWROOT/etc/fstab"
            sed -i '/\/boot\t/d' "$NEWROOT/etc/fstab"
            sed -i '/\/ /d' "$NEWROOT/etc/fstab"
            sed -i '/\/\t/d' "$NEWROOT/etc/fstab"
            cat /dev/null > "$NEWROOT/root/.bash_history"
            umount /mytmpfs

3: create new initrd
4: update grub boot menu
5: boot parameter parsing
imgfile=$(getarg imgfile=)
[ -f "$imgfile" ] && tar zxvf $imgfile -C $NEWROOT

http://linux.die.net/man/1/bash
Conditional Expressions boot process stage https://www.kernel.org/pub/linux/utils/boot/dracut/dracut.html#stages