%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
Â
LXC on Ganeti
1. LXC on Ganeti
Yuto Kawamura(kawamuray)
Email: kawamuray.dadada@gmail.com
2. Google summer of Code
â Global program that offers students stipends to write code for open source
projects
â 4420 students
â 6313 proposals
â (donât know how much of them were accepted)
3. Improve LXC support
â Project accepted by Ganeti on Google Summer of Code 2014
â Objective
â Improve LXC support on Ganeti
â Fix existing broken implementation
â Add unit tests
â Setup QA
â Add additional features
â Mentored by Hrvoje Ribicic
4. LXC
â An userspace interface for the Linux kernel containment features
â Operating System level virtualization
â Composed by two Linux kernel features
â Namespace - Isolation for pid, ipc, mount, net, user between
containers and the host
â Cgroup - Hardware resource limitation
6. â cpu_mask: Cpu mask. Same as other hypervisors but no support for vcpus
pinning
â lxc_cgroup_use: Cgroup subsystems list. Correspond to lxc.cgroup.use
configuration parameter
â lxc_devices: The list of devices ACL.
â lxc_drop_capabilities: List of capabilities which should be dropped for an
instance.
â lxc_tty: Number of ttys(ptys) created for an instance.
â lxc_startup_wait: Time to wait until a daemonized instance state changed
to RUNNING.
Parameters
7. Parameters
â lxc_extra_config: For extra configuration parameters
â Separating by comma is problem
â Planning to introduce a functionality to read the value from a file
8. How to try?
1. Enable cgroup subsystems(if needed)
a. cpuset, memory, devices, cpuacct
b. Debian: Need âcgroup_enable=memoryâ in kernel argument
2. Install LXC >= 1.0.0
3. Check your kernel configuration
$ sudo lxc-checkconfig
--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled
Multiple /dev/pts instances:
enabled ...
9. 4.
5.
6.
How to try?
gnt-cluster modify
--enabled-hypervisors=kvm,lxc # enable lxc hv
--nic-parameters mode=bridge,link=br0 # your brdige interface
gnt-cluster verify # Checks your environment by LXC hypervisor
gnt-instance add
-H lxc # use LXC hypervisor!
-o debootstrap+default
-t drbd -s 8G lxc-instance1.ganeti
â will automatically modify the fstab of the instance to make it proper for the
LXC container(riba is working for it)
Itâs all! Easy right?
10. â Log file
Tips - How to debug?
â $PREFIX/var/log/ganeti/lxc/<instance name>.<instance uuid>.log
â Logs from lxc-start command
â Mainly used when your instance didnât started
â $PREFIX/var/run/ganeti/lxc/<instance name>.console
â Logs from processes inside the container(init, rc, etc.)
â See when you cannot use `gnt-instance console âŠ`
12. Basic policy
â LXC containers are managed by command line tools
â lxc-start, lxc-stop, lxc-ls, lxc-info âŠ
â Not using lxc python library(written in python3)
13. Instance Start
1. lxc-start -n <instance name> âŠ
-f $PREFIX/var/run/ganeti/lxc/<instance name>.conf
2. lxc-wait -n <instance name> -s RUNNING
a. with timeout specified by the lxc_startup_wait hvparam
14. Instance Stop
â Soft shutdown
â lxc-stop --nokill -n <instance name>
â Send SIGPWR to the init
â Overwritable with lxc.haltsignal
â Wait until init exits
â Hard(forced) shutdown
â lxc-stop --kill -n <instance name>
â Send SIGKILL to the init
â Container killed without normal shutting down process
15. Instance Reboot
â Soft reboot
â lxc-stop --reboot -n <instance name>
â Send SIGINT to the init
â Requires CAP_SYS_BOOT
â Hard rebot
â Same as other hypervisors
â soft shutdown && start
16. Memory ballooning
â Use Cgroup memory subsystem
â memory.limit_in_bytes
â memory.memsw.limit_in_bytes
â Shrinking memory will fail if the kernel canât reclaim memory from
processes inside the container
17. Cgroup filesystem
â Differences between distributions/rc system
1. Search /proc/mounts for line having type == cgroup && <subsystem> in
options
2. If not found, do: mount -t cgroup -o <subsystem> <subsystem>
$PREFIX/var/run/lxc/cgroup/<subsystem>
18. Storage mount
â LXC canât handle raw storage device
â Need to prepare per-pertition(mountable) device file for a LXC rootfs
â Use kpartx(1)
â Create device maps for over partitions segments detected
â Suppose every environment have it(because OS scripts uses it)
kpartx -a
/dev/drbdX /dev/loopY
/dev/mapper/loopYp1
/dev/mapper/loopYp1
Per-partition
device files
19. Storage mount
â Use first partition as a rootfs
â lxc.rootfs = /dev/mapper/loopNp1
â You may have to increase the number of max_loop
20. Quality
â Tested on Debian Linux/Gentoo Linux with LXC 1.0.0 and 1.0.5
â Passed basic QA
but âŠ
â Need more example of use
â Possibly contains platform(distribution) related problems
â especially around the cgroup
22. Future candidates
â Two level isolation(IOW, containers inside a Virtual Machine)
â LXC instance migration
23. Two level isolation
KVM1 for userA
Container 1 Container 2
Container 3
KVM2 for userB
Container 1 Container 2
Hardware/OS Container 3
level Isolation
noded
with attribute?
noded
with attribute?
24. Instance Migration
â LXC has no migration functionality
â Have to implement it on our own
25. Checkpoint/Restore
â Implemented by project CRIU (http://criu.org/)
â Able to dump a running processes into serialized images
â Serialized in protocol buffers format
/path/to/checkpoint/dir
----- Memory image
----- Namespaces info
----- File descriptors
----- Cgroup info
----- TCP sockets
----- Network informations
28. Instance migration
â Already have working implementation
â https://github.com/kawamuray/ganeti/commits/topic-lxc-migration
â but âŠ
â its temporaily implementation
â still have lot of restrictions
â requires patched criu
â https://github.com/kawamuray/criu
â requires patched lxc
â https://github.com/kawamuray/lxc
â will be solved soon(by sending patches to originals and by fixing
implementation)
30. We really want
â Example of use
â on many platforms(Debian, RHEL6/7, âŠ)
â Bug report
â Realistic use case
â Desired functionality
â possibly I can handle it from both Ganeti and LXC aspect