Rationale

On the platform, having a customized environment is usually provided by the Kadepoy3 tool. It provides bare-metal provisionning of a limited number of Linux distributions (including Debian, Ubuntu) that can be adapted to the user need.

Some time ago, booting virtual machines in the production environment has been enabled.

  • It’s fast : we just need to get a job from the scheduler and run a kvm process inside. Usually few seconds are required to have a virtual machine ready.
  • It’s flexible : theorically any virtual machine can be used.

In addition to that, a ceph cluster is now available. This can be used to store virtual machine images.

Vagrant-g5k

Vagrant-g5k is a plugin for vagrant to easily deploy virtual machines on Grid’5000. From a high level point of view, each kvm process is run on a physical node after being scheduled by OAR.

The plugin is far from perfect : ideas, contribution are very welcome. Let’s take a look to its main features. An example of Vagrantfile can be found here.

Disk image

The virtual machine uses a base disk image that can be back by the frontend filesystem or by a RBD stored in the Ceph cluster. The plugin exposes different strategies to manage the disk :

  • copy : will copy the disk to a dedicated location. The copy will be created in your home directory on the frontend or in the same pool as the original image. This mimics the behaviour of vagrant which imports the base box in the .vagrant directory of your project.
  • cow : will create a Copy On Write clone of the base image. The original image will be used in read-only mode.
  • snapshot : will create an ephemeral Copy On Write clone of the base image.
  • direct : will use the base image in read-write mode.

Network

In the current implementation, network traffic from the virtual machine is NATed to the outside world. Inter virtual machines SSH communication could be added in a near future using Grid’5000 subnets. Ports can be forwarded from the virtual machine to its host machine. vagrant ssh command tunnels automatically SSH communication from your local machine to the virtual machine.

Job / virtual machine

The plugin is build based on the idea that one virtual machine correspond to one OAR job. vagrant up will submit a job to the platform. When the job is ready it will execute the kvm process. vagrant halt will delete the job. Termination signals is trapped to gracefully shutdown the virtual machine. vagrant destroy will additionally destroy the virtual machine disk image.

Currently a whole node is reserved and the virtual machine CPU/RAM is set to the maximum available. In the future a virtual machine resource demand could be map to a more complex resource selection to let the scheduler assign them.

Wish list

  • Automatic port forwarding to your local machine is appealing. Except for SSH, you still need to create manually a tunnel from your local machine and the physical host (ports are forwarded from the physical host to the virtual machine by kvm)

  • Support for parallel operation. The thread safety of the current implementation is questionnable. Adding this we should be able to start a mass of virtual machines and manage them with Vagrant.

  • Support for CPU/RAM customization.

  • Support for inter virtual machines communication

  • Improve code quality - be more idiomatic.

Thanks

pmorillon for (among other things) maintaining the Ceph cluster, and providing inputs to this project (especially the wrapper script around kvm)