Redesigning the fedimg AMI creation process - Part 1

What is fedimg?

Fedimg is a Python-powered service that uploads the Fedora Cloud images on various Cloud providers. such as Amazon Web Services (AWS) .

How does it currently work?

Fedimg listens to fedmsg for nightly as well as the production compose builds messages. On receiving a new message, the FedimgConsumer triggers the upload process. fedimg uses fedfind to extract the metadata for the compose from where it get’s the URL for the compressed raw image files.

The AWS EC2Service then starts the process of creating the AMI.

  • Booting up the utility instance.
    The EC2Service parses the fedimg configuration file to get the list of configuration regions mapped with the RHEL AMIs, AKIs, architecture etc. EC2Service.upload starts the uploading by booting a utility instance in the first region(determined from the config file). The process continuously keeps trying to ssh into the machine to check if the machine has boot-ed up.

  • Downloading image & taking snapshot
    Once the utility instance is ready to use the compressed RAW image is downloaded in the secondary volume. A snapshot is taken out of the volume and the AMI is created out of it.

  • The AMIs are then copied to the other regions

  • Testing the AMIs Once the AMIs are created, a very basic /bin/true test is performed on a test instance created using one of the created AMIs. Once the tests passes, the AMIs are made public.

At any point doing the execution, if the code throws an exception and even if the AMIs fails the _clean_up method is fired. The method cleans up the utility instance, test instance and the volumes which were created.

The next blog post will discuss the issues related to the current architecture and about the new architecture.

comments powered by Disqus