Deploy Software Factory

This section presents how to properly deploy Software Factory services.

Overview

Follow these steps for a successful deployment:

  • Use a CentOS-7 system or grab the provided disk image with all the components pre-installed.
  • Create as many host instances as needed according to the recommended requirements.
  • Setup Network Access Control and make sure the install-server can ssh into the other instance using the root user ssh key.
  • On the install server, do a minimal configuration.
  • Run sfconfig. It will execute the following steps:
    • Generate an Ansible playbook based on the arch.yaml file,
    • Generate secrets and group_vars based on the sfconfig.yaml file, and
    • Run the playbook to:
      • Install the services if needed.
      • Configure and start the services.
      • Setup the config and zuul-jobs repositories.
      • Validate the deployment with test-infra.

Alternatively, you can also follow one of these automated deployments guide based on a custom disk image.

Requirements

Minimum

Software Factory needs at least one instance, referred to as the install-server.

An all-in-one deployment requires at least a single instance with 4CPU, 8GB of memory and 20GB of disk.

Network Access Control

All external network access goes through the gateway instance and the FQDN needs to resolve to the instance IP:

Port Service
443 the web interface (HTTPS)
29418 gerrit access to submit code review
1883 and 1884 MQTT events (Firehose)
64738 (TCP) and 64738 (UDP) mumble (the audio conference service)

Operators will need SSH access to the install-server to manage the services.

Internal instances need to be accessible to each other for many shared services, so it is recommended to run all the services instances within a single service network. For example, the following services are consumed by:

Service name Consumers
SQL server Most services
Gearman/Zookeeper Zuul/Nodepool
Gearman/Elasticsearch Log-gearman for logstash

Test instances (slaves) need to be isolated from the service network; however the following ports must be open:

Provider type TCP Port access for Zuul
OpenStack 22
OpenContainer 22022 - 65035

Deployment configuration

Deployment and maintenance tasks (such as backup/restore or upgrade) are performed through a node called the install-server which acts as a jump server. Make sure this instance can ssh into the other instances via public key authentication. The steps below need to be performed on the install-server.

If you used a vanilla CentOS-7 system, you have to install the sf-config package on the install-server first:

yum install -y https://softwarefactory-project.io/repos/sf-release-2.7.rpm
yum update -y
yum install -y sf-config

To enable extra services (such as logstash) or to distribute services on multiple instances, you have to edit the arch.yaml file (see the architecture documentation for more details). For example to add a logstash service on a dedicated instance, edit the /etc/software-factory/arch.yaml file like this:

- name: elk
  ip: 192.168.XXX.YYY
  roles:
    - elasticsearch
    - job-logs-gearman-client
    - job-logs-gearman-worker
    - logstash
    - kibana

Note

You can find reference architectures in /usr/share/sf-config/refarch, for example the softwarefactory-project.io.yaml is the architecture we use in our production deployment.

From the install-server, you can also set operator settings, such as external service credentials, in the sfconfig.yaml file (see the configuration documentation for more details). For example, to define your fqdn, the admin password and an OpenStack cloud providers, edit the /etc/software-factory/sfconfig.yaml file like this:

fqdn: example.com
authentication:
  admin_password: super_secret
nodepool3:
  providers:
    - name: default
      auth_url: https://cloud.example.com/v3
      project_name: tenantname
      username: username
      password: secret
      region_name: regionOne
      user_domain_name: Default
      project_domain_name: Default

Finally, to setup and start the services, run:

sfconfig

Access Software Factory

The Dashboard is available at https://FQDN and the admin user can authenticate using “Internal Login”. If you used the default domain sftests.com then the default admin password is userpass.

Congratulations, you successfully deployed Software Factory. You can now head over to the architecture documentation to check what services can be enabled, or read the configuration documentation to check all services settings.

Lastly you can learn more about operations such as maintenance, backup and upgrade in the management documentation.

Otherwise you can find below some guides to help you automate deployment steps so that you can easily reproduce a deployment.

Image based deployment

This documentation describes 3 solutions to install Software Factory using images provided by the project:

OpenStack based deployment

To simplify and speed up the deployment process, a pre-built image should be used. A new diskimage is created for each release.

Prepare the installation image

The Software Factory base image first needs to be created in Glance:

$ curl -O https://softwarefactory-project.io/releases/sf-2.7/sf-2.7.qcow2
$ openstack image create sf-2.7.0 --disk-format qcow2 --container-format bare --file softwarefactory-C7.0-2.7.0.img.qcow2

Deploying with Heat

Heat templates are available to automate the deployment process of different reference architectures.

These templates require the following parameters:

  • image_id: The Software Factory image UUID. This is obtained when uploading the installation image.
  • external_network: The external Neutron network UUID. This is obtained by querying Neutron with openstack network list.
  • domain: The fully qualified domain name (FQDN) of the deployment.
  • key_name: The name of the keypair to provision on the servers. You can import a keypair in Nova with openstack keypair create or list existing keypairs with openstack keypair list.

First, retrieve the template you’re interested in, for example ‘all in one’:

$ curl -O https://softwarefactory-project.io/releases/sf-2.7/sf-2.7-allinone.hot

Then, create the Heat stack:

$ openstack stack create sf_stack --template softwarefactory-C7.0-2.7.0-allinone.hot \
    --parameter key_name=<key-name> \
    --parameter domain=<fqdn> \
    --parameter image_id=<glance image UUID> \
    --parameter external_network=<neutron external network uuid> \
    --parameter flavor=<flavor>

Once the stack is created jump to the section Configuration and reconfiguration.

Deploying with Nova

When Heat is not available, Software Factory can also be deployed manually using the Nova CLI, or using the web UI of your cloud provider. You should first install the software factory image

Once the VM is created jump to the section Configuration and reconfiguration. Don’t forget to manage by yourself the security groups for the SF deployment Network Access.

KVM based deployment

Prerequisites

Ensure the following packages are installed (example for CentOS7 system)

$ sudo yum install -y libvirt virt-install genisoimage qemu-img
$ sudo systemctl start libvirtd && sudo systemctl enable libvirtd

Note

when you start libvirtd, a bridge named virbr0 is created. (using 192.168.122.0/24 or 192.168.124.0/24 networks).

Prepare the installation image

The Software Factory image needs to be downloaded on your kvm host

$ curl -O https://softwarefactory-project.io/releases/sf-2.7/sf-2.7.qcow2
$ sudo mv sf-2.7.qcow2 /var/lib/libvirt/images
$ sudo qemu-img resize /var/lib/libvirt/images/sf-2.7.qcow2 +20G

Prepare the cloud-init configuration files

It’s possible to use cloud-init without running a network service by providing the meta-data and user-data files to the local vm on a iso9660 filesystem.

First, you have to adapt the following values:

$ my_hostname=managesf
$ my_domain=sftests.com
$ my_ssh_pubkey=$(cat ~/.ssh/id_rsa.pub)
  • create the user-data file
$ cat << EOF >> user-data
#cloud-config
hostname: $my_hostname
fqdn: $my_hostname.$my_domain

groups:
  - centos

users:
  - default
  - name: root
    ssh-authorized-keys:
      - $my_ssh_pubkey
  - name: centos
    gecos: RedHat Openstack User
    shell: /bin/bash
    primary-group: centos
    ssh-authorized-keys:
      - $my_ssh_pubkey
    sudo:
      - ALL=(ALL) NOPASSWD:ALL

write_files:
  - path: /etc/sysconfig/network-scripts/ifcfg-eth0
    content: |
      DEVICE="eth0"
      ONBOOT="yes"
      TYPE="Ethernet"
      BOOTPROTO="none"
      IPADDR=192.168.124.10
      PREFIX=24
      GATEWAY=192.168.124.1
      DNS1=192.168.124.1
  - path: /etc/sysconfig/network
    content: |
      NETWORKING=yes
      NOZEROCONF=no
      HOSTNAME=$my_hostname
  - path: /etc/sysctl.conf
    content: |
      net.ipv4.ip_forward = 1

runcmd:
  - /usr/sbin/sysctl -p
  - /usr/bin/sed  -i "s/\(127.0.0.1\)[[:space:]]*\(localhost.*\)/\1 $my_hostname.$my_domain $my_hostname \2/" /etc/hosts
  - /usr/bin/systemctl restart network
  - /usr/bin/sed  -i "s/requiretty/\!requiretty/" /etc/sudoers
EOF
  • create the meta-data file
$ cat << EOF >> meta-data
instance-id: $my_hostname-01
local-hostname: $my_hostname.$my_domain
EOF
  • generate an iso image with user-data and meta-data files
$ sudo genisoimage -output /var/lib/libvirt/images/$my_hostname.iso -volid cidata -joliet -rock user-data meta-data
  • create a storage disk for the instance
$ sudo qemu-img create -f qcow2 -b /var/lib/libvirt/images/sf-2.7.qcow2 /var/lib/libvirt/images/$my_hostname.qcow2
  • boot the instance
$ sudo virt-install --connect=qemu:///system --accelerate --boot hd --noautoconsole --graphics vnc --disk /var/lib/libvirt/images/$my_hostname.qcow2 --disk path=/var/lib/libvirt/images/$my_hostname.iso,device=cdrom --network bridge=virbr0,model=virtio --os-variant rhel7 --vcpus=4 --cpu host --ram 4096 --name $my_hostname
  • You can connect to your instance using ssh, it’s possible to use “virsh console $my_hostname” during the boot process to following the boot sequence.
$ ssh 192.168.124.10 -l centos

Once the virtual machine is available, jump to the section Configuration and reconfiguration.