概述¶
本操作指南仍在进行中，基于 https://pagure.io/fm-orchestrator
设置如下： * 目前我们正在测试 mbs 2.32 * 已安装 CentOS-8 * 防火墙允许主机间通信 * 用于 Koji、psql 和 Kerberos 通信的公网访问和私网通信 * mbs.gnulab.org/10.10.10.4 主机提供 mbs 和 fedmsg * Web 面向服务使用 Let's Encrypt 设置了 SSL，并且有目录服务设置，您需要解决这个问题
安装 mbs 和 fedmsg¶
!!! 本指南仅用于快速测试！！！ * 目前我们通过 pip 从源码安装，但最新版本可在以下地址获取： https://copr.fedorainfracloud.org/coprs/nalika/rockylinux-tools/packages/ * 通过运行 dnf copr enable nalika/rockylinux-tools 来启用当前的临时 rockylinux-tools 仓库 * 安装所需的软件包： dnf install epel-release -y && dnf install fedmsg python3-gssapi git httpd mod_ssl python3-mod_wsgi python3-solv python3-pungi python3-psycopg2 mod_auth_gssapi -y
git clone https://pagure.io/fm-orchestrator.git
cd fm-orchestrator
git checkout v2.32.0
pip3 install .
启动 fedmsg¶
据我所知，我们需要 fedmsg-hub 和 fedmsg-relay 来启动和运行，但我禁用 fedora 的入站消息和消息签名 * 编辑 /etc/fedmsg.d/endpoints.py 并注释掉 "tcp://hub.fedoraproject.org:9940" * 如下编辑 /etc/fedmsg.d/module_build_service.py "保持其他所有内容不变"
# "tcp://stg.fedoraproject.org:9940"
]
},
# Start of code signing configuration
'sign_messages': False,
'validate_signatures': False,
config = dict(
sign_messages=False,
validate_signatures=False,
topic_prefix="org.gnulab" 和 environment="prod" * 启动 fedmsg-hub 和 fedmsg-relay 服务systemctl enable fedmsg-hub --now
systemctl enable fedmsg-relay --now
mbs-frontend 的 Apache 配置¶
- 创建新文件 /etc/httpd/conf.d/mbs.conf，内容如下
<IfModule mod_ssl.c> <VirtualHost *:443> ServerName mbs.gnulab.org WSGIDaemonProcess mbs user=mbs group=mbs threads=5 WSGIScriptAlias / /etc/module-build-service/mbs.wsgi WSGIPassAuthorization on <Directory /etc/module-build-service> WSGIProcessGroup mbs WSGIApplicationGroup %{GLOBAL} Require all granted </Directory> <Location /> AuthType GSSAPI AuthName "GSSAPI Single Sign On Login" GssapiCredStore keytab:/etc/koji.keytab Require valid-user </Location> SSLCertificateFile /etc/letsencrypt/live/mbs.gnulab.org/fullchain.pem SSLCertificateKeyFile /etc/letsencrypt/live/mbs.gnulab.org/privkey.pem Include /etc/letsencrypt/options-ssl-apache.conf </VirtualHost> </IfModule> - 创建 wsgi 文件 /etc/module-build-service/mbs.wsgi，内容如下
import logging logging.basicConfig(level=logging.DEBUG) from module_build_service import app as application
Kerberos 设置¶
- 确保 /etc/krb5.conf 具有正确的领域和设置，如下所示
# To opt out of the system crypto-policies configuration of krb5, remove the # symlink at /etc/krb5.conf.d/crypto-policies which will not be recreated. includedir /etc/krb5.conf.d/ [logging] default = FILE:/var/log/krb5libs.log kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log [libdefaults] dns_lookup_realm = false ticket_lifetime = 24h renew_lifetime = 7d forwardable = true rdns = false pkinit_anchors = FILE:/etc/pki/tls/certs/ca-bundle.crt spake_preauth_groups = edwards25519 default_realm = GNULAB.ORG default_ccache_name = KEYRING:persistent:%{uid} [realms] GNULAB.ORG = { kdc = koji.gnulab.org admin_server = koji.gnulab.org } [domain_realm] .gnulab.org = GNULAB.ORG gnulab.org = GNULAB.ORG
mbs 的通用设置¶
- 创建 mbs 用户并设置密码，如下所示
useradd mbs passwd mbs - 修复 /etc/module-build-service/ 的权限给 mbs：
chown -R mbs:mbs /etc/module-build-service/
Postgresql 配置¶
我使用与 Koji 相同的数据库服务器，该服务器托管在 koji.gnulab.org * 通过编辑 /var/lib/pgsql/data/pg_hba.conf 创建所需的访问权限，如下所示
# IPv4 local connections:
host mbs mbs 10.10.10.4/32 md5
host all all 127.0.0.1/32 ident
listen_addresses = 'localhost,10.10.10.2' * 创建 mbs pgsql 用户和数据库createuser --no-superuser --no-createrole --no-createdb mbs
createdb -O mbs mbs
psql -c "alter user mbs with encrypted password 'mysupersecretepasswordmbs';"
systemctl restart postgresql
mbs 配置¶
- 如下编辑 /etc/module-build-service/koji.conf
[koji] ;configuration for koji cli tool ;url of XMLRPC server server = https://koji.gnulab.org/kojihub ;url of web interface weburl = https://koji.gnulab.org/koji ;url of package download site topurl = https://koji.gnulab.org/ authtype = kerberos krb_rdns = false use_fast_upload = true [staging] server = https://koji.gnulab.org/kojihub weburl = https://koji.gnulab.org/koji topurl = https://koji.gnulab.org/ authtype = kerberos krb_rdns = false use_fast_upload = true - 如下编辑 /etc/module-build-service/config.py
# -*- coding: utf-8 -*- # SPDX-License-Identifier: MIT from os import environ, path # FIXME: workaround for this moment till confdir, dbdir (installdir etc.) are # declared properly somewhere/somehow confdir = path.abspath(path.dirname(__file__)) # use parent dir as dbdir else fallback to current dir dbdir = path.abspath(path.join(confdir, "..")) if confdir.endswith("conf") else confdir class ProdConfiguration(object): DEBUG = True # Make this random (used to generate session keys) SECRET_KEY = "74d9e9f9cd40e66fc6c4c2e9987dce48df3ce98542529126" #SQLALCHEMY_DATABASE_URI = "sqlite:///{0}".format(path.join(dbdir, "module_build_service.db")) SQLALCHEMY_DATABASE_URI = 'postgresql://mbs:mysupersecretepasswordmbs@koji.gnulab.org/mbs' SQLALCHEMY_TRACK_MODIFICATIONS = True # Where we should run when running "manage.py run" directly. HOST = "0.0.0.0" PORT = 5000 # Global network-related values, in seconds NET_TIMEOUT = 120 NET_RETRY_INTERVAL = 30 #DISTGITS = {"git+https://git.centos.org": ("git clone {repo_path}", "get_sources.sh")} SYSTEM = "koji" MESSAGING = "fedmsg" # or amq MESSAGING_TOPIC_PREFIX = ["org.gnulab.prod"] KOJI_CONFIG = "/etc/module-build-service/koji.conf" KOJI_PROFILE = "koji" ARCHES = ["x86_64"] ALLOW_ARCH_OVERRIDE = False KOJI_REPOSITORY_URL = "https://koji.gnulab.org/kojifiles/repos" KOJI_TAG_PREFIXES = ["module", "scrmod"] KOJI_ENABLE_CONTENT_GENERATOR = True CHECK_FOR_EOL = False PDC_URL = "https://pdc.fedoraproject.org/rest_api/v1" PDC_INSECURE = False PDC_DEVELOP = True SCMURLS = ["git+https://git.centos.org/", "https://git.centos.org/"] YAML_SUBMIT_ALLOWED = False # How often should we resort to polling, in seconds # Set to zero to disable polling POLLING_INTERVAL = 600 # Determines how many builds that can be submitted to the builder # and be in the build state at a time. Set this to 0 for no restrictions NUM_CONCURRENT_BUILDS = 5 ALLOW_CUSTOM_SCMURLS = False RPMS_DEFAULT_REPOSITORY = "git+https://git.centos.org/rpms/" RPMS_ALLOW_REPOSITORY = False #RPMS_DEFAULT_CACHE = "http://pkgs.fedoraproject.org/repo/pkgs/" RPMS_ALLOW_CACHE = False MODULES_DEFAULT_REPOSITORY = "git+https://git.centos.org/modules/" MODULES_ALLOW_REPOSITORY = False MODULES_ALLOW_SCRATCH = True ALLOW_ONLY_COMPATIBLE_BASE_MODULES = True ALLOWED_GROUPS_TO_IMPORT_MODULE = set() # Available backends are: console and file LOG_BACKEND = "file" # Path to log file when LOG_BACKEND is set to "file". LOG_FILE = "/tmp/module_build_service.log" # Available log levels are: debug, info, warn, error. LOG_LEVEL = "debug" # Allow stream override ALLOW_STREAM_OVERRIDE_FROM_SCM = True # Settings for Kerberos KRB_KEYTAB = "/etc/mbs.keytab" KRB_PRINCIPAL = "mbs@GNULAB.ORG" # AMQ prefixed variables are required only while using 'amq' as messaging backend # Addresses to listen to AMQ_RECV_ADDRESSES = [ "amqps://messaging.mydomain.com/Consumer.m8y.VirtualTopic.eng.koji", "amqps://messaging.mydomain.com/Consumer.m8y.VirtualTopic.eng.module_build_service", ] # Address for sending messages AMQ_DEST_ADDRESS = \ "amqps://messaging.mydomain.com/Consumer.m8y.VirtualTopic.eng.module_build_service" AMQ_CERT_FILE = "/etc/module_build_service/msg-m8y-client.crt" AMQ_PRIVATE_KEY_FILE = "/etc/module_build_service/msg-m8y-client.key" AMQ_TRUSTED_CERT_FILE = "/etc/module_build_service/Root-CA.crt" # Disable Client Authorization NO_AUTH = False AUTH_METHOD = "kerberos" LDAP_URI = "ldap://koji.gnulab.org" LDAP_GROUPS_DN = "ou=group,dc=gnulab,dc=org" ADMIN_GROUPS = {"packageradmin"} ALLOWED_GROUPS = {"packager"} KOJI_CG_DEVEL_MODULE = True KOJI_PROXYUSER = True REBUILD_STRATEGY = 'only-changed' REBUILD_STRATEGY_ALLOW_OVERRIDE = True KOJI_CG_BUILD_TAG_TEMPLATE = "{}-modular-updates-candidate" KOJI_CG_DEFAULT_BUILD_TAG = "modular-updates-candidate" # Extra options set for newly created Koji tags KOJI_TAG_EXTRA_OPTS = { "mock.package_manager": "dnf", # This is needed to include all the Koji builds (and therefore # all the packages) from all inherited tags into this tag. # See https://pagure.io/koji/issue/588 and # https://pagure.io/fm-orchestrator/issue/660 for background. "repo_include_all": True, # Has been requested by Fedora infra in # https://pagure.io/fedora-infrastructure/issue/7620. # Disables systemd-nspawn for chroot. "mock.new_chroot": 0, # Works around fail-safe mechanism added in DNF 4.2.7 # https://pagure.io/fedora-infrastructure/issue/8410 "mock.yum.module_hotfixes": 1, } # DEFAULT_DIST_TAG_PREFIX = 'module_' - 创建日志文件 /tmp/module_build_service.log 并设置正确的权限
touch /tmp/module_build_service.log chown mbs:fedmsg /tmp/module_build_service.log chmod 664 /tmp/module_build_service.log - 重启服务
systemctl restart fedmsg-hub systemctl restart fedmsg-relay systemctl restart httpd
测试模块构建服务¶
我们需要创建几个 tag 才能使其正常工作，所以在 koji 管理机上运行以下命令： * 创建 tag
koji add-tag module-centos-8.2.0-build
koji add-tag module-centos-8.3.0-build
koji add-external-repo -m bare -t module-centos-8.2.0-build module-cent-8.2-baseos-external http://mirror.centos.org/centos-8/8.3.2011/BaseOS/\$arch/os/
koji add-external-repo -m bare -t module-centos-8.2.0-build module-cent-8.2-appstream-external http://mirror.centos.org/centos-8/8.3.2011/AppStream/\$arch/os/
koji add-external-repo -m bare -t module-centos-8.2.0-build module-cent-8.2-devel-external http://mirror.centos.org/centos-8/8.3.2011/Devel/\$arch/os/
koji add-external-repo -m bare -t module-centos-8.2.0-build module-cent-8.2-ha-external http://mirror.centos.org/centos-8/8.3.2011/HighAvailability/\$arch/os/
koji add-external-repo -m bare -t module-centos-8.2.0-build module-cent-8.2-pt-external http://mirror.centos.org/centos-8/8.3.2011/PowerTools/\$arch/os/
koji add-external-repo -m bare -t module-centos-8.2.0-build module-cent-8.2-cp-external http://mirror.centos.org/centos-8/8.3.2011/centosplus/\$arch/os/
koji add-external-repo -m bare -t module-centos-8.2.0-build module-cent-8.2-cr-external http://mirror.centos.org/centos-8/8.3.2011/cr/\$arch/os/
koji add-external-repo -m bare -t module-centos-8.2.0-build module-cent-8.2-extras-external http://mirror.centos.org/centos-8/8.3.2011/extras/\$arch/os/
koji add-external-repo -m bare -t module-centos-8.2.0-build module-cent-8.2-fasttrack-external http://mirror.centos.org/centos-8/8.3.2011/fasttrack/\$arch/os
koji add-external-repo -m bare -t module-centos-8.2.0-build module-cent-8.2-debuginfo-external http://debuginfo.centos.org/8/\$arch/
koji add-external-repo -m bare -t module-centos-8.2.0-build module-cent-8.2-sheriflocalrepo-external https://koji.gnulab.org/localrepo/\$arch/
koji add-external-repo -m bare -t module-centos-8.3.0-build module-cent-8.3-baseos-external http://mirror.centos.org/centos-8/8.3.2011/BaseOS/\$arch/os/
koji add-external-repo -m bare -t module-centos-8.3.0-build module-cent-8.3-appstream-external http://mirror.centos.org/centos-8/8.3.2011/AppStream/\$arch/os/
koji add-external-repo -m bare -t module-centos-8.3.0-build module-cent-8.3-devel-external http://mirror.centos.org/centos-8/8.3.2011/Devel/\$arch/os/
koji add-external-repo -m bare -t module-centos-8.3.0-build module-cent-8.3-ha-external http://mirror.centos.org/centos-8/8.3.2011/HighAvailability/\$arch/os/
koji add-external-repo -m bare -t module-centos-8.3.0-build module-cent-8.3-pt-external http://mirror.centos.org/centos-8/8.3.2011/PowerTools/\$arch/os/
koji add-external-repo -m bare -t module-centos-8.3.0-build module-cent-8.3-cp-external http://mirror.centos.org/centos-8/8.3.2011/centosplus/\$arch/os/
koji add-external-repo -m bare -t module-centos-8.3.0-build module-cent-8.3-cr-external http://mirror.centos.org/centos-8/8.3.2011/cr/\$arch/os/
koji add-external-repo -m bare -t module-centos-8.3.0-build module-cent-8.3-extras-external http://mirror.centos.org/centos-8/8.3.2011/extras/\$arch/os/
koji add-external-repo -m bare -t module-centos-8.3.0-build module-cent-8.3-fasttrack-external http://mirror.centos.org/centos-8/8.3.2011/fasttrack/\$arch/os
koji add-external-repo -m bare -t module-centos-8.3.0-build module-cent-8.3-debuginfo-external http://debuginfo.centos.org/8/\$arch/
koji add-external-repo -m bare -t module-centos-8.3.0-build module-cent-8.3-sheriflocalrepo-external https://koji.gnulab.org/localrepo/\$arch/
koji add-user mbs
koji call addBType module
koji grant-cg-access mbs module-build-service --new
现在，在 mbs 服务器上，我们需要执行以下操作： * 升级 mbs 数据库 mbs-manager db upgrade`` * 下载平台模块 `git clone https://git.centos.org/modules/platform` * 切换到所需的分支并加载它 `
git branch -a
git checkout c8-stream-el8.2.0
mbs-manager import_module platform.yaml
git checkout c8-stream-el8.3.0
mbs-manager import_module platform.yaml
{ "scmurl": "https://git.centos.org/modules/redis.git?#f042b198bcbc0fe410be219f27cd041deea7bcc3", "branch": "c8-stream-5" }