repoman.common.stores.RPM package

This module holds the class and methods to manage an rpm store and it’s sources.

In our case an rpm store is not just a yum repository but a set of them and src files, in the following structure:

repository_dir
├── rpm
│   ├── $dist1  <- this is a yum repository
│   │   ├── repodata
│   │   │   └── ...
│   │   ├── SRPMS
│   │   │   ├── $srcrpm1
│   │   │   ├── $srcrpm2
│   │   │   └── ...
│   │   ├── $arch1
│   │   │   ├── $rpm1
│   │   │   ├── $rpm2
│   │   │   └── ...
│   │   ├── $arch2
│   │   └── ...
│   ├── $dist2  <- another yum reposiory
│   │   └── ...
│   └── ...
└── src
    ├── $project1
    │   │   ├── $source1
    │   │   ├── $source1.sig
    │   │   ├── $source2
    │   │   └── ...
    │   └── ...
    └── ...

exception repoman.common.stores.RPM.CreaterepoError

Bases: exceptions.Exception

exception repoman.common.stores.RPM.CreatereposError

Bases: exceptions.Exception

class repoman.common.stores.RPM.RPMStore(config, repo_path=None)

Bases: repoman.common.stores.ArtifactStore

Represents the repository sctructure, it does not require that the repo has the structure specified in the module doc when loading it, but when adding new rpms or generating the sources it will create the new files in that directory structure.

You can pass rpm properties (like version, distro, arch or major_version) as python’s format variables and they will be expanded at runtime for each rpm using the expansion as store path, for example ‘/myrepo/{major_version}’ will create all the repository structure under that path for each rpm (if you have multiple independent rpms that does not make much senes though, but you get the idea)

Configuration options:

• distro_reg

  Regular expression to extract the distribution from the release string

• extra_symlinks

  Comma separated list of orig:symlink pairs to create links, the paths

• on_wrong_distro

  Action to execute when a package has an incorrect distro (it’s release string does not match the distro_reg regular expression). Possible values are ‘fail’, ‘copy_to_all’ or anything else. The default is ‘fail’, if ‘copy_to_all’ specified it will copy the rpm to all the distros (it needs to have any other distros in the dst repo, or other rpms with a defined distro). If anything else specified, it will warn and skip that rpm.

• path_prefix

  Prefixes of this store inside the globl artifact repository, separated by commas

• rpm_dir

  name of the directory that will contain the rpms (rpm by default), if empty, it will not create a subdirectory for the rpms and will be put on the root of the repo (root/$dist/$arch/*rpm)

• signing_key

  Path to the gpg keey to sign the rpms with, will not sign them if not set

• signing_passphrase

  Passphrase for the above key

• temp_dir

  Temporary dir to store any transient downloads (like rpms from urls). The caller should make sure it exists and clean it up if needed.

• with_sources

  If true, will extract the sources form the scrrpms

• with_srcrpms

  If false, will ignore the srcrpms will be relative to the store root path.

CONFIG_SECTION = 'RPMStore'

DEFAULT_CONFIG = {'distro_reg': '\\.(fc|el)\\d+(?=\\w*)', 'extra_symlinks': '', 'on_wrong_distro': 'fail', 'path_prefix': 'rpm,src', 'rpm_dir': 'rpm', 'signing_key': '', 'signing_passphrase': 'ask', 'temp_dir': 'generate', 'with_sources': 'false', 'with_srcrpms': 'true'}

add_artifact(pkg, **args)

This method adds an artifact to the store

Parameters:artifact – full path or url to the artifact

add_rpm(pkg, onlyifnewer=False, to_copy=True, hidelog=False)

Generic functon to add an rpm package to the repo.

Parameters:

• pkg – path or url to the rpm file to add
• onlyifnewer – If set to True, will only add the package if it’s not there already or the version is newer than the on already there.
• to_copy – If set to True, will add that package to the list of packages to copy into the repo when saving, usually used when adding new packages to the repo.
• hidelog – If set to True will not show the extra information (used when loading a repository to avoid verbose output)

change_path(new_path)

Changes the store path to the given one, copying any artifacts if needed

Parameters:new_path (str) – New path to set Returns:None

create_symlinks()

Creates all the symlinks to the dirs passed on the config

static createrepo(dst_dir)

createrepos()

Generate the yum repositories metadata

delete_old(keep=1, noop=False)

Delete the oldest versions for each package from the repo

Parameters:

• keep – Maximium number of versions to keep of each package
• noop – If set, will only log what will be done, not actually doing anything.

generate_sources(with_patches=False, key=None, passphrase=None)

Generate the sources directory from all the srcrpms

Parameters:

• with_patches – If set, will also extract the .patch files from the srcrpm
• key – If set to the path of a gpg key, will use that key to create the detached signatures of the extracted sources
• passphrase – Passphrase to unlock the key

get_artifacts(regmatch=None, fmatch=None)

Returns the list of artifacts matching the params

Parameters:

• regmatch – Regular expression to filter the rpms path with
• fmatch – Filter function, must return True for packages to be included, or False to be excluded. The package object will be passed as parameter

get_latest(regmatch=None, fmatch=None, num=1)

Return the num latest versions for each rpm in the repo

Parameters:num – number of latest versions to return Return type:repoman.common.artifact.Artifact

get_rpms(regmatch=None, fmatch=None, latest=0)

Get the list of rpms, filtered or not.

Parameters:

• regmatch – Regular expression that will be applied to the path of each package to filter it
• fmatch – Filter function that must return True for a package to be selected, will be passed the RPM object as only parameter
• latest – If set to N>0, it will return only the N latest versions for each package

get_store_path(pkg)

handles_artifact(artifact)

This method must return True if the given artifact (as a path or url) can be handled by the implemented store

Parameters:artifact_str – full path or url to the artifact

is_latest_version(pkg)

Check if the given package is the latest version in the repo :pram pkg: RPM instance of the package to compare

path_prefix

save(**args)

Realizes the changes made to the store, usually writing the artifacts to disk or any other operation required to persist the store state

sign_rpms()

Sign all the unsigned rpms in the repo.

Submodules

repoman.common.stores.RPM.RPM module

This module holds the helper classes to represent a repository, that in our case (oVirt) is a set of repositories, in the form:

Base_dir
├── rpm
│   └── $dist
│       ├── repodata
│       ├── SRPMS
│       └── $arch
└── src
    └── $name
        ├── $name-$version-src.tar.gz
        └── $name-$version-src.tar.gz.sig

This module has the classess that manage a set of rpms, ina hierarchical fashion, in the order:

name 1-* version 1-* inode 1-* rpm-instance

So that translated to classes, with the first being the placeholder for the whole data structure, is:

RPMList 1-* RPMName 1-* RPMVersion 1-* RPMInode 1-* RPM

All except the RPM class are implemented as subclasses of the python dict, so as key-value stores.

For clarification, here’s a dictionary like diagram:

RPMList{
    name1: RPMName{
        version1: RPMVersion{
            inode1: RPMInode[RPM, RPM, ...]
            inode2: RPMInode[...]
        },
        version2: RPMVersion{...}
    },
    name2: RPMName{...}
}

class repoman.common.stores.RPM.RPM.RPM(path, temp_dir='/tmp', distro_reg='\.(fc|el)\d+', to_all_distros=(), verify_ssl=True)

Bases: repoman.common.artifact.Artifact

__str__()

This string uniquely identifies a rpm file, if two rpms have the same string representation, the must point to the same file or a copy of it, if not, you wrongly generated two rpms with the same version/release and different content, or you signed them with different keys

extension

full_name

Unique RPM Name.

This property should uniquely identify a rpm entity, in the sense that if you have two rpms with the same full_name they must package the same content or one of them is wrongly generated (the version was not bumped or something).

generate_path(base_dir='rpm')

Returns the theoretical path that the rpm should be, instead of the current path it is. As explained at the module docs.

If the package has to go to all distros, a placeholder for it will be set in the string

static get_distro(release, distro_reg)

name

sign(key_path, passwd)

Defines how to sign this artifact, by default with detached signature

type

version

class repoman.common.stores.RPM.RPM.RPMList(name_class=<class 'repoman.common.stores.RPM.RPM.RPMName'>)

Bases: repoman.common.artifact.ArtifactList

List of rpms, separated by name

class repoman.common.stores.RPM.RPM.RPMName(name, version_class=<class 'repoman.common.artifact.ArtifactVersion'>)

Bases: repoman.common.artifact.ArtifactName

List of available versions for a package name

add_pkg(pkg, onlyifnewer)

get_latest(num=1)

Returns the list of available inodes for the latest version if any

exception repoman.common.stores.RPM.RPM.WrongDistroException

Bases: exceptions.Exception