diff options
Diffstat (limited to 'posts/2022')
| -rw-r--r-- | posts/2022/managing-binary-package-repositories.rst | 193 | ||||
| -rw-r--r-- | posts/2022/new-pgp-key-id-1793dad5d803a8ffd7451697bb992f9864fad168.rst | 182 |
2 files changed, 375 insertions, 0 deletions
diff --git a/posts/2022/managing-binary-package-repositories.rst b/posts/2022/managing-binary-package-repositories.rst new file mode 100644 index 0000000..89f203f --- /dev/null +++ b/posts/2022/managing-binary-package-repositories.rst @@ -0,0 +1,193 @@ +.. title: Managing binary package repositories +.. slug: managing-binary-package-repositories +.. date: 2022-07-02 13:19:59 UTC+02:00 +.. tags: arch linux, packaging, repod, dbscripts +.. category: archlinux +.. link: +.. description: +.. type: text + +In `Packaging for Arch Linux +<https://sleepmap.de/2022/packaging-for-arch-linux>`_ I described the ins and +outs of binary repository management and some of the issues that come with the +tooling currently used by |arch linux|. + +In this article I will highlight the work on new tooling and its features. + +Since my last write-up on this topic, the project formerly known as +``arch-repo-management`` has been renamed to ``repod`` (as in *repo-d*) and has +just seen its first minor release. 🎉 + +You can find its documentation at https://repod.archlinux.page. + +.. TEASER_END + +.. note:: + + The https://archlinux.page domain has recently been acquired by Arch Linux to + serve as the common platform for documentation of projects driven by the + distribution. + +Please note, that |repod| 0.1.0 is still alpha grade software and **should +not** be used to actually manage binary package repositories at this point in +time! + +However, it is already possible to do a few things with the software and if you +are able to test it or are interested in helping develop it, that is very much +welcomed! + +On Arch Linux you can install it using |pacman|: + +.. code:: sh + + pacman -S repod + +On other distributions (even on macOS!) you may install it using |pip| (|repod| +is |available on pypi|) until your respective package management system makes +the software available to you on a system level: + +.. code:: sh + + pip install --user repod + +Working on repod +---------------- + +Since 2021 I have been on and off working on what is now |repod|. The project +is written in typed |python| and is extensively tested using |pytest|. + +Work first began after |arch conf| 2019, at which a working group had looked +into improving the workflows currently employed by the distribution and pushing +for tooling that would allow moving away from an |svn| monorepo based approach +to a deconstructed |git| setup. + +A proof of concept (PoC) to mimic the behavior of |dbscripts| had been created, +but after 2019 this work laid dormant. + +Over 2021 I have spent time to transform parts of the PoC into a Python project +following best practices for development (e.g. type hints following |pep0484|, +100% test coverage, data validation using |pydantic| models), exposing first +features in scripts. + +In 2022 more work has been done to extend validation and transform the project +into a package based setup for easier handling and extension in the future. + +Concepts of repod +----------------- + +Contrary to |dbscripts|, |repod| follows a paradigm in which it is largely +decoupled from the |source repository| of the binary packages it maintains and +aims at becoming a self-contained service. + +Package files and their signatures are consumed, relevant metadata is extracted +and transferred to a |management repository|, which is where the state of +each |binary repository| is kept. + +Available packages in a given |binary repository| are exposed to |pacman| via +|sync database| files. The |management repository| contents can be +(transparently and reproducibly) transformed into |sync database| files and +vice versa. + +The above functionality is exposed on the command line via |repod-file|. + +Upcoming work +------------- + +As mentioned above, |repod| is not yet stable and still misses quite a few +features. +The following topics (and more) will be worked on in the next milestones (not +necessarily in this order): + +* file handling (moving package files from staging areas to actual repositories) +* signature validation (PGP signature validation of package files) +* handling of debug packages +* consolidate schema of |management repository| +* improve logging throughout the project +* git backend to transparently expose changes to the |management repository| +* caching for |management repository| state (e.g. to allow fast searches) +* API to interact with a |repod| instance over the wire in an authenticated fashion +* client-side tooling to interact with |repod|'s API + +There is still a lot of work to be done, so if you have a background in +|Python| development and are interested in working on a project that is very +close to the distribution and will likely improve the workflow of many people +while making binary repository management more transparent to the user, have a +look at the project's |contribution guidelines| and do not hesitate to reach +out! + +.. |arch linux| raw:: html + + <a target="blank" href="https://archlinux.org">Arch Linux</a> + +.. |repod| raw:: html + + <a target="blank" href="https://gitlab.archlinux.org/archlinux/repod">repod</a> + +.. |pacman| raw:: html + + <a target="blank" href="https://man.archlinux.org/man/pacman.8">pacman</a> + +.. |pip| raw:: html + + <a target="blank" href="https://man.archlinux.org/man/pip.1">pip</a> + +.. |available on pypi| raw:: html + + <a target="blank" href="https://pypi.org/project/repod/">available on pypi</a> + +.. |python| raw:: html + + <a target="blank" href="https://www.python.org/">Python</a> + +.. |pytest| raw:: html + + <a target="blank" href="https://docs.pytest.org/en/latest/">pytest</a> + +.. |arch conf| raw:: html + + <a target="blank" href="https://conf.archlinux.org/">Arch Conf</a> + +.. |svn| raw:: html + + <a target="blank" href="https://man.archlinux.org/man/svn.1">svn</a> + +.. |git| raw:: html + + <a target="blank" href="https://man.archlinux.org/man/git.1">git</a> + +.. |dbscripts| raw:: html + + <a target="blank" href="https://gitlab.archlinux.org/archlinux/dbscripts">dbscripts</a> + +.. |pep0484| raw:: html + + <a target="blank" href="https://peps.python.org/pep-0484/">PEP 0484</a> + +.. |pydantic| raw:: html + + <a target="blank" href="https://pydantic-docs.helpmanual.io/">pydantic</a> + +.. |source repository| raw:: html + + <a target="blank" href="https://repod.archlinux.page/repositories/source_repository.html">source repository</a> + +.. |management repository| raw:: html + + <a target="blank" href="https://repod.archlinux.page/repositories/management_repository.html">management repository</a> + +.. |binary repository| raw:: html + + <a target="blank" href="https://repod.archlinux.page/repositories/binary_repository.html">binary repository</a> + +.. |sync database| raw:: html + + <a target="blank" href="https://repod.archlinux.page/repositories/sync_database.html">sync database</a> + +.. |repod-file| raw:: html + + <a target="blank" href="https://man.archlinux.org/man/repod-file.1">repod-file</a> + +.. |contribution guidelines| raw:: html + + <a target="blank" href="https://gitlab.archlinux.org/archlinux/repod/-/blob/79a0fa9aa4a4d9def6dad24805f9bbff0388f734/CONTRIBUTING.md">contribution guidelines</a> + diff --git a/posts/2022/new-pgp-key-id-1793dad5d803a8ffd7451697bb992f9864fad168.rst b/posts/2022/new-pgp-key-id-1793dad5d803a8ffd7451697bb992f9864fad168.rst new file mode 100644 index 0000000..15ce8fb --- /dev/null +++ b/posts/2022/new-pgp-key-id-1793dad5d803a8ffd7451697bb992f9864fad168.rst @@ -0,0 +1,182 @@ +.. title: New PGP key ID 1793DAD5D803A8FFD7451697BB992F9864FAD168 +.. slug: new-pgp-key-id-1793dad5d803a8ffd7451697bb992f9864fad168 +.. date: 2022-04-30 10:35:57 UTC+02:00 +.. tags: chain of trust, gnupg, gpg, infrastructure, openpgp, sequoia, sq, web key directory, web of trust +.. category: admin +.. link: +.. description: +.. type: text + +As my current |PGP| key ``91BD8815FE0040FA7FF5D68754C28F4FF5A1A949`` will be +expired soon, I have created a new one to replace it. + +You can get my new key ``1793DAD5D803A8FFD7451697BB992F9864FAD168`` as well as +the old one and the cross-signatures required to establish the |chain of trust| +between the two via Web Key Directory (|WKD|) (which should be used +automatically by ``gpg >= 2.1.23``). + +To not deal with the rather convoluted |gnupg| tooling I have created a +deployment method for this using |sequoia-pgp|'s |sq|, about which you can read +in the rest of this article. + +.. TEASER_END + +Key servers +=========== + +Historically, there has been a set of |key servers|, which have been used to +distribute the public keys of users centrally and/ or in a synchronized +fashion. These key servers have been widely relied upon, but they suffer(ed) +from a lot of issues in regards to privacy, stability and speed. Most notably +the Synchronized Key Server (|SKS|) system collapsed under its technical debt +and had to be shut down. + +To this day there are still other large, non-synchronized keyserver systems +around (e.g. |hockeypuck|, which drives https://keyserver.ubuntu.com), but they +all suffer from the fact, that a large centralized setup, in which keys are +only ever appended, does not scale. + +Additionally, not all keyserver systems support all key types or signatures on +keys, which is problematic, as they can not reflect upon |chain of trust| +between two or more keys. This is very problematic for the |web of trust| of +|PGP|. + +Web Key Directory +================= + +At the time of writing the |WKD| system is formalized in +|draft-koch-openpgp-webkey-service-11|. It describes a decentralized way of +providing public key material via a given domain's webserver, by exposing +specially crafted files. + +WKD Deployment +============== + +Personally, I believe that the |gpg| commandline interface is incredibly +convoluted and very complex. I therefore used |sequoia-pgp|'s |sq| instead to +combine PGP public key material and prepare the required directory structure. + +In my personal |wkd| project I assembled a simple system consisting of easy to +prepare directories in which to place PGP public keys, which are converted to a +keyring using |sq keyring join|, converted into a |WKD| structure using |sq wkd +generate| and synchronized using |rsync|. + +As I do not provide an ``openpgpkey`` subdomain, I am using a direct domain +directory structure (see |WKDHosting| for further details). + +Using keys from WKD +=================== + +In theory all that is required for |gpg| to make use of |WKD| is a version +``>=2.1.23``. However, its use can be somewhat confusing: + +.. code:: sh + + gpg --locate-keys dave@sleepmap.de + +The above only returns the new key +``1793DAD5D803A8FFD7451697BB992F9864FAD168``, but not the old +``91BD8815FE0040FA7FF5D68754C28F4FF5A1A949``. It is entirely opaque to the user +as to why. + +Meanwhile with |sq wkd get| it is possible to return the public key material of +both keys: + +.. code:: sh + + sq wkd get dave@sleepmap.de + +The certificates can be inspected directly using |sq inspect|: + +.. code:: sh + + sq inspect --certifications <(sq wkd get dave@sleepmap.de) + +Additionally, the certificates can be imported into an existing |gnupg| based +keyring: + +.. code:: sh + + gpg --import <(sq wkd get dave@sleepmap.de) + + +Using |WKD|, the need for providing a separate PGP public key file in the +context of this website has been made obsolete and I have therefore instead +replaced it with plain mentions of the PGP key IDs on the `about page +<https://sleepmap.de/about/>`_. + +.. |PGP| raw:: html + + <a target="blank" href="https://en.wikipedia.org/wiki/Pretty_Good_Privacy">PGP</a> + +.. |chain of trust| raw:: html + + <a target="blank" href="https://en.wikipedia.org/wiki/Chain_of_trust">chain of trust</a> + +.. |web of trust| raw:: html + + <a target="blank" href="https://en.wikipedia.org/wiki/Web_of_trust">web of trust</a> + +.. |WKD| raw:: html + + <a target="blank" href="https://wiki.gnupg.org/WKD">WKD</a> + +.. |gnupg| raw:: html + + <a target="blank" href="https://gnupg.org/">gnupg</a> + +.. |sequoia-pgp| raw:: html + + <a target="blank" href="https://sequoia-pgp.org/">sequoia-pgp</a> + +.. |sq| raw:: html + + <a target="blank" href="https://man.archlinux.org/man/sq.1">sq</a> + +.. |key servers| raw:: html + + <a target="blank" href="https://en.wikipedia.org/wiki/Key_server_(cryptographic)">key servers</a> + +.. |SKS| raw:: html + + <a target="blank" href="https://github.com/SKS-Keyserver/sks-keyserver">SKS</a> + +.. |hockeypuck| raw:: html + + <a target="blank" href="https://github.com/hockeypuck/hockeypuck">hockeypuck</a> + +.. |draft-koch-openpgp-webkey-service-11| raw:: html + + <a target="blank" href="https://tools.ietf.org/id/draft-koch-openpgp-webkey-service-11.html">draft-koch-openpgp-webkey-service-11</a> + +.. |gpg| raw:: html + + <a target="blank" href="https://man.archlinux.org/man/gpg.1">gpg</a> + +.. |wkd| raw:: html + + <a target="blank" href="https://git.sleepmap.de/dave/wkd.git/">wkd</a> + +.. |sq keyring join| raw:: html + + <a target="blank" href="https://man.archlinux.org/man/sq-keyring-join.1">sq keyring join</a> + +.. |sq wkd generate| raw:: html + + <a target="blank" href="https://man.archlinux.org/man/sq-wkd-generate.1">sq wkd generate</a> + +.. |rsync| raw:: html + + <a target="blank" href="https://man.archlinux.org/man/rsync.1">rsync</a> + +.. |WKDHosting| raw:: html + + <a target="blank" href="https://wiki.gnupg.org/WKDHosting">WKDHosting</a> + +.. |sq wkd get| raw:: html + + <a target="blank" href="https://man.archlinux.org/man/sq-wkd-get.1">sq wkd get</a> + +.. |sq inspect| raw:: html + + <a target="blank" href="https://man.archlinux.org/man/sq-inspect.1">sq inspect</a> |