1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
|
.. title: Managing binary package repositories
.. slug: managing-binary-package-repositories
.. date: 2022-07-02 13:19:59 UTC+02:00
.. tags: arch linux, packaging, repod, dbscripts
.. category: archlinux
.. link:
.. description:
.. type: text
In `Packaging for Arch Linux
<https://sleepmap.de/2022/packaging-for-arch-linux>`_ I described the ins and
outs of binary repository management and some of the issues that come with the
tooling currently used by |arch linux|.
In this article I will highlight the work on new tooling and its features.
Since my last write-up on this topic, the project formerly known as
``arch-repo-management`` has been renamed to ``repod`` (as in *repo-d*) and has
just seen its first minor release. 🎉
You can find its documentation at https://repod.archlinux.page.
.. TEASER_END
.. note::
The https://archlinux.page domain has recently been acquired by Arch Linux to
serve as the common platform for documentation of projects driven by the
distribution.
Please note, that |repod| 0.1.0 is still alpha grade software and **should
not** be used to actually manage binary package repositories at this point in
time!
However, it is already possible to do a few things with the software and if you
are able to test it or are interested in helping develop it, that is very much
welcomed!
On Arch Linux you can install it using |pacman|:
.. code:: sh
pacman -S repod
On other distributions (even on macOS!) you may install it using |pip| (|repod|
is |available on pypi|) until your respective package management system makes
the software available to you on a system level:
.. code:: sh
pip install --user repod
Working on repod
----------------
Since 2021 I have been on and off working on what is now |repod|. The project
is written in typed |python| and is extensively tested using |pytest|.
Work first began after |arch conf| 2019, at which a working group had looked
into improving the workflows currently employed by the distribution and pushing
for tooling that would allow moving away from an |svn| monorepo based approach
to a deconstructed |git| setup.
A proof of concept (PoC) to mimic the behavior of |dbscripts| had been created,
but after 2019 this work laid dormant.
Over 2021 I have spent time to transform parts of the PoC into a Python project
following best practices for development (e.g. type hints following |pep0484|,
100% test coverage, data validation using |pydantic| models), exposing first
features in scripts.
In 2022 more work has been done to extend validation and transform the project
into a package based setup for easier handling and extension in the future.
Concepts of repod
-----------------
Contrary to |dbscripts|, |repod| follows a paradigm in which it is largely
decoupled from the |source repository| of the binary packages it maintains and
aims at becoming a self-contained service.
Package files and their signatures are consumed, relevant metadata is extracted
and transferred to a |management repository|, which is where the state of
each |binary repository| is kept.
Available packages in a given |binary repository| are exposed to |pacman| via
|sync database| files. The |management repository| contents can be
(transparently and reproducibly) transformed into |sync database| files and
vice versa.
The above functionality is exposed on the command line via |repod-file|.
Upcoming work
-------------
As mentioned above, |repod| is not yet stable and still misses quite a few
features.
The following topics (and more) will be worked on in the next milestones (not
necessarily in this order):
* file handling (moving package files from staging areas to actual repositories)
* signature validation (PGP signature validation of package files)
* handling of debug packages
* consolidate schema of |management repository|
* improve logging throughout the project
* git backend to transparently expose changes to the |management repository|
* caching for |management repository| state (e.g. to allow fast searches)
* API to interact with a |repod| instance over the wire in an authenticated fashion
* client-side tooling to interact with |repod|'s API
There is still a lot of work to be done, so if you have a background in
|Python| development and are interested in working on a project that is very
close to the distribution and will likely improve the workflow of many people
while making binary repository management more transparent to the user, have a
look at the project's |contribution guidelines| and do not hesitate to reach
out!
.. |arch linux| raw:: html
<a target="blank" href="https://archlinux.org">Arch Linux</a>
.. |repod| raw:: html
<a target="blank" href="https://gitlab.archlinux.org/archlinux/repod">repod</a>
.. |pacman| raw:: html
<a target="blank" href="https://man.archlinux.org/man/pacman.8">pacman</a>
.. |pip| raw:: html
<a target="blank" href="https://man.archlinux.org/man/pip.1">pip</a>
.. |available on pypi| raw:: html
<a target="blank" href="https://pypi.org/project/repod/">available on pypi</a>
.. |python| raw:: html
<a target="blank" href="https://www.python.org/">Python</a>
.. |pytest| raw:: html
<a target="blank" href="https://docs.pytest.org/en/latest/">pytest</a>
.. |arch conf| raw:: html
<a target="blank" href="https://conf.archlinux.org/">Arch Conf</a>
.. |svn| raw:: html
<a target="blank" href="https://man.archlinux.org/man/svn.1">svn</a>
.. |git| raw:: html
<a target="blank" href="https://man.archlinux.org/man/git.1">git</a>
.. |dbscripts| raw:: html
<a target="blank" href="https://gitlab.archlinux.org/archlinux/dbscripts">dbscripts</a>
.. |pep0484| raw:: html
<a target="blank" href="https://peps.python.org/pep-0484/">PEP 0484</a>
.. |pydantic| raw:: html
<a target="blank" href="https://pydantic-docs.helpmanual.io/">pydantic</a>
.. |source repository| raw:: html
<a target="blank" href="https://repod.archlinux.page/repositories/source_repository.html">source repository</a>
.. |management repository| raw:: html
<a target="blank" href="https://repod.archlinux.page/repositories/management_repository.html">management repository</a>
.. |binary repository| raw:: html
<a target="blank" href="https://repod.archlinux.page/repositories/binary_repository.html">binary repository</a>
.. |sync database| raw:: html
<a target="blank" href="https://repod.archlinux.page/repositories/sync_database.html">sync database</a>
.. |repod-file| raw:: html
<a target="blank" href="https://man.archlinux.org/man/repod-file.1">repod-file</a>
.. |contribution guidelines| raw:: html
<a target="blank" href="https://gitlab.archlinux.org/archlinux/repod/-/blob/79a0fa9aa4a4d9def6dad24805f9bbff0388f734/CONTRIBUTING.md">contribution guidelines</a>
|