[PureOS] Next steps for Laniakea: The archive problem

[Matthias Klumpp]

Am Di., 2. März 2021 um 12:35 Uhr schrieb Guido Günther <guido.gunther at puri.sm>:
>
> Hi Matthias,
> On Mon, Feb 01, 2021 at 08:47:43PM +0100, Matthias Klumpp wrote:
> [..snip..]
> > So, what do you think? Am I crazy, or is this a good thing to finally attempt?
>
> Thanks for the explanations, your reasoning makes sense to me. Two
> things kept me wondering:
>
> 1.) Could that be built so we can share more things with other Debian
>     downstreams to put long term maintenance on broader shoulders?

Absolutely, I think Laniakea itself could be extremely useful for
other downstreams as well. However, there's one catch: At the moment,
Laniakea can theoretically work with Aptly/reprepro/dak etc. for
repository management. After this change though, you would pretty much
need to use its built-in archive service. I do absolutely think that
that's a price worth paying, but it may make Lk harder to adopt.

> 2.) How would that help (or delay) other discussed developments in PureOS like
>
>     - being able to block packages from migration by bug reports (as in
>        Debian) (https://tracker.pureos.net/T872)

That one is unrelated to this proposal, so would neither be blocked
nor helped by it (even though it may benefit from more accurate
database information).

>     - automatic QA via autopkgtests, reproducibility tests (and
>       rejection of packages when those fail)

The autopkgtests/reproducibility stuff itself wouldn't be impacted,
any automation based on that would be easier to implement though (so
automatic package rejection etc.), except for things touching
migration control, which is unrelated (again).

>       (https://tracker.pureos.net/T649, https://tracker.pureos.net/T1013)
>     - rebuilding packages on import
>       (https://tracker.pureos.net/T1016)

The changes would help, but we could actually do this right now as
well - *but* rebuilding all packages needs a lot more maintenance from
the PureOS team as we would need to handle all transitions on our own.
At the moment, we don't have tooling set up for this task to make it
easier to achieve.

>     - building modified packages from git tags
>       (https://tracker.pureos.net/T1014))

Would be far easier with the changes implemented.

>     - automatic forward porting of downstream changes
>       (https://tracker.pureos.net/T1015)

That would probably be unrelated, as handled by a separate tool.

>     and how would that fit in the overall PureOS roadmap. (i filed
>     bugs for each bullet point above if my search didn't bring one up
>     already).

Many of these features could already be implemented, but would
actually be harder to write, because most need some synchronization
with dak's state (which is the #1 problem source).
My first priority would be to streamline the migration process and
also get feature-parity with what we have today. Then, the other
features could be implemented on top, or even in parallel in case of
the QA bits.

> As daily PureOS consumer (and user) i'm curious how it will affect these
> developments?

Ideally you shouldn't notice anything. To make sure we don't break
installations, we would likely need to run a copy of the archive in
parallel that's created by the new system, and then at some point make
the switch. When that's done, user shouldn't notice any changes, if
things are done right, with possibly two exceptions: I'm not planning
to implement index diffs or by-hash initially, just to keep the scope
of this manageable. So, and apt update would have a bigger download
size. On the upside though, we would have a Contents file again for
those people who need it, and software.pureos.net would always reflect
the current state of the archive exactly.

Cheers,
    Matthias