From agx at sigxcpu.org Tue Mar 2 05:16:01 2021 From: agx at sigxcpu.org (Guido =?iso-8859-1?Q?G=FCnther?=) Date: Tue, 2 Mar 2021 14:16:01 +0100 Subject: [PureOS] Next steps for Laniakea: The archive problem In-Reply-To: References: Message-ID: Hi Matthias, On Mon, Feb 01, 2021 at 08:47:43PM +0100, Matthias Klumpp wrote: [..snip..] > So, what do you think? Am I crazy, or is this a good thing to finally attempt? Thanks for the explanations, your reasoning makes sense to me. Two things kept me wondering: 1.) Could that be built so we can share more things with other Debian downstreams to put long term maintenance on broader shoulders? 2.) How would that help (or delay) other discussed developments in PureOS like - being able to block packages from migration by bug reports (as in Debian) (https://tracker.pureos.net/T872) - automatic QA via autopkgtests, reproducibility tests (and rejection of packages when those fail) (https://tracker.pureos.net/T649, https://tracker.pureos.net/T1013) - rebuilding packages on import (https://tracker.pureos.net/T1016) - building modified packages from git tags (https://tracker.pureos.net/T1014)) - automatic forward porting of downstream changes (https://tracker.pureos.net/T1015) and how would that fit in the overall PureOS roadmap. (i filed bugs for each bullet point above if my search didn't bring one up already). As daily PureOS consumer (and user) i'm curious how it will affect these developments? Cheers, -- Guido From matthias.klumpp at puri.sm Tue Mar 2 05:31:26 2021 From: matthias.klumpp at puri.sm (Matthias Klumpp) Date: Tue, 2 Mar 2021 14:31:26 +0100 Subject: [PureOS] Next steps for Laniakea: The archive problem In-Reply-To: References: Message-ID: Am Di., 2. M?rz 2021 um 12:35 Uhr schrieb Guido G?nther : > > Hi Matthias, > On Mon, Feb 01, 2021 at 08:47:43PM +0100, Matthias Klumpp wrote: > [..snip..] > > So, what do you think? Am I crazy, or is this a good thing to finally attempt? > > Thanks for the explanations, your reasoning makes sense to me. Two > things kept me wondering: > > 1.) Could that be built so we can share more things with other Debian > downstreams to put long term maintenance on broader shoulders? Absolutely, I think Laniakea itself could be extremely useful for other downstreams as well. However, there's one catch: At the moment, Laniakea can theoretically work with Aptly/reprepro/dak etc. for repository management. After this change though, you would pretty much need to use its built-in archive service. I do absolutely think that that's a price worth paying, but it may make Lk harder to adopt. > 2.) How would that help (or delay) other discussed developments in PureOS like > > - being able to block packages from migration by bug reports (as in > Debian) (https://tracker.pureos.net/T872) That one is unrelated to this proposal, so would neither be blocked nor helped by it (even though it may benefit from more accurate database information). > - automatic QA via autopkgtests, reproducibility tests (and > rejection of packages when those fail) The autopkgtests/reproducibility stuff itself wouldn't be impacted, any automation based on that would be easier to implement though (so automatic package rejection etc.), except for things touching migration control, which is unrelated (again). > (https://tracker.pureos.net/T649, https://tracker.pureos.net/T1013) > - rebuilding packages on import > (https://tracker.pureos.net/T1016) The changes would help, but we could actually do this right now as well - *but* rebuilding all packages needs a lot more maintenance from the PureOS team as we would need to handle all transitions on our own. At the moment, we don't have tooling set up for this task to make it easier to achieve. > - building modified packages from git tags > (https://tracker.pureos.net/T1014)) Would be far easier with the changes implemented. > - automatic forward porting of downstream changes > (https://tracker.pureos.net/T1015) That would probably be unrelated, as handled by a separate tool. > and how would that fit in the overall PureOS roadmap. (i filed > bugs for each bullet point above if my search didn't bring one up > already). Many of these features could already be implemented, but would actually be harder to write, because most need some synchronization with dak's state (which is the #1 problem source). My first priority would be to streamline the migration process and also get feature-parity with what we have today. Then, the other features could be implemented on top, or even in parallel in case of the QA bits. > As daily PureOS consumer (and user) i'm curious how it will affect these > developments? Ideally you shouldn't notice anything. To make sure we don't break installations, we would likely need to run a copy of the archive in parallel that's created by the new system, and then at some point make the switch. When that's done, user shouldn't notice any changes, if things are done right, with possibly two exceptions: I'm not planning to implement index diffs or by-hash initially, just to keep the scope of this manageable. So, and apt update would have a bigger download size. On the upside though, we would have a Contents file again for those people who need it, and software.pureos.net would always reflect the current state of the archive exactly. Cheers, Matthias