- PureOS-project - Purism Mailing Lists

Re: [PureOS] Updating localechooser in landing/byzantium
by Jeremiah C. Foster 08 Oct '21

08 Oct '21

On 10/8/21 3:11 AM, Guido Günther wrote: > Hi, > On Thu, Oct 07, 2021 at 02:58:54PM -0400, Jeremiah C. Foster wrote: >>> We can forward all of gitlab traffic to from the PureOS space to this >>> (or another) list or better explain how to subscribe to all changes there. >> >> So you'd rather de-commission or deprecate this list? Isn't following even >> just the PureOS space via a RSS type feed a _lot_ of mail? I'll import it >> into my inbox to see how much myself. I can see a benefit of having fewer >> communication streams. > > Not at all. I'd keep the list for sure as a discussion point. My point > was just that you can also have email for gitlab discussions forwarded > to a list. The /pureos space currently doesn't generate much mail but > I'd rather keep `-project` as low volume and have all changes forwarded > to another list. Okay, sounds good. Do we then leave the status quo and ask those who wish to follow everything packaging related to PureOS via the Gitlab feed[0] (which is down at the moment) or should we redirect the PureOS activity to the pureos-changes[1] mailing list? I'd prefer just keeping the status quo. Cheers, Jeremiah 0. https://source.puri.sm/pureos.atom?feed_token=FyqHac77BYBaZh9ZAXtM 1. https://lists.puri.sm/listinfo/pureos-changes

1 0

Re: [PureOS] Updating localechooser in landing/byzantium
by Carsten Schoenert 08 Oct '21

08 Oct '21

Guido, Am 06.10.21 um 16:38 schrieb Guido Günther: >> I've currently decided against this option because to keep the existing old >> history within the same tree as base for the current new added branch. > > …but that wouldn't change since the current history is on `master` and > you'd merge that plus your changes / the update into `pureos/latest`, > right? no, my current way of doing nor your point would break or drop history, how you are seeing it, or your pov of doing is just another way to get the same goal. The merge of the old master branch is just into another direction than I've done it currently. > Am i missing something? Your merge of the old 'master' wouldn't > get broken as long as the merge of the MR to `pureos/latest` is > fast-forward. Sure, to get things merge-able by a MR the repo on core-packages would need an existing branch pureos/latest. That isn't existing right now. I've tried the readjust the way you would make update for the fork. 1. Create new branch pureos/latest based on Debian tag 2.93 2. Merge in the (old pureos) master branch 3. Add additional stuff as gbp.conf and CI yaml file The outcome and the required work is the same to my current kind of doing, most of the work is to solve the merge conflicts as this is exact same todo on both ways. But maybe I'm the one that is missing something. ;) > Again I have no strong opinion here but rather try to understand why one > would use a different workflow as with other packages (which > e.g. results in the discussion being split between mailing list and > giltab). Well, the thing is still, there is no branch I can start a MR against (at least I think). There is not even yet a tag 2.93 on soure.p.s., there is currently only the old master branch which isn't connected anyhow to the "mother" tree on Salsa. If it would exist at all this mail traffic wouldn't needed and I've started directly any discussion within a MR. If the outcome is there is no problem to create a starting branch pureos/latest by preparing the tree on source.p.s then I'm of course fine with this. I'm unsure if I would can do that, given to the permissions for Developers I should. > My take for using merge requests for about everything is that it keeps > the discussion close to the code (and searchable). I fully understand this. But I'm quite conservative about possible direct action on "hot" repositories as I'm still new and like to prevent any damages. -- Regards Carsten Schoenert

3 7

Re: [PureOS] Updating localechooser in landing/byzantium
by Jeremiah C. Foster 07 Oct '21

07 Oct '21

Hello again! On 10/7/21 3:51 AM, Guido Günther wrote: > Hi Jeremiah, > On Wed, Oct 06, 2021 at 04:22:55PM -0400, Jeremiah C. Foster wrote: >> Hello, >> >> On 10/6/21 12:29 PM, Carsten Schoenert wrote: >>> Guido, >>> >>> Am 06.10.21 um 16:38 schrieb Guido Günther: >> <snip> >>>> Again I have no strong opinion here but rather try to understand why one >>>> would use a different workflow as with other packages (which >>>> e.g. results in the discussion being split between mailing list and >>>> giltab). >>> Well, the thing is still, there is no branch I can start a MR against >>> (at least I think). There is not even yet a tag 2.93 on soure.p.s., >>> there is currently only the old master branch which isn't connected >>> anyhow to the "mother" tree on Salsa. >> This is an issue with all packages on source.puri.sm - both pureos/core and >> pureos/packages are not connected to Salsa or to Laniakea. > By far not all. /Librem5/debs space uses full sources mirrored from > salsa so do many projects I've encountered in the PureOS /core and > /packages spaces. Just one exmple from each: > > https://source.puri.sm/pureos/core/dpkg/-/commits/pureos/master > https://source.puri.sm/pureos/packages/gnome-control-center > > There's many more and new ones also use this pattern to my > knowledge. That is fantastic! In my experience that was not the case but looking at the package lists of core and packages now I can see that there are many updated as you say. Other's say "fork from Debian" and some are years old or month old but they may have newer branches, etc. If we could use those two package repos going forward, we might be able to determine a minimum set of PureOS forked or modified packages that we can evaluate for support over the longer term if we chose to support Amber longer than it's current security service going forward. We have more "PureOS only" stuff in Byzantium I assume so the sooner we can define that minimum set of things we need to support (and for how long) the better we can allocate resources that work. >> These repo groupings (core and packages) likely no longer serve the same >> purpose as originally - namely to be "complete and corresponding" source >> code for all the packages in PureOS. We likely ought to have a plan or >> create one for how we can have a one-to-one correspondence between source >> code in a git repo and the binaries in the archives to make support easier. >> >>> If it would exist at all this mail traffic wouldn't needed and I've >>> started directly any discussion within a MR. >>> >>> If the outcome is there is no problem to create a starting branch >>> pureos/latest by preparing the tree on source.p.s then I'm of course >>> fine with this. >>> I'm unsure if I would can do that, given to the permissions for >>> Developers I should. >>> >>>> My take for using merge requests for about everything is that it keeps >>>> the discussion close to the code (and searchable). >> This makes sense but this list can be a very valuable resource for those new >> to PureOS, for making policy, and for other co-ordination. So I for one >> welcome Carsten's mail to the list. > We can forward all of gitlab traffic to from the PureOS space to this > (or another) list or better explain how to subscribe to all changes there. So you'd rather de-commission or deprecate this list? Isn't following even just the PureOS space via a RSS type feed a _lot_ of mail? I'll import it into my inbox to see how much myself. I can see a benefit of having fewer communication streams. Regards, Jeremiah

1 0

Re: [PureOS] Updating localechooser in landing/byzantium
by Carsten Schoenert 06 Oct '21

06 Oct '21

Hello Guido, Am 06.10.21 um 14:44 schrieb Guido Günther: >> So why I'm posting here? >> Simply because there is no target branch on the destination tree I can > > Wouldn't a branch `pureos/latest` at the same commit as Debian's 2.93 be > the target branch to create an MR against? Similar to what happened for > seahorse and lollypop? Why wouldn't that work? of course that would work too. > It's a small diff and i don't want to complicate things, just trying to > figure out where default workflows break. I've currently decided against this option because to keep the existing old history within the same tree as base for the current new added branch. I think currently it makes no difference if we would stay on the current created pureos/latest branch. If you all think it would make more sense to branch away pureos/latest starting from the relevant tag 2.93 I'll can rework the trees. That's why I've posted here, to get a common opinion on how to proceed. -- Regards Carsten Schoenert

2 1

Byzantium plasma builds
by Jeremiah C. Foster 05 Oct '21

05 Oct '21

Hiya, Do we plan to support KDE Plasma builds for Byzantium? I don't see it here; https://downloads.puri.sm/byzantium/ I don't know how much usage they get or how much work it is to support them so I'm open to dropping if it makes sense. Regards, Jeremiah

2 2

Re: [PureOS] Droping the Uploader field in d/control within PureOS?
by Carsten Schoenert 27 Sep '21

27 Sep '21

Hi Guido, Am 25.09.21 um 12:09 schrieb Guido Günther: >> What is the rationale behind that note? > > This documents current practice (I documented what I found when looking > into getting packages into PureOS proper). > > If you think it's a bad idea just > > - make a proposal (e.g. drop, change to XSBC-*) on tracker.pureos.net. > - if there's a discussion let's find consensus > - if there's no discussion even better (i'd wait for >2w for feedback) > - adjust the documentation > - close the issue thanks, so there is no hard reasoning for the current state. I'll follow your suggestion within the next days, the mentioned solution is btw. the best way to get this done I think. -- Regards Carsten Schoenert

4 7

Droping the Uploader field in d/control within PureOS?
by Carsten Schoenert 24 Sep '21

24 Sep '21

Hi, as I'm still quite new with contributing to PureOS I faced a question yesterday while working on T1076 and T1079. While preparing seahorse for byzantium lintian was complaining about 'invalid-field-for-derivative' for the still existing Uploaders field. https://lintian.debian.org/tags/invalid-field-for-derivative Looking at https://tracker.pureos.net/w/development/packaging_overview/ there is mentioned "Note that the Uploaders: field remains untouched." What is the rationale behind that note? Personally I believe that Lintian is of course right here, I myself e.g have users that email me from time to time about some problems in KiCad e.g. which aren't rooted in Debian as they use the PPA for Ubuntu, but got my email address from the package data. I quite almost need to point the users to the PPA owner. It's not that difficult but sometimes annoying me. So why should a user of PureOS get in contact with the uploaders of an package in Debian if he correctly should contact the packagers for PureOS? Why is this error message in lintian currently ignored for PureOS? -- Regards Carsten Schoenert

2 1

Lollypop in byzantium has a bug that should be fixed
by Andreas Ronnquist 23 Sep '21

23 Sep '21

Bullseye is released, and byzantium seems to get closer and closer to being released - congratulations! In the Lollypop package there's a bug in the package in byzantium/bullseye which affects the adaptive mode of the application (which is bad for the phone interface of course). Upstream bug #2778 [1] (fixed in 1.4.19-2 in debian, in upstream in 1.4.20, while 1.4.14-1 is in byzantium/bullseye). It's two commits upstream [2][3] merged to one patch in the debian fix [4]. If I can, I would like to fix this in byzantium - what's the preferred way to do this? Cherry-picking the fix and adding to 1.4.14-1, or simply backporting the latest (or maybe just the first one with the fix) to byzantium? I would gladly provide an updated package. regards -- Andreas Rönnquist mailinglists(a)gusnan.se andreas(a)ronnquist.net [Please don't CC me, if I mail to a mailinglist, I am subscribed to it.] 1: https://gitlab.gnome.org/World/lollypop/-/issues/2778 2: https://gitlab.gnome.org/World/lollypop/-/commit/5e0cbc686e3754a5a3c6e05315… 3: https://gitlab.gnome.org/World/lollypop/-/commit/8cd6117a597be5ee276dca7b99… 4: https://salsa.debian.org/python-team/packages/lollypop/-/commit/bdad22d541d…

3 4

Re: [PureOS] Next steps for Laniakea: The archive problem
by Matthias Klumpp 02 Mar '21

02 Mar '21

Am Di., 2. März 2021 um 12:35 Uhr schrieb Guido Günther <guido.gunther(a)puri.sm>: > > Hi Matthias, > On Mon, Feb 01, 2021 at 08:47:43PM +0100, Matthias Klumpp wrote: > [..snip..] > > So, what do you think? Am I crazy, or is this a good thing to finally attempt? > > Thanks for the explanations, your reasoning makes sense to me. Two > things kept me wondering: > > 1.) Could that be built so we can share more things with other Debian > downstreams to put long term maintenance on broader shoulders? Absolutely, I think Laniakea itself could be extremely useful for other downstreams as well. However, there's one catch: At the moment, Laniakea can theoretically work with Aptly/reprepro/dak etc. for repository management. After this change though, you would pretty much need to use its built-in archive service. I do absolutely think that that's a price worth paying, but it may make Lk harder to adopt. > 2.) How would that help (or delay) other discussed developments in PureOS like > > - being able to block packages from migration by bug reports (as in > Debian) (https://tracker.pureos.net/T872) That one is unrelated to this proposal, so would neither be blocked nor helped by it (even though it may benefit from more accurate database information). > - automatic QA via autopkgtests, reproducibility tests (and > rejection of packages when those fail) The autopkgtests/reproducibility stuff itself wouldn't be impacted, any automation based on that would be easier to implement though (so automatic package rejection etc.), except for things touching migration control, which is unrelated (again). > (https://tracker.pureos.net/T649, https://tracker.pureos.net/T1013) > - rebuilding packages on import > (https://tracker.pureos.net/T1016) The changes would help, but we could actually do this right now as well - *but* rebuilding all packages needs a lot more maintenance from the PureOS team as we would need to handle all transitions on our own. At the moment, we don't have tooling set up for this task to make it easier to achieve. > - building modified packages from git tags > (https://tracker.pureos.net/T1014)) Would be far easier with the changes implemented. > - automatic forward porting of downstream changes > (https://tracker.pureos.net/T1015) That would probably be unrelated, as handled by a separate tool. > and how would that fit in the overall PureOS roadmap. (i filed > bugs for each bullet point above if my search didn't bring one up > already). Many of these features could already be implemented, but would actually be harder to write, because most need some synchronization with dak's state (which is the #1 problem source). My first priority would be to streamline the migration process and also get feature-parity with what we have today. Then, the other features could be implemented on top, or even in parallel in case of the QA bits. > As daily PureOS consumer (and user) i'm curious how it will affect these > developments? Ideally you shouldn't notice anything. To make sure we don't break installations, we would likely need to run a copy of the archive in parallel that's created by the new system, and then at some point make the switch. When that's done, user shouldn't notice any changes, if things are done right, with possibly two exceptions: I'm not planning to implement index diffs or by-hash initially, just to keep the scope of this manageable. So, and apt update would have a bigger download size. On the upside though, we would have a Contents file again for those people who need it, and software.pureos.net would always reflect the current state of the archive exactly. Cheers, Matthias

1 0

Next steps for Laniakea: The archive problem
by Matthias Klumpp 02 Mar '21

02 Mar '21

Hi! I would like to share a part of the plan for future Laniakea improvements which might likely bind a significant amount of my time, but may also provide a huge amount of improvements for the PureOS tooling. What is Laniakea? ------------------------- In order to build a Debian derivative, many tools are needed - besides managing the archive, you also want to do QA, migrate packages between suites, sync packages from Debian, show some web UI about packages, etc. If all tools are separate and don't know of each other, you have to manage them separately, which is a lot of work, but also coordinate them somehow and manage their configuration (e.g. if a suite stabilizes, you need to adjust the new development target 5 times for 5 different tools). Laniakea is a suite of tools which either provides features for archive management on its own, or wraps existing tools and manages their configuration. All tools share one database for configuration which is the single source of truth in the system, and individual Laniakea modules can also communicate directly via a ZeroMQ-based messaging protocol. The Problem ----------------- Laniakea was originally created as part of the Tanglu Debian derivative, and then adapted for PureOS. One of the original ideas of Laniakea was to work with any archive management software, bei it dak (Debian Archive Kit), reprepro, aptly, ... and just wrap it like it wraps any other module. Because that's what it's design was based on: Be very modular and make each module easily replaceable in case something better comes along. This also enabled Laniakea to for a while host two ways to get packages built, while nowadays this has pretty much been collapsed into one singular job management system, which handles package builds as well as image builds. The problem with making the archive manager pluggable is that it goes against Laniakea's principle of having a single source of truth in its database. Currently, only dak is properly supported as archive manager for Laniakea, which keeps its own package database, handles all incoming packages and generates data which Laniakea then imports into its own copy of the archive database. This is terrible for multiple reasons: * Laniakea and dak can - even if just briefly - desynchronize. In reality this makes people wonder which version of a package currently is in what suite, as the web UI does not reflect the actual archive status. * Due to potentially desynchronized states, Laniakea can not trigger certain actions, or can only trigger them in a static sequence of operation when it *knows* that the archive data is currently the same as the data in its database. This results in a lot of archive operations being executed in strict sequences and some changes will not take effect until those were run (like package removals, migrations, syhfornizations, ...) * Laniakea does not know of certain actions taken by dak: For example, if dak rejects a package from upload, Laniakea can only guess that this has happened as there is no direct communication channel that would be race-free. This results in heuristics for the package build system to figure out whether an upload actually reached its destination suite. * The constant interactions between Laniakea and dak are a huge performance hit - Laniakea needs to constantly synchronize its database with the dak data, which can occasionally take more than 20min. That combined with the sequences that Laniakea has to run anyway destroys performance and the "immediateness" that archive actions could have, as everything runs at timed intervals by cron jobs. The Solution ----------------- The solution seems simple: Just implement archive management in Laniakea itself! I did shy away from this for years though. Reasons are that creating the apt archive actually is not an easy task, and a lot of work went into dak for a lot of nice features that would all have to be implemented again. Also, the Debian archive is constantly evolving, and by detaching from dak we would not be able to benefit from them unless we put in work from our side to implement them again. Another reason why this is a bad idea is that it would bind my time, so other features which I consider important would get realized much later. There's also personal reasons for not attempting that rewrite: Dak has some insane SQL to generate indices performantly directly from database tables, and me being no experienced database developer would not help with achieving a better result. However, despite all of these concerns I think I should attempt to write the Laniakea archive module, with the aim to replace dak mid-term. Reasons being the following: * The issues due to interfacing with dak are touching every area of our archive, and it is honestly a pain to work with. Also, 90% of feature requests brought forth by users directly tie in with this issue and could be easily addressed if Laniakea could control the archive directly, and would be the only entity doing so (instead of having some external thing to synchronize with). * Ubuntu's Launchpad is a good example of what can be built if the archive is integrated a bit more tightly with other distribution management features. * Launchpad also uses apt-ftparchive for some metadata read actions. Performance and flexibility is a huge worry for me in an endeavour like this, so I looked into using apt-ftparchive for a "Laniakea archive" prototype. It's not a tool that makes sense to run directly on the archive, but it is great to obtain exactly the metadata we need to build archive index files. Those can be transformed into JSON and stored in Postgres JSON table elements, so we can very easily extend the extra metadata elements in future, if we need to. Also, reading a large quantity of data elements from Postgres tables with SQLAlchemy's ORM works very quickly, and converting and writing them with C++ code is almost instant. Building on top of apt-ftparchive and APT's libapt-pkg also ensures we benefit from improvements made on these tools in Debian. * We don't actually need to support all features initially. Some features we would certainly lose initially are pdiffs, by-hash etc, but a functioning APT archive, without any legacy support but also without any of the very fancy new features, could actually be built in a reasonable amount of time. Of course that would be annoying for some users, but the audience for PureOS would unlikely mind these changes much. And we could gradually add them back later. So, due to that I think it makes sense to spend some serious time to look into implementing this module. I would build on top of apt-pkg and apt-ftparchive, using C++ code where necessary for performance, and Python code everywhere else. I would also make heavy use of json/jsonb entries in Postgres for many of the metadata elements, as it's a convenient, performant but most of all easily extensible way to store most of the auxiliary metadata. For actions such as dependency-resolution I would also use the existing APT library. There are a few open questions on design details that I haven't nailed down, but unlike a year ago, I think a project like this is actually doable and will not take years to complete. But as always with these things, what may look easily achievable now can turn out to be much harder in future ;-) So, what do you think? Am I crazy, or is this a good thing to finally attempt? Cheers, Matthias P.S: To address the obvious question "Why not adjust dak?" - I looked into this, but the amount of changes we would need is quite big, and since there is no chance to merge these upstream, we would essentially be forking dak and still having a separate database and separate-ish tooling. Also, dak serves its own purpose and serves it well for Debian - trying to make it something that it isn't just to work for Laniakea is not a great plan.

2 1

2025

2024

2023

2022

2021

2020

2019

2018

PureOS-project