- PureOS-project - Purism Mailing Lists

Fwd: Reproducible Builds in August 2019
by Chris Lamb 06 Sep '19

06 Sep '19

----- Original message ----- From: Chris Lamb <lamby(a)debian.org> Subject: Reproducible Builds in August 2019 Date: Friday, 6 September 2019 1:23 PM ================================== Reproducible Builds in August 2019 ================================== Welcome to the August 2019 report from the Reproducible Builds [0] project! In these monthly reports we outline the most important things that have happened in the world of Reproducible Builds and we have been up to. You can find a HTML version of this report at the following URI: https://reproducible-builds.org/reports/2019-08/ As a quick recap of our project, whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed to end users or systems as precompiled binaries. The motivation behind the reproducible builds effort is to ensure zero changes have been introduced during these compilation processes. This is achieved by promising identical results are always generated from a given source thus allowing multiple third-parties to come to a consensus on whether a build was changed or even compromised. In August's month's report, we cover: * Media coverage & events — *Webmin, CCCamp, etc.* * Distribution work — *The first fully-reproducible package sets, openSUSE update, etc* * Upstream news — *libfaketime updates, gzip, ensuring good definitions, etc.* * Software development — *More work on diffoscope, new variations in our testing framework, etc.* * Misc news — *From our mailing list, etc.* * Getting in touch — *How to contribute, etc.* If you are interested in contributing to our project, please visit our *Contribute* [1] page on our website. [ 0] https://reproducible-builds.org [ 1] https://reproducible-builds.org/contribute/ Media coverage & events ======================= A backdoor was found in Webmin [2] a popular web-based application used by sysadmins to remotely manage Unix-based systems. Whilst more details can be found on upstream's dedicated exploit page [3], it appears that the build toolchain was compromised. Especially of note is that the exploit "did not show up in any Git diffs" and thus would not have been found via an audit of the source code. The backdoor would allow a remote attacker to execute arbitrary commands with superuser privileges on the machine running Webmin. Once a machine is compromised, an attacker could then use it to launch attacks on other systems managed through Webmin or indeed any other connected system. Techniques such as reproducible builds can help detect exactly these kinds of attacks that can lay dormant for years. (LWN comments [4]) In a talk titled *There and Back Again, Reproducibly!* [5] Holger Levsen and Vagrant Cascadian presented at the 2019 edition of the Linux Developer Conference [6] in São Paulo, Brazil on Reproducible Builds. LWN [7] posted and hosted an interesting summary and discussion on *Hardening the "file" utility for Debian* [8]. In July, Chris Lamb had cross-posted his reply to the "Re: file(1) now with seccomp support enabled [9]" thread, originally started on the "debian-devel" [10] mailing list. In this post, Chris refers to our "strip-nondeterminism" tool not being able to accommodate the additional security hardening in "file(1)" [11] and the changes made to the tool in order to do fix this issue which was causing a huge number of regressions in our testing framework [12]. The Chaos Communication Camp [13] — an international, five-day open-air event for hackers that provides a relaxed atmosphere for free exchange of technical, social, and political ideas — hosted its 2019 edition [14] where there were many discussions and meet-ups at least partly related to Reproducible Builds. This including the titular Reproducible Builds Meetup [15] session which was attended by around twenty-five people where half of them were new to the project as well as a session dedicated to all Arch Linux related issues [16]. [ 2] http://www.webmin.com/ [ 3] http://www.webmin.com/exploit.html [ 4] https://lwn.net/Articles/796951/ [ 5] https://cfp.linuxdev-br.net/2019/talk/VH9CCY/ [ 6] https://linuxdev-br.net/ [ 7] https://lwn.net [ 8] https://lwn.net/Articles/796108 [ 9] https://lists.reproducible-builds.org/pipermail/rb-general/2019-July/001612… [10] https://lists.debian.org/debian-devel/2019/07/msg00391.html [11] http://darwinsys.com/file/ [12] http://tests.reproducible-builds.org/ [13] https://en.wikipedia.org/wiki/Chaos_Communication_Camp [14] https://events.ccc.de/camp/2019/ [15] https://events.ccc.de/camp/2019/wiki/Session:Reproducible_Builds_Meetup [16] https://events.ccc.de/camp/2019/wiki/Session:Arch_Linux_Meetup Distribution work ================= In Debian, the first "package sets" — ie. defined subsets of the entire archive — have become 100% reproducible including as the so-called "essential" set for the bullseye distribution on the "amd64" [17] and the "armhf" [18] architectures. This is thanks to work by Chris Lamb on "bash" [19], "readline" [20] and other low-level libraries and tools. Perl still has issues on "i386" [21] and "arm64" [22], however. Dmitry Shachnev filed a bug report [23] against the "debhelper" utility that speaks to issues around using the date from the "debian/changelog" file as the source for the "SOURCE_DATE_EPOCH" [24] environment variable as this can lead to non-intuitive results when package is automatically rebuilt via so-called binary (NB. not "source" [25]) NMUs. A related issue was later filed against "qtbase5-dev" [26] by Helmut Grohne as this exact issue led to an issue with co-installability across architectures. Lastly, 115 reviews of Debian packages were added, 45 were updated and 244 were removed this month, appreciably adding to our knowledge about identified issues [27]. Many issue types were updated by Chris Lamb, including "embeds_build_data_via_node_preamble" [28], "embeds_build_data_via_node_rollup" [29], "captures_build_path_in_beam_cma_cmt_files" [30], "captures_varying_number_of_build_path_directory_components" [31] (discussed later), "timezone_specific_files_due_to_haskell_devscripts" [32], etc. Bernhard M. Wiedemann posted his monthly Reproducible Builds status update [33] for the openSUSE [34] distribution. New issues were found from enabling Link Time Optimization [35] (LTO) in this distribution's *Tumbleweed* [36] branch. This affected, for example, "nvme-cli" [37] as well as "perl-XML-Parser" and "pcc" [38] with packaging issues. [17] https://tests.reproducible-builds.org/debian/bullseye/amd64/pkg_set_essenti… [18] https://tests.reproducible-builds.org/debian/bullseye/armhf/pkg_set_essenti… [19] https://bugs.debian.org/935127 [20] https://bugs.debian.org/935363 [21] https://tests.reproducible-builds.org/debian/rb-pkg/bullseye/i386/diffoscop… [22] https://tests.reproducible-builds.org/debian/rb-pkg/bullseye/arm64/diffosco… [23] https://bugs.debian.org/934405 [24] https://reproducible-builds.org/docs/source-date-epoch/ [25] https://wiki.debian.org/NonMaintainerUpload [26] https://bugs.debian.org/934511 [27] https://tests.reproducible-builds.org/debian/index_issues.html [28] https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/5d91… [29] https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/e6b6… [30] https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/850d… [31] https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/c0c7… [32] https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/a1a6… [33] https://lists.opensuse.org/opensuse-factory/2019-08/msg00186.html [34] https://opensuse.org/ [35] https://gcc.gnu.org/wiki/LinkTimeOptimization [36] https://software.opensuse.org/distributions/tumbleweed [37] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91307 [38] https://bugzilla.opensuse.org/show_bug.cgi?id=1146634 Upstream news ============= * "libfaketime" [39] is a tool to trick programs into believing that the current system time is actually one specified by the user. This month, Bernhard M. Wiedemann requested the ability to track and intercept calls that change file timestamps [40] which can help better debug or fix reproducibility issues in software. * Chris Lamb requested that the "molior" build tool [41] prefers to use the term "repeatable build" [42] in order to avoid confusion over the term "reproducible." * The "gzip [43]" program is commonly used to compress artifacts such as the the source code archives generated by Sourcehut [44] hosting platform, but depending on the specific program used, the output may be different. Daniel Edgecumbe has submitted patches [45] to the BusyBox [46] suite of tools to ensure the output of its version of "gzip" matches the output of GNU gzip [47] when using the same options regardless of the configuration of BusyBox. In the process, an off-by-one error [48] in the default settings was also fixed. * There was more progress on ensuring that the "gem" tool in rubygems respects [49] the "SOURCE_DATE_EPOCH" [50] environment variable. * A request to include ".buildinfo" files [51] in the OpenWRT [52] operating system that targets embedded devices such as routes, etc. was accepted and merged upstream. [39] https://github.com/wolfcw/libfaketime [40] https://github.com/wolfcw/libfaketime/issues/183 [41] https://github.com/molior-dbs/molior [42] https://github.com/molior-dbs/molior/issues/3 [43] https://www.gzip.org/ [44] https://todo.sr.ht/~sircmpwn/git.sr.ht/232 [45] http://lists.busybox.net/pipermail/busybox/2019-September/087438.html [46] https://busybox.net/ [47] https://www.gnu.org/software/gzip/ [48] https://en.wikipedia.org/wiki/Off-by-one_error [49] https://github.com/rubygems/rubygems/issues/2290#issuecomment-522206365 [50] https://reproducible-builds.org/docs/source-date-epoch/ [51] https://github.com/openwrt/openwrt/pull/2121 [52] https://openwrt.org/ Software development ==================== The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. In August we wrote a large number of such patches, including: * Bernhard M. Wiedemann: * "buildad" [53] (date) * "dracut" [54] (CPU influences build result) * "fwupd" [55] (unreproducible LTO [56] data) * "gnutls" [57] (date / copyright year) * "katacontainers-image-initrd/osbuilder" [58] (shell date; new variant with nanoseconds) * "kernel-obs-build" [59] (date from "/etc/shadow") * "kernel-vanilla" [60] (drop number of CPUs) * "libfaketime" [61] (toolchain: fix various builds under "libfaketime" [62]) * "nethack" [63] (date and "tar(1)" [64])) * "pcc" [65] (unreproducible when building with LTO [66]) * "python-ipyparallel" [67] (Fails to build with a single CPU / "-j1") * "python-pytest-httpserver" [68] (renew SSL certs to fix FTBFS after September 2019) * "python-python3-saml" [69] (Fails to build in 2020) * "sblim-cmpi-base" [70] (Disable parallel "make" [71]) due to broken build dependencies) [53] https://github.com/containers/buildah/pull/1805 [54] https://github.com/dracutdevs/dracut/issues/617 [55] https://bugzilla.opensuse.org/show_bug.cgi?id=1143905 [56] https://gcc.gnu.org/wiki/LinkTimeOptimization [57] https://gitlab.com/gnutls/gnutls/merge_requests/1058 [58] https://github.com/kata-containers/osbuilder/pull/340 [59] https://lists.opensuse.org/opensuse-kernel/2019-08/msg00001.html [60] https://lists.opensuse.org/opensuse-kernel/2019-08/msg00000.html [61] https://github.com/wolfcw/libfaketime/issues/183 [62] https://github.com/wolfcw/libfaketime [63] https://build.opensuse.org/request/show/722212 [64] https://en.wikipedia.org/wiki/Tar_(computing [65] https://bugzilla.opensuse.org/show_bug.cgi?id=1146634 [66] https://gcc.gnu.org/wiki/LinkTimeOptimization [67] https://github.com/ipython/ipyparallel/issues/380 [68] https://github.com/csernazs/pytest-httpserver/pull/22 [69] https://github.com/onelogin/python3-saml/pull/156 [70] https://build.opensuse.org/request/show/726294 * Chris Lamb: * #872728 [72] filed against "desktop-file-utils" [73] (closed) * #933783 [74] filed against "virulencefinder" [75]. * #933790 [76] filed against "norsnet" [77]. * #933834 [78] filed against "haskell-devscripts" [79]. * #933838 [80] filed against "superlu-dist" [81]. * #934120 [82] filed against "python-bleach" [83]. * #934697 [84] filed against "re2c" [85] (filed upstream [86]). * #934698 [87] filed against "libchamplain" [88] (filed upstream [89]) * #934699 [90] filed against "scons" [91]. * #934767 [92] filed against "ecbuild" [93]. * #934918 [94] filed against "python-etcd3gw" [95]. * #934919 [96] filed against "omnidb" [97]. * #935127 [98] filed against "bash" [99]. * #935361 [100] filed against "node-autoprefixer" [101]. * #935362 [102] filed against "gdbm" [103]. * #935363 [104] filed against "readline" [105]. * #935790 [106] filed against "node-package-preamble" [107]. * #935846 [108] filed against "musescore-snapshot" [109]. * #936452 [110] filed against "ust-fs-extra" [111]. * #936453 [112] filed against "litl" [113]. [71] https://en.wikipedia.org/wiki/Make_(software [72] https://bugs.debian.org/872728 [73] https://tracker.debian.org/pkg/desktop-file-utils [74] https://bugs.debian.org/933783 [75] https://tracker.debian.org/pkg/virulencefinder [76] https://bugs.debian.org/933790 [77] https://tracker.debian.org/pkg/norsnet [78] https://bugs.debian.org/933834 [79] https://tracker.debian.org/pkg/haskell-devscripts [80] https://bugs.debian.org/933838 [81] https://tracker.debian.org/pkg/superlu-dist [82] https://bugs.debian.org/934120 [83] https://tracker.debian.org/pkg/python-bleach [84] https://bugs.debian.org/934697 [85] https://tracker.debian.org/pkg/re2c [86] https://github.com/skvadrik/re2c/pull/258 [87] https://bugs.debian.org/934698 [88] https://tracker.debian.org/pkg/libchamplain [89] https://gitlab.gnome.org/GNOME/libchamplain/merge_requests/9 [90] https://bugs.debian.org/934699 [91] https://tracker.debian.org/pkg/scons [92] https://bugs.debian.org/934767 [93] https://tracker.debian.org/pkg/ecbuild [94] https://bugs.debian.org/934918 [95] https://tracker.debian.org/pkg/python-etcd3gw [96] https://bugs.debian.org/934919 [97] https://tracker.debian.org/pkg/omnidb [98] https://bugs.debian.org/935127 [99] https://tracker.debian.org/pkg/bash [100] https://bugs.debian.org/935361 [101] https://tracker.debian.org/pkg/node-autoprefixer [102] https://bugs.debian.org/935362 [103] https://tracker.debian.org/pkg/gdbm [104] https://bugs.debian.org/935363 [105] https://tracker.debian.org/pkg/readline [106] https://bugs.debian.org/935790 [107] https://tracker.debian.org/pkg/node-package-preamble [108] https://bugs.debian.org/935846 [109] https://tracker.debian.org/pkg/musescore-snapshot [110] https://bugs.debian.org/936452 [111] https://tracker.debian.org/pkg/rust-fs-extra [112] https://bugs.debian.org/936453 [113] https://tracker.debian.org/pkg/litl * Mathieu Parent: "php-pear" [114] — Fixes over 150 packages with date issues. [114] https://github.com/pear/pear-core/pull/96 diffoscope ---------- "diffoscope" [115] is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. It is run countless times a day on our testing infrastructure [116] and is essential for identifying fixes and causes of non-deterministic behaviour. This month, Chris Lamb made the following changes: * Improvements: * Don't fallback to an unhelpful raw hexdump when, for example, "readelf(1)" reports an minor issue in a section in an ELF binary. For example, when the ".frames" section is of the "NOBITS" type its contents are apparently "unreliable" and thus "readelf(1)" returns 1. (#58 [117], #931962 [118]) * Include either standard error or standard output (not just the latter) when an external command fails. [119] * Bug fixes: * Skip calls to "unsquashfs" when we are neither root nor running under "fakeroot". (#63 [120]) * Ensure that all of our artificially-created "subprocess.CalledProcessError" [121] instances have "output" instances that are "bytes" objects, not "str". [122] * Correct a reference to "parser.diff"; "diff" in this context is a Python function in the module. [123] * Avoid a possible traceback caused by a "str"/"bytes" type confusion when handling the output of failing external commands. [124] * Testsuite improvements: * Test for "4.4" in the output of "squashfs -version", even though the Debian package version is "1:4.3+git190823-1". [125] * Apply a patch from László Böszörményi to update the "squashfs" test output and additionally bump the required version for the test itself. (#62 [126] & #935684 [127]) * Add the "wabt" Debian package to the test-dependencies so that we run the WebAssembly [128] tests on our continuous integration platform, etc. [129] * Improve debugging: * Add the containing module name to the (eg.) "Using StaticLibFile for ..." debugging messages. [130] * Strip off trailing "original size modulo 2^32 671" (etc.) from "gzip" compressed data as this is just a symptom of the contents itself changing that will be reflected elsewhere. (#61 [131]) * Avoid a lack of space between "... with return code 1" and "Standard output". [132] * Improve debugging output when instantantiating our "Comparator" object types. [133] * Add a literal "eg." to the comment on stripping "original size modulo..." text to emphasise that the actual numbers are not fixed. [134] * Internal code improvements: * No need to parse the section group from the class name; we can pass it via "type" built-in "kwargs" argument. [135] * Add support to "Difference.from_command_exc" and friends to ignore specific returncodes from the called program and treat them as "no" difference. [136] * Simplify parsing of optional "command_args" argument to "Difference.from_command_exc". [137] * Set "long_description_content_type" to "text/x-rst" to appease the PyPI.org [138] linter. [139] * Reposition a comment regarding an exception within the indented block to match Python code convention. [140] [115] https://diffoscope.org [116] https://tests.reproducible-builds.org/debian/reproducible.html [117] https://salsa.debian.org/reproducible-builds/diffoscope/issues/58 [118] https://bugs.debian.org/931962 [119] https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/4689755 [120] https://salsa.debian.org/reproducible-builds/diffoscope/issues/63 [121] https://docs.python.org/3/library/subprocess.html [122] https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/eb02809 [123] https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/8eb9e39 [124] https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/b803d43 [125] https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/7cecd8a [126] https://salsa.debian.org/reproducible-builds/diffoscope/issues/62 [127] https://bugs.debian.org/935684 [128] https://webassembly.org/ [129] https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/84ad96d [130] https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/2f101b8 [131] https://salsa.debian.org/reproducible-builds/diffoscope/issues/61 [132] https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/ffa22f8 [133] https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/1647da8 [134] https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/18e3526 [135] https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/5261096 [136] https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/d3c7ac8 [137] https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/cc9a730 [138] https://pypi.org/ [139] https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/7583af2 [140] https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/ec86443 In addition, Mattia Rizzolo made the following changes: * Now that we install wabt, expect its tools to be available. [141] * Bump the Debian backport check. [142] Lastly, Vagrant Cascadian updated diffoscope to versions 120 [143], 121 [144] and 122 [145] in the GNU Guix [146] distribution. [141] https://salsa.debian.org/reproducible-builds/diffoscope/commit/f2e72a8 [142] https://salsa.debian.org/reproducible-builds/diffoscope/commit/9591cfb [143] https://git.savannah.gnu.org/cgit/guix.git/commit/?id=c91364d36cf6c8fc4c696… [144] https://git.savannah.gnu.org/cgit/guix.git/commit/?id=8c1379ba404b4db2f0afc… [145] https://git.savannah.gnu.org/cgit/guix.git/commit/?id=b126f41b301a5ac13835b… [146] https://guix.gnu.org/ strip-nondeterminism -------------------- strip-nondeterminism [147] is our tool to remove specific non- deterministic results from a completed build. This month, Chris Lamb made the following changes. * Add support for enabling and disabling specific normalizers via the command line. (#10 [148]) * Drop accidentally-committed warning emitted on every fixture-based test. [149] * Reintroduce the ".ar" normalizer [150] but disable it by default so that it can be enabled with "--normalizers=+ar" or similar. (#3 [151]) * In verbose mode, print the normalizers that "strip-nondeterminism" will apply. [152] In addition, there was some movement on an issue in the "Archive::Zip" Perl module [153] that "strip-nondeterminism" uses regarding the lack of support for "bzip" compression [154] that was originally filed in 2016 [155] by Andrew Ayer [156]. [147] https://tracker.debian.org/pkg/strip-nondeterminism [148] https://salsa.debian.org/reproducible-builds/strip-nondeterminism/issues/10 [149] https://salsa.debian.org/reproducible-builds/strip-nondeterminism.git/commi… [150] https://salsa.debian.org/reproducible-builds/strip-nondeterminism.git/commi… [151] https://salsa.debian.org/reproducible-builds/strip-nondeterminism#3 [152] https://salsa.debian.org/reproducible-builds/strip-nondeterminism.git/commi… [153] https://metacpan.org/pod/Archive::Zip [154] https://en.wikipedia.org/wiki/Bzip2 [155] https://github.com/redhotpenguin/perl-Archive-Zip/issues/26 [156] https://www.agwa.name/ Test framework -------------- We operate a comprehensive Jenkins [157] based testing framework that powers tests.reproducible-builds.org [158]. This month Vagrant Cascadian suggested and subsequently implemented [159] that we additionally test a varying build directory of different string lengths (eg. "/path/to/123" vs "/path/to/123456" but we also vary the number of directory *components* within this, eg. "/path/to/dir" vs. "/path/to/parent/subdir". Curiously, whilst it was *a priori* believed that was rather unlikely to yield differences, Chris Lamb has managed to identify approximately twenty packages [160] that are affected by this issue. It was also noticed that our testing of the Coreboot free software firmware [161] fails to build the toolchain since we switched to building on the Debian "buster" distribution. The last successful build was on August 7th [162] but all newer builds have failed. [157] https://jenkins.io/ [158] https://tests.reproducible-builds.org [159] https://salsa.debian.org/qa/jenkins.debian.net/commit/94469490 [160] https://tests.reproducible-builds.org/debian/issues/unstable/captures_varyi… [161] https://tests.reproducible-builds.org/coreboot/coreboot.html [162] https://jenkins.debian.net/job/reproducible_coreboot/356 was t In addition, the following code changes were performed in the last month: * Chris Lamb: Ensure that the size the log for the second build in HTML pages was also correctly formatted (eg. "12KB" vs "12345"). [163] * Holger Levsen: * Many changes related to updating our build nodes to the "buster" distribution for Debian. [164][165][... [166][167][168][169][170] * Attempt to automatically fixup spurious build failures. [171] * Update the maintainer address for the Debian team tasked with maintaining [172] the MATE desktop [173]. [174] * Try not to build all the release tags of tools such as diffoscope [175], etc.. [176] * Use a newer kernel to support building the latest Arch Linux [177] packages. [178] * Re-add checks for "zombie" and log file size sanity checks. [179][180][181][182] * Vary the choice of kernel on the "amd64" again by using the kernel from Debian "backports" [183]. [184] * Drop some ancient Debian "jessie"-related configuration. [185][186][187] * Mathieu Parent: Update the contact details for the Debian PHP Group [188]. [189] * Mattia Rizzolo: * Update our Postfix [190] email server configuration. [... [191][192][193] * Use the "safe_load" function of PyYAML [194] when parsing YAML- formatted [195] files. [196] [163] https://salsa.debian.org/reproducible-builds/jenkins.debian.net.git/commit/… [164] https://salsa.debian.org/qa/jenkins.debian.net/commit/a97c97ec [165] https://salsa.debian.org/qa/jenkins.debian.net/commit/6fb3ee7b [166] https://salsa.debian.org/qa/jenkins.debian.net/commit/1fad75e4 [167] https://salsa.debian.org/qa/jenkins.debian.net/commit/7fef98af [168] https://salsa.debian.org/qa/jenkins.debian.net/commit/da55be7a [169] https://salsa.debian.org/qa/jenkins.debian.net/commit/28941bc2 [170] https://salsa.debian.org/qa/jenkins.debian.net/commit/309a1d66 [171] https://salsa.debian.org/qa/jenkins.debian.net/commit/82bee189 [172] https://wiki.debian.org/Teams/pkg-mate [173] https://mate-desktop.org/ [174] https://salsa.debian.org/qa/jenkins.debian.net/commit/e6f7c6d4 [175] https://diffoscope.org [176] https://salsa.debian.org/qa/jenkins.debian.net/commit/974b699e [177] https://www.archlinux.org/ [178] https://salsa.debian.org/qa/jenkins.debian.net/commit/7e575590 [179] https://salsa.debian.org/qa/jenkins.debian.net/commit/f17552ad [180] https://salsa.debian.org/qa/jenkins.debian.net/commit/30049d46 [181] https://salsa.debian.org/qa/jenkins.debian.net/commit/9fbf8d2c [182] https://salsa.debian.org/qa/jenkins.debian.net/commit/42d7c71a [183] https://backports.debian.org/ [184] https://salsa.debian.org/qa/jenkins.debian.net/commit/b2870778 [185] https://salsa.debian.org/qa/jenkins.debian.net/commit/96cbb81e [186] https://salsa.debian.org/qa/jenkins.debian.net/commit/7e37c5a4 [187] https://salsa.debian.org/qa/jenkins.debian.net/commit/87840dae [188] https://wiki.debian.org/Teams/DebianPHPGroup [189] https://salsa.debian.org/qa/jenkins.debian.net/commit/03510cdf [190] http://www.postfix.org/ [191] https://salsa.debian.org/qa/jenkins.debian.net/commit/61ceaf5d [192] https://salsa.debian.org/qa/jenkins.debian.net/commit/8780a849 [193] https://salsa.debian.org/qa/jenkins.debian.net/commit/3b964081 [194] https://pyyaml.org/wiki/PyYAMLDocumentation [195] https://en.wikipedia.org/wiki/YAML [196] https://salsa.debian.org/qa/jenkins.debian.net/commit/fa720775 The usual node maintenance was performed by Holger Levsen [... [197][198] and Vagrant Cascadian [199]. [197] https://salsa.debian.org/qa/jenkins.debian.net/commit/961d70a6 [198] https://salsa.debian.org/qa/jenkins.debian.net/commit/25f295b7 [199] https://salsa.debian.org/qa/jenkins.debian.net/commit/30f567ff Misc news ========= There was a yet more effort put into our our website [200] this month, including misc copyediting by Chris Lamb [201], Mathieu Parent referencing his fix for "php-pear" [202] and Vagrant Cascadian updating a link to his homepage. [203]. On our mailing list [204] this month Santiago Torres Arias started a *Setting up a MS-hosted rebuilder with in-toto metadata* [205] thread regarding Microsoft's interest in setting up a rebuilder for Debian packages touching on issues of transparency logs and the integration of in-toto [206] by the Secure Systems Lab [207] at New York University [208]. In addition, Lars Wirzenius [209] continued conversation regarding various questions about reproducible builds [210] and their bearing on building a distributed continuous integration system. Lastly, in a thread titled *Reproducible Builds technical introduction tutorial* [211] Jathan asked whether anyone had some "easy" Reproducible Builds tutorials in slides, video or written document format. [200] https://reproducible-builds.org/ [201] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/a9… [202] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/e4… [203] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/7f… [204] https://lists.reproducible-builds.org/pipermail/rb-general/ [205] https://lists.reproducible-builds.org/pipermail/rb-general/2019-August/0016… [206] https://in-toto.io/ [207] https://ssl.engineering.nyu.edu/ [208] https://engineering.nyu.edu/ [209] https://liw.fi/ [210] https://lists.reproducible-builds.org/pipermail/rb-general/2019-August/0016… [211] https://lists.reproducible-builds.org/pipermail/rb-general/2019-August/0016… Getting in touch ================ If you are interested in contributing the Reproducible Builds project, please visit our *Contribute* [212] page on our website. However, you can get in touch with us via: * IRC: "#reproducible-builds" on "irc.oftc.net". * Twitter: @ReproBuilds [213] * Mailing list: "rb-general(a)lists.reproducible-builds.org" [214] [212] https://reproducible-builds.org/contribute/ [213] https://twitter.com/ReproBuilds [214] https://lists.reproducible-builds.org/listinfo/rb-general This month's report was written by Bernhard M. Wiedemann, Chris Lamb, Eli Schwartz, Holger Levsen, Jelle van der Waa, Mathieu Parent and Vagrant Cascadian. Wiedemann, Chris Lamb, Holger Levsen, Mathieu Parent and Vagrant Cascadian. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list. Best wishes, -- Chris Lamb https://puri.sm

1 0

Bits from PureOS | Compensatory Spoonful version
by Jeremiah C. Foster 27 Aug '19

27 Aug '19

Hello! It appears as if Amber is on it's way. What is Amber? Amber is the code name for the new stable version of PureOS. As one can see from the PureOS changes mailing list, Matthias has added Amber to debootstrap; https://lists.puri.sm/pipermail/pureos-changes/2019-August/000414.html I look forward to testing it on my Librem 15. I've already tested it a bit in a Docker container (see deets in pureos-dev chat on matrix.) Speaking of the PureOS changes mailing list, there is now a bridge from Laniakea to Matrix. Ask Matthias or myself for an invite. In discussing the new Buster based version of Amber internally, I note that there continues to be interest in having Green be the stable version of PureOS. I worry that if we don't use Green, lots of users who would benefit from stability will get a huge update once we move back to a rolling release. Right now, using a Docker image of an updated Green, I note there are 114[0] packages waiting to be updated if I switch over to Debian Testing, and that is just for a minimal image. I imagine those who have a lot more packages installed will see a lot more updates. This might get messy quickly. Fortunately I beleive that Matthias and I have a plan that will work and be minimaly invasive to users. The plan is to _append_ the various Amber suites to /etc/apt/sources.list and have users update that way. There are some new Docker images for 'green' and 'landing' available here: https://cloud.docker.com/u/pureos/repository/docker/pureos/green I plan to create one for Amber and Purple as well. I'm automating the process to build these images since updated images will likely get consumed in other parts of our CI/CD. Lots of testing coming up to ensure Amber is easy to update to. Feedback most welcome. Cheers, Jeremiah [0.] See attached list of upated package to minimal PureOS image

1 0

Lollypop stuck in the New queue
by Jeremiah C. Foster 14 Aug '19

14 Aug '19

Hiya, I think my package 'lollypop' is still in the New Queue - how long might one expect it to be there? Regards, Jeremiah

2 2

Bits from PureOS | Sad GNOME edition
by Jeremiah C. Foster 13 Aug '19

13 Aug '19

Hi, Welcome to another bits from PureOS email. Your irregular email for all things PureOS! Or maybe for _some_ things PureOS. Okay, mostly just what I've been doing and noticed happening. Firstly, I can't make GUADEC despite the fact that my talk has been approved and I'd really like to go. I just cannot find child care for my daughter. I would bring her but she starts school on the 28th and that's the day that there is flatpak hacking, including work on a store. *sigh* Secondly, I'd like to point out that our intrepid dak hacker (dacker?) Matthias has already begun work on a new stable distro called "amber". This is going to be pretty exciting stuff I think, at least I'm excited to offer a more stable distro to users who're asking for a more stable distro; https://forums.puri.sm/t/would-you-use-a-pureos-rolling-release-or-do-you-w… Thirdly, I've uploaded a couple packages (and am working on a couple more.) See https://lists.puri.sm/pipermail/pureos-changes/2019-August/thread.html for details but what's happened is I've made some tiny config changes to Gnupg to support the made-in-America Librem Key and pushed a new version of flashrom into PureOS. I'm trying to upload Lollypop but having some problems getting the orig taball uploaded, not sure what triggers that even though its in my .changes file. Hopefully the dak gods will assist. - There's a meeting being planned for next week to talk about the design of software.pureos.net - The download ISO of pureos.net/download has finally been updated to the image that Matthias created in July - Lots of meetings this week including a infrastructure meeting which will hopefully lay the ground work for bringing the work done on L5 into purple. - We have a patch upstream in Gnupg to add the Librem Key into Debian, as soon as that package is live we'll no longer need the Gnupg package I uploaded That's it for this bits email, as always, feedback welcome.

3 4

Bits frm PureOS | Buster edition
by Jeremiah C. Foster 20 Jul '19

20 Jul '19

Congratulations to all the DDs on this list for their work on Buster! Thank you! I'm sure there are many, many people out there grateful for your contribution. My next question naturally is; is PureOS Green going to continue to track Buster? This was a question we brought up in March: https://lists.puri.sm/pipermail/pureos-project/2019-March/000069.html A broader discussion followed that thread and while the shape of consensus seemed to emerge, there were some threads left dangling. Now, as we're being asked by other stakeholders what are plans are, I'd like to be able to announce that PureOS Green is tracking Debian Buster. May I make that announcement? Regards, Jeremiah

6 16

PureOS 2019/07/14 images ready for testing
by Matthias Klumpp 16 Jul '19

16 Jul '19

Hi! Primarily due to an initramfs permission issue where reading disk encryption keys from the initramfs was possible in certain configurations[1], but also because the previous images were getting old, we have new PureOS installation images for testing. Thinks to look at, as always: * Does the image boot cleanly? * Does the Live installer work? * Does the OEM installer work? * Does the OEM initial setup work as expected? * Does the respective desktop function as expected? * Any noticeable bugs/regressions? You can find the images for the respective flavors here: GNOME Live: https://downloads.puri.sm/live/gnome/2019-07-14/ GNOME OEM: https://downloads.puri.sm/oem/gnome/2019-07-14/ Plasma Live: https://downloads.puri.sm/live/plasma/2019-07-14/ I already tested the images and found no noticeable issues, but having more eyes on them before we change the default download links is always good. Happy testing! Cheers, Matthias [1]: https://calamares.io/calamares-cve-2019/

1 0

Using SOURCE_DATE_EPOCH env var in PureOS builds
by Jeremiah C. Foster 12 Jul '19

12 Jul '19

Hi, I impatiently added the SOURCE_DATE_EPOCH to the version of make-live that we use to build PureOS images. It seems to have made a difference though not completely eliminated our blockers to reproducibility; http://45.33.86.90/pureos-8.0-images.html While the byte-for-byte difference now is much smaller, and the date is consistent across builds in the ISO at least, other parts of the build are getting the date from the file system I believe. For example, under the section casper/filesystem.squashfs[0.] unsqaushfs -s {} says; 2 Creation·or·last·append·time·Fri·Jul·12·07:12:11·2019 2 Creation·or·last·append·time·Fri·Jul·12·05:11:57·2019 The leads me to believe that I need to ensure that SOURCE_DATE_EPOCH is being seen when the squashfs file system is written using Lamby's debug method of printing "echo SDE=$SOURCE_DATE_EPOCH" in various places to determine if it's being seen in each build context. After that I think the next step, or even a step to take anyway, is to run strip-nondeterminism over the resulting ISOs[1.] 0. http://45.33.86.90/pureos-8.0-images.html#casper-filesystem.squashfs 1. https://packages.debian.org/sid/strip-nondeterminism Feedback, corrections most welcome. Regards, Jeremiah

2 3

Bits from PureOS
by Jeremiah C. Foster 12 Jul '19

12 Jul '19

Hi! There's been some discussion amongst various folks that our communication culture can be suboptimal at times. I thought I would try and encourage greater use of our mailing list to compliment our chat channel and this irregular (but hopefully frequent) email is a collection of "bits" in the spirit of a Debian's "Bits from . . . " In PureOS we have a number of current topics which I'll enumerate here; 1. Reproducible builds 2. Supporting Pureboot and Purism hardware 3. Continuous delivery of PureOS 4. PureOS store (and application discovery) These are high level topics which are organized in no particular priority. I thought that I'd try to have updates in each of these areas and to tie in the work being done in other parts of Purism that may have an impact on those topics expanding the topic list where necessary. In regard to reproducible builds, Chris Lamb and I have been meeting regularly to discuss PureOS builds and their reproducibility. We've managed to create a automated scan of PureOS builds via diffoscope; http://45.33.86.90/pureos-8.0-images.html That page is created automatically each day from two builds done on that host. I believe that we'll be able to say that the PureOS ISO is reproducible when the builds are byte-for-byte identical. We hold some documentation here; https://tracker.pureos.net/w/development/reproducible_isos/ The next topic I'd like to talk about is the delta between PureOS and Debian. I mention this because I feel it fits into the Continuous Delivery topic above and is something that I've been talking to Matthias and Gunther about, albeit breifly. The reason to understand the delta between PureOS and Debian is to be able to calculate roughly the effort needed to move our currently Phone OS from Debian to PureOS. Hopefully being able to specify what is different between the two distros will enable us to create a PureOS image that matches the versions and/or the functionality of the work being done in the L5 team. I'll send a separate email regarding this topic as it is important and we may need to gather a number of people to participate. Work is ongoing on the PureOS Store. Rodolfo has been working recently on adaptive design and it looks good. More work to be done of course. I'm working on getting a round trip for Animatch and other flatpaks, like Lollypop. At this stage I'm looking to export the built flatpak to the server and see if I can't use the "install" button to install it on the devkit. More updates soon. We may have an additional colleague if the org chart is any indication, I think that will be a welcome addition. I'll let everyone know more as soon as I know more. Well, just a medium length email this time, if they get too long they tend to get a bit boring. There's still lots to write about but hopefullly another "Bits from PureOS" will be coming soon to an inbox near you. Cheers, Jeremiah

3 9

[Fwd: Suite name for security updates changing with Debian 11 "bullseye"]
by Jeremiah C. Foster 11 Jul '19

11 Jul '19

Speaking of suite name changes . . .

1 0

Re: [PureOS] "PureOS changes" notifications
by Chris Lamb 11 Jul '19

11 Jul '19

Matthias Klumpp wrote: > I didn't actually understand your mails as "I want Git notifications" > but rather as "I want notifications on archive uploads and other > archive actions", so I never actually directly asked our sysadmins for > the former. (Sure. The [sysadmin] ticket I filed has this and other details in so I felt no need to duplicate its contents here.) Best wishes, -- Chris Lamb https://puri.sm

2 1

2025

2024

2023

2022

2021

2020

2019

2018

PureOS-project