<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">Hi guys,<br>
<br>
Thank you both for your question about the worker requirement, it
is a great question to ask.<br>
<br>
To me, the biggest argument for such a requirement is that the bus
factor on these ports is really low, usually just one person.
Making sure we always have a worker image helps to reduce the risk
of dead ports by ensuring that at least the build process for all
of the “official” (i.e. in our tree) ports is documented,
reproducible, and (strongly preferably) updatable, so binaries can
be created by automation or any other team member instead of
relying on a single person. There are at least four examples I can
point to that indicate a real need for this kind of strict
requirement for any of our official ports:<br>
<br>
1. Code for a new “official” 3DS port was added to our tree in
2016, and then we never actually released for this platform
because the porter disappeared before any release could be made;<br>
2. When the usual Windows porter was lost around the 1.9 release
time, there was apparently a non-trivial amount of time &
effort needed to get a working build system up and running again,
and then I essentially had to re-do this work yet again to update
Buildbot, which just wastes the core team’s limited time;<br>
3. Porters do stuff which is totally undocumented, like the Nero
images which are currently being created for the Dreamcast port.
If the porter disappears, it’s unclear if anybody is going to be
able to reproduce this. I was only even able to create a Dreamcast
worker because digitall sent me some scripts that they’d used to
create a cross-compiler for this platform in the past;<br>
3. We *still* don’t have any binaries available of ScummVM 2.0 for
Ubuntu/Debian, iOS, PSP, or Wii, even though these are our top
platforms after Windows, macOS, and Android. We wouldn’t even have
an Android package if I hadn’t already done the Buildbot work, and
it was unclear for a time whether we would have a Windows package
ready to go for 2.0 either.<br>
<br>
If we don’t have a policy like this, these exact same issues will
recur eventually with every newly accepted port. It does no good
for us to say we “officially” support some platform when we can’t
actually make a release binary for it, and it is damaging to the
ScummVM brand and to our users when we can’t even get our most
popular ports’ binaries ready by release day. This change won’t
fix all the release process issues (this is probably impossible),
but it does at least get us to the point where we don’t ever have
to worry if we’re even going to be able to actually *release*
something at release time.<br>
<br>
A secondary reason for such a requirement is build security.
There’s a small but sadly non-zero risk that bad code could be
introduced by a porter. Since we are giving these binaries our
stamp of approval by calling them “official”, and we have no good
way for the core team to actually vet them, we can partially
mitigate the risk of a bad actor introducing something ugly by
making the process more transparent and robust against subversion.
In the current global environment I feel that it is important to
try to reduce these kinds of secondary risks where possible, and
having a transparent and reproducible build system which fixes the
above issues will also help to ensure the integrity of our
releases.<br>
<br>
I understand that there are some potential downsides to this
proposal, I just can’t think of another more reasonable approach
that won’t leave us right back where we are today. If you can, let
me know of the alternative. We could grandfather some of the
manual ports, but that still doesn’t really solve the bus factor
problem. Also, it’s important to acknowledge that this policy
change would only move unbuildable ports outside our “official”
realm, so we do not have a direct hand in managing them or
maintaining their code/tracking their bugs in our bug tracker any
longer; it won’t prevent them from existing, or even from being
linked on our downloads page, or from being pulled back in again
in the future if someone makes the effort later on to create a
functioning worker image.<br>
<br>
With regards to the specific technical questions: For anything
that can’t be cross-compiled from Linux for some reason, there is
always the nuclear option of running qemu in a worker. With
regards to macOS, I don’t think there is any particular reason why
those releases need to come in a dmg any more than they need to be
delivered as pkg/mpkg installers, though there is a project called
libdmg-hfsplus which can also create dmgs.<br>
<br>
Best,<br>
<br>
On 2017-12-18 16:54, Thierry Crozat wrote:<br>
</div>
<blockquote type="cite"
cite="mid:B9DD97A5-E9DD-44E5-864D-C4D66A56DF7C@scummvm.org">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<div class="">Hi Colin,</div>
<div class=""><br class="">
</div>
<div class="">I was about to write almost the same thing.</div>
<div class=""><br class="">
</div>
<div class="">This looks like a great idea overall, and thank you
for all your work on this.</div>
<div class=""><br class="">
</div>
<div class="">I am however not sure having a buildbot worker for
all ports should be a requirement. Can you give your reasoning
for this suggestion?</div>
<div class="">I am worried this would be a death sentence for a
few ports (you mentioned difficulties with Samsung TV and OS/2,
and I have also had difficulties getting mac PPC compilation to
work with anything remotely modern). There is probably not many
users for those, but I am a bit uncomfortable with killing them
if we have porters willing to build those manually. I am a bit
less worried for ports on more modern platforms as
cross-compilation on Linux seems to be usually well supported.
Packaging might be an issue though (I don't know of any way to
create a dmg image on Linux for example, which might force us to
use zip instead for macOS packages).</div>
<div class=""><br class="">
</div>
<div class="">Thierry</div>
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On 18 Dec 2017, at 22:21, Eugene Sandulenko <<a
href="mailto:sev@scummvm.org" class=""
moz-do-not-send="true">sev@scummvm.org</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" style="font-family: Helvetica; font-size:
12px; font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; text-align:
start; text-indent: 0px; text-transform: none;
white-space: normal; word-spacing: 0px;
-webkit-text-stroke-width: 0px;" class="">
<div class="gmail_extra">
<div class="gmail_quote">On 17 December 2017 at 15:36,
Willem Jan Palenstijn<span
class="Apple-converted-space"> </span><span
dir="ltr" class=""><<a
href="mailto:wjp@usecode.org" target="_blank"
class="" moz-do-not-send="true">wjp@usecode.org</a>></span><span
class="Apple-converted-space"> </span>wrote:<br
class="">
<blockquote class="gmail_quote" style="margin: 0px 0px
0px 0.8ex; border-left-width: 1px;
border-left-style: solid; border-left-color:
rgb(204, 204, 204); padding-left: 1ex;"><span
class="">On Sat, Dec 16, 2017 at 09:08:15PM -0600,
Colin Snover wrote:<br class="">
> Team,<br class="">
><br class="">
> One of my goals for the next release of
ScummVM is to automate the<br class="">
> process of building and generating release
packages. My hope is that<br class="">
> this will allow us to have a faster release
cycle, more consistent<br class="">
> release quality, and free up manpower to
focus on the many parts of<br class="">
> ScummVM which cannot be so easily automated
(like adding more engines! :)).<br class="">
><br class="">
> The first two major parts of this project are
nearly complete: the<br class="">
> upgrade of Buildbot, and the introduction of
infrastructure management<br class="">
> using Ansible.<br class="">
<br class="">
</span>Hi Colin,<br class="">
<br class="">
Let me start by saying: Thank you! This is great
work, and will really help<br class="">
keep ScummVM portable and maintainable. I'm very
enthusiastic about these<br class="">
changes.</blockquote>
<div class=""><br class="">
</div>
<div class="">In short, I also like the idea, and I
see no problem with granting you the needed
permissions.</div>
<div class=""><br class="">
</div>
<div class="">The only thing which I would like to
discuss, is the requirements for every port to have
a buildbot instance. I have no real data, but I
would be not surprised that in some cases some
toolchains will not be easy to install on Linux
environment. And of course, I could be wrong in this
regard.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">Eugene </div>
</div>
</div>
</div>
<span style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; text-align:
start; text-indent: 0px; text-transform: none;
white-space: normal; word-spacing: 0px;
-webkit-text-stroke-width: 0px; float: none; display:
inline !important;" class="">_______________________________________________</span><br
style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; text-align:
start; text-indent: 0px; text-transform: none;
white-space: normal; word-spacing: 0px;
-webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; text-align:
start; text-indent: 0px; text-transform: none;
white-space: normal; word-spacing: 0px;
-webkit-text-stroke-width: 0px; float: none; display:
inline !important;" class="">Scummvm-devel mailing list</span><br
style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; text-align:
start; text-indent: 0px; text-transform: none;
white-space: normal; word-spacing: 0px;
-webkit-text-stroke-width: 0px;" class="">
<a href="mailto:Scummvm-devel@lists.scummvm.org"
style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-size-adjust: auto;
-webkit-text-stroke-width: 0px;" class=""
moz-do-not-send="true">Scummvm-devel@lists.scummvm.org</a><br
style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; text-align:
start; text-indent: 0px; text-transform: none;
white-space: normal; word-spacing: 0px;
-webkit-text-stroke-width: 0px;" class="">
<a href="http://lists.scummvm.org/listinfo/scummvm-devel"
style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-size-adjust: auto;
-webkit-text-stroke-width: 0px;" class=""
moz-do-not-send="true">http://lists.scummvm.org/listinfo/scummvm-devel</a></div>
</blockquote>
</div>
<br class="">
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Scummvm-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Scummvm-devel@lists.scummvm.org">Scummvm-devel@lists.scummvm.org</a>
<a class="moz-txt-link-freetext" href="http://lists.scummvm.org/listinfo/scummvm-devel">http://lists.scummvm.org/listinfo/scummvm-devel</a>
</pre>
</blockquote>
<p><br>
</p>
<pre class="moz-signature" cols="72">--
Colin Snover
<a class="moz-txt-link-freetext" href="https://zetafleet.com">https://zetafleet.com</a></pre>
</body>
</html>