I’m Looking Forward To The Future Of CentOS Stream

Home » CentOS » I’m Looking Forward To The Future Of CentOS Stream
CentOS 42 Comments

Personally, I think that changing focus on CentOS Stream is going to make CentOS (and maybe even RHEL) better in the same way and for the same reasons that Fedora is a better distribution than Red Hat Linux was. CentOS Stream should fix the biggest problems that CentOS has had in the past, providing a reliable, free LTS distribution with community participation.

Having read the announcement, along with hundreds of reactions in blogs, forums, and mailing lists, I am of the opinion that all (or a reasonable approximation thereof) of the vocal concern is the result of the overloaded term “stable” as it applies to software distributions. If we imagine a spectrum of individuals in which one end of the spectrum is individuals whose primary occupation is in release engineering for software distributions and the other end is individuals who primarily consume software distributions, I would expect that individuals on one end to mostly use the term “stable” in the sense of compatibility and semantic versioning, and the other end to use the same term in the sense of having fewer bugs. The use of that word causes people at one end of the spectrum to infer a completely different message than people at the other end intend to communicate.

If we never use the ambiguous term “stable” and instead use the terms compatibility and reliability (the two common meanings of “stable” at different ends of the spectrum), I think that various aspects of CentOS
Stream will be better than CentOS, or the same as CentOS.

With respect to compatibility:

I think most developers are familiar with semantic versioning. Semantic versioning is used by some applications and libraries to indicate the nature and extent of changes. The version is presented numerically as Major.Minor.Revision. When new interfaces are added, the minor number is increased. Those changes don’t affect backward compatibility. When individual interfaces are changed or removed, the major number is increased. Those changes aren’t backward compatible. That allows consumers to infer that if an application is compatible with 8.1, then it is also compatible with 8.2 and later, but might not be compatible with 9.0.

Red Hat Enterprise Linux applies that concept to the entire software distribution, providing independent software vendors a convenient means of communicating their compatibility. If they’ve tested their application on RHEL 8.2, then any RHEL 8 host patched to at least that release is expected to run the application. Moreover, Red Hat will continue to publish security patches to each given minor release’s channel, allowing consumers to “pin” a host to a minor release. Those hosts will not receive feature updates, but will mitigate vulnerabilities.

CentOS Stream isn’t going to feature minor releases, and isn’t going to provide semantic versioning of the distribution. The same application that the vendor has validated on RHEL 8.2 will run on a fully patched CentOS Stream 8 host, but might not run on a host that isn’t fully patched. On the surface, it appears that CentOS users will lose the convenience provided by semantic versioning. I would simply point out that the CentOS developers have never supported running CentOS in any state other than fully patched. They don’t publish security information in the package repositories, and they don’t support any means of pinning a host to a minor release.

For practical purposes, CentOS Stream will need to be fully patched for compatibility purposes, just like CentOS is, and will be equally suited for production purposes.

To put a really find point on that: Semantic versioning is only meaningful for hosts that are not fully patched. A fully patched host is expected to be compatible with any application validated for that major release.

With respect to reliability:

Many of the people concerned about the change in focus refer to CentOS
Stream as a “beta” for RHEL. That is not how Red Hat or the CentOS
maintainers describe CentOS Stream( anywhere that I’ve seen), and I
think it ignores most of the development, testing, and distribution pipeline.

At the risk of oversimplifying that pipeline a whole lot, in the future Free Software will pass through several stages on the way to RHEL:

Stage 1: (Software Development) The majority of development and testing is done in individual upstream projects, outside of Red Hat.

Stage 2: (Release Development, aka Rawhide) The initial work to build and integrate individual packages with the rest of the software distribution is done in what is essentially a development branch of the software distribution.

Stage 3: (Stable[1], aka Fedora) Packages that have passed through review and QA are published for general use. There is no minor release, as major releases occur every 6 months and are supported for only 13
months, anyway. Compatibility is maintained by prohibiting significant version changes for applications and libraries “whenever possible.”
Users expect no new features during a release, but no breaking changes either.

Stage 4: (Long Term Support, aka CentOS Stream) Packages that CentOS Stream includes from Fedora have proven reliable in a variety of workloads managed by many users of Fedora. These packages are expected to be very reliable as a result of testing by their developers, by package maintainers, and by real-world users. They are included in CentOS Stream when they are ready.

Stage 5: (Long Term Support with Semantic Versioning of the OS, aka Red Hat Enterprise Linux) Packages that RHEL includes from CentOS
Stream have a similar level of QA, but package updates that introduce new features and interfaces are accepted only once every six months when Red Hat publishes a minor release.

There’s a lot of concern that CentOS Stream will offer less reliability than RHEL, but there’s currently no reason to believe that will be true.
There is no evidence that the minor releases that are part of the RHEL
lifecycle improve reliability, and as far as I know that’s not the reason they’re used. Their function is to offer semantic versioning of the OS.

CentOS Stream will probably offer the same level of compatibility and reliability that CentOS does today, and should be equally appropriate or inappropriate for production use in the future as CentOS is now in that respect. And that brings me to the one aspect where I think CentOS
Stream will resolve the one major, glaring problem that CentOS has today, that most users ignore: Security.

With respect to security:

Today, CentOS is a release stage after Stage 5 described above. The CentOS maintainers begin work on a minor release after that release is available to RHEL consumers, and the process of rebuilding those packages is often very time consuming. CentOS maintainers have to reverse-engineer the exact order in which packages are built, with the exact set of installed and available packages in the build environment in order to ensure that the resulting package actually uses the same interfaces that RHEL’s packages do. All packages require that ordering and build environment matching, but most packages are published in small sets and ordering is much easier to identify than it is when they are published in a large batch.

As a result, security updates can’t be published for CentOS while the maintainers are rebuilding the minor release, because the build dependencies aren’t available yet. Those windows occur every six months, and are typically a month or more in length. [2]

Today, CentOS users accept the risk that for roughly two months out of the year, their systems may have known vulnerabilities with no patch to remediate the problem. Personally, I think that’s a huge risk that needs to be weighed against the costs of RHEL licenses whenever CentOS
is used in production.

The good news is that CentOS Stream looks like it won’t have that problem. CentOS Stream updates still won’t be prepared early, while vulnerability details are embargoed, but there aren’t any windows in which CentOS Stream can’t immediately begin work on preparing updates once the embargo ends. That means that CentOS Stream will be as secure as CentOS is today in between minor updates, and significantly more secure than CentOS is today while its maintainers prepare minor releases.

The trade-off of significantly improving security in exchange for giving up semantic versioning of the OS is a huge improvement for production purposes.

In addition, the announcement of the change in focus indicates that Fedora maintainers should have much better visibility into work going on within the RHEL release engineering process, and more opportunity to participate in that work, and I look forward to that. CentOS’s big security problem has always been compounded by the lack of any external visibility into the process.

Based on the information available today, I expect CentOS to be a very reliable, reasonably secure distribution of GNU/Linux with Long Term Support. And judging by Red Hat’s mention that Facebook’s internal groups either are already using an internally curated OS built from CentOS Stream, or will be using it soon, I think I’m not alone in believing that.

1: I’m using the term “stable” here because I expect it to have both common meanings: compatibility isn’t broken within a release, and the software is expected to be reliable.

2: https://en.wikipedia.org/wiki/CentOS#CentOS_version_8

42 thoughts on - I’m Looking Forward To The Future Of CentOS Stream

  • […]

    Allow me to disagree. We both trust Chris Wright’s words, don’t we? CTO
    won’t lie. Citing him:

    “To be exact, CentOS Stream is an upstream development platform for ecosystem developers. It will be updated several times a day. This is not a production operating system.”

    […]

    I do not wish to argue with all your statements. Mostly they look reasonable. However, there’s an unpredictable variable in this equation, namely RH.

    The major problem here is the breach of trust. A year ago RH’s CTO is singing charming songs that CentOS won’t go, now we see an abrupt direction change. This time, CTO keeps silent (I wonder why).

    Also, there’s change in patterns. With CentOS, I reduce updates to minimal ones. That’s significant: the management doesn’t like the idea that updates can be applied daily, and glitches may happen at any moment. The management prefers the known devil.

    With current CentOS life cycle the number of upgrades is typically small. And even if I reduce the number of CentOS Stream upgrades to minimal one, the base advantage of CentOS is lost: predictability. At any given moment I could be sure that it has the same quirks and bugs the matching RHEL has.

    CentOS Stream has its advantages and use cases. The problem is, no one cared to estimate what use cases of majority of current CentOS users are.

    Damn, RH could at least bring formal apologies for changing the promised lifecycle. Instead we see the typical marketing blah-blah-blah of how that would benefit everyone. Nothing shows better the actual RH attitude towards the CentOS community.

    —
    Sincerely,

    Konstantin Boyandin system administrator (ProWide Labs Ltd. – IPHost Network Monitor)

  • So, like Fedora?  People run servers on Fedora now, and I think that’s fine.

    Does he say that CentOS is a production operating system?

    As far as I know, Red Hat has never endorsed running CentOS in production, so I don’t understand why it’s significant that they also don’t endorse running CentOS Stream in production.

    It’s really difficult for me to look at a distribution that just stops getting updates for 4-6 weeks, twice a year, and use the word
    “predictable” to describe it.

    My first reaction to the announcement was pretty negative, too. But when I stepped back and looked at the current situation *real* honestly, I
    had to admit that CentOS just doesn’t offer any of the things that people are complaining about losing.

    And I hope that the CentOS maintainers don’t interpret that as criticism, because it isn’t intended to be.  They’ve always maintained that if you need updates/patches in a timely manner, then you should be paying Red Hat for RHEL.  I agreed with them then, and I still do.

  • On a production server, where no surprises are expected? That may be. People often act very, so to say, strangely.

    I am telling about other people. I doubt those actively running Fedora on production systems do participate in these threads.

    Is RHEL itself suitable for running on production servers?

    If not, my argument is weak. If yes, then CentOS, bug-to-bug compatible, is suitable, too.

    RH won’t ever endorse running CentOS (more generally, anything free of charge) for obvious reasons, so I don’t care about their opinion on this subject.

    Well, it’s not at all difficult for me. Tastes differ.

    My primary objection is breach of trust. RH shouldn’t have lied at least to CentOS community.

    Other bug-to-bug compatible RHEL clones will replace the CentOS, so this is the part I am less worried about. If someone is happy with CentOS
    Stream, that’s fine. I am not, but that’s (not only) my problem.

    —
    Sincerely,

    Konstantin Boyandin system administrator (ProWide Labs Ltd. – IPHost Network Monitor)

  • Am 11.12.20 um 09:23 schrieb Gordon Messmer:

    To be honest, such argumentation is pointless because anyone knowns that grey shades in beetween exits. CentOS Linux was more on the bright side, then CentOS Stream will be (in terms of current usage scenarios).

    I think a main point(s) at this all is the timing (communication)!

    —
    Leon

  • Il 11/12/20 10:24, Konstantin Boyandin via CentOS ha scritto:

    This.

    CentOS Stream is NOT a REPLACEMENT of CentOS, it is a different
    “product” used as a rhel preview (and testing platform for next rhel releases [minor/major]). This is a simple direction change for a corporation. I accept this without any problem, they have not any legal duty with CentOS community. Ethically, wow…they should ask itself WTF
    did they done. But no problem..many of us have imagined this since IBM
    ops (also if this is a CentOS board decision), today it is reality. Really there is nothing new for me (This is why I started to find alternatives for my case usage since 8 was released waiting the switch to see if direction was good)

    The days you install CentOS as server distro for stability and compatibility are gone. I always used CentOS and not rhel because I
    don’t need support. I don’t need CentOS Stream so  I will not use it. CentOS 8, with its all defects, was enough for me and I think I will not use rhel until forced. So for me (and many) there is not an alternative then to change ship and switch to Debian/Ubuntu LTS that are not bad systems. Intended, there are other alternatives like SUSE/OpenSUSE, OL
    and other…

    I read many times that Debian/Ubuntu LTS are not CentOS/rhel, this is true (they are different products) but please, stop saying this, them are not shitty distro..but when I read that many users use fedora as server distro I laugh.

    I’m not in enterprise so I don’t need this type of “support” and can change the distro without any problem but I will stay away from CentOS/RH products due to 0 trust in them.

    For the past years, for CentOS 6,7 and 8 thank you Johnny, Rich and all other maintainers. You did a great job.

  • I started intensively using Debian, Ubuntu and Kali 3+ years ago. So far, they are solid enough (talking of LTS) and quite reliable, as CentOS 6 was.

    I agree, I think I leave this thread on a good line. The maintainers did great job (and I hope they will keep doing it). But “tempora mutantur, and nos mutamus in illis”. Good luck to all of us.

    —
    Sincerely,

    Konstantin Boyandin system administrator (ProWide Labs Ltd. – IPHost Network Monitor)

  • Yeah, I too think this is important context. I don’t think you’ll ever find anyone from the business side ever even suggesting that they think CentOS
    Linux, the rebuild, was *ever* something Red Hat recommended to run in production.

  • In early 2000 I don’t think you’ll ever find anyone from the business side ever even suggesting that they think Linux (in general) was *ever*
    something vendors recommended to run in production… but here we are now
    ;-)
    And bye bye to AIX, HP-UX, Sun Solaris, Digital Unix, Tru64 Unix (only to mention the OSes I had been involved in at different levels); and I would like to notice that each one of those had its strong points anyway and let me learn much.

    Business men joked with me when I asked about considering Linux in some context and they replied “Eh, Linus? The cartoon guy?”

    So what?

    Please leave business to business

  • El vie, 11 dic 2020 a las 12:33, Matthew Miller ()
    escribió:

    With all due respect. Please don’t mix topics. Everyone is grateful about the effort of CentOS developers in the last 16
    years. The problem is not only the announce and decision of RH, but also the unfortunate PR of CentOS, how could you ask trust and confidence with something like that:

    concerned that CentOS Stream will not meet your needs, we encourage you to contact Red Hat about options.

    Please don’t blame the rest of the world for things that you wrote.

    Regards

    —
    —
    Sergio Belkin LPIC-2 Certified – http://www.lpi.org

  • I’ll repeat what I said earlier, CentOS has never offered the things people are complaining about losing.  They’ve never asked for your trust and confidence.  Both Red Hat and the CentOS maintainers have always referred users who needed “trust and confidence” to RHEL.

  • I’m happy you made this point. Yes, CentOS is asssumed to be as
    “stable”  as the release it’s based on, but there are changes.

    I think it’s good to keep this in mind and consider an actual RH license if 100% stability and compatibility are the goals.

    —
    “DO or DO NOT; there is no try.”
    — Yoda

    Kay

  • Exactly.  That’s why it’s so weird that those people, today, think that CentOS Stream won’t be usable, based on their interpretation of the official statements from Red Hat.  Red Hat’s statements weren’t taken into consideration before, but now they’re a sign of doom?

    I’m going to stop you there, because the CentOS maintainers kept that process out of public visibility long before Red Hat was ever involved. 
    If you think users should know more about the process, then you are pointing fingers at the *wrong* people.

    I don’t want this to become a flame war.  So rather than pointing fingers, let’s focus on the fact that CentOS Stream promises to be developed in the open, resolving the problem that you’re describing.

    Red Hat is fixing the thing you’re complaining about.

    Red Hat is giving us the thing that has been requested more often, by more people, than any other change in CentOS, and the result is that the press is full of stories about users being angry, because five people on the mailing lists sent a lot of messages.  (About half of the traffic in the threads on CentOS and CentOS-devel comes from five people, and various people replying to them.)

    You’re misinformed.  Kernels are still built from SRPM, but the archive used is no longer an upstream release with a series of patches.

    The reason for the change is not insidious.  It’s unfortunate that the pristine source + patches can’t be maintained, I agree, but speaking as a developer: maintaining hundreds of patches that touch intersecting files and rebasing them all when earlier patches are updated is an incredibly difficult and time consuming task.  And, if I remember correctly, applying all of those patches took almost as long as building the kernel.  If it takes that long to just prepare the source code, that’s a major hit to productivity when a developer needs to work on the code or build the SRPM to validate changes.

    And, ultimately, there’s very little value in shipping those patches when the vast majority of them are already in the current version of the upstream kernel, and they’re merely backported to the older release that Red Hat supports.  It’s an entirely different story when distributions are shipping patches that they don’t push upstream, but that’s not generally what you see with the kernel package.

  • Who exactly said “doom”?

    CentOS Stream won’t match corresponding stable RHEL version, that’s all. While CentOS was matching it, it was stable enough to use it safely on production.

    Now that it becomes constant beta, it might be considered less stable, all the compatibility arguments have been uttered many a time.

    Doom or no doom, everyone decides for oneself.

    The primary problem is breach of trust. All the other consequences are mostly technical.

    —
    Sincerely,

    Konstantin Boyandin system administrator (ProWide Labs Ltd. – IPHost Network Monitor)

  • Le 11/12/2020 à 18:25, Gordon Messmer a écrit :

    For the last 16 years, the explicit scope of the CentOS project has been to rebuild RHEL “bug by bug”. No more no less. A fact that has been stressed repeatedly by the maintainers on this list. So admins all over the world trusted this.

    Words do have a meaning.

    —
    Microlinux – Solutions informatiques durables
    7, place de l’église – 30730 Montpezat Site : https://www.microlinux.fr Blog : https://blog.microlinux.fr Mail : info@microlinux.fr Tél. : 04 66 63 10 32
    Mob. : 06 51 80 12 12

  • +1

    And what is worse: RH are pushing their users to their competitors
    (read: OL).

    RH are pulling their own eyes out… It’s a shame.

    And all that because they decided to stop supporting a quite extensive worldwide amount of orgs and sysadmins who need a safe and dependable production OS that does not cost a fortune, although many of those at some point in time might become RH support customers!

    Now RH earned discontent and distrust from a large part of their FOSS
    community.

    Such a sad end for CentOS… (In fact it is an end indeed.)

    OL, Rocky Linux (when fully established) and other such projects
    (mentioned in various threads) will gain large parts of this extensive group. Some many end up in using CentOS Stream, but the core part of this vibrant community will probably lost by RH and the good old CentOS
    group (now in RH). Time will show.

    That’s a pity, because a large and conscious part of this community indeed has some affinity to Karanbir et al, greatly respecting their history and efforts…

    RH (& CentOS) still has some small window of opportunity to announce full support of CentOS 8 to its EOL.

    Cheers, Nick

  • Do not turn this upside down. Many obviously smarter then me saw this coming. Us more gullible believed when corporation told us nothing will change, while they took control of entire project. Death by thousand cuts is always more preferable by corporations, less bad PR. And they are not sign of doom but death, but of only this particular clone, others will take it’s mantle. The reason I an agitated is I
    believed and with all my hearth supported this project, and now is owned by heartless profit-driven corporation….

    Don’t be silly. They wanted to control the process, and to prevent anyone else from cutting into their process. You should read their manifesto for stream, only RH employees will be able to change that code, others will only be able to report bugs/issues and to sit and watch what RH does with them. Only real difference with stream is for those few hundred developers that are developing software that runs on RHEL so their code can be deployed as soon as new point release is launched.

    And even that RH does more for it’s own gain, so that (opensurce?)
    projects/software important to RH can be deployed without delay, lest users ditch RH and go elsewhere.

    When people are happy with something they do not voice their content on the mailing list, mailing list is only to voice your discontent. You heard about “silent majority”, right? Ever though why it is called that?

    Not misinformed, I was part of that discussion 9 years ago, but my memory was little rusty, but I was part of that discussion and It did not bother be to track down one of the comments about kernel tarballs being published as monolith code vs trackable patches, to make problems for builders of RHEL clones:

    “There is the kernel tarball issue. That’s understandable, but real. Red Hat is publishing their kernels as tarballs, rather than as the vanilla tarball with a list of ordered patches against it. This makes layering in, or out, specific patches much more awkward. Was Oracle being any better about this? Does anyone have actual pointers to their SRPM’s?”
    (https://lists.CentOS.org/pipermail/CentOS-devel/2011-March/063442.html)

    Point is that RH DID slow down build of clones due to this change, and for several months. I can not say about intent, but in hindsight it fits the pattern, I even thought the same back then (mail in link was direct reply to my mail).

    —
    Ljubomir Ljubojevic
    (Love is in the Air)
    PL Computers Serbia, Europe

    StarOS, Mikrotik and CentOS/RHEL/Linux consultant

  • As one of those “five people” I assure you, this is not just a few angry voices. If you, or anyone at Red Hat believe this is the case, you are very sadly mistaken.

    Here is the problem: When IBM took over Red Hat, and hence CentOS, these words were posted on the CentOS Blog:

    “What does this mean for Red Hat’s contributions to the CentOS project?

    In short, nothing.

    Red Hat always has and will continue to be a champion for open source and projects like CentOS. IBM is committed to Red Hat’s independence and role in open source software communities so that we can continue this work without interruption or changes.

    Our mission, governance, and objectives remain the same. We will continue to execute the existing project roadmap.”

    This was *last year*. (CF
    https://blog.CentOS.org/2019/07/ibm-red-hat-and-CentOS/) Note the last sentence. The roadmap then had CentOS 8 supported through May 2029.

    The simple fact is Red Hat reneged on a promise that hordes of us believed and made a lot of plans on. It is now going to be very expensive, and stress inducing to have to completely rethink everything we have done, and are doing.

    You damn right we are angry.

    And there’s *a lot* more than five of us.

    —

    *Matt Phelps*

    *Information Technology Specialist, Systems Administrator*

    (Computation Facility, Smithsonian Astrophysical Observatory)

    Center for Astrophysics | Harvard & Smithsonian

    60 Garden Street | MS 39 | Cambridge, MA 02138
    email: mphelps@cfa.harvard.edu

    cfa.harvard.edu | Facebook <http://cfa.harvard.edu/facebook> | Twitter
    <http://cfa.harvard.edu/twitter> | YouTube <http://cfa.harvard.edu/youtube>
    | Newsletter <http://cfa.harvard.edu/newsletter>

  • Just one of those groups energised from this decision is Rocky Linux. There are 4,606 people on their Slack right now, which did not even exist a week ago.

    Yeah, it’s more than five.

  • So, the majority of users are silent, because they’re happy? Cool.

    Your memory is still rusty.  Early accusations were that this would impact developers (such as Oracle) who were adding additional patches to the kernel, or other maintenance.  It never impacted “clones” like CentOS at all.

  • And you’re incorrect; CentOS publishes a CentOSplus kernel that was impacted to some extent by the kernel tarball changes I believe.

    There are no absolutes in life, saying “never impacted” is almost surely untrue for at least one group of people.

    John

  • IIRC, one of the reasons cited that CentOS „merged“ with RedHat back then was that a lot of people were using CentOS, but there wasn’t enough money generated to pay the developers.

    A lot of them were basically working for free.

    That is never sustainable. At least not for a long time.

    It’s also not often the case that you can split this kind of work into a thousand work-packages and have everybody just work 1/2 hour a day on it.

  • The workflow is very different. For a primary distribution, updates to different packages happen at different times. Contributors can do that work when they have the time.

    For a rebuild, work must happen as fast as possible after RHEL has released an update. Much harder for volunteers to contribute to.

    There are other support roles that volunteers can hopefully do, but the core mission doesn’t really align well with that.

    —
    Orion Poplawski Manager of NWRA Technical Systems 720-772-5637
    NWRA, Boulder/CoRA Office FAX: 303-415-9702
    3380 Mitchell Lane orion@nwra.com Boulder, CO 80301 https://www.nwra.com/

  • HAHAHAHAHA, what a wonderful imaginary world you live in :-D
    530 negative commends on the blog, 7.800 signed the petition, 100+
    negative mails on the CentOS mailing list, and at least 200 negative comments only in CentOS Facebook group, not counting comments in other
    20-30 groups.

    So yeah, lets go with only 5 complainers :-D

    So long, there is nothing more to be said on this topic, except that I
    will be soon leaving Facebook group admin team.

    —
    Ljubomir Ljubojevic
    (Love is in the Air)
    PL Computers Serbia, Europe

    StarOS, Mikrotik and CentOS/RHEL/Linux consultant

  • It’s to be expected. What I find surprising is the shilling for RH and this decision that I am seeing. Oh well, such is life.

  • I’m just trying to determine whether you were making the argument you intended to, because you are literally suggesting that the majority is silent, and the people who are silent are the ones that are happy with something.

  • Silence doesn’t confer anything. People can be displeased, cautiously optimistic, and silent, all at the same time.

  • OT excuse to rant, because I get so mad at this.

    Remember when slack first came out, and said, Oh, we’re gonna be compatible with your irc stuff. At that point, you could use it with irssi and weechat. Then they changed it, you could still sorta use it with irssi, but it was cumbersome. Thank goodness, someone made a plugin to work for weechat which still works at the moment, though, as they’ve done something to tokens, it’s harder than it used to be, and no doubt, they’ll eventually eliminate it so that you have to use the slow, high resource, web app. Talk about embrace, engulf, extinguish, they are a role model for it.

    OK, sorry. But, I feel so much better now.

    —
    Scott Robbins PGP keyID EB3467D6
    ( 1B48 077D 66F6 9DB0 FDC2 A409 FA54 EB34 67D6 )
    gpg –keyserver pgp.mit.edu –recv-keys EB3467D6

  • Since the release of CentOS 8, I have been moving my stuff over to Fedora. The combination of modularity and missing -devel packages make developing and building software on EL8 impractical. As a result, EL8 is poor choice for deploying custom software.

    Fedora has other advantages.

    1. More changes. Bugs are likely to be addressed sooner and I find addressing small changes one at a time is more manageable than many big changes all at once. Having a good test suite helps. Our sysadmin at work spent most of 2020 doing the upgrade from CentOS 6 to
    8. I like to think there were better uses of his time.

    2. More software. Fedora packages much more software than CentOS. Even adding in EPEL leaves a big gap and EPEL is Fedora, not RHEL. I
    spend less time building dependencies and more time adding value.

    3. Easy licensing. Fedora may be used anywhere for anything. We have a RHEL license at work, but I don’t use it because I do not want the headache of tracking where and how it is deployed. I’ve wasted too many days fighting licensing and compliance issues to want to ever do it again. It is huge advantage for Free Software.

    Your needs may differ, but it is not an insane choice, so please stop insulting us.

    Jim

    P.S. It seems to me that compared to Fedora, Stream has the disadvantages of RHEL but not the advantages. It’s not clear to me how Stream will be an improvement.

  • While I don’t use Fedora as a production server, I will say that ever since Adam Williamson joined them, the QA has been quite good. I used to worry about an update breaking things. Now I use it as my go to Linux on laptops, and have successfully upgraded, using their instructions for CLI updates, with no problems.

    I do use openbox and dwm (which I install from source) rather than Gnome, which might have something to do with my painless updates.

    Not to say it’s a good server OS (though not saying it isn’t, I don’t have enough knowledge of it in that situation to say), but it’s not the always on the edge of breaking that it used to be.

  • The main issue against using Fedora in production environments is the short lifecycle. Forcing an upgrade, and all the associated testing, auditing, etc. of the base version every year or so is not tenable for most organizations.

  • on.  The best protest is carried by moving feet.

    I ditched CentOS for Uuntu back in 2016 and haven’t looked back. I only have one last machine still running CentOS – the firewall. When that EOL’s, mine will be a 100% Ubuntu shop.  But, not knowing what would happen to Canonical in the future, I’ve also started toying with Arch and FreeBSD…

  • While I agree with your entire post, Gordon, this specific point I think is the most critical. In our environment, we already need to look to the Continuous Release repos to get critical security updates during this embargo period. I’m betting Stream will be no less well vetted than the CR
    repos, and likely will be better. In any case, the burden for tracking down the updates will be much less with Stream: we’ll just get the packages through our normal channels, rather than going on a hunt through CVEs and Bugzilla, then temporarily enabling the CR repos for just the period of time when we need to get the updates before disabling them again.

  • No, not at all like Debian.  Debian doesn’t have to try to match the unattainable goal of 100% binary compatibility with an upstream source. 
    I’ve seen a small part of this first-hand, and deducing the build order to gain binary compatibility is the one thing that can single-thread the build process quicker than anything else.  RHEL doesn’t even have the same need; an RHEL rebuild that didn’t have the goal to be bug-compatible near the 100% level doesn’t, either, and can be built by a lot of people.

    Try it yourself: go back to CentOS 5.5 and attempt to rebuild the released sources for 5.6 and get a binary compatible build.  I’ve done it myself for IA64; it was a pain.

    All of the upstream distributions, Debian, Fedora, etc, have a lot of latitude that CentOS never has enjoyed.

  • Given that rebuilds /had/ been taking longer and longer lengths of time, it was clearly hop(e)xpected that bringing CentOS into the RedHat fold in an official partnership in 2014 would allow for direct access to the RHEL engineers and a path toward making the CentOS Project’s rebuilds easier. Indeed, although it’s been six years, moving to integrating EL/ELN/CS and RHEL in some way in the pipeline is precisely what had been desired… BUT, defining the problem out of existence while also defining “CentOS Linux” out of existence helps absolutely no one (on the outside, at least). And cutting the support period for 8 in half is so disruptive as to outweigh the progress in other areas.

    -jc