Blog Article About The State Of CentOS

Home » CentOS » Blog Article About The State Of CentOS
CentOS 56 Comments

Hi,

I just read this blog article from austrian Linux expert Michael Kofler. For those among you who don’t know the guy, he’s my home country’s number one Linux expert (known as “der Kofler”) and most notably the author of a series of excellent books about Linux over the last 25 years.

https://kofler.info/CentOS-8-wertlose-langzeitunterstuetzung/

Disclaimer : I’ve been a CentOS user (and fan) since 4.x, I’m using it on all my servers, and yes, I know the difference between upstream RHEL and CentOS.

The article is in german, but the statistics graph is eloquent enough for the non-german-speaking users. It focuses on updates for CentOS 8, and more exactly the extended periods of time where there have been no updates available.

The author’s theory (“unspoken truth”): while it’s a positive thing that Red Hat is sponsoring CentOS, the amount of sponsoring is just insufficient enough so that the product is “starved to death” by Red Hat (e. g. IBM) to encourage users to move to RHEL.

The author’s conclusion is quite severe: in the current state of things, CentOS
8 is not recommendable for production as updates are lagging too much behind. While CentOS 7 may be usable, CentOS 8 has been “degraded to teaching and testing purposes”.

Still according to Mister Kofler, this “sorry state of things” will probably encourage users to move to Oracle Linux, the other big RHEL clone.

After some hesitation, I decided to share this on the mailing list. Since this raises some concerns, I’d be curious to have your take on this.

Cheers from the sunny South of France,

Niki


Microlinux – Solutions informatiques durables
7, place de l’église – 30730 Montpezat Site : https://www.microlinux.fr Blog : https://blog.microlinux.fr Mail : info@microlinux.fr Tél. : 04 66 63 10 32
Mob. : 06 51 80 12 12

56 thoughts on - Blog Article About The State Of CentOS

  • Il 17/06/20 09:16, Nicolas Kovacs ha scritto:

    Hi Niki, this is a sad thing but I’m not surprised by this and this is what I’m thinking about CentOS since 8 was released. The CentOS team does a huge work to release CentOS8, CentOS Stream and maintaining 7 and 6. Thank you so much for this CentOS Team. But since the core team is composed by 6(?) member is obvious that they can’t maintain all this versions without being in late and because it is supported (but maybe supported is the wrong word in the current case) by RH, if RH really interest about CentOS release they should put more effort on CentOS Development. This is not what is happening and since 8
    is released, it seems a try and buy distro. I walked through 6.5 to 7 and 8. In 6.5 I was new, in 7 I really appreciated CentOS but with 8 I noticed that it is not supported like CentOS 7.

    I use (like you) CentOS on all my server, some are anchored to the old stable and other on 8 (but not production). I liked it very much that I
    use it also on my workstation.

    About update blackout, this is a great problem for production system faced on Internet. SELinux can mitigate the risk (but also selinux got security fix) but this is always an issue. Many report that the gap between RHEL release and CentOS release is too much but I’m not interested in how much time CentOS releases get through the build process but I’m interested that, when a new CentOS release (major/minor)
    are on the way, all others releases (current stable inside) will get security updates during the build process and not left alone. Another minor issue about updates are relative announces on ml that works for C7
    but not for C8 (I read in a reddit AMA why this happens). This is annoying because I need to follow every RHBA to see what happened and why a package is updated or blindly install updates. This is another thing that is not so good for a server.

    If the theory “unspoken truth” is real I don’t like how RH is trying to encourage me to switch to RHEL and this is a bad way to do this.

    About Oracle as alternative. Oracle Linux is not an alternative to CentOS but for RHEL and if I will force to pay for enteprise system currently I will pay RHEL, not OL. Over this, OL is not the only enterprise distro that a “user” could choose. If support is needed there are SUSE (SLES) and Ubuntu. For who that don’t need support there are Debian, Ubuntu, OpenSUSE (I’m talking about the most used but you know that slackware,FreeBSD are in that list), so many alternatives are in place.

    I hope that things will be better but in case of “failure” I will evaluate other alternatives to CentOS for this “sorry state of things”.

    Thank you for sharing, I found it interesting.

    My 2 Cent

  • *snip

    +1

    Longtime user but with the current sturcture they have turned it (CentOS8)
    into really a non production release. I’ve always tried to use it on internal servers but these are small businesses (non-profit) that will pay for a windows server license vs a RHEL. Microsoft has actually expanded and opened it up even more. I really thought it was a good thing for RHEL to support CentOS since they had made the builds more difficult but it doesn’t appear that they do not want to support it too much. As it currently stands I could not or will not deploy CentOS8 in a internet facing environment.

    I continue to run 8 at home but do not see myself deploying it into production anywhere. I deeply appreciate all the hard work that goes into putting it out, Karanbir, Fabian, Johnny, Ralph and Tru and all the others put in to produce and support it. :)

    My hope is that RHEL changes the level of support they provide to CentOS
    but not confident it’s going to happen here’s hoping.

    ;)

  • I think if Red Hat really wanted to improve the situation, they could integrate the building of CentOS into the EL build system to produce both versions, RHEL and CentOS, at the same time. In the end 99% of the bits are the same anyway. If the delay of CentOS builds is really wanted by Red Hat, it would be nice of them to speak it out – and change the name to COS, because the ent is not true anymore :-)

    Up to now I thought the big delay with 8 is more an accident than wanted. Would be nice to hear what Red Hat says about it. Maybe the problem is not known well enough in the Red Hat universe.

    Regards, Simon

  • I think it’s particularly disappointing *if* this is a “policy” from RH
    since the other major RHEL clone, Scientific Linux, has not produced an EL8 offering in favour of using CentOS.

    I think all of us here understand the hugely complex process of producing a quality OS, even when it’s “just” a clone of another one. The official sanctioning from RH was touted as a two-way process:
    community input into RHEL and RH support and help of the cloning and build process. It would be a bit underhand if it turned out that it was RH’s way of creating a two tier system: buy RHEL+support and get timely updates; use CentOS for free, get security updates, but wait two months for each upgrade.

    P.

  • Keep in mind that as soon as Scientific Linux started taking off, IIRC, because CentOS was late with a release, RH quickly hired its main developer. I don’t think we can really expect RH to act differently than most corporations.

  • A major reason various people were hired from Fermi labs was very much that DOE(*) was cutting back funding on Fermilab and there were a lot of layoffs for anything which could not be the ‘core’ mission of Fermi. Making operating systems by any of the labs in the DOE complex was considered to be not a core mission, and multiple groups were either laid off or ‘asked’ to retire. The hiring of people was mainly because people were soon to be out of a job.

    DOE – US Department of Energy which funds Fermi and Argonne National Labs in Illinois as physics science institutes.

  • Il 17/06/20 15:42, Scott Robbins ha scritto:

    Yes but today Scientific Linux is out of games and CERN made CentOS CERN
    based on C8 so there is no more a valid competitor for this target (in the RH family based distro)

    CentOS at the current status (third 8 release) is not suitable for production with the current issues and we are forced to buy rhel license
    (and every needed extension).

    I would to know what think all ISPs that offer CentOS 8 on their VPS, Dedicated Server, Shared Hosting and how they handle this problem
    (probably they don’t because the choice is on the customer but maybe…)

    As you said, probably nothing will change for the corporation “limit”
    concept but at this point, probably, many users will migrate to another platform with less limitation.

  • No one is trynig to make anything slower.

    And CentOS Stream ‘is’ going to be how RHEL is developed in the future. So, all this is happening.

    But modules introduced in RHEL 8 requires a who new build system (as koji set up) and a whole new module build back end (MBS).

    If you would rather use Oracle for your open source needs .. well, that is a decision you can make and be responsible for.

    If you would instead have feedback directly into the RHEL development process as a community .. then CentOS is where you want to be.

    This stuff is open source, and you are all gown ups who can make your own decisions.

    I can assure you .. I am working my butt off everyday to make CentOS
    Linux the best it can be. If you want to compare what the CentOS team
    (a small team) can do compared to Oracle (a tmulti billin dollar corporation who bought Sun Microsystems .. took over Java and Open Office, etc) .. well, we can not provide the resources they can provide.

    But Red hat heard the community .. that the community and Red Hat customers want more direct input into RHEL development. And Red Hat is taking action to open up RHEL development in CentOS Stream.

    You get to decide who you trust.

    Thanks, Johnny hUghes

  • Am 17.06.20 um 09:16 schrieb Nicolas Kovacs:

    Site note: Despite the implications of update delays. The numbers provided by Michael are not comparable. The needed effort between every major release to build the OS is different. So, the numbers should be normalized.

    Quote from https://wiki.CentOS.org/About/Building_8:

    “The differences … has changed drastically, the repository format has added ‘modules’ and RPMS have grown many features that EL7 and before do not have.”

    I wonder about the authors conclusion; the fact that RHEL is the choice for critical applications (what ever critical is) is known since the early days. This applies randomly to C5.11, C4.9 or C8.2.2004.

    So – cold soup get cooked again :-)

  • Thank you, Johnny!

    I was going to reply to original post that: Nobody ever promised to pay CentOS team to do what they do. That was my understanding always.

    Changes in the OS, new components (do not mean systemd explicitly), and how new release is built may or may not be part of longer/bigger plan of RHEL owner.

    Without regard to the above, RedHat always was following meticulously the letter of GNU license. And the reputation IBM has (in my book), does not raise red flag after they bought out RedHat. Time will show, but I’m sure with acquisition of RedHat by IBM “nothing changed” for open source community in general, and CentOS existence in particular. Not dramatically at least. And someone mentioned Oracle, and I agree, Oracle has quite opposite reputation in my boot to that of IBM.

    Thanks, Johnny, very much again!!

    Note: I’m just humble outsider, not CentOS project member.

    Valeri

  • “Scientific” linux, with all due respect to one of US national laboratories situated nearby that fathered it, was never viable from the long term use standpoint system in my book.

    I got experimental proof about 5 or 6 years ago when “Scientific” linux (no, I’m not misplacing the second quote mark ;-) didn’t release updates for some 6 Months at least. I am more unimpressed by CERN’s nearsightness, than I am impressed by my own long term predictive power. The thruth is: “Scientific” linux is the creature of small team, situated in one place, and those who put it together have primamry dedication to their job place, NOT the the “scientific” linux project and its existense. And I am not mentioning what I was mentioning to my researchers when they asked to install “scientific” linux, and I was giving them a list of arguments why they will be better off with CentOS – in addition to the above, CentOS is much reacher without extra repositories (which when there is large enogh number tend to create mess for you).

    Just my 2 cents.

    Valeri

  • And there was also Tao Linux, which I was using back in the day. another one-person project, which that one person did a good job with, but when time came to change jobs he had to let the project go. He had the foresight and grace to work with Johnny Hughes to define a way to transition to CentOS, which worked fine, so here I am still in the CentOS camp.

    And my thanks to that original developer (whose name I cannot dredge out of my back-brain right now), to Johnny, and all the other folks who make CentOS great!

    Fred

  • Hi Johnny, thank you for your and all CentOS team works.

    Many of us know how much work is needed for building new releases and maintaining C6 and C7, plus CentOS Stream and modules (Appstream). This is a huge work for a small team. Again thank you.

    For me OL is not an alternative.

    As reported in my previous message I’m not worried about how much time is required to build the new (major/minor) release, it will be ready when it will be. My major concern is about the “security update blackout” that take long as the build process.

    I would ask to you:

    1. Why all security fix are stopped when a new release building process is started? There is a way or possibility to run the two process in parallel?

    2. When a build process is started and a security fix released there is a way for your team to “suspend” the building process, release security updates (for 6/7.x or 8.1) and resume the builing process? I think that many users (included me) will have less disappointment having security updates instead of receiving a “signal lost” when building process takes its way.

    Thank you for your time.

    Alessandro.

    Il Mer 17 Giu 2020, 16:15 Johnny Hughes ha scritto:

  • Once upon a time, Alessandro Baggi said:

    I’m not involved in building CentOS, but the issue is that it is a rebuild of upstream. When RHEL 8.2 is released, there are no more upstream updates released for RHEL 8.1; they are all on top of the RHEL
    8.2 release. So, until the time that CentOS can rebuild RHEL 8.2 and make a new CentOS release, there can’t be any updates for CentOS 8.1.

    RHEL 8 introduced modules, which complicated the build system and required new tooling, so CentOS has had a bunch of “under the hood” work to catch up. Hopefully, once that’s ironed out, the gap between a RHEL
    8.x release and the corresponding CentOS release will drop.

  • Hi,

    I am the author of said blog article.

    FIRST: It was never my intention to criticize the CentOS
    team. I appreciate the hard work you are doing. If my blog text (which is in German langugage) gave a wrong impression, I apologize.

    SECOND: I LOVE CentOS. Otherwise it would not matter to me. I use CentOS to teach Linux administration at university, I promote CentOS in my books and I use it personally on some servers.

    THIRD: It is a fact that the update gaps for CentOS 8 are currently too long for productive use. Basically, that’s why I now warn against using CentOS 8 on live systems.

  • Some of us fled servers from Linux (of whichever flavor) to one of BSD
    descendants. That has more to do with frequent reboots due to glibc or kernel security updates or other unpleasantnesses of Linux. Many of those who did this, still use/support Linux on workstations and laptops.

    RedHat has not and never had any obligation to financially or otherwise support its clones (released free for use). To think they do is delusional. And CentOS project I’m sure never expected that (I am not part of CentOS project, though I greatly appreciate what they do).

    Valeri


    ++++++++++++++++++++++++++++++++++++++++
    Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247
    ++++++++++++++++++++++++++++++++++++++++

  • Most of the security fixes after the minor release are based off being built with the code from that minor release. Sometimes the code may even not compile without the rest of the packages involved. Deciding which packages can and can not be affected by the minor release can be a full time job.

    It would require a second build system which built against the older set of packages, it would also require someone to edit those packages so they can be replaced by later released ones after the main release is done. It also requires work on making sure conflicts and broken packages are dealt with.

    A lot of this is sausage making.. We all want the end product of a cooked sausage.. but there is a lot of messy work which adding more people to doesn’t help unless you can add a lot of other things which are supposed to appear ex-nihilo and fully formed. You can have a lot of people volunteer to stand around but there are only so many places to push meat on one side, turn cranks in the middle and one tube which the mess gets stuffed into. Every extra thing requires months and months of work and preparation to set up while still trying to make the other items. However the only time people seem to get bothered enough to even try to set up their own build system and find out how much mess it is.. is after the current release is delayed.

  • IMHO this is the crux of the problem.  I feel for the CentOS team every time they get beat up by users asking why things take so long, and they’re forced to explain over and over again how they have to re-engineer processes that the RHEL team has already engineered.

    In theory, both RHEL and CentOS start from the same sources —
    git.CentOS.org — which is a great thing.  But, RHEL obviously has package build infrastructure, release composition, release management, and QA (among other) systems that are requisite steps to building and releasing, say, RHEL 8.2.  It makes me sad that the CentOS devs (most of whom are Red Hat employees, as I understand it) are forced to re-implement what the RHEL team has already implemented, without any advice, guidance, or tooling from the RHEL engineering team.  (i.e. the CentOS team has to discover that “these packages have to be built in this order, or with this modified build environment”, etc. on their own)

    It’s not clear why 2 different groups at the same company doing the same thing can’t combine resources.  Why can’t one group at Red Hat produce binary RPMs from git.CentOS.org that find their way into both a RHEL
    compose and a CentOS compose?  And would the composes then be so different if the only thing that varied was the package set and branding?

    Perhaps the duplication of engineering effort stems from the history of CentOS being a separate organization that’s still undergoing integration with other Red Hat teams.  And I’d love to be enlightened if any or all of my assumptions above are wrong; my perspective is just that of a long-time Red Hat Linux, RHEL, Fedora, and CentOS user (since 1998 or so).

    -Greg

  • As far as I’m aware that has always been the case. Johnny has never been slow in coming forward and saying if you need updates, or a service level agreement, or support then you should buy RHEL. That is what it is for. If not, then use CentOS for free. But don’t use CentOS for free on production servers and then shout or act surprised when you don’t have updates on a timescale you consider appropriate.

    Nothing has changed in this regard for as long as I’ve been a CentOS
    user or been involved in the CentOS community. If you are now having to rethink your approach then you probably either haven’t given it sufficient thought in the first place or you originally came to the wrong conclusion.

    This is a non-issue. Nothing has changed. Things were exactly the same with CentOS 4, CentOS 5, CentOS 6, CentOS 7, and by it’s very nature it will be the same in CentOS 9… The simple matter is it takes time to rebuild a complete OS and there will always be a lag. Either that is acceptable to you and you use it, or you purchase a RHEL license for your publicly facing infrastructure. The only issue here is people’s unrealistic expectations, and to be fair the CentOS Project can hardly be accused of falsely raising peoples expectations having consistently stated it will be ready when it’s done for at least the last 15 years.

  • This is the essence of the question, to me. I agree that _in_principle_ nothing has changed, and I don’t even see any disagreement with that in the list. However, there is a separate question, and that is whether _in_practice_ the lag between RHEL and CentOS updates has increased with CentOS 8. I don’t know what the answer is, because I’m not paying attention since I’m far from adopting CentOS 8, but it’s a legitimate (and in fact empirical) question.

    Noam

  • I get what you are saying, but what difference does it make if it has?
    What does it matter if the lag is 1 week, or 1 month, or more? The only reason it will matter to you is if you are trying to do something with CentOS that is time critical – e.g, publicly facing server that needs security updates, using CentOS on test servers to validate production releases for RHEL, etc. At which point you probably should be using RHEL
    if it is important to you, not CentOS, and it was a mistake to deploy CentOS in those roles in the first place.

    People need to hold their hands up and say, I took a gamble that CentOS
    would get updates out quick and I wouldn’t get hacked in a week, and now updates are taking a lot longer my gamble is no longer paying off and I
    need to get myself a RHEL subscription or switch to Ubuntu or whatever other flavor you like the taste of. Coming here and complaining when
    (you) made a bet and lost doesn’t achieve anything.

    On my home file server for example, which is not connected to the internet, what does it matter if the release is 1 month or 3 months out of date? I can install the server in the knowledge it’s going to work, and be supported with updates for 10 years and I can largely forget about it. My el5 box ran for more than 10 years until the hardware eventually died.

  • And yet in practice many of us have found CentOS to be perfectly adequate for such applications in the past, up to and including CentOS 7. If this is no longer true for CentOS 8, for whatever reason, it’s useful to know. I’m not saying RHEL doesn’t have its place – just that perhaps the boundary in the range of applicability between it and CentOS has therefore also changed.

    Noam

  • Am 17.06.20 um 21:37 schrieb Noam Bernstein via CentOS:

    The answer is not inherently in the distribution itself. Make your analysis about your needs an requirements and the choice is then yours.

    One could argue that the gap between disclosure of one security issues and the update via RHEL subscription is to big. Then a contract with the upstream developer of the corresponding software component is a better choice then relying in RHEL, right?

  • Of course. My only question is whether the observation that the gap for CentOS 8 is indeed larger than we have come to be used to for CentOS 7. I’m certainly not, and I don’t think anyone is, claiming that the CentOS teams owes us any particular response time. I just want to know if the claim that it’s systematically significantly longer for 8 than 7 is in fact (empirically) true.

    Noam

  • Once upon a time, Noam Bernstein said:

    So, I took a look… and the answer is “it’s not” (with a small sample set). I took dates from Wikipedia for RHEL and the archived release notes for CentOS. I didn’t bother with the .0 releases (since that’s a lot of new work anyway). Right now, CentOS 8 is far faster than CentOS
    7 and 6 were at this stage.

    release RHEL date CentOS date days
    6.1 2011-05-19 2011-12-12 207
    6.2 2011-12-06 2012-07-24 231
    6.3 2012-05-20 2012-09-30 133
    6.4 2013-02-21 2013-05-21 89
    6.5 2013-11-21 2014-02-26 97
    6.6 2014-10-13 2014-11-15 33
    6.7 2015-07-22 2015-09-05 45
    6.8 2016-05-10 2016-07-28 79
    6.9 2017-03-21 2017-04-05 15
    6.10 2018-06-19 2018-07-03 14

    7.1 2015-03-05 2015-10-11 220
    7.2 2015-11-19 2016-02-19 92
    7.3 2016-11-03 2016-12-21 48
    7.4 2017-08-01 2018-03-21 232
    7.5 2018-04-10 2018-10-30 203
    7.6 2018-10-30 2019-01-28 90
    7.7 2019-08-06 (didn’t find release notes)
    7.8 2020-03-31 2020-04-27 27

    8.1 2019-11-05 2020-01-15 71
    8.2 2020-04-28 2020-06-15 48

  • Indeed!

    You have clearly had more success with the longevity of your hardware than me. I was plagued with expanding capacitors about 15 years ago which killed off most of my hardware at the time, and the replacements were taken out of service as they subsequently died meaning I’m exclusively running el7|8 now :-)

  • Did you look at the German blog that started this discussion? I don’t know what determines the archived release notes dates, but I just picked the longest delay, CentOS 7.4. You list:
    which is indeed the “last updated” dated on the archived notes. However, the German blog post that started this thread lists the much earlier 2017-09-13 for CentOS, 43 days. On the mailing list this message https://lists.CentOS.org/pipermail/CentOS-announce/2017-September/022532.html <https://lists.CentOS.org/pipermail/CentOS-announce/2017-September/022532.html>appears to confirm the earlier date.

    I’m not sure the difference shown in that blog (assuming the other dates are also correct) is really quite so dramatic as to justify the conclusion that CentOS 8 is now too slow in getting updates for a substantial number of situations where the CentOS 7 lag was acceptable, but it’s apparently not faster.

    Noam

  • So .. when a point release happens .. say 7.8 to 7.9 (just an example .. could be 6.10 to 6.11 or 8.1 to 8.2, etc)

    Those packages are built against EACH other, one at a time. Once we build the new gcc, new kernel, and new glibc (if they are reqruies) .. then all the OTHER updated packages are built against those new libraries.. they therefore need those NEW shared libraries to run. So the new files have to be released as a set, not individually.

    It makes no difference if the update is a bugfix update or a security update. If 500 packages get released at the same time, they have to be built in a specific order in order to match how they were built in RHEL.

    We have to build them, one at a time, then individually test them to make sure they LINK against the proper new libraries and not older libraries.

    Also any UPDATES released to the new version , after RHEL does the point release (so updates FOR 7.9 after the 7.9 release) need to wait until the 7.9 release is done and tested to be built .. as they were built against RHEL 7.9 and not RHEL 7.8

    So, you can’t just build items out of order at point release time.

    We have to build the 500 packages , in a specific order. We then have to test the packages, and usually rebuild several of them again for bad links, etc.

    This is the process that takes time .. testing and getting the proper links to the proper shared libraries.

    If we quickly release bad files .. then we have to rebuild them and re-release them with different versions that RHEL has (because they have to replace our previuosly BAD release). That is not good for anyone.

    Hopefully this answers your question.

  • While both of those would be helpful .. neither will really SOLVE the issue. This is an iterative process. You build something, test it, rebuild all the bad parts, test it again .. rinse, repeat

    We usually have the first build 3 or so days after they release.

    Many things build out of order. They get tested and rebuilt.

  • How is this going to be fixed .. Welcome to CentOS Stream

    Stream will be , once it is fully implemented, the ACTUAL development of RHEL the ‘next point release’ on git.CentOS.org in the open.

    It will be a rolling distro that is GOING to be the Source Code used for next RHEL point release.

    Therefore, we will have all package as they are being worked on by the RHEL Engineers .. and you can see it happen in progress. You can also use it however you want. There will be no delay i this at all. It will be constantly moving. There will be no 500 pacakges drop or delays.

  • Your dates are significantly off Wikipedia has a delay listed in a table:

    It is, for CentOS-7, For example:

    7.0 27
    7.1 26
    7.2 25
    7.3 39
    7.4 43
    7.5 31
    7.6 34
    7.7 42
    7.8 28

    For 6 .. since 6.2, it has bee3n between 10 and 18 days.

    For 8:

    8.0 140
    8.1 71
    8.2 48

    And EL8 is exponentially harder with an entirely new build system and the requirement to build modules.

  • Le 19/06/2020 à 17:29, Johnny Hughes a écrit :

    Reading this, I just want to say a big warm THANK YOU for churning out one of the finest Linux distributions.

    Keep up the good work.

    Cheers from the sunny South of France,

    Niki


    Microlinux – Solutions informatiques durables
    7, place de l’église – 30730 Montpezat Site : https://www.microlinux.fr Blog : https://blog.microlinux.fr Mail : info@microlinux.fr Tél. : 04 66 63 10 32
    Mob. : 06 51 80 12 12

  • Il 19/06/20 17:15, Johnny Hughes ha scritto:

    Hi Johnny, thank you for your answer. This is more clear to me now.

    Alessandro.

  • So basically stream is a testing ground for RHEL. It’s not actually a rebuild of RHEL since it’s what comes *before* RHEL, not after.

    This is all well and good, but I don’t think that CentOS was ever meant to be a testing ground for RHEL. As the name actually stands for it is a “Community Enterprise OS” and it has always been a rebuild of the RHEL
    sources. Stream is basically RHEL Rolling Beta, and that can hardly be considered “Enterprise”.

    I and I think many others find this focus on Stream to be rather distressing, and it does have the appearance to be taking focus away from the core OS. This is further evidenced by the long wait times for release.

    The way I see it, Red Hat pays the bills now, Red Hat employs the core team, and Red Hat wants a RHEL Beta platform, so that is what they have decreed that CentOS will become. Now I could be wrong here because I
    certainly don’t have any inside information about this, but it seems from teh outside looking in that any progress on the core OS is incidental and time spent on it has to be time leftover after any work is done on Stream.

    Now I don’t have an issue with Stream, in fact I think taht Stream can be beneficial to CentOS, but it hsould not be at the expense of the core OS, imo. The core OS should take priority over any other CentOS
    project, whether it be streams, or SIGs or anything else, because we can’t really have a Community Enterprise OS without the core OS.

    Peter

  • So the delays for 8 are significantly longer than they ever were for 7.

    But it seems like every major release has had reasons to be exponentially harder than the last. With 7 it was the shift to using the git sources instead of the SRPMS that were previously provided by Red Hat, thereby necessitating an entirely new build system, plus the change to systemd and the introduction of a new point release numbering scheme, not to mention the move to entirely new infrastructure because of the change to Red Hat sponsorship. So given those I find it hard to believe that the changes in 8 are so much different as to have had such longer delays than 7.

    I’d also like to point out something else, from:
    https://wiki.CentOS.org/About/Building_8.x#Current_Timeline_8.2.2004

    It would appear that the package build was completed on the 4th of May, why didn’t we get a CR repo dump this time around so that CentOS users wouldn’t have to wait another month before getting critical updates?

    Peter

  • I should also say that the time lag for each point release of 8 seems to be dropping exponentially. Hopefully this means that 8.3 will have a significantly shorter delay and we will be in the range of less than 30
    days for that, if so I can accept the delays up to this point.

    Peter

  • +1 Streams is not for a production workload, if I wanted that I can easily deploy an Arch instance if I want or need a rolling distro (it’s not Redhat etc but still). If Redhat wanted CentOS to be released near the same time line they could help make that happen, although that wouldn’t be in there best financial interest.

    Now maybe there will be a way to set streams up to only get security updates and then when they release the .1 release you could update and have everything update. If something like that could be worked out that would work for me but I would only want security updates in between and I’m not sure if that is possible.

  • I have to shake my head at this. You’re telling the guy who builds, and had built, CentOS for years what the project is and where it is going? He told you what Stream is and what it will be used for already. It is the new future of the OS. That’s it. You’re arguing semantics about having Enterprise in the name, while ignoring that it’s still Enterprise Linux, even if it’s not the final end product because it is the future source project of Enterprise Linux.

  • think of it this way … when the rolling beta is done, the final release will be done with no further delay.

  • No, Stream and the core OS are built form a completely separate set of sources. I highly doubt that we could use binaries built from stream to populate the final release. The whole thing has to be rebuilt for the final release regardless of the status of Stream.

    Peter

  • but the build process should be the same, no? I can’t believe RH would use a completely different build process for the release than for the beta/development stuff.

  • The packages still have to be built as a whole, they need to go through QA testing, isos need to be built and tested. The only thing that I can think of that Stream benefits this process is to help Red Hat find the odd bug here and there before their final release (after which CentOS
    still has to do everything listed above).

    Peter

  • Il 21/06/20 03:28, Peter ha scritto:

    I have a question. Why CentOS Stream is needed and what problem it solves?

  • As I understand it the whole full build and QA and whatever may still be done again. The big difference is that the whole work of how to build and setting up the build infrastructure has already been done and is known and tested. So the complete build is going quite fast and the big delays are a thing of the past.

    If it’s going to be like that it sounds very good.

    Regards, Simon

  • exactly, that was my point. I remember 8.0 was very delayed by how much harder and different the build process was.

  • That work was completed in the build of 8.0 and to a smaller extent 8.1.
    Stream doesn’t really add anything here.

    Peter

  • There are 2 sets of work.
    1. There is the work on the tools which were slapped together as an emergency from parts before 8.0. Those mbboxx tools are getting a rewrite and upgrade currently by the CPE team to make them more useful in the future. Stream only helps in that it is the excuse for that work to be done versus it molding and falling apart right after every
    8.x release comes out.
    2. There is the work that happens because various things are rebased and you need to figure out the HTF you get from build A to build A+1
    by rebuilding N packages. That is work that Stream should help on because this is then knowledge is being done in stream before hand. If you know that package A went to A+1 then to A+2 and then back to A+1
    but you learned how to do the second A+1 from a flag you used with A+2, then the amount of time reinventing the wheel is shortened.

  • I didn’t know that a rewrite is still needed on the current tool set and granted Stream can help with this, but I hardly think that it’s necessary and the tool set can always be tested against the current release (8.2) from git.

    This I do realize and it’s the one exception I considered where Stream might come in handy, but not handy enough to justify its existence, imo.
    Usually in a new point release there might be a small handful of packages that need re-basing, out of those the number of packages that would need to have the spec file tweaked to build them would be minimal
    (at a complete guess three or less) and out of those the number that would require a change to the tool set would likely average out to be less than one per point release. In a worst-case scenario it might save a day or two on a particularly nasty point release, and this would easily be recouped in the amount of time it would save if the CentOS
    team did not have to maintain Stream at all.

    Now these are just semi-educated guesses and I don’t have the experience to justify this so I’m happy to consider real numbers that prove me wrong.

    Peter

  • Now I have RHEL 8 installed for my test machine and some test Virtual Machines.

    I then subscribed to the RHSA-announce mailing list.

    Now I wonder why a particular package has not been released for CentOS 8
    while it has been some time on RHEL OS and mailing list.

    With CentOS 7, I had no RHEL developer access, so I never wondered why a particular update has not been released CentOS 7

    I just used CentOS 7 and was happy.

    Is this a valid reason for my impatience?


    Lee

  • Using git to build sources from, while a change, doesn’t significantly increase the number of packages that need building. The actual package building – compilation – stage can take days on even the fastest hardware.

  • Am 24.06.20 um 18:37 schrieb Lamar Owen:

    Additionally; I think the “build loop 0” is build on Fedora and loop 1
    on the output of loop 0 and so on … which makes things more difficult.


    Leon