All our servers and company laptops went down at pretty much the same time. Laptops have been bootlooping to blue screen of death. It’s all very exciting, personally, as someone not responsible for fixing it.

Apparently caused by a bad CrowdStrike update.

Edit: now being told we (who almost all generally work from home) need to come into the office Monday as they can only apply the fix in-person. We’ll see if that changes over the weekend…

      • @Blackmist@feddit.uk
        link
        fedilink
        English
        1211 months ago

        Yep, anything done on Friday can enter the world on a Monday.

        I don’t really have any plans most weekends, but I sure as shit don’t plan on spending it fixing Friday’s fuckups.

        • @sugar_in_your_tea@sh.itjust.works
          link
          fedilink
          English
          211 months ago

          And honestly, anything that can be done Monday is probably better done on Tuesday. Why start off your week by screwing stuff up?

          We have a team policy to never do externally facing updates on Fridays, and we generally avoid Mondays as well unless it’s urgent. Here’s roughly what each day is for:

          • Monday - urgent patches that were ready on Friday; everyone WFH
          • Tuesday - most releases; work in-office
          • Wed - fixing stuff we broke on Tuesday/planning the next release; work in-office
          • Thu - fixing stuff we broke on Tuesday, closing things out for the week; WFH
          • Fri - documentation, reviews, etc; WFH

          If things go sideways, we come in on Thu to straighten it out, but that almost never happens.

      • @sasquash@sopuli.xyz
        link
        fedilink
        English
        111 months ago

        Actually I was not even joking. I also work in IT and have exactly the same opinion. Friday is for easy stuff!

    • @merc@sh.itjust.works
      link
      fedilink
      English
      311 months ago

      You posted this 14 hours ago, which would have made it 4:30 am in Austin, Texas where Cloudstrike is based. You may have felt the effect on Friday, but it’s extremely likely that the person who made the change did it late on a Thursday.

      • @Hotzilla@sopuli.xyz
        link
        fedilink
        English
        211 months ago

        This is AV, and even possible that it is part of definitions (for example some windows file deleted as false positive). You update those daily.