No Stupid Questions
No such thing. Ask away!
!nostupidquestions is a community dedicated to being helpful and answering each others' questions on various topics.
The rules for posting and commenting, besides the rules defined here for lemmy.world, are as follows:
Rules (interactive)
Rule 1- All posts must be legitimate questions. All post titles must include a question.
All posts must be legitimate questions, and all post titles must include a question. Questions that are joke or trolling questions, memes, song lyrics as title, etc. are not allowed here. See Rule 6 for all exceptions.
Rule 2- Your question subject cannot be illegal or NSFW material.
Your question subject cannot be illegal or NSFW material. You will be warned first, banned second.
Rule 3- Do not seek mental, medical and professional help here.
Do not seek mental, medical and professional help here. Breaking this rule will not get you or your post removed, but it will put you at risk, and possibly in danger.
Rule 4- No self promotion or upvote-farming of any kind.
That's it.
Rule 5- No baiting or sealioning or promoting an agenda.
Questions which, instead of being of an innocuous nature, are specifically intended (based on reports and in the opinion of our crack moderation team) to bait users into ideological wars on charged political topics will be removed and the authors warned - or banned - depending on severity.
Rule 6- Regarding META posts and joke questions.
Provided it is about the community itself, you may post non-question posts using the [META] tag on your post title.
On fridays, you are allowed to post meme and troll questions, on the condition that it's in text format only, and conforms with our other rules. These posts MUST include the [NSQ Friday] tag in their title.
If you post a serious question on friday and are looking only for legitimate answers, then please include the [Serious] tag on your post. Irrelevant replies will then be removed by moderators.
Rule 7- You can't intentionally annoy, mock, or harass other members.
If you intentionally annoy, mock, harass, or discriminate against any individual member, you will be removed.
Likewise, if you are a member, sympathiser or a resemblant of a movement that is known to largely hate, mock, discriminate against, and/or want to take lives of a group of people, and you were provably vocal about your hate, then you will be banned on sight.
Rule 8- All comments should try to stay relevant to their parent content.
Rule 9- Reposts from other platforms are not allowed.
Let everyone have their own content.
Rule 10- Majority of bots aren't allowed to participate here. This includes using AI responses and summaries.
Credits
Our breathtaking icon was bestowed upon us by @Cevilia!
The greatest banner of all time: by @TheOneWithTheHair!
view the rest of the comments
I'm not going to come running to the defense of private equity (PE) firms, but compared to so-called AI companies, the PE firms are at least building tangible things that have an ostensible alternative use. A physical data center building -- even one located far away from the typical metropolitan area that have better connectivity to the world's fibre networks -- will still be an asset with some utility, when/if the AI bubble pops.
In that scenario, the PE firm would certainly take a haircut on their investment, but they'd still get something because an already-built data center will sell for some non-zero price, with possible buyers being the conventional, non-AI companies that just happen to need some cheap rack space. Looking at the AI companies though, what assets do they have which carry some intrinsic value?
It is often said that during the California Gold Rush, the richest people were not those which staked out the best gold mining sites, but those who sold pickaxes to miners. At least until gold fever gave way to sober realization that it was overhyped. So too would PE firms pivot to whatever comes next, selling their remaining interest from the prior hype cycle and moving to the next.
I've opined before that because no one knows when the bubble will burst, it is simultaneously financially dangerous to: 1) invest into that market segment, but also 2) to exit from that market segment. And so if a PE firm has already bet most of the farm, then they might just have to follow through with it and pray for the best.
It's important to note that in some previous bubbles, the leftovers of the crash ended up spurring new beneficial growth after.
GPUlike computing power available at scape for essentially free after the ai crash could be used in all sorts of potential ways.
Maybe it makes rending movies with special effects super cheap, and available even to tiny indie studios. Maybe scientists grab it for running physics simulations or disease treatment computations.
The problem is that the deprecation/obsolescence/lifetime cycles of GPUs are WAY more rapid than anyone in the “AI” circlejerk bubble is willing to admit. Aside from the generational upgrades that you tend to see in GPUs, which make older models far less valuable in terms of investment, server hardware simply cannot function at peak load indefinitely - and running GPUs at peak load constantly MASSIVELY shortens the MTBF.
TL;DR: the way GPUs are used in ML applications mean that they tend to cook themselves WAY quicker than the GPU you have in your gaming machine or console - as in, they often have a couple of years lifetime, max, and that failure rate is a bell curve.
You're pulling shit out of your ass at this point, there are some doom reports out of people suggesting that may be a problem, but there are also reports out of other companies(meta for example) with documentation saying the rate is much lower and the mean failure is 6+ years.
The other leftovers from the crash also won't have that problem. It's not just about GPUs. Datacenters and their infrastructure last a lot longer, and the electric generation/transportation networks will also potentially be useful for various alternative applications if the AI use case flops.
MTBF is absolutely not six years if you’re running your H100 nodes at peak load and heat soaking the shit out of them. ML workloads are particularly hard on GPU RAM in particular, and sustained heat load on that particular component type on the board is known to degrade performance and integrity.
As to Meta’s (or MS, or OpenAI, or what have you) doc on MTBF: I don’t really trust them on that, because they’re a big player in the “AI” bubble, so of course they’d want to give the impression that the hardware they’re using in their data centers still have a bunch of useful life left. That’s a direct impact to their balance sheet. If they can misrepresent extremely expensive components that they have a shitload of as still being worth a lot, instead of being essentially being salvage/parts only, I would absolutely expect them to do that. Especially in the regulatory environment in which we now exist.
I mean, we really don't have the data to prove this either way.
https://www.tomshardware.com/tech-industry/artificial-intelligence/faulty-nvidia-h100-gpus-and-hbm3-memory-caused-half-of-the-failures-during-llama-3-training-one-failure-every-three-hours-for-metas-16384-gpu-training-cluster
Meta's training of Llama3 405B model had a 1.34% failure rate for GPUs over the 54 days it ran, across 16387 gpus. It's not likely that all of those faults led to bricked hardware either, they could have just lost part of their performance or memory.
The real question is does that test scale to the long term, often with hardware like this there's a bathtub curve for failure. If those units used were brand new, many of the failures could have just been the initial wave of failures, and there could be a long period of relative stability that hadn't even been seen yet.
GPU based coin mining demonstrated that GPUs often had a lifespan over 5 years of constant use before failure on consumer cards in often less than ideal operating conditions.