this post was submitted on 17 May 2025
301 points (94.9% liked)

Technology

70942 readers
3875 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws
L4s@hackingne.ws
301
The Collapse of GPT: Will future artificial intelligence systems perform increasingly poorly due to AI-generated material in their training data? (cacm.acm.org)
submitted 3 weeks ago by Pro@programming.dev to c/technology@lemmy.world
57 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] toastmeister@lemmy.ca 2 points 2 weeks ago

To make this more precise, we say that original data follows a normal distribution 

collapsed inline media{\displaystyle X^{0}\sim {\mathcal {N}}(\mu ,\sigma ^{2})}
, and we possess 
collapsed inline media{\displaystyle M\_{0}}
 samples 
collapsed inline media{\displaystyle X\_{j}^{0}}
 for 
collapsed inline media{\displaystyle j\in {\\{\\,1,\dots ,M\_{0}\\,{}\\}}}
. Denoting a general sample 
collapsed inline media{\displaystyle X\_{j}^{i}}
 as sample 
collapsed inline media{\displaystyle j\in {\\{\\,1,\dots ,M\_{i}\\,{}\\}}}
 at generation 
collapsed inline media{\displaystyle i}
, then the next generation model is estimated using the sample mean and variance:

collapsed inline media{\displaystyle \mu \_{i+1}={\frac {1}{M\_{i}}}\sum \_{j}X\_{j}^{i};\quad \sigma \_{i+1}^{2}={\frac {1}{M\_{i}-1}}\sum \_{j}(X\_{j}^{i}-\mu \_{i+1})^{2}.}

Leading to a conditionally normal next generation model 

collapsed inline media{\displaystyle X\_{j}^{i+1}|\mu \_{i+1},\\;\sigma \_{i+1}\sim {\mathcal {N}}(\mu \_{i+1},\sigma \_{i+1}^{2})}
. In theory, this is enough to calculate the full distribution of 
collapsed inline media{\displaystyle X\_{j}^{i}}
. However, even after the first generation, the full distribution is no longer normal: It follows a variance-gamma distribution.