04 · crop

previewcode

9:41

the garden3 members

t

M

Melon

you have to watch this

M

Melon

youtube.com/watch?v=zjkBMFhNj_g

YouTube

[1hr Talk] Intro to Large Language Models

Andrej Karpathy

1

Melonyoutube.com/watch?v=zjkBMFhNj_g

@xtldrbot

xtldr

🤖LLMs: The 140GB "Zip File" of the Internet

• An LLM is just 2 files: a 140GB parameters file (70B numbers) and ~500 lines of C code to run it.0:29

• Training is lossy compression of ~10TB of internet text into those weights: a ~100x compression ratio.4:34

🛠️The Future: An "LLM OS"

• Think of an LLM as the kernel of an emerging operating system, coordinating tools and memory (context window = RAM).42:33

It's kind of like a lossy compression of the internet.6:02

Andrej Karpathy

🔗 YouTube⏱️ 59m 48s saved

what does he say about hallucinations?

xtldr

💭Hallucinations

• The model dreams internet documents: a made-up ISBN looks exactly like a real one to it.17:35

• Tool use is the fix: browse, run code, look it up, rather than recall from weights.32:03

M

Melon

you have to watch this

M

Melon

youtube.com/watch?v=zjkBMFhNj_g

YouTube

[1hr Talk] Intro to Large Language Models

Andrej Karpathy

1

Melonyoutube.com/watch?v=zjkBMFhNj_g

@xtldrbot

xtldr

🤖LLMs: The 140GB "Zip File" of the Internet

• An LLM is just 2 files: a 140GB parameters file (70B numbers) and ~500 lines of C code to run it.0:29

• Training is lossy compression of ~10TB of internet text into those weights: a ~100x compression ratio.4:34

🛠️The Future: An "LLM OS"

• Think of an LLM as the kernel of an emerging operating system, coordinating tools and memory (context window = RAM).42:33

It's kind of like a lossy compression of the internet.6:02

Andrej Karpathy

🔗 YouTube⏱️ 59m 48s saved

21

what does he say about hallucinations?

xtldr

💭Hallucinations

• The model dreams internet documents: a made-up ISBN looks exactly like a real one to it.17:35

• Tool use is the fix: browse, run code, look it up, rather than recall from weights.32:03

Message

9:41

the garden3 members

t

M

Melon

you have to watch this

M

Melon

youtube.com/watch?v=zjkBMFhNj_g

YouTube

[1hr Talk] Intro to Large Language Models

Andrej Karpathy

1

Melonyoutube.com/watch?v=zjkBMFhNj_g

@xtldrbot

xtldr

🤖LLMs: The 140GB "Zip File" of the Internet

• An LLM is just 2 files: a 140GB parameters file (70B numbers) and ~500 lines of C code to run it.0:29

• Training is lossy compression of ~10TB of internet text into those weights: a ~100x compression ratio.4:34

🛠️The Future: An "LLM OS"

• Think of an LLM as the kernel of an emerging operating system, coordinating tools and memory (context window = RAM).42:33

It's kind of like a lossy compression of the internet.6:02

Andrej Karpathy

🔗 YouTube⏱️ 59m 48s saved

what does he say about hallucinations?

xtldr

💭Hallucinations

• The model dreams internet documents: a made-up ISBN looks exactly like a real one to it.17:35

• Tool use is the fix: browse, run code, look it up, rather than recall from weights.32:03

Message

Reactions are buttons: press one and you count, press again and you leave. The emoji are Noto Animated Emoji (Google, Apache 2.0), animated WebP, no player; Telegram's own set lives behind its API and is its own. Each is 150-300 KB, lazy, which is why only reactions get them. The summaries are @xtldrbot's real output for these links; the follow-up answer is illustrative.