The phylosophy of AI

Be concise and different

We must teach ai which part should it keep the same with the data

which part should it be flexiable and different with the data

Human can precive something exist that cann’t be uniform defined.

I know it when I see it.

Pytorch版本的GPT2

GPT2

模型实现

Flash 注意力

混合精度

前向传播

反向传播

自动微分

梯度更新

内存管理

CUDA后端

单机多卡

多机多卡

LLama3