8 Commits

Author	SHA1	Message	Date
George Hotz	dca076dbf1	remove dumb nn ops	2021-11-29 18:05:31 -05:00
George Hotz	8097b8f7d6	vit works	2021-11-29 16:28:14 -05:00
George Hotz	f909ab194f	gelu with broken test	2021-11-29 15:00:50 -05:00
George Hotz	1eafa5580e	layernorm with learnable parameters	2021-11-29 13:03:57 -05:00
George Hotz	c7f795ca1e	added dot affine	2021-11-29 12:55:56 -05:00
George Hotz	30eb3afbe1	add bias term to transformer	2021-11-29 12:45:27 -05:00
George Hotz	99b6051467	add ff_dim to transformer	2021-11-29 12:40:52 -05:00
George Hotz	d3f169b267	move good models to models, add a training step test	2021-06-19 11:24:15 -07:00