17 Commits

Author	SHA1	Message	Date
George Hotz	c0c2c0b041	support larger ViT models	2021-12-12 10:45:10 -08:00
George Hotz	4b7c31b5b7	break vit into it's own file	2021-11-30 11:19:22 -05:00
George Hotz	835869974c	clean up vit code	2021-11-30 10:58:03 -05:00
George Hotz	535f02cc64	use sequential	2021-11-30 00:25:39 -05:00
George Hotz	de938c2d9d	vit is now tested	2021-11-30 00:23:06 -05:00
George Hotz	aff810e722	unify transformer block	2021-11-29 18:58:15 -05:00
George Hotz	58ed46963e	fix broadcastdot	2021-11-29 18:54:57 -05:00
George Hotz	125e74293f	promote layernorm to tensor op	2021-11-29 18:08:21 -05:00
George Hotz	dca076dbf1	remove dumb nn ops	2021-11-29 18:05:31 -05:00
George Hotz	70544e7e9f	sum hook override	2021-11-29 17:14:24 -05:00
George Hotz	8097b8f7d6	vit works	2021-11-29 16:28:14 -05:00
George Hotz	7c07c5efdd	plz fix vit	2021-11-29 15:45:19 -05:00
George Hotz	ca160504e1	affine is always the last dim	2021-11-29 15:22:49 -05:00
George Hotz	e86f7a4aa3	deterministic	2021-11-29 15:10:15 -05:00
George Hotz	f909ab194f	gelu with broken test	2021-11-29 15:00:50 -05:00
George Hotz	9ce881f88c	fix bug in getitem, drop int axis	2021-11-29 14:01:24 -05:00
George Hotz	99b6051467	add ff_dim to transformer	2021-11-29 12:40:52 -05:00