github/tinygrad
1
1
Fork 0
mirror of https://github.com/tinygrad/tinygrad.git synced 2026-01-10 23:48:01 -05:00
Code Issues Packages Projects Releases Wiki Activity
5,064 Commits 203 Branches 13 Tags
3ebf569f04579cf4f12b60b9817ecdd6cb2ffa4f
Commit Graph

5064 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
chenyu
3ebf569f04 relax fuzz transend math threshold a bit (#5442) 
* relax fuzz transend math threshold a bit

* fuzz more

* fuzz 50k
 2024-07-13 03:31:21 -04:00
chenyu
e398734890 fuzz test transcend math (#5383) 
* fuzz test transcend math

found something wrong with float64 sin reduction

```
from tinygrad import Tensor, dtypes
import numpy as np
print(Tensor([39800.0], dtype=dtypes.float64).sin().numpy())
print(Tensor([39800.0], dtype=dtypes.float32).sin().numpy())
print(Tensor([39800.0], dtype=dtypes.float16).sin().numpy())
print(np.sin(np.array([39800.0], dtype=np.float64)))
print(np.sin(np.array([39800.0], dtype=np.float32)))
print(np.sin(np.array([39800.0], dtype=np.float16)))
```

```
CLANG=1 python test.py
[0.92785633]
[0.7428573]
[-0.7705]
[0.74285722]
[0.7428572]
[-0.7705]
```

* fix test

* abs

* skip
 2024-07-13 01:54:52 -04:00
hikettei
3a7262d923 [Patch] Fixed an invaild value of fp64 xlog(DBL_MIN) (#5441) 
* [Patch] Removed weird NaN Handling in xlog2 resulting in different output around 1e-203

* Patch: compare the value of xlog(x) using y, allowing x <= 1e-200

* mypy

* fuzzer tests for log2

* fix tests: use approximate dbl_min, fp64 fails at nv

* update: gradually increment the scale (if y is not inf)
 2024-07-13 01:11:53 -04:00
wozeparrot
90f0e2fc49 db in wal mode (#5388) 2024-07-12 20:43:36 -07:00
George Hotz
414aa6ee98 minor improvments to matcher speed [run_process_replay] (#5439) 
* minor improvments to matcher speed [run_process_replay]

* oh, put that back
 2024-07-12 20:41:41 -07:00
chenyu
4df63da190 clean up rest of the loadop [run_process_replay] (#5440) 
to metaop and filter_sink
 2024-07-12 23:38:51 -04:00
hikettei
0795139f30 Fix TRANSCENDENTAL=2 fp64 sin (#5385) 
* fixes on transcendental: fix for fp64 payne hanek, refactor for fp16 sin

* revert the changes on test

* refactor on xsin: removed cody_waite_reduction, always use payne_hanek

* Revert "refactor on xsin: removed cody_waite_reduction, always use payne_hanek"

This reverts commit 2fd401f251.

* still need cody_waite_reduction for the very smaller range

* test: added a regression test for transcendental sin

* test: found the worse case ulp 3.5 only in numpy

* give the input as a valid dtype

---------

Co-authored-by: chenyu <chenyu@fastmail.com>
 2024-07-12 23:15:04 -04:00
George Hotz
fb3011ac61 improve matcher speed [run_process_replay] (#5438) 
* improve matcher speed [run_process_replay]

* don't use arg set in ptx
 2024-07-12 20:02:19 -07:00
George Hotz
03c2dc8bd7 lowerer is kernel [run_process_replay] (#5437) 2024-07-12 18:50:55 -07:00
George Hotz
b8342fb085 independent lowerer [run_process_replay] (#5434) 
* independent lowerer [run_process_replay]

* don't relinearize PTX

* fix ptx

* Revert "fix ptx"

This reverts commit f4e8e059c0.

* Revert "don't relinearize PTX"

This reverts commit f6c12c506c.

* parents is fine, no need for linearization

* remove loop local idxs

* recover stupid loop_idxs
 2024-07-12 18:08:43 -07:00
chenyu
9a187e6102 fix handcode_opt script (#5435) 
* fix handcode_opt script

* run in ci

* real run in ci

* HALF=0
 2024-07-12 20:52:28 -04:00
wozeparrot
b80fd7d23c allow benchmarking forward only (#5436) 2024-07-12 17:37:49 -07:00
chenyu
00813a92a0 update Tensor.eye api to match torch (#5433) 
* update Tensor.eye api to match torch

input is n for nrows and optional m for ncols

* space

* fix onnx
 2024-07-12 20:25:12 -04:00
George Hotz
cddfd8e25d bugfix: group for reduce should check all dimensions (#5431) 2024-07-12 17:02:40 -07:00
George Hotz
fbaf040baf compute full_shape from LazyOp [run_process_replay] (#5429) 
* compute full_shape from LazyOp

* put KernelInfo in the sink

* wrong but pass
 2024-07-12 16:47:08 -07:00
George Hotz
870dc8c350 s/Linearizer/Lowerer [run_process_replay] (#5428) 2024-07-12 15:54:07 -07:00
George Hotz
6707c778d0 scheduleitem is not Tuple [run_process_replay] (#5425) 
* scheduleitem is not Tuple [run_process_replay]

* fix tests

* fix op + fuzzers

* fix mop test
 2024-07-12 15:13:19 -07:00
chenyu
4cd1de038a smaller reshape_and_permute arg in shift_to (#5426) 
adding tuples directly
[run_process_replay]
 2024-07-12 17:46:48 -04:00
George Hotz
94599c0637 fixup ast in kernel to be MetaOps.SINK [run_process_replay] (#5424) 
* fixup ast in kernel to be MetaOps.SINK [run_process_replay]

* fix tests

* fix more tests
 2024-07-12 14:01:03 -07:00
George Hotz
b055ece550 hotfix: bump to cache gpuocelot 2024-07-12 13:54:14 -07:00
chenyu
d37056f3b1 pass Renderer.global_max / local_max into get_grouped_dims (#5423) 
[run_process_replay]
 2024-07-12 16:49:27 -04:00
George Hotz
4aefb1595d MetaOps.SINK [run_process_replay] (#5422) 
* s/loadops/metaops [run_process_replay]

* add metaops.sink [run_process_replay]
 2024-07-12 13:37:30 -07:00
George Hotz
f6ef283e6a s/loadops/metaops [run_process_replay] (#5421) 2024-07-12 13:26:50 -07:00
nimlgen
f4944ced09 tiny amd cleanups (#5420) 2024-07-12 22:54:42 +03:00
chenyu
b17e4adb3a add -c advice.detachedHead=false to process replay git checkout (#5419) 
remove the noisy `Note: switching to 'origin/master'.

You are in 'detached HEAD' state. You can look around, make experimental
changes...` in log
 2024-07-12 15:13:26 -04:00
wozeparrot
d1cbd6bb95 unity handcode_resnet_opt and handcode_bert_opt (#5418) 2024-07-12 12:05:01 -07:00
chenyu
a0dbe20dbd skip some redundant and slow tests in ci (#5416) 2024-07-12 14:43:13 -04:00
chenyu
76125c07be make some grouped_dim test work (#5415) 
next need to support max size per dim, splitting and correct way to do reverse or arbitrary permute global dims
 2024-07-12 14:22:50 -04:00
wozeparrot
b7cc75a9df usage summary in handcode opt (#5414) 2024-07-12 11:21:18 -07:00
uuuvn
3cb94a0a15 Rename tinygrad/runtime/driver to support (#5413) 2024-07-12 11:06:42 -07:00
nimlgen
6604d2b2c3 amd/nv respect visible devs (#5409) 
* nv/amd respect visible devices

* linter

* sort amd gpus

* env docs
 2024-07-12 20:02:12 +03:00
Roelof van Dijk
b18aa00bba refactor: consolidate replace [run_process_replay] (#5403) 2024-07-12 07:36:57 -07:00
chenyu
497274f663 add float64 to test_dtype_alu dtypes_float (#5410) 
* add float64 to test_dtype_alu dtypes_float

* CUDACPU float64 crashes

* real NV failed
 2024-07-12 10:21:32 -04:00
qazal
31fcc516dc more process replay tooling (#5407) 
* replays

* what's in there

* can it be up there

* sha is enough

* insert sha as the key

* fix str

* update reset utils

* that nested try/except was terrible

* github_context can go
 2024-07-12 13:11:34 +03:00
Roelof van Dijk
6ec7dbc287 ci: parallelize uops tests (#5405) 2024-07-12 11:22:41 +03:00
qazal
e22b377839 generalize FUSE_AS_ONE_KERNEL in the scheduler (#5397) 
* test: use const

* hotfix: base

* asserts

* dont push through reshape

* cleanup

* dont need the cache

* test_reduceop_reshape_dont_push and test_index_fused are next
 2024-07-12 10:23:16 +03:00
chenyu
6e0a523078 repro slow resnet kernel with 4 global dims (#5402) 
* repro slow resnet kernel with 4 global dims

* fix ruff
 2024-07-11 23:31:15 -04:00
George Hotz
8390feb7b9 optim.OptimizerGroup in hlb_cifar (#5401) 2024-07-11 20:14:36 -07:00
George Hotz
01fbd18209 metal compile fail 2024-07-11 19:27:05 -07:00
George Hotz
3a2b5a75d2 improve single kernel indexing (#5398) 
* improve single kernel indexing

* metadata in graph (#5399)

* indexing is O(1)

* add failing test

* ugh, that all needs to be replaced with symbolic

* broken on ptx, it's fine

---------

Co-authored-by: wozeparrot <wozeparrot@gmail.com>
 2024-07-11 19:00:57 -07:00
wozeparrot
c24d495ef9 metadata in handcode_opt (#5400) 2024-07-11 17:45:34 -07:00
wozeparrot
c60838594c metadata in graph (#5399) 2024-07-11 17:02:12 -07:00
George Hotz
c2da4454cd indexing getting better (#5389) 
* indexing getting better [run_process_replay] [no_assert]

* fix test

* test_arange_2_reduce is a simpler test

* put that print back, NOOPT

* don't merge reduces (they could be different reduces)

* FUSE_AS_ONE_KERNEL

* fix tests

* fix test_var_multireduce

* w/e put that there

* fails on others too

* fix test, revert UNMUL change

* in case order matters

* one kernel indexing works

* one kernel indexing works (test other)
 2024-07-11 16:41:51 -07:00
qazal
9712d9ffb6 pass lowering errors if not asserting process replay (#5395) 
* pass lowering errors if not asserting process replay

* ProcessReplayError
 2024-07-11 19:09:12 -04:00
wozeparrot
a02b38c0ac download openimages by running it (#5396) 2024-07-11 16:06:13 -07:00
qazal
0421f5d83e hotfix: compare test_var_multireduce against numpy (#5394) 2024-07-11 18:57:08 -04:00
qazal
b91a0ccdc3 make [run_process_replay] [no_assert] the default (#5390) 2024-07-11 22:36:59 +03:00
George Hotz
e8191479a3 add bigint type for indexing [run_process_replay] (#5387) 2024-07-11 11:37:10 -07:00
George Hotz
5232e405ce hotfix: add BS to beautiful_mnist 2024-07-11 10:55:05 -07:00
George Hotz
3e40211e45 add UOP_IS_SYMBOLIC [run_process_replay] [no_assert] (#5386) 
* cleanup a few things in uops [run_process_replay] [no_assert]

* add optional UOP_IS_SYMBOLIC
 2024-07-11 10:48:45 -07:00