uuuvn 6729f20aab Ring allreduce try 2 (#3852) ...

* Ring allreduce v3

* Configurable size, number of gpus and jit in benchmark

* ScheduleBarrier v0

* GB/s that make sense

* ScheduleBarrier v0.1

* Fallback on 2 GPUs

* ScheduleBarrier v0.2

* ScheduleBarrier v0.3

* ScheduleBarrier v0.3.1

* ScheduleBarrier v0.3.2

* Replace ScheduleBarrier with automatic optimization

* unused import

* fix comment

* typing

* better fallback

* python 3.8

* RING=2 and use ContextVar

* DEBUG >= 2 and change name

* linter

* type

---------

Co-authored-by: George Hotz <72895+geohot@users.noreply.github.com>
Co-authored-by: chenyu <chenyu@fastmail.com>
Co-authored-by: nimlgen <138685161+nimlgen@users.noreply.github.com>

2024-03-21 19:17:51 -04:00

..

external

Ring allreduce try 2 (#3852)

2024-03-21 19:17:51 -04:00

imported

fix pad 0 size (#3277)

2024-01-30 08:58:10 -08:00

models

move is_dtype_supported to test.helpers (#3762)

2024-03-15 14:33:26 -04:00

testextra

add Tensor.replace (#3738)

2024-03-14 13:34:14 -07:00

unit

factor out partial in SumNode div int (#3841)

2024-03-20 16:34:33 -04:00

web

fast path for copy (#2548)

2023-12-01 11:34:47 -08:00

__init__.py

All devices are equal! (#196)

2020-12-15 23:44:08 -08:00

Dockerfile

Docker fix (#1039)

2023-06-25 10:38:58 -07:00

helpers.py

threefry again (#3785)

2024-03-18 16:47:07 -04:00

test_arange.py

use old cumsum optimization for arange (#3813)

2024-03-18 20:01:03 -04:00

test_assign.py

restrict assignment to base (#3809)

2024-03-18 15:33:06 -07:00

test_conv_shapetracker.py

multioutput ScheduleItem (#3699)

2024-03-13 08:59:38 -07:00

test_conv.py

add back "CPU" in test_onnx_backend supports_device (#3426)

2024-02-16 00:49:30 -05:00

test_copy_speed.py

remove cpu and torch backends (#3399)

2024-02-15 16:55:39 +01:00

test_custom_function.py

support pickling tensors and others (#3787)

2024-03-17 18:29:14 -07:00

test_device_speed.py

cleanup tests Device[Device.DEFAULT] is always Compiled (#3645)

2024-03-07 11:15:42 -05:00

test_dtype_alu.py

remove hip backend (#3783)

2024-03-17 10:12:16 -07:00

test_dtype.py

use int32 instead of default_int in simplify_phi_loops (#3828)

2024-03-19 17:49:58 -04:00

test_fusion_op.py

Remove Interpreted device & remaining CPU/TORCH ref (#3423)

2024-02-16 00:30:21 -05:00

test_fuzz_shape_ops.py

fix: align .split, .chunk and .unsqueeze with torch, add fuzz tests (#3505)

2024-02-28 17:06:39 -08:00

test_gc.py

threefry again (#3785)

2024-03-18 16:47:07 -04:00

test_image_dtype.py

ban __bool__ on Tensor (#3632)

2024-03-06 17:12:35 -05:00

test_jit.py

threefry again (#3785)

2024-03-18 16:47:07 -04:00

test_kernel_cache.py

move the compiler cache to be global (#2957)

2024-01-01 10:59:56 -08:00

test_lazybuffer.py

multioutput ScheduleItem (#3699)

2024-03-13 08:59:38 -07:00

test_lazyop.py

move create schedule and delete old API (#3377)

2024-02-12 18:10:45 +01:00

test_linearizer_failures.py

reuse fuzz_linearizer.compare_linearizer in test_linearizer_failures (#3861)

2024-03-21 14:12:27 -04:00

test_linearizer_overflows.py

remove HIP in core tinygrad (#3810)

2024-03-18 18:19:27 -04:00

test_linearizer.py

Fix factoring bug (O(n) arange related) (#3817)

2024-03-19 11:49:42 -04:00

test_masked_st.py

multitensor start (#2676)

2023-12-07 17:07:05 -08:00

test_method_cache.py

simple LoadOps.ASSIGN (#3745)

2024-03-14 20:44:34 -07:00

test_multitensor.py

Ring allreduce try 2 (#3852)

2024-03-21 19:17:51 -04:00

test_net_speed.py

Remove pytest markers (#2831)

2023-12-18 18:53:28 -05:00

test_nn.py

fix SCE ignore_index with label_smoothing (#3574)

2024-03-01 22:19:45 -05:00

test_ops.py

user _resolve_dim in argmax (#3846)

2024-03-20 20:17:30 -04:00

test_optim.py

remove realize from optimizer (#2880)

2023-12-20 16:42:41 -08:00

test_pickle.py

hotfix: add test for JIT reset

2024-03-17 21:35:49 -07:00

test_randomness.py

threefry again (#3785)

2024-03-18 16:47:07 -04:00

test_sample.py

enable test_sample for all backend (#2593)

2023-12-03 17:20:27 -05:00

test_schedule.py

threefry again (#3785)

2024-03-18 16:47:07 -04:00

test_search.py

multioutput ScheduleItem (#3699)

2024-03-13 08:59:38 -07:00

test_specific_conv.py

move is_dtype_supported to test.helpers (#3762)

2024-03-15 14:33:26 -04:00

test_speed_v_torch.py

move graph.py and jit.py into features (#3376)

2024-02-12 17:34:34 +01:00

test_symbolic_jit.py

bring ptx back (#3623)

2024-03-06 13:34:21 -08:00

test_symbolic_ops.py

bring ptx back (#3623)

2024-03-06 13:34:21 -08:00

test_symbolic_shapetracker.py

unbind view or shapetracker also returns var_val (#3067)

2024-01-09 21:45:05 -05:00

test_tensor_data.py

dtype fmt (#3132)

2024-01-15 11:31:54 -08:00

test_tensor.py

threefry again (#3785)

2024-03-18 16:47:07 -04:00

test_to_numpy.py

Apply ruff linting rules to tests (#2473)

2023-11-27 21:24:06 -08:00

test_uop_graph.py

add some graph tests (#3702)

2024-03-12 09:49:47 -07:00

test_uops_stats.py

uops flop counter (#3373)

2024-02-20 09:36:30 +01:00

test_uops.py

support pickling tensors and others (#3787)

2024-03-17 18:29:14 -07:00

test_winograd.py

multioutput ScheduleItem (#3699)

2024-03-13 08:59:38 -07:00

test_zero_copy.py

remove numpy from device (#3123)

2024-01-14 19:36:05 -08:00