chenyu 90c3ed17c5 move cast to before softmax in attention (#9213) ...

* move cast to before softmax in attention

saved some memory because exp (which is used for backward) are done in half. training bert seems fine and can fit BS=78 now (from 66)

* test

2025-02-24 17:24:59 -05:00

..

__init__.py

more test cleanups (#2631)

2023-12-05 16:17:57 -08:00

test_allreduce.py

allow double copy [pr] (#8559)

2025-01-10 18:21:01 -05:00

test_attention.py

move cast to before softmax in attention (#9213)

2025-02-24 17:24:59 -05:00

test_device.py

fix device canonicalize for :0 in middle [pr] (#8193)

2024-12-12 16:32:36 -05:00

test_disk_cache.py

helpers.diskcache_clear (#4436)

2024-05-05 14:19:01 -04:00

test_disk_tensor.py

CLANG -> CPU (#9189)

2025-02-20 18:03:09 -05:00

test_elf.py

CLANG -> CPU (#9189)

2025-02-20 18:03:09 -05:00

test_gguf.py

add a missing unittest.main() [pr] (#8274)

2024-12-16 14:28:10 -05:00

test_gradient.py

allow VIEW(BUFFER) in Tensor UOps [pr] (#9210)

2025-02-24 13:06:15 +01:00

test_graph_rewrite.py

rewriter -> devectorizer [pr] (#9147)

2025-02-18 12:42:08 +08:00

test_helpers.py

run unit test on windows ci (#9187)

2025-02-20 14:40:41 -05:00

test_pattern_matcher.py

Remove UnaryOps, BinaryOps, TernaryOps, MetaOps [pr] (#7725)

2024-11-16 20:56:56 +08:00

test_qcom.py

BufferSpec and ProgramSpec [pr] (#7814)

2024-11-21 12:18:05 +08:00

test_rewrite_map.py

symbolic gets its own file [pr] (#9132)

2025-02-17 18:55:21 +08:00

test_shapetracker_math.py

[bounty] Stride is flip (#8792)

2025-01-31 11:34:10 +09:00

test_shapetracker.py

rewriter -> devectorizer [pr] (#9147)

2025-02-18 12:42:08 +08:00

test_shm_tensor.py

line reduction [pr] (#7296)

2024-10-25 17:05:09 +07:00

test_simplify_valid_idx.py

rewriter -> devectorizer [pr] (#9147)

2025-02-18 12:42:08 +08:00

test_tar.py

accept filename decorator [pr] (#8049)

2024-12-05 11:40:59 +08:00

test_tensor_io.py

tar_extract with Tensors (#7853)

2024-12-04 17:03:19 +08:00

test_tensor_uop_representation.py

allow VIEW(BUFFER) in Tensor UOps [pr] (#9210)

2025-02-24 13:06:15 +01:00

test_tqdm.py

Context manager support for tqdm (#7770)

2024-11-18 14:12:03 -05:00

test_transcendental_helpers.py

Prevent const folding in test_payne_hanek_reduction (#9088)

2025-02-17 17:31:10 +08:00

test_uop_resolve.py

rename MathTraits to maximum [pr] (#7356)

2024-10-29 16:43:04 +08:00

test_uop_symbolic.py

rewriter -> devectorizer [pr] (#9147)

2025-02-18 12:42:08 +08:00

test_uop_vmin_vmax.py

type annotation of resolve [pr] (#8467)

2025-01-01 10:21:59 -05:00

test_verify_ast.py

CLANG -> CPU (#9189)

2025-02-20 18:03:09 -05:00

test_view.py

Small bug in _reshape_mask (#8538)

2025-01-08 22:11:24 -05:00

test_viz.py

support custom name function in viz [pr] (#9219)

2025-02-24 03:03:25 +02:00