when testing alu for bfloat16, it should cast inputs to bfloat16 first, otherwise numpy has both errors from input and errors from alu which is more inaccurate
* Remote `.q(..., wait=True)`
Seems a bit cleaner than doing `.batch_request()` after `.q(...)` for
requests with return value.
* Remote finalize
---------
Co-authored-by: George Hotz <72895+geohot@users.noreply.github.com>
* Make `dev` a property of `Allocator`
(this is a prereq refactor for #10285)
At least `BufferXfer.copy` accesses it assuming it's always present,
currently most devices just add this property on their own repeating
the same code over and over again.
This is also a bit footguny, see `RemoteAllocator` that named this
property `device` instead of `dev`, i could obviously just change that
in one place but doing it globally seems like a better solution (and it
reduces code duplication too).
`MallocAllocator` is a bit special, but passing `None` works just fine.
* typing
* ignore type instead of cast
Should be a small speed improvement but the main reason this is needed
is to have a defined ordering of RemoteRequests within one host so that
transfers won't required doing something like:
```python
src_dev.batch_submit()
dest_dev.q(Transfer(dest, src_dev.session, src))
dest_dev.batch_submit()
```
for correctness.
Not strictly required for anything but soon there will be like 4 new
properties and having it be a huge json just seems like a bad taste.
It also seems right to not have a separate endpoint for this, just
`GetProperties` request that returns a repr of this similar to how
requests are sent in `BatchRequest`.
This will also make a switch to anything other than http much simpler
if it will be required for any reason, like just a tcp stream of
`BatchRequest`s
* Basic remote multi support
Simplest thing to be able to use remote with multiple gpus, very slow
because no transfers (copyin copyout for cross-device copies)
* tests
