mirror of
https://github.com/Significant-Gravitas/AutoGPT.git
synced 2026-04-08 03:00:28 -04:00
9a6ae90d12
Currently, we are using PyClamd to run a file anti-virus scan for all the files uploaded into the platform. We split the file into small chunks and serially check the chunks for the virus scan. The socket is not thread-safe, and we need to create multiple sockets across many threads to leverage concurrency. To make this step concurrent and keep it fully async, we need to migrate PyClamd to aioclamd. ### Changes 🏗️ Convert pyclamd to aioclamd, leverage chunk parallelism scan with a semaphore limiting the concurrency limit. #### Side Note Shout-out to @tedyu for raising this improvement idea. ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: <!-- Put your test plan here: --> - [x] Execute file upload into the platform