[BUG]: Fit File Import Results in Errors but no import #4

New Issue

AtHeartEngineer · 2025-07-08T08:37:19-04:00

AtHeartEngineer commented

2025-07-08 08:37:19 -04:00

Originally created by @TheBig-O on 7/5/2025

Checklist

I have searched existing issues to make sure this bug hasn't already been reported.
I have checked the documentation at https://docs.endurain.com and didn't find a solution.

Description

**** UPDATE
Turns out it was an issue with MacOS. When you copy files from a mac to Linux, Macs also copy a "dot" file with the same name as the .fit file with a dot on the front. These are AppleDouble files and store metadata (like resource forks) but are not supported by Linux systems and are useless but confuse the bulk importer when you have both the actual .fit file and these AppleDouble files.
Once I removed these files and ran the importer again, the process worked just fine.
Perhaps, the only thing to note from this is maybe to look at ignoring any AppleDouble files when the bulk importer runs. (Or maybe any "dot" files.)

+++++++++++
When attempting to bulk import 3 .fit files, the result is no imports and the following error in the logs

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/app/backend/fit/utils.py", line 479, in parse_fit_file
    for frame in fit_data:
                 ^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/fitdecode/reader.py", line 232, in __iter__
    yield from self._read_next()
  File "/usr/local/lib/python3.13/site-packages/fitdecode/reader.py", line 351, in _read_next
    self._read_header()
    ~~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/site-packages/fitdecode/reader.py", line 424, in _read_header
    raise FitHeaderError(f'not a FIT file @ {self._chunk_offset}')
fitdecode.exceptions.FitHeaderError: not a FIT file @ 0
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 74, in app
    await response(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/responses.py", line 160, in __call__
    await self.background()
  File "/usr/local/lib/python3.13/site-packages/starlette/background.py", line 41, in __call__
    await task()
  File "/usr/local/lib/python3.13/site-packages/starlette/background.py", line 28, in __call__
    await run_in_threadpool(self.func, *self.args, **self.kwargs)
  File "/usr/local/lib/python3.13/site-packages/starlette/concurrency.py", line 37, in run_in_threadpool
    return await anyio.to_thread.run_sync(func)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        func, args, abandon_on_cancel=abandon_on_cancel, limiter=limiter
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/anyio/_backends/_asyncio.py", line 967, in run
    result = context.run(func, *args)
  File "/app/backend/activities/activity/utils.py", line 364, in parse_and_store_activity_from_file
    raise http_err
  File "/app/backend/activities/activity/utils.py", line 292, in parse_and_store_activity_from_file
    parsed_info = parse_file(
        token_user_id,
    ...<3 lines>...
        db,
    )
  File "/app/backend/activities/activity/utils.py", line 540, in parse_file
    raise http_err
  File "/app/backend/activities/activity/utils.py", line 528, in parse_file
    parsed_info = fit_utils.parse_fit_file(filename, db)
  File "/app/backend/fit/utils.py", line 750, in parse_fit_file
    raise HTTPException(
    ...<2 lines>...
    ) from err
fastapi.exceptions.HTTPException: 500: Can't parse FIT file
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        self.scope, self.receive, self.send
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.13/site-packages/opentelemetry/instrumentation/asgi/__init__.py", line 743, in __call__
    await self.app(scope, otel_receive, otel_send)
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/base.py", line 183, in __call__
    raise app_exc
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/base.py", line 141, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/cors.py", line 93, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/cors.py", line 144, in simple_response
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 714, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 734, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 56, in wrapped_app
    raise RuntimeError("Caught handled exception, but response already started.") from exc
RuntimeError: Caught handled exception, but response already started.

Steps to Reproduce

Version: 12.7
Go to 'Integrations, Bulk Import'
Click on 'Bulk Import'
See error: Above

Expected Behavior

I expected the .fit files to be imported.

Screenshots, logs or files

see above

Additional Context

This worked as recently as the previous version

*Originally created by @TheBig-O on 7/5/2025* ## Checklist - [x] I have searched **existing issues** to make sure this bug hasn't already been reported. - [x] I have checked the **documentation** at [https://docs.endurain.com](https://docs.endurain.com) and didn't find a solution. ## Description **** UPDATE Turns out it was an issue with MacOS. When you copy files from a mac to Linux, Macs also copy a "dot" file with the same name as the `.fit` file with a dot on the front. These are AppleDouble files and store metadata (like resource forks) but are not supported by Linux systems and are useless but confuse the bulk importer when you have both the actual `.fit` file and these AppleDouble files. Once I removed these files and ran the importer again, the process worked just fine. Perhaps, the only thing to note from this is maybe to look at ignoring any AppleDouble files when the bulk importer runs. (Or maybe any "dot" files.) +++++++++++ When attempting to bulk import 3 .fit files, the result is no imports and the following error in the logs ``` ERROR: Exception in ASGI application Traceback (most recent call last): File "/app/backend/fit/utils.py", line 479, in parse_fit_file for frame in fit_data: ^^^^^^^^ File "/usr/local/lib/python3.13/site-packages/fitdecode/reader.py", line 232, in __iter__ yield from self._read_next() File "/usr/local/lib/python3.13/site-packages/fitdecode/reader.py", line 351, in _read_next self._read_header() ~~~~~~~~~~~~~~~~~^^ File "/usr/local/lib/python3.13/site-packages/fitdecode/reader.py", line 424, in _read_header raise FitHeaderError(f'not a FIT file @ {self._chunk_offset}') fitdecode.exceptions.FitHeaderError: not a FIT file @ 0 The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app await app(scope, receive, sender) File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 74, in app await response(scope, receive, send) File "/usr/local/lib/python3.13/site-packages/starlette/responses.py", line 160, in __call__ await self.background() File "/usr/local/lib/python3.13/site-packages/starlette/background.py", line 41, in __call__ await task() File "/usr/local/lib/python3.13/site-packages/starlette/background.py", line 28, in __call__ await run_in_threadpool(self.func, *self.args, **self.kwargs) File "/usr/local/lib/python3.13/site-packages/starlette/concurrency.py", line 37, in run_in_threadpool return await anyio.to_thread.run_sync(func) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.13/site-packages/anyio/to_thread.py", line 56, in run_sync return await get_async_backend().run_sync_in_worker_thread( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ func, args, abandon_on_cancel=abandon_on_cancel, limiter=limiter ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/usr/local/lib/python3.13/site-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread return await future ^^^^^^^^^^^^ File "/usr/local/lib/python3.13/site-packages/anyio/_backends/_asyncio.py", line 967, in run result = context.run(func, *args) File "/app/backend/activities/activity/utils.py", line 364, in parse_and_store_activity_from_file raise http_err File "/app/backend/activities/activity/utils.py", line 292, in parse_and_store_activity_from_file parsed_info = parse_file( token_user_id, ...<3 lines>... db, ) File "/app/backend/activities/activity/utils.py", line 540, in parse_file raise http_err File "/app/backend/activities/activity/utils.py", line 528, in parse_file parsed_info = fit_utils.parse_fit_file(filename, db) File "/app/backend/fit/utils.py", line 750, in parse_fit_file raise HTTPException( ...<2 lines>... ) from err fastapi.exceptions.HTTPException: 500: Can't parse FIT file The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.13/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ self.scope, self.receive, self.send ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/usr/local/lib/python3.13/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__ return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.13/site-packages/fastapi/applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "/usr/local/lib/python3.13/site-packages/starlette/applications.py", line 112, in __call__ await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.13/site-packages/starlette/middleware/errors.py", line 187, in __call__ raise exc File "/usr/local/lib/python3.13/site-packages/starlette/middleware/errors.py", line 165, in __call__ await self.app(scope, receive, _send) File "/usr/local/lib/python3.13/site-packages/opentelemetry/instrumentation/asgi/__init__.py", line 743, in __call__ await self.app(scope, otel_receive, otel_send) File "/usr/local/lib/python3.13/site-packages/starlette/middleware/base.py", line 183, in __call__ raise app_exc File "/usr/local/lib/python3.13/site-packages/starlette/middleware/base.py", line 141, in coro await self.app(scope, receive_or_disconnect, send_no_error) File "/usr/local/lib/python3.13/site-packages/starlette/middleware/cors.py", line 93, in __call__ await self.simple_response(scope, receive, send, request_headers=headers) File "/usr/local/lib/python3.13/site-packages/starlette/middleware/cors.py", line 144, in simple_response await self.app(scope, receive, send) File "/usr/local/lib/python3.13/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app raise exc File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app await app(scope, receive, sender) File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 714, in __call__ await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 734, in app await route.handle(scope, receive, send) File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 288, in handle await self.app(scope, receive, send) File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 76, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 56, in wrapped_app raise RuntimeError("Caught handled exception, but response already started.") from exc RuntimeError: Caught handled exception, but response already started. ``` ## Steps to Reproduce 1. Version: 12.7 2. Go to 'Integrations, Bulk Import' 3. Click on 'Bulk Import' 4. See error: Above ## Expected Behavior I expected the .fit files to be imported. ## Screenshots, logs or files see above ## Additional Context This worked as recently as the previous version

AtHeartEngineer closed this issue

2025-07-08 08:37:19 -04:00

AtHeartEngineer added the bug bug labels 2025-07-08 08:37:20 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github/endurain#4