[FEATURE]: Implement throttling against geocode.maps.co API #35

New Issue

AtHeartEngineer · 2025-07-08T08:38:12-04:00

AtHeartEngineer commented

2025-07-08 08:38:12 -04:00

Originally created by @fyksen on 6/18/2025

Summary

It doesn't seem to be any throttling implemented against the geocode API. This makes it unreliable to import many activities.

Proposed Solution

Maybe add a global throttling on the backend/app/activities/activity/utils.py
And make it a environment variable that defaults to 1 (API call per second).
Make it configurable, if people pay for the service.

Something like:

import threading
import time

# Throttling settings for Geocode maps API
_GEOCODES_MAPS_RATE_LIMIT = float(os.environ.get("GEOCODES_MAPS_RATE_LIMIT", 1))
_GEOCODES_MIN_INTERVAL = 1.0 / _GEOCODES_MAPS_RATE_LIMIT if _GEOCODES_MAPS_RATE_LIMIT > 0 else 0
_GEOCODES_LOCK = threading.Lock()
_GEOCODES_LAST_CALL = 0.0

def location_based_on_coordinates(latitude, longitude) -> dict | None:
    if latitude is None or longitude is None:
        return None

    if os.environ.get("GEOCODES_MAPS_API") == "changeme":
        return None

    # Create a dictionary with the parameters for the request
    url_params = {
        "lat": latitude,
        "lon": longitude,
        "api_key": os.environ.get("GEOCODES_MAPS_API"),
    }

    # Create the URL for the request
    url = f"https://geocode.maps.co/reverse?{urlencode(url_params)}"

    # Throttle requests according to configured rate limit
    global _GEOCODES_LAST_CALL
    if _GEOCODES_MIN_INTERVAL > 0:
        with _GEOCODES_LOCK:
            now = time.monotonic()
            interval = _GEOCODES_MIN_INTERVAL - (now - _GEOCODES_LAST_CALL)
            if interval > 0:
                time.sleep(interval)
            _GEOCODES_LAST_CALL = time.monotonic()

    # Make the request and get the response
    try:
        # Make the request and get the response
        response = requests.get(url)
        response.raise_for_status()

        # Get the data from the response
        data = response.json().get("address", {})

        # Return the data
        return {
            "city": data.get("city"),
            "town": data.get("town"),
            "country": data.get("country"),
        }
    except requests.exceptions.RequestException as err:
        # Log the error
        core_logger.print_to_log_and_console(
            f"Error in location_based_on_coordinates - {str(err)}", "error"
        )
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail=f"Error in location_based_on_coordinates: {str(err)}",
        )

Remove the throttling on the parse .fit file. Remove this?

                            # Wait for 1 second (for geocoding API rate limiting)
                            timelib.sleep(1)

*Originally created by @fyksen on 6/18/2025* ## Summary It doesn't seem to be any throttling implemented against the geocode API. This makes it unreliable to import many activities. ## Proposed Solution Maybe add a global throttling on the `backend/app/activities/activity/utils.py` And make it a environment variable that defaults to 1 (API call per second). Make it configurable, if people pay for the [service](https://geocode.maps.co/plans/). Something like: ``` import threading import time # Throttling settings for Geocode maps API _GEOCODES_MAPS_RATE_LIMIT = float(os.environ.get("GEOCODES_MAPS_RATE_LIMIT", 1)) _GEOCODES_MIN_INTERVAL = 1.0 / _GEOCODES_MAPS_RATE_LIMIT if _GEOCODES_MAPS_RATE_LIMIT > 0 else 0 _GEOCODES_LOCK = threading.Lock() _GEOCODES_LAST_CALL = 0.0 def location_based_on_coordinates(latitude, longitude) -> dict | None: if latitude is None or longitude is None: return None if os.environ.get("GEOCODES_MAPS_API") == "changeme": return None # Create a dictionary with the parameters for the request url_params = { "lat": latitude, "lon": longitude, "api_key": os.environ.get("GEOCODES_MAPS_API"), } # Create the URL for the request url = f"https://geocode.maps.co/reverse?{urlencode(url_params)}" # Throttle requests according to configured rate limit global _GEOCODES_LAST_CALL if _GEOCODES_MIN_INTERVAL > 0: with _GEOCODES_LOCK: now = time.monotonic() interval = _GEOCODES_MIN_INTERVAL - (now - _GEOCODES_LAST_CALL) if interval > 0: time.sleep(interval) _GEOCODES_LAST_CALL = time.monotonic() # Make the request and get the response try: # Make the request and get the response response = requests.get(url) response.raise_for_status() # Get the data from the response data = response.json().get("address", {}) # Return the data return { "city": data.get("city"), "town": data.get("town"), "country": data.get("country"), } except requests.exceptions.RequestException as err: # Log the error core_logger.print_to_log_and_console( f"Error in location_based_on_coordinates - {str(err)}", "error" ) raise HTTPException( status_code=status.HTTP_401_UNAUTHORIZED, detail=f"Error in location_based_on_coordinates: {str(err)}", ) ``` Remove the throttling on the parse .fit file. Remove this? ``` # Wait for 1 second (for geocoding API rate limiting) timelib.sleep(1) ```

AtHeartEngineer added the enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement labels 2025-07-08 08:38:14 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github/endurain#35