Pagination and Rate limiting

.gitignore

@@ -46,10 +46,6 @@ htmlcov/
 .mypy_cache/
 *,cover
 
-# Mac
-.DS_Store
-*,cover
-
 # Mac
 .DS_Store
 

README.md

@@ -19,7 +19,7 @@ OAuth2 PKCE authentication and resource-based API interactions.
 
 ## Installation
 
-This package requires Python 3.13 or later.
+This package requires Python 3.13 (or later, when there is a later).
 
 Once published, install like this:
 
@@ -59,13 +59,13 @@ except Exception as e:
 ```
 
 The response will always be the body of the API response, and is almost always a
-`Dict`, `List` or `None`. `nutrition.get_activity_tcx` is the exception. It
-returns XML (as a `str`).
+`JSONDict`, `JSONList` or `None`. `nutrition.get_activity_tcx` is the exception.
+It returns XML (as a `str`).
 
 ## Method Aliases
 
-All resource methods are available directly from the client instance. This means
-you can use:
+All API methods are available directly from the client instance. This means you
+can use:
 
 ```python
 # Short form with method aliases
@@ -87,7 +87,7 @@ Both approaches are equivalent, but aliases provide a more concise syntax.
 
 ## Authentication
 
-Uses a local callback server to automatically handle the OAuth2 flow:
+Authentication wses a local callback server to handle the OAuth2 flow:
 
 ```python
 client = FitbitClient(
@@ -97,7 +97,7 @@ client = FitbitClient(
     token_cache_path="/tmp/fb_tokens.json" 
 )
 
-# Will open browser and handle callback automatically
+# This will open browser and handle callback automatically:
 client.authenticate()
 ```
 
@@ -129,7 +129,10 @@ Where secrets.json contains:
 }
 ```
 
-You can also include the optional token_cache_path:
+Using this strategy, you can initialize the client with several additional
+parameter arguments (such as [Rate Limiting](#rate-limiting) and language/locale
+options; see the [FitbitClient](fitbit_client/client.py) initializer). Perhaps
+the most useful of these is the `token_cache_path`:
 
 ```json
 {
@@ -144,18 +147,65 @@ The `token_cache_path` parameter allows you to persist authentication tokens
 between sessions. If provided, the client will:
 
 1. Load existing tokens from this file if available (avoiding re-authentication)
-
 2. Save new or refreshed tokens to this file automatically
-
 3. Handle token refresh when expired tokens are detected
 
 ## Setting Up Your Fitbit App
 
 1. Go to dev.fitbit.com and create a new application
-2. Set OAuth 2.0 Application Type to "Personal"
+2. Set OAuth 2.0 Application Type to "Personal" (or other types, if you know
+   what you're doing)
 3. Set Callback URL to "https://localhost:8080" (or your preferred local URL)
 4. Copy your Client ID and Client Secret
 
+## Pagination
+
+Some Fitbit API endpoints support pagination for large result sets. With this
+client, you can work with paginated endpoints in two ways:
+
+```python
+# Standard way - get a single page of results
+sleep_logs = client.get_sleep_log_list(before_date="2025-01-01")
+
+# Iterator way - get an iterator that fetches all pages automatically
+for page in client.get_sleep_log_list(before_date="2025-01-01", as_iterator=True):
+    for sleep_entry in page["sleep"]:
+        print(sleep_entry["logId"])
+```
+
+Endpoints that support pagination:
+
+- `get_sleep_log_list()`
+- `get_activity_log_list()`
+- `get_ecg_log_list()`
+- `get_irn_alerts_list()`
+
+For more details, see [PAGINATION.md](docs/PAGINATION.md).
+
+## Rate Limiting
+
+The client includes automatic retry handling for rate-limited requests. When a
+rate limit is encountered, the client will:
+
+1. Log the rate limit event
+2. Wait using an exponential backoff strategy
+3. Automatically retry the request
+
+You can configure rate limiting behavior:
+
+```python
+client = FitbitClient(
+    client_id="YOUR_CLIENT_ID",
+    client_secret="YOUR_CLIENT_SECRET",
+    redirect_uri="https://localhost:8080",
+    max_retries=5,                # Maximum number of retry attempts (default: 3)
+    retry_after_seconds=30,       # Base wait time in seconds (default: 60)
+    retry_backoff_factor=2.0      # Multiplier for successive waits (default: 1.5)
+)
+```
+
+For more details, see [RATE_LIMITING.md](docs/RATE_LIMITING.md).
+
 ## Additional Documentation
 
 ### For API Library Users
@@ -165,6 +215,9 @@ between sessions. If provided, the client will:
 - [NAMING.md](docs/NAMING.md): API method naming conventions
 - [VALIDATIONS.md](docs/VALIDATIONS.md): Input parameter validation
 - [ERROR_HANDLING.md](docs/ERROR_HANDLING.md): Exception hierarchy and handling
+- [PAGINATION.md](docs/PAGINATION.md): Working with paginated endpoints
+- [RATE_LIMITING.md](docs/RATE_LIMITING.md): Rate limit handling and
+  configuration
 
 It's also worth reviewing
 [Fitbit's Best Practices](https://dev.fitbit.com/build/reference/web-api/developer-guide/best-practices/)

docs/DEVELOPMENT.md

@@ -307,6 +307,33 @@ The OAuth callback mechanism is implemented using two main classes:
 
 This section will be documented as we near our first release.
 
+## Pagination Implementation
+
+The pagination implementation uses the following approach:
+
+### Pagination Iterator
+
+- Uses the `PaginatedIterator` class that implements the Python `Iterator`
+  protocol
+- Automatically handles fetching the next page when needed using the `next` URL
+  from pagination metadata
+- Properly handles edge cases like invalid responses, missing pagination data,
+  and API errors
+
+### Type Safety
+
+- Uses `TYPE_CHECKING` from the typing module to avoid circular imports at
+  runtime
+- Maintains complete type safety and mypy compatibility
+- All pagination-related code has 100% test coverage
+
+### Resource Integration
+
+Each endpoint that supports pagination has an `as_iterator` parameter that, when
+set to `True`, returns a `PaginatedIterator` instead of the raw API response.
+This makes it easy to iterate through all pages of results without manually
+handling pagination.
+
 ## Intraday Data Support
 
 This client implements intraday data endpoints (detailed heart rate, steps, etc)

docs/NAMING.md

@@ -2,10 +2,10 @@
 
 ## Naming Principles
 
-The method names in this library are designed to align with the official Fitbit
-Web API Documentation. When there are inconsistencies in the official
-documentation, we prioritize the URL slug. For example, if the documentation
-page title says "Get **Time Series** by Date" but the URL is
+The API method names are designed to align with the official Fitbit Web API
+Documentation. When there are inconsistencies in the official documentation, we
+prefer the URL slug for the page. For example, if the documentation page title
+says "Get **Time Series** by Date" but the URL is
 ".../get-azm-timeseries-by-date/", our method will be named
 `get_azm_timeseries_by_date()`. (not `get_azm_time_series_by_date()`).
 
@@ -20,13 +20,16 @@ you navigate the API more effectively:
 
 ### Method Name vs. Functionality Inconsistencies
 
+Examples:
+
 - `create_activity_goals` creates only one goal at a time, despite the plural
   name
-- `add_favorite_foods` adds one food at a time, while all other creation methods
-  start with "create".
+- `add_favorite_foods` adds one food at a time; also, all other creation/POST
+  methods start with "`create_`".
 - `get_sleep_goals` returns a single goal, not multiple goals
 - Additionally, some pluralized methods return lists, while others return
-  dictionaries containing lists
+  dictionaries containing lists (see
+  [Response Structure Inconsistencies](#response-structure-inconsistencies))
 
 For user convenience, these inconsistencies have aliases:
 

docs/PAGINATION.md

@@ -0,0 +1,80 @@
+# Pagination
+
+Some API endpoints return potentially large result sets and support pagination.
+We provide an easy and pythonic way to work with these paginated endpoints as
+iterators.
+
+## Supported Endpoints
+
+The following endpoints support pagination:
+
+- `client.get_sleep_log_list()`
+- `client.get_activity_log_list()`
+- `client.get_ecg_log_list()`
+- `client.get_irn_alerts_list()`
+
+## Usage
+
+### Standard Mode
+
+By default, all endpoints return a single page of results with pagination
+metadata:
+
+```python
+# Get a single (the first) page of sleep logs
+sleep_data = client.get_sleep_log_list(
+    before_date="2025-01-01",
+    sort=SortDirection.DESCENDING,
+    limit=10
+)
+```
+
+### Iterator Mode
+
+When you need to process multiple pages of data, use iterator mode:
+
+```python
+iterator = client.get_sleep_log_list(
+    before_date="2025-01-01",
+    sort=SortDirection.DESCENDING,
+    limit=10,
+    as_iterator=True # Creates an iterator for all pages
+)
+
+# Process all pages - the iterator fetches new pages as needed
+for page in iterator:
+    # Each page has the same structure as the standard response
+    for sleep_entry in page["sleep"]:
+        print(f"Sleep log ID: {sleep_entry['logId']}")
+```
+
+## Pagination Parameters
+
+Different endpoints support different pagination parameters, but they generally
+follow these patterns:
+
+| Parameter     | Description                     | Constraints                                                 |
+| ------------- | ------------------------------- | ----------------------------------------------------------- |
+| `before_date` | Return entries before this date | Must use with `sort=SortDirection.DESCENDING`               |
+| `after_date`  | Return entries after this date  | Must use with `sort=SortDirection.ASCENDING`                |
+| `limit`       | Maximum items per page          | Varies by endpoint (10-100)                                 |
+| `offset`      | Starting position               | Usually only `0` is supported                               |
+| `sort`        | Sort direction                  | Use `SortDirection.ASCENDING` or `SortDirection.DESCENDING` |
+
+## Endpoint-Specific Notes
+
+Each paginated endpoint has specific constraints:
+
+### `get_sleep_log_list`
+
+- Max limit: 100 entries per page
+- Date filtering: `before_date` or `after_date` (must specify one but not both)
+
+### `get_activity_log_list`
+
+- Max limit: 100 entries per page
+- Date filtering: `before_date` or `after_date` (must specify one but not both)
+
+### `get_ecg_log_list` and `get_irn_alerts_list`
+
+- Max limit: 10 entries per page