Fixed incorrect API endpoint paths that were causing JSON parsing errors:
- Changed /api/channels/stream-groups/ to /api/channels/streams/groups/ (correct endpoint)
- Changed /api/channels/m3us/ to /api/m3u/accounts/ (correct endpoint)
- Removed pagination loops since both endpoints return simple arrays, not paginated responses
Fixed stream filtering to use correct field names:
- Changed stream_group_id to channel_group (streams use channel_group FK)
- Changed m3u_id to m3u_account (streams use m3u_account FK)
- Use group_name_to_id instead of stream_group_name_to_id (stream groups endpoint returns strings, not objects)
These changes resolve the "Expecting value: line 1 column 1 (char 0)" JSON errors.
Added proper error handling for API endpoints that return empty or invalid JSON responses. This prevents JSONDecodeError exceptions when the StreamMaster API returns empty content.
Changes:
- Added empty response checking before JSON parsing in _get_api_data, _patch_api_data, and _post_api_data methods
- Added try-except blocks to catch json.JSONDecodeError with helpful error messages
- Return appropriate empty values ([] for GET, {} for PATCH/POST) when response is empty
- Added debug logging to show response content when JSON parsing fails
Fixes errors like: "Expecting value: line 1 column 1 (char 0)" when calling /api/channels/stream-groups/ and /api/channels/m3us/ endpoints
- Add new 'selected_m3us' setting field to filter streams by M3U source
- Fetch M3U sources from API endpoint /api/channels/m3us/
- Filter streams based on selected M3U sources (comma-separated or blank for ALL)
- Update CSV header comments to display selected M3U sources
- Store selected_m3us in processed_data for persistence
- Apply filter in both Preview Changes and Add Streams actions
This allows users to limit which streams are considered during matching
by specifying M3U source names (e.g., "IPTV Provider 1, Local M3U") or
leaving blank to use all available M3U sources. Works in conjunction with
the stream group filter for fine-grained stream selection.
- Add new 'selected_stream_groups' setting field to filter streams by group
- Fetch stream groups from API endpoint /api/channels/stream-groups/
- Filter streams based on selected stream groups (comma-separated or blank for ALL)
- Update CSV header comments to display selected stream groups
- Store selected_stream_groups in processed_data for persistence
- Apply filter in both Preview Changes and Add Streams actions
This allows users to limit which streams are considered during matching
by specifying stream group names (e.g., "TVE, Cable") or leaving blank
to use all available stream groups.
Additional logs moved to DEBUG level:
- Settings and configuration operations (loaded/saved settings, schedule config)
- API request details (PATCH/POST requests, token caching)
- Internal operations (frontend refresh, WebSocket updates)
- Pagination details (reached last page messages for groups/streams)
- Schedule syncing and viewing operations
- Channel database reloading in fuzzy matcher
These changes ensure that:
- INFO level shows only user-facing events and important state changes
- DEBUG level contains detailed operational and internal information
- Logs are consistent across all actions (preview_changes, add_streams, etc.)
- Cleaner log output for production use while maintaining debuggability
Benefits:
- Reduced log noise in production environments
- Easier troubleshooting with DEBUG level when needed
- Consistent logging patterns across the entire codebase
- Better separation between user-facing and internal operations
Changes to logging levels:
- Changed fuzzy_matcher channel database loading logs to DEBUG level
- Changed plugin validation and API authentication logs to DEBUG level
- Changed notification sending log to DEBUG level
- Changed per-file channel loading log to DEBUG level
Enhanced INFO logs:
- Updated total channels loaded log to include database names
- Added channel group statistics logging showing:
* Total number of groups processed
* For each group: name, channel count, and matched stream count
* Shows first 10 groups with summary for remaining groups
Benefits:
- Cleaner INFO logs show only essential information
- Detailed debugging information available at DEBUG level
- Better visibility into channel group processing and stream matching
- Easier troubleshooting with group-level statistics
Changes:
- Modified preview_changes_action to always reload channels before previewing
- Modified add_streams_to_channels_action to always reload channels before adding streams
- Removed file existence check that prevented reload when data was stale
- This ensures deleted channels (like ES DAZN 09/10) won't cause warnings
The processed data file is now refreshed on every preview/add operation,
ensuring users always work with current channel data from the Dispatcharr API.
Addresses issue where stale cached channel data causes foreign key errors when
attempting to create ChannelStream relationships for channels that no longer
exist in the database.
Changes:
- Add pre-validation check using Channel.objects.filter(id).exists() before
attempting any ChannelStream operations
- Gracefully skip deleted channels with warning log message including channel
name and ID
- Track count of skipped channels and report in final success message
- Suggest reloading channels when skipped channels are detected
This prevents the error:
"insert or update on table dispatcharr_channels_channelstream violates
foreign key constraint" when channel_id is not present in the channels table.
Integrates robust pagination error handling from PR #16 (by wdmitchell.uk)
into the current v0.6.0b codebase.
Changes:
- Add try-except error handling for all pagination API calls (groups and streams)
- Gracefully handle 404 errors when attempting to fetch non-existent pages
- Add empty result set detection to prevent unnecessary API calls
- Improve logging with detailed pagination status messages
- Re-raise errors only on first page (indicates real problem vs. end of pagination)
Fixes pagination bug where total records = exact multiple of page_size
(e.g., 700 streams with page_size=100 would crash on page 8)
Applied to 3 pagination loops:
1. Group fetching in validate_settings_action (line ~1693)
2. Group fetching in load_process_channels_action (line ~2019)
3. Stream fetching in load_process_channels_action (line ~2053)
Related: PR #16
# Stream-Mapparr Plugin Changelog (v0.6.0a -> v0.6.0b)
## New Features
* **Smart Analysis & Recommendations:**
* Added "What-if" analysis: Automatically tests matching at lower fuzzy thresholds (down to 65) to identify potential matches missed by current settings.
* Added Token Mismatch detection: Analyzes unmatched streams to suggest specific prefixes or suffixes (e.g., "US:", "HD") to add to the "Ignore Tags" list.
* **Enhanced CSV Export:**
* CSV files now include a detailed header section containing active settings, version info, and specific optimization recommendations based on the Smart Analysis.
* Rows now include data on potential matches found at lower thresholds.
* **Rate Limiting:** Added a "None (Disabled)" option to remove all artificial delays for faster processing on local networks.
* **Notifications:** Implemented WebSocket-based frontend notifications to display progress bars and success/error toasts during long-running operations.
## Improvements
* **Matching Logic:** Improved fuzzy matching algorithm to handle token overlap, better detecting matches where word order differs (e.g., "Channel Name US" vs "US Channel Name").
* **Maintenance:** Added a "Cleanup Orphaned Tasks" action to remove stale Celery Beat schedules left behind by older plugin versions.
* **Scheduling:** Updated schedule saving logic to provide immediate feedback on the next calculated run times in the local timezone.
## Internal
* Added `_get_matches_at_thresholds` and `_analyze_token_mismatch` methods to power the new recommendation engine.
* Refined `_clean_channel_name` to support optional country prefix removal.
- Update plugin.py version from 0.5.0 to 0.5.1
- Update fuzzy_matcher.py version from 25.314.1907 to 25.317.1200
- Remove 📚 Channel Databases header field from settings (keep dynamic database fields)
Root cause: Stream names were normalizing to empty strings through
aggressive pattern stripping, causing multiple unrelated channels
(GNT, MTV, TLC, BIS, TNT, etc.) to incorrectly match each other
with 100% similarity scores.
Changes made:
1. Fixed calculate_similarity() to return 0.0 for empty string
comparisons instead of 1.0, preventing false positives
2. Added validation in normalize_name() to log warnings when
normalization results in empty strings
3. Added empty string checks (< 2 chars) in all matching stages:
- fuzzy_match() Stage 1 (exact match)
- fuzzy_match() Stage 2 (substring match)
- find_best_match() (token-sort matching)
4. Added validation in plugin.py _match_streams_to_channel() to
skip streams with empty/short cleaned names in:
- Fuzzy matcher result collection
- JSON exact match section
- Basic substring matching fallback
5. Fixed country prefix regex pattern from [:|\s] to [:\s]
(removed incorrect pipe and backslash characters)
Testing: Added comprehensive test suite (test_fuzzy_matcher_fix.py)
that verifies empty strings don't match and valid matches still work.
All tests pass.
Changed the following log entries to DEBUG level to reduce log verbosity:
- API request logs
- Page fetch progress logs
- Fuzzy matcher usage logs
- Fuzzy match result logs
These logs are useful for debugging but too verbose for normal operation.
Implements automatic version checking that displays update status on the plugin settings page.
Features:
- Fetches latest release version from GitHub using stdlib (urllib.request)
- Displays version status at top of settings page (Update Available / Up to Date)
- Caches version check results for 24 hours to minimize API calls
- Re-checks automatically when plugin version changes
- Fails gracefully if network unavailable or GitHub API errors occur
Technical Details:
- Uses only Python standard library (no requests dependency)
- Version check triggered via fields property (runs when settings opened)
- Cache stored in /data/stream_mapparr_version_check.json
- Timeout set to 5 seconds for API requests
- All errors logged at debug level to avoid cluttering logs
Enhancements:
- Add database validation to settings validation
* Checks if at least one database file exists
* Validates JSON format of enabled databases
* Ensures at least one database is enabled
* Reports invalid/malformed database files
- Add database information to CSV exports
* Show which database was used for each channel match
* Add "database_used" column to preview and update CSVs
* Include enabled databases list in CSV header comments
* Track database source through matching pipeline
- Shorten validation success message
* Condensed format with key info only
* Separate success items and info items
* More readable for small notification areas
* Shows database count in validation results
Changes to _match_streams_to_channel():
- Now returns 5-tuple: (streams, cleaned_name, cleaned_stream_names, match_reason, database_used)
- Tracks country_code from channel database entry
- Returns "N/A" if channel not found in any database
CSV export improvements:
- Preview CSV includes database_used column
- Update CSV includes database_used column
- Header comments show: "Channel Databases Loaded: [list]"
- Update plugin.py version from 0.6.0 to 0.5.0a
- Update fuzzy_matcher.py version to 25.314.1907 (Nov 10, 2025 19:07)
- Update all references to v0.6.0 in README.md to v0.5.0a
Changed approach from modifying global GEOGRAPHIC_PATTERNS (which affects
all plugins using fuzzy_matcher.py) to adding a new optional parameter.
Changes:
- Reverted GEOGRAPHIC_PATTERNS to original US-only patterns
- Added new remove_country_prefix parameter to normalize_name() (default: False)
- Updated _clean_channel_name() to use remove_country_prefix=True by default
- Implemented smart prefix detection that avoids removing quality tags (HD, SD, UHD, FHD)
- Added fallback country prefix removal in basic cleaning code
- Updated README to clarify country code prefix handling approach
This ensures backward compatibility for other plugins while enabling
multi-country support for Stream-Mapparr.
This update introduces GUI-based channel database management, allowing users to enable or disable specific country databases for channel matching.
Key Changes:
- Convert fields from static list to @property method for dynamic database detection
- Add _get_channel_databases() method to scan and extract database metadata
- Update _load_channels_data() to filter by enabled databases from settings
- Support new database format with country_code, country_name, and version metadata
- Maintain backward compatibility with legacy array format
- Add dynamic checkbox fields for each detected database in plugin settings
- Default behavior: US enabled, all others disabled (or enable if only one database exists)
- Update fuzzy_matcher.py GEOGRAPHIC_PATTERNS to handle any country code prefix (CC:, CC , CCC:, CCC )
- Add comprehensive README documentation for new database format and GUI management
- Include sample CA_channels.json demonstrating new metadata format
Features:
- Selectable channel databases through GUI settings
- Multi-country support with automatic country code prefix handling
- Clear database labels showing country name and version in settings
- Improved matching accuracy by enabling only relevant regional databases
Version: 0.6.0
- Document the *_channels.json file format and naming pattern
- Provide step-by-step guide for creating country-specific databases
- Include field descriptions and examples (UK_channels.json)
- Add Docker commands for installation and verification
- Include tips for better channel matching
- Update field description and placeholder to show quotation usage
- Add _parse_tags() static method to parse comma-separated tags with quote support
- Support both single and double quotes to preserve spaces and special characters
- Update all ignore_tags parsing locations to use new helper function
- Examples: "4K, \" East\", \"[Dead]\"" -> ["4K", " East", "[Dead]"]
Changes:
1. Extract validation logic into _validate_plugin_settings helper method:
- Returns (has_errors, validation_results, token) tuple
- Validates API connection, profiles, groups, thresholds, etc.
- Reusable across multiple action methods
2. Update validate_settings_action:
- Now calls _validate_plugin_settings helper
- Formats results for UI display
3. Add validation to load_process_channels_action:
- Validates settings before loading channels
- Prevents processing with invalid profiles or groups
- Returns clear error messages if validation fails
4. Add validation to preview_changes_action:
- Validates settings before previewing changes
- Ensures settings are still valid when preview is run
- Returns clear error messages if validation fails
5. Update version to 0.5.0d
Benefits:
- Prevents users from proceeding with invalid settings
- Provides immediate feedback on configuration errors
- Ensures channel profiles and groups exist before processing
- Improves user experience with clear error messages
- Maintains DRY principle with shared validation logic
Changes:
1. Add four new UI settings for granular pattern control:
- ignore_quality_tags: Control quality pattern removal ([4K], HD, etc.)
- ignore_regional_tags: Control regional pattern removal (East)
- ignore_geographic_tags: Control geographic prefix removal (US:, USA:)
- ignore_misc_tags: Control miscellaneous pattern removal ((CX), (Backup), etc.)
2. Update _clean_channel_name method:
- Add parameters: ignore_quality, ignore_regional, ignore_geographic, ignore_misc
- Pass these to fuzzy_matcher.normalize_name
- Defaults to True for backward compatibility
3. Update _match_streams_to_channel method:
- Accept category flag parameters
- Pass them through to all _clean_channel_name calls
4. Store category settings in processed_data:
- Save settings when loading/processing channels
- Retrieve settings when executing actions
- Apply settings consistently across all operations
5. Update all action methods:
- preview_changes_action
- add_streams_to_channels_action
- manage_channel_visibility_action
- All retrieve and use category settings from processed_data
This enables users to fine-tune pattern matching behavior through the UI
while maintaining full backward compatibility with existing configurations.
Features:
- Multi-profile support: Profile Name field now accepts comma-separated list
- Validate Settings button: New first action to validate all settings
- Checks API connection
- Validates all profiles exist
- Validates all groups exist
- Verifies fuzzy matcher initialization
- CSV headers: All generated CSV files now include comment headers with:
- Plugin version
- All applicable settings (Overwrite, Threshold, Profiles, Groups, Tags, Limit)
- Statistics (visible channels, total streams, matched streams)
Changes:
- Updated Profile Name field help text and placeholder
- Modified load_process_channels_action to handle multiple profiles
- Added validate_settings_action method with comprehensive validation
- Added _generate_csv_header_comment helper method
- Updated all three CSV exports (preview, update, visibility) to include headers
