Joren/Stream-Mapparr
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #11 from PiratesIRC/claude/selectable-channel-databases-011CV1BxggpwpUe3u86UDzvN

Browse Source 
Add selectable channel databases to GUI
This commit is contained in:
Pirates IRC
2025-11-10 19:34:52 -06:00
committed by GitHub
parent 38ae6893c0 b625e4f55b
commit a71e8f0382
4 changed files with 657 additions and 196 deletions
Download Patch File Download Diff File Expand all files Collapse all files

155
README.md
View File

@@ -15,6 +15,8 @@ Before installing or using this plugin, it is **highly recommended** that you cr
* **Advanced Fuzzy Matching**: Automatically finds and assigns streams to channels using an advanced fuzzy-matching engine (`fuzzy_matcher.py`).
* **Unlimited Stream Support**: Fetches and processes ALL available streams regardless of quantity (no 10,000 stream limit).
* **Enhanced OTA Callsign Matching**: Uses a robust `*_channels.json` database for superior callsign extraction and matching for Over-The-Air broadcast channels.
* **Selectable Channel Databases** *(NEW v0.5.0a)*: Enable or disable specific channel databases through the GUI settings.
* **Multi-Country Support** *(NEW v0.5.0a)*: Support for multiple country databases with automatic country code prefix handling (e.g., `CA:`, `UK `).
* **Multi-Stream Assignment**: Assigns **all** matching streams to each channel (e.g., 4K, FHD, HD versions), sorted by quality.
* **Quality Prioritization**: Sorts matched streams by quality (4K → FHD → HD → (H) → (F) → (D) → SD → Slow).
* **Channel Visibility Management**: Automatically enables/disables channels based on stream assignments and duplicate detection.
@@ -44,11 +46,44 @@ Before installing or using this plugin, it is **highly recommended** that you cr
Stream-Mapparr uses `*_channels.json` files to improve OTA (Over-The-Air) and cable channel matching. The plugin includes `US_channels.json` by default, but you can create additional database files for other countries or regions.
**NEW in v0.5.0a**: Channel databases are now **selectable within the GUI**! You can enable or disable specific databases in the plugin settings.
### Database File Format
Channel database files follow the naming pattern: `[COUNTRY_CODE]_channels.json` (e.g., `US_channels.json`, `CA_channels.json`, `UK_channels.json`)
Each file contains a JSON array of channel objects with three required fields:
#### Recommended Format (v0.5.0a+)
The recommended format includes metadata at the top level:
```json
{
"country_code": "CA",
"country_name": "Canada",
"version": "2025-11-10",
"channels": [
{
"channel_name": "CBC",
"category": "News",
"type": "National"
},
{
"channel_name": "CTV",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "Global",
"category": "Entertainment",
"type": "National"
}
]
}
```
#### Legacy Format (Still Supported)
The legacy format is still supported and uses a direct array:
```json
[
@@ -61,17 +96,25 @@ Each file contains a JSON array of channel objects with three required fields:
"channel_name": "CTV",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "Global",
"category": "Entertainment",
"type": "National"
}
]
```
**Note**: If using the legacy format without metadata, the database will be displayed in settings using the filename.
### Field Descriptions
#### Metadata Fields (Recommended Format Only)
| Field | Required | Description | Examples |
|:---|:---|:---|:---|
| **country_code** | Recommended | Two-letter ISO country code | `US`, `CA`, `UK`, `AU`, `DE` |
| **country_name** | Recommended | Full country/region name | `United States`, `Canada`, `United Kingdom` |
| **version** | Optional | Database version or date | `2025-11-10`, `1.0`, `v2` |
| **channels** | Yes | Array of channel objects | See below |
#### Channel Object Fields
| Field | Required | Description | Examples |
|:---|:---|:---|:---|
| **channel_name** | Yes | The channel name or callsign | `CBC`, `BBC One`, `WSBT`, `Sky Sports` |
@@ -113,58 +156,98 @@ Each file contains a JSON array of channel objects with three required fields:
```
* The plugin will automatically detect and use all `*_channels.json` files in the directory
### Example: Creating UK_channels.json
### Example: Creating UK_channels.json (Recommended Format)
```json
[
{
"channel_name": "BBC One",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "BBC Two",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "ITV",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "Channel 4",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "Sky Sports",
"category": "Sports",
"type": "National"
}
]
{
"country_code": "UK",
"country_name": "United Kingdom",
"version": "2025-11-11",
"channels": [
{
"channel_name": "BBC One",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "BBC Two",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "ITV",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "Channel 4",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "Sky Sports",
"category": "Sports",
"type": "National"
}
]
}
```
### Managing Channel Databases in the GUI
**NEW in v0.5.0a**: All channel databases are now manageable through the plugin settings!
1. **Viewing Available Databases**
* Navigate to **Plugins** → **Stream-Mapparr** → **Settings**
* Scroll to the **"📚 Channel Databases"** section
* All detected `*_channels.json` files will be listed with checkboxes
2. **Enabling/Disabling Databases**
* Check the box next to a database to enable it for matching
* Uncheck the box to disable it
* By default, only the **US** database is enabled
* If only one database exists, it will be enabled by default
3. **Database Labels**
* Databases using the **recommended format** show: `Country Name (vVersion)`
* Example: `Canada (v2025-11-10)`
* Databases using the **legacy format** show: `Filename`
* Example: `UK_channels.json`
4. **Country Code Prefix Handling**
* Stream names may be prefixed with country codes (e.g., `CA: CBC`, `UK BBC One`, `USA News`)
* The plugin automatically removes these prefixes during matching
* Supported formats: `CC:` or `CC ` (2-letter codes), `CCC:` or `CCC ` (3-letter codes)
* Smart detection avoids removing quality tags like HD, SD, UHD, FHD
### Tips for Better Matching
* Include all variations of channel names (e.g., `BBC 1`, `BBC One`, `BBC1`)
* Add both full names and abbreviations (e.g., `The Sports Network`, `TSN`)
* Include regional variants if applicable (e.g., `BBC One London`, `BBC One Scotland`)
* Use the exact callsigns for OTA broadcast stations
* Test your database by running the plugin and checking the logs for matching activity
* Enable only the databases relevant to your region for better matching accuracy
* Use the recommended format with metadata for clearer identification in the GUI
* Test your database by enabling it in settings and checking the logs for matching activity
## Settings Reference
| Setting | Type | Default | Description |
|:---|:---|:---|:---|
| **Fuzzy Match Threshold** | `number` | 85 | Minimum similarity score (0-100) for fuzzy matching. Higher values require closer matches. |
| **Overwrite Existing Streams** | `boolean` | True | If enabled, removes all existing streams and replaces with matched streams |
| **Fuzzy Match Threshold** | `number` | 85 | Minimum similarity score (0-100) for fuzzy matching. Higher values require closer matches |
| **Dispatcharr URL** | `string` | - | Full URL of your Dispatcharr instance (e.g., `http://192.168.1.10:9191`) |
| **Dispatcharr Admin Username** | `string` | - | Username for API authentication |
| **Dispatcharr Admin Password** | `password` | - | Password for API authentication |
| **Profile Name** | `string` | - | Name of an existing Channel Profile to process (e.g., "Primary", "Sports") |
| **Channel Groups** | `string` | - | Comma-separated group names to process, or leave empty for all groups |
| **Ignore Tags** | `string` | - | Comma-separated tags to ignore during matching (e.g., `4K, [4K], [Dead]`) |
| **Ignore Quality Tags** | `boolean` | True | Remove quality-related patterns like [4K], HD, (SD) during matching |
| **Ignore Regional Tags** | `boolean` | True | Remove regional indicators like "East" during matching |
| **Ignore Geographic Tags** | `boolean` | True | Remove geographic prefixes like US:, CA:, UK: during matching |
| **Ignore Miscellaneous Tags** | `boolean` | True | Remove miscellaneous tags like (CX), (Backup) during matching |
| **Visible Channel Limit** | `number` | 1 | Number of channels per matching group that will be visible and have streams added |
| **Enable [Database]** *(v0.5.0a)* | `boolean` | US: True, Others: False | Enable or disable specific channel databases for matching |
## Usage Guide

107
Stream-Mapparr/CA_channels.json Normal file
View File

@@ -0,0 +1,107 @@
{
"country_code": "CA",
"country_name": "Canada",
"version": "2025-11-11",
"channels": [
{
"channel_name": "CBC",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "CBC News Network",
"category": "News",
"type": "National"
},
{
"channel_name": "CTV",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "CTV News Channel",
"category": "News",
"type": "National"
},
{
"channel_name": "Global",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "Citytv",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "TSN",
"category": "Sports",
"type": "National"
},
{
"channel_name": "The Sports Network",
"category": "Sports",
"type": "National"
},
{
"channel_name": "Sportsnet",
"category": "Sports",
"type": "National"
},
{
"channel_name": "TVA",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "Ici Radio-Canada",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "CTV Comedy Channel",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "CTV Drama Channel",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "Discovery Channel",
"category": "Documentary",
"type": "National"
},
{
"channel_name": "History",
"category": "Documentary",
"type": "National"
},
{
"channel_name": "Food Network Canada",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "HGTV Canada",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "W Network",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "Showcase",
"category": "Entertainment",
"type": "National"
},
{
"channel_name": "Space",
"category": "Entertainment",
"type": "National"
}
]
}

19
Stream-Mapparr/fuzzy_matcher.py
View File

@@ -11,7 +11,7 @@ import logging
from glob import glob
# Version: YY.DDD.HHMM (Julian date format: Year.DayOfYear.Time)
__version__ = "25.313.1157"
__version__ = "25.314.1907"
# Setup logging
LOGGER = logging.getLogger("plugins.fuzzy_matcher")
@@ -193,7 +193,7 @@ class FuzzyMatcher:
return callsign
def normalize_name(self, name, user_ignored_tags=None, ignore_quality=True, ignore_regional=True,
ignore_geographic=True, ignore_misc=True, remove_cinemax=False):
ignore_geographic=True, ignore_misc=True, remove_cinemax=False, remove_country_prefix=False):
"""
Normalize channel or stream name for matching by removing tags, prefixes, and other noise.
@@ -205,6 +205,7 @@ class FuzzyMatcher:
ignore_geographic: If True, remove geographic prefix patterns (e.g., US:, USA)
ignore_misc: If True, remove miscellaneous patterns (e.g., (CX), (Backup), single-letter tags)
remove_cinemax: If True, remove "Cinemax" prefix (useful when channel name contains "max")
remove_country_prefix: If True, remove country code prefixes (e.g., CA:, UK , DE: ) from start of name
Returns:
Normalized name
@@ -215,6 +216,20 @@ class FuzzyMatcher:
# Remove leading parenthetical prefixes like (SP2), (D1), etc.
name = re.sub(r'^\([^\)]+\)\s*', '', name)
# Remove country code prefix if requested (e.g., "CA:", "UK ", "USA: ")
# This handles multi-country databases where streams may be prefixed with country codes
if remove_country_prefix:
# Known quality tags that should NOT be removed (to avoid false positives)
quality_tags = {'HD', 'SD', 'FD', 'UHD', 'FHD'}
# Check for 2-3 letter prefix with colon or space at start
prefix_match = re.match(r'^([A-Z]{2,3})[:|\s]\s*', name)
if prefix_match:
prefix = prefix_match.group(1).upper()
# Only remove if it's NOT a quality tag
if prefix not in quality_tags:
name = name[len(prefix_match.group(0)):]
# Remove "Cinemax" prefix if requested (for channels containing "max")
if remove_cinemax:
name = re.sub(r'\bCinemax\b\s*', '', name, flags=re.IGNORECASE)

538
Stream-Mapparr/plugin.py
View File

@@ -31,106 +31,149 @@ class Plugin:
"""Dispatcharr Stream-Mapparr Plugin"""
name = "Stream-Mapparr"
version = "0.5.0d"
version = "0.5.0a"
description = "🎯 Automatically add matching streams to channels based on name similarity and quality precedence with enhanced fuzzy matching"
# Settings rendered by UI
fields = [
{
"id": "overwrite_streams",
"label": "🔄 Overwrite Existing Streams",
"type": "boolean",
"default": True,
"help_text": "If enabled, all existing streams will be removed and replaced with matched streams. If disabled, only new streams will be added (existing streams preserved).",
},
{
"id": "fuzzy_match_threshold",
"label": "🎯 Fuzzy Match Threshold",
"type": "number",
"default": 85,
"help_text": "Minimum similarity score (0-100) for fuzzy matching. Higher values require closer matches. Default: 85",
},
{
"id": "dispatcharr_url",
"label": "🌐 Dispatcharr URL",
"type": "string",
"default": "",
"placeholder": "http://192.168.1.10:9191",
"help_text": "URL of your Dispatcharr instance (from your browser address bar). Example: http://127.0.0.1:9191",
},
{
"id": "dispatcharr_username",
"label": "👤 Dispatcharr Admin Username",
"type": "string",
"help_text": "Your admin username for the Dispatcharr UI. Required for API access.",
},
{
"id": "dispatcharr_password",
"label": "🔑 Dispatcharr Admin Password",
"type": "string",
"input_type": "password",
"help_text": "Your admin password for the Dispatcharr UI. Required for API access.",
},
{
"id": "profile_name",
"label": "📋 Profile Name",
"type": "string",
"default": "",
"placeholder": "Sports, Movies, News",
"help_text": "*** Required Field *** - The name(s) of existing Channel Profile(s) to process channels from. Multiple profiles can be specified separated by commas.",
},
{
"id": "selected_groups",
"label": "📁 Channel Groups (comma-separated)",
"type": "string",
"default": "",
"placeholder": "Sports, News, Entertainment",
"help_text": "Specific channel groups to process, or leave empty for all groups.",
},
{
"id": "ignore_tags",
"label": "🏷️ Ignore Tags (comma-separated)",
"type": "string",
"default": "",
"placeholder": "4K, [4K], \" East\", \"[Dead]\"",
"help_text": "Tags to ignore when matching streams. Use quotes to preserve spaces/special chars (e.g., \" East\" for tags with leading space).",
},
{
"id": "ignore_quality_tags",
"label": "🎬 Ignore Quality Tags",
"type": "boolean",
"default": True,
"help_text": "If enabled, hardcoded quality tags like [4K], [HD], (UHD), etc., will be ignored during matching.",
},
{
"id": "ignore_regional_tags",
"label": "🌍 Ignore Regional Tags",
"type": "boolean",
"default": True,
"help_text": "If enabled, hardcoded regional tags like 'East' will be ignored during matching.",
},
{
"id": "ignore_geographic_tags",
"label": "🗺️ Ignore Geographic Tags",
"type": "boolean",
"default": True,
"help_text": "If enabled, hardcoded geographic prefixes like 'US:', 'USA:' will be ignored during matching.",
},
{
"id": "ignore_misc_tags",
"label": "🏷️ Ignore Miscellaneous Tags",
"type": "boolean",
"default": True,
"help_text": "If enabled, miscellaneous tags like (CX), (Backup), and single-letter tags will be ignored during matching.",
},
{
"id": "visible_channel_limit",
"label": "👁️ Visible Channel Limit",
"type": "number",
"default": 1,
"help_text": "Number of channels that will be visible and have streams added. Channels are prioritized by quality tags, then by channel number.",
},
]
@property
def fields(self):
"""Dynamically generate settings fields including channel database selection."""
# Static fields that are always present
static_fields = [
{
"id": "overwrite_streams",
"label": "🔄 Overwrite Existing Streams",
"type": "boolean",
"default": True,
"help_text": "If enabled, all existing streams will be removed and replaced with matched streams. If disabled, only new streams will be added (existing streams preserved).",
},
{
"id": "fuzzy_match_threshold",
"label": "🎯 Fuzzy Match Threshold",
"type": "number",
"default": 85,
"help_text": "Minimum similarity score (0-100) for fuzzy matching. Higher values require closer matches. Default: 85",
},
{
"id": "dispatcharr_url",
"label": "🌐 Dispatcharr URL",
"type": "string",
"default": "",
"placeholder": "http://192.168.1.10:9191",
"help_text": "URL of your Dispatcharr instance (from your browser address bar). Example: http://127.0.0.1:9191",
},
{
"id": "dispatcharr_username",
"label": "👤 Dispatcharr Admin Username",
"type": "string",
"help_text": "Your admin username for the Dispatcharr UI. Required for API access.",
},
{
"id": "dispatcharr_password",
"label": "🔑 Dispatcharr Admin Password",
"type": "string",
"input_type": "password",
"help_text": "Your admin password for the Dispatcharr UI. Required for API access.",
},
{
"id": "profile_name",
"label": "📋 Profile Name",
"type": "string",
"default": "",
"placeholder": "Sports, Movies, News",
"help_text": "*** Required Field *** - The name(s) of existing Channel Profile(s) to process channels from. Multiple profiles can be specified separated by commas.",
},
{
"id": "selected_groups",
"label": "📁 Channel Groups (comma-separated)",
"type": "string",
"default": "",
"placeholder": "Sports, News, Entertainment",
"help_text": "Specific channel groups to process, or leave empty for all groups.",
},
{
"id": "ignore_tags",
"label": "🏷️ Ignore Tags (comma-separated)",
"type": "string",
"default": "",
"placeholder": "4K, [4K], \" East\", \"[Dead]\"",
"help_text": "Tags to ignore when matching streams. Use quotes to preserve spaces/special chars (e.g., \" East\" for tags with leading space).",
},
{
"id": "ignore_quality_tags",
"label": "🎬 Ignore Quality Tags",
"type": "boolean",
"default": True,
"help_text": "If enabled, hardcoded quality tags like [4K], [HD], (UHD), etc., will be ignored during matching.",
},
{
"id": "ignore_regional_tags",
"label": "🌍 Ignore Regional Tags",
"type": "boolean",
"default": True,
"help_text": "If enabled, hardcoded regional tags like 'East' will be ignored during matching.",
},
{
"id": "ignore_geographic_tags",
"label": "🗺️ Ignore Geographic Tags",
"type": "boolean",
"default": True,
"help_text": "If enabled, hardcoded geographic prefixes like 'US:', 'USA:' will be ignored during matching.",
},
{
"id": "ignore_misc_tags",
"label": "🏷️ Ignore Miscellaneous Tags",
"type": "boolean",
"default": True,
"help_text": "If enabled, miscellaneous tags like (CX), (Backup), and single-letter tags will be ignored during matching.",
},
{
"id": "visible_channel_limit",
"label": "👁️ Visible Channel Limit",
"type": "number",
"default": 1,
"help_text": "Number of channels that will be visible and have streams added. Channels are prioritized by quality tags, then by channel number.",
},
]
# Add channel database section header
static_fields.append({
"id": "channel_databases_header",
"type": "info",
"label": "📚 Channel Databases",
})
# Dynamically add channel database enable/disable fields
try:
databases = self._get_channel_databases()
if databases:
for db_info in databases:
db_id = db_info['id']
db_label = db_info['label']
db_default = db_info['default']
static_fields.append({
"id": f"db_enabled_{db_id}",
"type": "boolean",
"label": f"Enable {db_label}",
"help_text": f"Enable or disable the {db_label} channel database for matching.",
"default": db_default
})
else:
static_fields.append({
"id": "no_databases_found",
"type": "info",
"label": "⚠️ No channel databases found. Place XX_channels.json files in the plugin directory.",
})
except Exception as e:
LOGGER.error(f"[Stream-Mapparr] Error loading channel databases for settings: {e}")
static_fields.append({
"id": "database_error",
"type": "info",
"label": f"⚠️ Error loading channel databases: {e}",
})
return static_fields
# Actions for Dispatcharr UI
actions = [
@@ -204,6 +247,68 @@ class Plugin:
LOGGER.info(f"[Stream-Mapparr] {self.name} Plugin v{self.version} initialized")
def _get_channel_databases(self):
"""
Scan for channel database files and return metadata for each.
Returns:
List of dicts with 'id', 'label', 'default', and 'file_path' keys
"""
plugin_dir = os.path.dirname(__file__)
databases = []
try:
from glob import glob
pattern = os.path.join(plugin_dir, '*_channels.json')
channel_files = sorted(glob(pattern))
for channel_file in channel_files:
try:
filename = os.path.basename(channel_file)
# Extract country code from filename (e.g., "US" from "US_channels.json")
country_code = filename.split('_')[0].upper()
# Try to read the file and extract metadata
with open(channel_file, 'r', encoding='utf-8') as f:
file_data = json.load(f)
# Check if it's the new format with metadata
if isinstance(file_data, dict) and 'country_code' in file_data:
country_name = file_data.get('country_name', filename)
version = file_data.get('version', '')
if version:
label = f"{country_name} (v{version})"
else:
label = country_name
else:
# Old format or missing metadata - use filename
label = filename
# Determine default value: US enabled by default, or if only one database, enable it
# We'll check the count later
default = (country_code == 'US')
databases.append({
'id': country_code,
'label': label,
'default': default,
'file_path': channel_file,
'filename': filename
})
except Exception as e:
LOGGER.warning(f"[Stream-Mapparr] Error reading database file {channel_file}: {e}")
continue
# If only one database exists, enable it by default
if len(databases) == 1:
databases[0]['default'] = True
except Exception as e:
LOGGER.error(f"[Stream-Mapparr] Error scanning for channel databases: {e}")
return databases
def _initialize_fuzzy_matcher(self, match_threshold=85):
"""Initialize the fuzzy matcher with configured threshold."""
if self.fuzzy_matcher is None:
@@ -445,7 +550,7 @@ class Plugin:
return tags
def _clean_channel_name(self, name, ignore_tags=None, ignore_quality=True, ignore_regional=True,
ignore_geographic=True, ignore_misc=True, remove_cinemax=False):
ignore_geographic=True, ignore_misc=True, remove_cinemax=False, remove_country_prefix=True):
"""
Remove brackets and their contents from channel name for matching, and remove ignore tags.
Uses fuzzy matcher's normalization if available, otherwise falls back to basic cleaning.
@@ -458,6 +563,7 @@ class Plugin:
ignore_geographic: If True, remove geographic prefix patterns (e.g., US:, USA)
ignore_misc: If True, remove miscellaneous patterns (e.g., (CX), (Backup), single-letter tags)
remove_cinemax: If True, remove "Cinemax" prefix (for streams when channel contains "max")
remove_country_prefix: If True, remove country code prefixes (e.g., CA:, UK ) from start of name
"""
if self.fuzzy_matcher:
# Use fuzzy matcher's normalization
@@ -467,15 +573,27 @@ class Plugin:
ignore_regional=ignore_regional,
ignore_geographic=ignore_geographic,
ignore_misc=ignore_misc,
remove_cinemax=remove_cinemax
remove_cinemax=remove_cinemax,
remove_country_prefix=remove_country_prefix
)
# Fallback to basic cleaning
if ignore_tags is None:
ignore_tags = []
cleaned = name
# Remove country code prefix if requested
if remove_country_prefix:
quality_tags = {'HD', 'SD', 'FD', 'UHD', 'FHD'}
prefix_match = re.match(r'^([A-Z]{2,3})[:|\s]\s*', cleaned)
if prefix_match:
prefix = prefix_match.group(1).upper()
if prefix not in quality_tags:
cleaned = cleaned[len(prefix_match.group(0)):]
# Remove anything in square brackets or parentheses at the end
cleaned = re.sub(r'\s*[\[\(][^\[\]\(\)]*[\]\)]\s*$', '', name)
cleaned = re.sub(r'\s*[\[\(][^\[\]\(\)]*[\]\)]\s*$', '', cleaned)
# Keep removing until no more brackets at the end
while True:
new_cleaned = re.sub(r'\s*[\[\(][^\[\]\(\)]*[\]\)]\s*$', '', cleaned)
@@ -545,30 +663,83 @@ class Plugin:
return sorted(streams, key=get_quality_index)
def _load_channels_data(self, logger):
"""Load channel data from *_channels.json files."""
def _load_channels_data(self, logger, settings=None):
"""
Load channel data from enabled *_channels.json files.
Args:
logger: Logger instance
settings: Plugin settings dict (optional, for filtering by enabled databases)
Returns:
List of channel data from enabled databases
"""
plugin_dir = os.path.dirname(__file__)
channels_data = []
try:
# Find all *_channels.json files
from glob import glob
pattern = os.path.join(plugin_dir, '*_channels.json')
channel_files = glob(pattern)
# Get all available databases
databases = self._get_channel_databases()
if channel_files:
for channel_file in channel_files:
try:
with open(channel_file, 'r', encoding='utf-8') as f:
file_data = json.load(f)
channels_data.extend(file_data)
logger.info(f"[Stream-Mapparr] Loaded {len(file_data)} channels from {os.path.basename(channel_file)}")
except Exception as e:
logger.error(f"[Stream-Mapparr] Error loading {channel_file}: {e}")
logger.info(f"[Stream-Mapparr] Loaded total of {len(channels_data)} channels from {len(channel_files)} file(s)")
else:
if not databases:
logger.warning(f"[Stream-Mapparr] No *_channels.json files found in {plugin_dir}")
return channels_data
# Filter to only enabled databases
enabled_databases = []
for db_info in databases:
db_id = db_info['id']
setting_key = f"db_enabled_{db_id}"
# Check if this database is enabled in settings
if settings:
is_enabled = settings.get(setting_key, db_info['default'])
else:
# No settings provided, use default
is_enabled = db_info['default']
if is_enabled:
enabled_databases.append(db_info)
if not enabled_databases:
logger.warning("[Stream-Mapparr] No channel databases are enabled. Please enable at least one database in settings.")
return channels_data
# Load data from enabled databases
for db_info in enabled_databases:
channel_file = db_info['file_path']
db_label = db_info['label']
country_code = db_info['id']
try:
with open(channel_file, 'r', encoding='utf-8') as f:
file_data = json.load(f)
# Handle both old and new format
if isinstance(file_data, dict) and 'channels' in file_data:
# New format with metadata
channels_list = file_data['channels']
# Add country_code to each channel for prefix handling
for channel in channels_list:
channel['_country_code'] = country_code
elif isinstance(file_data, list):
# Old format - direct array
channels_list = file_data
# Add country_code to each channel for prefix handling
for channel in channels_list:
channel['_country_code'] = country_code
else:
logger.error(f"[Stream-Mapparr] Invalid format in {channel_file}")
continue
channels_data.extend(channels_list)
logger.info(f"[Stream-Mapparr] Loaded {len(channels_list)} channels from {db_label}")
except Exception as e:
logger.error(f"[Stream-Mapparr] Error loading {channel_file}: {e}")
logger.info(f"[Stream-Mapparr] Loaded total of {len(channels_data)} channels from {len(enabled_databases)} enabled database(s)")
except Exception as e:
logger.error(f"[Stream-Mapparr] Error loading channel data files: {e}")
@@ -595,7 +766,11 @@ class Plugin:
def _match_streams_to_channel(self, channel, all_streams, logger, ignore_tags=None,
ignore_quality=True, ignore_regional=True, ignore_geographic=True,
ignore_misc=True, channels_data=None):
"""Find matching streams for a channel using fuzzy matching when available."""
"""Find matching streams for a channel using fuzzy matching when available.
Returns:
tuple: (matching_streams, cleaned_channel_name, cleaned_stream_names, match_reason, database_used)
"""
if ignore_tags is None:
ignore_tags = []
if channels_data is None:
@@ -606,6 +781,9 @@ class Plugin:
# Get channel info from JSON
channel_info = self._get_channel_info_from_json(channel_name, channels_data, logger)
# Determine which database was used (if any)
database_used = channel_info.get('_country_code', 'N/A') if channel_info else 'N/A'
# Check if channel name contains "max" (case insensitive) - used for Cinemax handling
channel_has_max = 'max' in channel_name.lower()
@@ -645,7 +823,7 @@ class Plugin:
) for s in sorted_streams]
match_reason = "Callsign match"
return sorted_streams, cleaned_channel_name, cleaned_stream_names, match_reason
return sorted_streams, cleaned_channel_name, cleaned_stream_names, match_reason, database_used
else:
logger.info(f"[Stream-Mapparr] No callsign matches found for {callsign}")
# Fall through to fuzzy matching
@@ -693,11 +871,11 @@ class Plugin:
) for s in sorted_streams]
match_reason = f"Fuzzy match ({match_type}, score: {score})"
return sorted_streams, cleaned_channel_name, cleaned_stream_names, match_reason
return sorted_streams, cleaned_channel_name, cleaned_stream_names, match_reason, database_used
# No fuzzy match found
logger.info(f"[Stream-Mapparr] No fuzzy match found for channel: {channel_name}")
return [], cleaned_channel_name, [], "No fuzzy match"
return [], cleaned_channel_name, [], "No fuzzy match", database_used
# Fallback to basic substring matching if fuzzy matcher unavailable
logger.info(f"[Stream-Mapparr] Using basic substring matching for channel: {channel_name}")
@@ -705,7 +883,7 @@ class Plugin:
if not all_streams:
logger.warning("[Stream-Mapparr] No streams available for matching!")
return [], cleaned_channel_name, [], "No streams available"
return [], cleaned_channel_name, [], "No streams available", database_used
# Try exact channel name matching from JSON first
if channel_info and channel_info.get('channel_name'):
@@ -732,7 +910,7 @@ class Plugin:
) for s in sorted_streams]
match_reason = "Exact match (channels.json)"
return sorted_streams, cleaned_channel_name, cleaned_stream_names, match_reason
return sorted_streams, cleaned_channel_name, cleaned_stream_names, match_reason, database_used
# Fallback to basic substring matching
for stream in all_streams:
@@ -755,10 +933,10 @@ class Plugin:
) for s in sorted_streams]
match_reason = "Basic substring match"
return sorted_streams, cleaned_channel_name, cleaned_stream_names, match_reason
return sorted_streams, cleaned_channel_name, cleaned_stream_names, match_reason, database_used
# No match found
return [], cleaned_channel_name, [], "No match"
return [], cleaned_channel_name, [], "No match", database_used
def _get_channel_info_from_json(self, channel_name, channels_data, logger):
"""Find channel info from channels.json by matching channel name."""
@@ -953,24 +1131,75 @@ class Plugin:
self._initialize_fuzzy_matcher(match_threshold)
if self.fuzzy_matcher:
validation_results.append(f"✅ Fuzzy Matcher: SUCCESS - Initialized with threshold {match_threshold}")
validation_results.append(f"✅ Fuzzy Matcher: Initialized (threshold: {match_threshold})")
else:
validation_results.append("⚠️ Fuzzy Matcher: WARNING - Could not initialize (will use fallback matching)")
except Exception as e:
validation_results.append(f"⚠️ Fuzzy Matcher: WARNING - {str(e)} (will use fallback matching)")
# 7. Check other settings
# 7. Validate Channel Databases
logger.info("[Stream-Mapparr] Validating channel databases...")
try:
databases = self._get_channel_databases()
if not databases:
validation_results.append("❌ Channel Databases: FAILED - No *_channels.json files found in plugin directory")
has_errors = True
else:
# Check which databases are enabled
enabled_databases = []
invalid_databases = []
for db_info in databases:
db_id = db_info['id']
setting_key = f"db_enabled_{db_id}"
is_enabled = settings.get(setting_key, db_info['default'])
if is_enabled:
# Validate JSON format
try:
with open(db_info['file_path'], 'r', encoding='utf-8') as f:
file_data = json.load(f)
# Check format
if isinstance(file_data, dict):
if 'channels' not in file_data:
invalid_databases.append(f"{db_info['label']} (missing 'channels' key)")
elif not isinstance(file_data['channels'], list):
invalid_databases.append(f"{db_info['label']} ('channels' must be an array)")
else:
enabled_databases.append(db_info['label'])
elif isinstance(file_data, list):
enabled_databases.append(db_info['label'])
else:
invalid_databases.append(f"{db_info['label']} (invalid format)")
except json.JSONDecodeError as e:
invalid_databases.append(f"{db_info['label']} (JSON error: {str(e)[:50]})")
except Exception as e:
invalid_databases.append(f"{db_info['label']} (error: {str(e)[:50]})")
if invalid_databases:
validation_results.append(f"❌ Channel Databases: FAILED - Invalid database(s): {', '.join(invalid_databases)}")
has_errors = True
elif not enabled_databases:
validation_results.append("❌ Channel Databases: FAILED - No databases enabled. Enable at least one database in settings.")
has_errors = True
else:
validation_results.append(f"✅ Channel Databases: {len(enabled_databases)} enabled")
except Exception as e:
validation_results.append(f"❌ Channel Databases: FAILED - {str(e)}")
has_errors = True
# 8. Check other settings
overwrite_streams = settings.get('overwrite_streams', True)
if isinstance(overwrite_streams, str):
overwrite_streams = overwrite_streams.lower() in ('true', 'yes', '1')
validation_results.append(f"ℹ️ Overwrite Existing Streams: {'Enabled' if overwrite_streams else 'Disabled'}")
ignore_tags_str = settings.get("ignore_tags", "").strip()
if ignore_tags_str:
ignore_tags = self._parse_tags(ignore_tags_str)
validation_results.append(f"ℹ️ Ignore Tags: {len(ignore_tags)} tag(s) configured: {', '.join(repr(tag) for tag in ignore_tags)}")
else:
validation_results.append("ℹ️ Ignore Tags: None configured")
validation_results.append(f"ℹ️ {len(ignore_tags)} ignore tag(s) configured")
# Return validation results
return has_errors, validation_results, token
@@ -991,8 +1220,14 @@ class Plugin:
message += "\n\nPlease fix the errors above before proceeding."
return {"status": "error", "message": message}
else:
message = "All settings validated successfully!\n\n" + "\n".join(validation_results)
message += "\n\nYou can now proceed with 'Load/Process Channels'."
# Condensed success message - only show key items
success_items = [item for item in validation_results if item.startswith("✅")]
info_items = [item for item in validation_results if item.startswith("ℹ️")]
message = "Settings validated! " + " | ".join(success_items)
if info_items:
message += "\n" + " | ".join(info_items)
message += "\n\nReady to proceed with 'Load/Process Channels'."
return {"status": "success", "message": message}
def load_process_channels_action(self, settings, logger):
@@ -1251,6 +1486,21 @@ class Plugin:
overwrite_streams = overwrite_streams.lower() in ('true', 'yes', '1')
fuzzy_match_threshold = settings.get('fuzzy_match_threshold', 85)
# Get enabled databases
try:
databases = self._get_channel_databases()
enabled_dbs = []
for db_info in databases:
db_id = db_info['id']
setting_key = f"db_enabled_{db_id}"
is_enabled = settings.get(setting_key, db_info['default'])
if is_enabled:
enabled_dbs.append(db_info['label'])
db_info_str = ', '.join(enabled_dbs) if enabled_dbs else 'None'
except Exception:
db_info_str = 'Unknown'
# Build header lines
header_lines = [
f"# Stream-Mapparr Export",
@@ -1264,6 +1514,7 @@ class Plugin:
f"# Channel Groups: {', '.join(selected_groups) if selected_groups else 'All groups'}",
f"# Ignore Tags: {', '.join(ignore_tags) if ignore_tags else 'None'}",
f"# Visible Channel Limit: {visible_channel_limit}",
f"# Channel Databases Loaded: {db_info_str}",
f"#",
f"# Statistics:",
f"# Total Visible Channels: {total_visible_channels}",
@@ -1313,7 +1564,7 @@ class Plugin:
logger.info("[Stream-Mapparr] Settings validated successfully, proceeding with preview...")
# Load channel data from channels.json
channels_data = self._load_channels_data(logger)
channels_data = self._load_channels_data(logger, settings)
# Load processed data
with open(self.processed_data_file, 'r') as f:
@@ -1377,7 +1628,7 @@ class Plugin:
sorted_channels = self._sort_channels_by_priority(group_channels)
# Match streams for this channel group (using first channel as representative)
matched_streams, cleaned_channel_name, cleaned_stream_names, match_reason = self._match_streams_to_channel(
matched_streams, cleaned_channel_name, cleaned_stream_names, match_reason, database_used = self._match_streams_to_channel(
sorted_channels[0], streams, logger, ignore_tags,
ignore_quality, ignore_regional, ignore_geographic, ignore_misc,
channels_data
@@ -1398,6 +1649,7 @@ class Plugin:
"stream_names": [s['name'] for s in matched_streams],
"stream_names_cleaned": cleaned_stream_names,
"match_reason": match_reason,
"database_used": database_used,
"will_update": True
}
all_matches.append(match_info)
@@ -1419,6 +1671,7 @@ class Plugin:
"stream_names": [s['name'] for s in matched_streams],
"stream_names_cleaned": cleaned_stream_names,
"match_reason": f"Skipped (exceeds limit of {visible_channel_limit})",
"database_used": database_used,
"will_update": False
}
all_matches.append(match_info)
@@ -1455,6 +1708,7 @@ class Plugin:
'channel_number',
'matched_streams',
'match_reason',
'database_used',
'stream_names'
]
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
@@ -1469,6 +1723,7 @@ class Plugin:
'channel_number': match.get('channel_number', 'N/A'),
'matched_streams': match['matched_streams'],
'match_reason': match['match_reason'],
'database_used': match['database_used'],
'stream_names': '; '.join(match['stream_names']) # Show all streams
})
@@ -1524,7 +1779,7 @@ class Plugin:
return {"status": "error", "message": error}
# Load channel data from channels.json
channels_data = self._load_channels_data(logger)
channels_data = self._load_channels_data(logger, settings)
# Load processed data
with open(self.processed_data_file, 'r') as f:
@@ -1596,7 +1851,7 @@ class Plugin:
sorted_channels = self._sort_channels_by_priority(group_channels)
# Match streams for this channel group
matched_streams, cleaned_channel_name, cleaned_stream_names, match_reason = self._match_streams_to_channel(
matched_streams, cleaned_channel_name, cleaned_stream_names, match_reason, database_used = self._match_streams_to_channel(
sorted_channels[0], streams, logger, ignore_tags,
ignore_quality, ignore_regional, ignore_geographic, ignore_misc,
channels_data
@@ -1650,7 +1905,8 @@ class Plugin:
update_details.append({
'channel_name': channel_name,
'stream_names': stream_names_list,
'matched_streams': len(matched_streams)
'matched_streams': len(matched_streams),
'database_used': database_used
})
if overwrite_streams:
@@ -1711,7 +1967,7 @@ class Plugin:
csvfile.write(header_comment)
# Write CSV data
fieldnames = ['channel_name', 'stream_names', 'matched_streams']
fieldnames = ['channel_name', 'stream_names', 'matched_streams', 'database_used']
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
writer.writeheader()