Compare commits

..
47 Commits
Author SHA1 Message Date
Maksim Totmin 598bcc69d3 chore: remove empty tests that covered 0% of codebase 2026-06-02 09:25:10 +07:00
Maksim Totmin 8b122aa938 docs: correct README — player supports iPod Nano 7G only, not 6G 2026-06-02 09:19:20 +07:00
Maksim Totmin 71f35c53cb refactor: vendor iOpenPod core modules instead of pip dependency
Vendored 5 packages from iOpenPod (TheRealSavi/iOpenPod):
  iTunesDB_Writer/  — binary iTunesDB generation
  iTunesDB_Shared/  — field defs, constants, album identity
  SQLiteDB_Writer/  — SQLite databases for Nano 6G/7G
  iTunesDB_Parser/  — play stats reading from iPod
  ipod_device/      — device detection, FireWire ID, capabilities

Rationale: iOpenPod is a standalone iPod manager app, not a library.
Pulling it as a pip dependency is heavy and confusing for users.

Changes:
  + src/vendor/   — 77 .py files + calcHashAB.wasm (506 KB)
  ~ ipod_nano7_db.py  — _find_iop_root() now returns src/vendor/
  ~ requirements.txt   — iopenpod replaced with wasmtime + pycryptodome
  ~ .gitignore         — removed iOpenPod/ exclusion
  (setup.py reads requirements.txt automatically)
2026-06-02 09:16:19 +07:00
Maksim Totmin 222c2fcffb fix: add iOpenPod dependency as pip installable from GitHub
iopenpod is not published on PyPI — it lives at
https://github.com/TheRealSavi/iOpenPod.git.  The old
requirement 'iopenpod>=1.0.53' would always fail with
'Package not found'.

Using the PEP 508 @ git+ URL ensures pip fetches and
installs iOpenPod directly from the upstream repo.
2026-06-02 09:05:25 +07:00
Maksim Totmin 8317ad1fe4 docs: rewrite README.md with detailed project description
- Update entry point from main.py to app.py
- Document all modules: artwork pipeline, iOpenPod integration,
  SQLite cache, session store, device monitor, playlist manager,
  hotkeys, UI widgets (8 tabs), worker thread
- Add architecture overview and technology table
- Add configuration paths section
2026-06-02 08:58:13 +07:00
Maksim Totmin 54cd87654f fix: reliable play count sync with auto-sync + deferred iTunesDB commit
Stops double-counting when iPod Nano 7G regenerates Play Counts
file after a DB write (firmware quirk, new mtime with stale data).

Architecture:
- Mount: only max(cache, base) — never reads Play Counts delta
- 2s after mount → auto-sync worker: reads delta, subtracts stored
  (persistent guard), applies effective to cache, defers DB write
- iTunesDB commit deferred to next stable mount (checks W_OK first)
- read_play_stats() uses 3s retry + read_play_counts_delta() fallback
  (filesystem-based PC parsing, independent of iTunesDB I/O errors)

Changes:
- ipod_nano7_db.py: sync_itunescdb() returns bool; new
  read_play_counts_delta() method; get_all_tracks() no longer adds
  delta; _merge_play_stats() drops delta addition; suppress iOpenPod
  ERROR logs to CRITICAL
- library_cache.py: schema v2 with playcounts_log table +
  get/set_last_synced_delta, migration from v1
- worker.py: session guards (_processed_fwids, _auto_synced_fwids,
  _pending_commit_fwids); auto_sync_playcounts task; deferred commit
  in mount
- app.py: QTimer.singleShot(2000) after mount for auto-sync
- ipod_tab.py: _on_auto_state_changed rewritten (no unmount, mp guard)
- library_tab.py: refresh_play_counts_from_cache() method
2026-06-02 00:54:18 +07:00
Maksim Totmin d08125c7b3 fix: use correct case for Location key in read_play_stats 2026-06-01 22:32:17 +07:00
Maksim Totmin ba7144c88f feat: migrate LibraryCache to SQLite, add SessionStore + splash overlay
- Rewrite LibraryCache from JSON (cache dir) to SQLite (data dir with WAL)
- Each track gets a stable UUID so play stats survive file renames/re-encodes
- Thread-safe via PRAGMA busy_timeout + RLock
- Automatic backup rotation (.backup, .bak) on every write
- One-shot migration from old ~/.cache/library_cache.json
- Add SessionStore (~/.local/share/session.json) for playback state
- Add SplashOverlay with spinning animation on first (empty-DB) launch
- _scan_library() now accepts on_progress callback for the overlay
- Fix _mtime → mtime SQLite column name lookup in recently-added filter
- PlaylistManager path migrated to ~/.local/share/ with auto-migration
- Metadata sync: remove dead consume_play_deltas(), use library API
2026-06-01 22:12:56 +07:00
Maksim Totmin 2d77b3ea3e feat: modernize iPod tab with table view, search, and sorting, fix LibraryCache API 2026-06-01 22:11:36 +07:00
Maksim Totmin 597603f90e feat: add ready audio files (m4a/mp3/aac) to library with structured copy
- Expand Add Files dialog to accept all supported audio formats
- Copy ready files to output_dir/Artist/Album/{track} - Title.ext
- Fallback to output_dir root when metadata is missing
- Handle filename collisions with numeric suffix
2026-06-01 21:37:41 +07:00
Maksim Totmin b7367e63c4 fix: correct playlist creation on iPod (real db_track_id, metadata fallback, Settings hover fix) 2026-06-01 21:05:39 +07:00
Maksim Totmin 8861e55f32 Add shuffle and repeat controls with end-of-playlist fix 2026-06-01 20:32:56 +07:00
Maksim Totmin aaaf354c9d fix: restore JumpStyle seek slider and fix fractional truncation bug
- Add JumpStyle(QProxyStyle) to player_header.py so clicking the seek
  bar jumps to the clicked position (lost during player_header extraction)
- Fix _on_slider_user_action: int(frac) always truncated to 0
  (e.g. 500/1000=0.5 → int(0.5)=0), now emits raw value 0-1000
- Fix _on_user_seek_frac: properly converts slider value (0-1000) to
  playback position with bounds clamping
2026-06-01 20:09:52 +07:00
Maksim Totmin a66446120c refactor: remove dead code — unused tab_switch_requested signal, scope concept in hotkeys, redundant ipod_tab_index 2026-06-01 20:04:58 +07:00
Maksim Totmin b68b86929b feat: drag-and-drop tracks onto playlist (sidebar) and iPod (tab + nav icon) 2026-06-01 20:00:56 +07:00
Maksim Totmin bfd146a4a0 feat: add star rating to metadata editor with write support to audio tags
- ClickableStarRating widget — 5 stars with hover preview, toggle to unrate
- Rating field in MetadataEditorDialog (0–5 stars), saved to LibraryCache (0–100)
- MetadataHandler writes rating to MP3 (POPM), M4A (\xa9rt), FLAC/Ogg (RATING)
2026-06-01 19:59:56 +07:00
Maksim Totmin 309ba994a5 fix: rating no longer reset to -1 during mount/sync from iPod Play Counts
The Play Counts parser uses rating=-1 as sentinel for 'no change',
but the merge logic used  which treats -1 as truthy, overwriting
real ratings with -1 in all three paths: mount-time sync (worker.py),
DB regeneration (_merge_play_stats), and track listing (get_all_tracks).
2026-06-01 19:45:32 +07:00
Maksim Totmin a36312c9d7 fix: QThread crash on iPod connect and device detection — proper wait() lifecycle
Root cause: QThread C++ destructor ran while underlying OS thread was
still terminating. The pattern of checking isRunning() and using
short timeouts (wait(3000/5000)) left a window where the Python
reference was dropped and Shiboken destroyed the C++ object before
the thread fully exited.

Fixes:
- Remove all self.finished.connect(self.deleteLater) — unsafe,
  deleteLater from within run() or from worker thread races with
  thread termination
- _cleanup_worker: always call wait() (no timeout) before nulling
  the Python reference — guarantees OS thread is fully dead
- Add _is_worker_running() helper with try/except RuntimeError
  guard to safely check stale C++ objects
- DeviceMonitor._poll/stop/wait: use wait() with no timeout
- closeEvent: guard isRunning() with try/except RuntimeError
2026-06-01 19:25:30 +07:00
Maksim Totmin dd93440146 fix: hide vertical header (row numbers) in library table 2026-06-01 15:18:25 +07:00
Maksim Totmin a1d6465730 fix: sort by # now uses track_num from UserRole instead of cell text, so EQ animation doesn't break sorting 2026-06-01 15:13:00 +07:00
Maksim Totmin 199fb43f04 feat: iTunes-style sidebar + global player header + album grid
- New: sidebar_widget.py — iTunes-like sidebar with library/playlist/nav sections, GTK palette
- New: player_header.py — global player header (cover, now playing, controls) above splitter
- New: playlist_manager.py — playlist CRUD with JSON persistence
- Refactor: moved QMediaPlayer from LibraryTab to MainWindow for clean architecture
- Refactor: LibraryTab — removed all player code, added play_requested signal
- Feat: album grid (IconMode) — grouped by album name, rounded covers 8px, Spotify-like
- Feat: NumericTableItem — numeric sort for track # column (1,2,3...10,11 not 1,10,2)
- Feat: track_num animation fixed — finds row by path, not index (no freeze on sort)
- Feat: album_artist tag extracted and stored in TrackInfo
- Feat: state persistence — view_mode/filtered_album saved to config.ini on close
- Feat: Ctrl+B toggle sidebar
- Fix: sidebar splitter handle colour from GTK palette
2026-06-01 14:46:58 +07:00
Maksim Totmin 4c703db802 Add incremental metadata/artwork sync to iPod (iTunes-style)
- content_hash() — SHA1 of first 64KB for matching local↔iPod tracks
- Nano7Database.update_track_metadata() — update DB records only,
  never touch audio files on the iPod
- Nano7Database.find_track_by_content_hash() + find_track_by_metadata()
- _write_databases_from_tracks() extracted from sync_itunescdb,
  supports artwork_overrides parameter
- LibraryCache stores content_hash for cached tracks
- Worker + UI: 'Sync Metadata to iPod' button + context menu item
- _REMOVE_ARTWORK sentinel for explicit artwork removal
- track_info.py: add play_count, rating, last_played, skip_count fields
2026-06-01 13:21:22 +07:00
Maksim Totmin 3bfb0ebbfe Fix UI freeze on iPod mount/track-load and add content_hash/sync-metadata
- Move mount_device, get_device_info, and track scanning to background
  WorkerThread (new 'mount_and_load' task type) so the UI stays responsive
- Fix DeviceMonitor auto-poll to run detect_devices() in a background thread
- Refactor _on_device_detection_finished and _on_mount_clicked to use the
  new worker; remove synchronous _set_device_mounted
- Avoid double scan (get_track_count + get_all_tracks) — scan once in worker
- Add content_hash fingerprint to _scan_ipod_files and library_cache for
  track matching
- Add update_track_metadata / _write_databases_from_tracks with artwork
  override support in Nano7Database
- Add 'Sync Metadata' button and context menu action in LibraryTab
2026-06-01 12:37:49 +07:00
Maksim Totmin 25bb8105ac feat: auto-detect iPod connection via QTimer polling + fix AlreadyMounted in _mount_linux 2026-06-01 12:29:43 +07:00
Maksim Totmin 71d162520c refactor: extract main.py into ui/ tabs + app.py, extract search_itunes into services/, remove MusicBrainz 2026-06-01 09:59:00 +07:00
Maksim Totmin e569f8c731 refactor: remove YouTube download feature, extract TrackInfo dataclass, clean up config and hotkeys 2026-06-01 09:19:14 +07:00
Maksim Totmin 9cdb27dace refactor: replace JumpSlider subclass with QProxyStyle, clamp slider seek bounds 2026-06-01 08:57:55 +07:00
Maksim Totmin d153d69bfd feat: hide library toolbar buttons option, Ctrl+O hotkey, iTunes-style equalizer animation on playing track 2026-06-01 08:56:03 +07:00
Maksim Totmin ec797b8074 feat: persist play/pause state — auto-resume playback or pause-on-load on restart 2026-06-01 01:25:34 +07:00
Maksim Totmin 06f6529cbe feat: remember last played track, playback position, and volume across restarts 2026-06-01 01:22:16 +07:00
Maksim Totmin 949783e76c feat: hide library progress bar and status when iPod not connected 2026-06-01 01:14:17 +07:00
Maksim Totmin 61bcaec7a3 feat: hide iPod tab/button/menu when no device + fix settings persistence
- Hide iPod tab when no iPod connected, show on device detection
- Disable Transfer button and context menu item when no iPod mounted
- Add missing settings load/save: video_resolution, use_musicbrainz, show_download
- Fix clean_temp section mismatch (was [General], now [Advanced])
- Add _save_settings() called on closeEvent
2026-06-01 01:05:05 +07:00
Maksim Totmin 1ae1e90ae3 chore: remove redundant wasmtime install from build script (now a transitive dep of iopenpod) 2026-06-01 00:58:59 +07:00
Maksim Totmin a0b1aba50e fix: resolve iOpenPod import conflict by removing src/ from sys.path during iop imports
iTunesDB_Writer's internal 'from ipod_device import ChecksumType' was resolving
to our local src/ipod_device.py instead of iOpenPod's version, even with
iop path at sys.path[0] (Python 3.14 import resolution quirk).

Now temporarily removes project src/ from sys.path during iOpenPod imports,
restoring it afterwards. The validated flow: main.py imports local ipod_device
first, then _import_iop evicts and replaces it when iPod tab is used.
2026-06-01 00:55:12 +07:00
Maksim Totmin 5ba346cc84 fix: add project-local iOpenPod/ fallback so /tmp cleanup won't break the app 2026-06-01 00:46:44 +07:00
Maksim Totmin e3666c8c64 refactor: dynamic iOpenPod discovery, add to requirements
- Replace hardcoded /tmp/iOpenPod with importlib.util.find_spec lookup
  with fallback to /tmp/iOpenPod and ~/iOpenPod for dev/backward compat
- Add iopenpod>=1.0.53 to requirements.txt
- Remove deprecated mkdir temp/downloads/converted from install.sh
  (all paths now XDG-compliant)
2026-06-01 00:40:06 +07:00
Maksim Totmin cbc888660f fix: hotkey capture compatibility, add library metadata caching
- Fix KeyCaptureDialog for both old (int) and new (enum) PyQt6 enum types
  using universal _to_int() helper
- New library_cache.py: JSON cache (~/.cache/neo-pod-desktop/library_cache.json)
  with mtime-based invalidation — second launch skips tag extraction for
  unchanged files (instant load for large libraries)
- _scan_library(force=False) uses cache by default; Refresh/F5 uses force=True
- _incremental_update_after_edit() updates cache + refreshes UI without
  full filesystem walk
- Cache entries invalidated on file deletion
2026-06-01 00:19:44 +07:00
Maksim Totmin ba6935420d fix: use .value for KeyboardModifier/Key enum in keyPressEvent (PyQt6+Python 3.11 compat) 2026-06-01 00:09:23 +07:00
Maksim Totmin a93bc7907d feat: add metadata editor dialog with cover art and batch editing support
- Add update_tags() method to MetadataHandler for writing arbitrary tag
  changes to MP3/M4A/FLAC/OGG files (title, artist, album, album_artist,
  genre, date, track/disc number, comment, cover art)
- New MetadataEditorDialog (QDialog) with: cover art preview (drag-drop,
  click-to-change, remove), form fields for all common tags, F2 / context
  menu / toolbar button integration
- Batch editing: select multiple tracks to edit common fields (title and
  track numbers hidden), mixed values shown as placeholder
- Search now filters across Artist, Title, and Album columns
- Fix: layout warning from double setLayout in dialog
- Fix: cover cache invalidation after metadata edit
- Fix: missing QDialog import causing crash in shortcut remapping
- Add F2 hotkey default for Edit Metadata action
2026-06-01 00:07:42 +07:00
Maksim Totmin 1ace4ad98c feat: add configurable keyboard shortcut system
- src/hotkeys.py: HotkeyManager with 17 default shortcuts, config
  load/save via config.ini, KeyCaptureDialog for reassignment UI
- src/config_loader.py: [Hotkeys] section defaults, _set_defaults
  called on every load to backfill missing sections
- src/main.py: integrate HotkeyManager, seek_forward/backward (+/-5s),
  context-aware select_all/delete_selected, Settings tab UI with
  shortcut table and Reset to Defaults button
2026-05-31 23:26:41 +07:00
Maksim Totmin 8cc4ef5279 Move cover cache and temp files from /tmp to XDG cache (~/.cache/neo-pod-desktop/)
- Add xdg_cache_path() utility respecting
- CoverCache now stores covers in ~/.cache/neo-pod-desktop/covers/
- MetadataHandler temp files in ~/.cache/neo-pod-desktop/thumbnails/
- Survives reboot, no more lost cover_path during process_file
2026-05-31 23:16:42 +07:00
Maksim Totmin 6a841afaa0 Add -bitexact to MP3 encoding to suppress ffmpeg timestamp warning 2026-05-31 22:52:30 +07:00
Maksim Totmin 340b0dea9e Switch default output to MP3, fix AAC for iPod Nano 7G, auto-remove gray source rows after convert
- Separate AAC and MP3 ffmpeg paths: remove +global_header and +genpts from MP3
- Clamp AAC bitrate to 160 kbps (iPod Nano 7G max); warn on override
- Make mp3 the default format in UI, converter, and CLI
- Auto-remove converted source tracks from library UI (keep files on disk)
- Bulk delete_tracks with single scan+sync (progress support)
- Async duplicate removal with progress via WorkerThread
2026-05-31 22:44:02 +07:00
Maksim Totmin 63d72adae6 fix: move iPod track deletion to background thread with progress bar, batch DB sync 2026-05-31 22:43:59 +07:00
Maksim Totmin 3da5e15fe0 fix: add +global_header +genpts ffmpeg flags, preserve cover art streams, gapless metadata for iPod Nano 7 2026-05-31 22:13:47 +07:00
Maksim Totmin 71928d3478 feat: integrate artwork module into sync pipeline for Nano 7G 2026-05-31 21:45:51 +07:00
Maksim Totmin 9da689782a fix: clamp default audio bitrate to 160 kbps for iPod Nano 7 compatibility 2026-05-31 21:45:49 +07:00
117 changed files with 32962 additions and 2756 deletions
+1 -1
View File
@@ -15,7 +15,6 @@ ENV/
# Temporary files
temp/
downloads/
converted/
# IDE files
@@ -37,3 +36,4 @@ converted/
*.log
specs.md
*.AppImage
squashfs-root/
+108 -61
View File
@@ -1,102 +1,149 @@
# neo-pod-desktop
Desktop application for downloading, converting, managing and transferring music to iPod Nano 7G. Includes YouTube audio extraction, format conversion, metadata management, local library with playback, cover art pipeline, and full iTunesDB/ArtworkDB generation.
Настольное приложение для управления музыкой на iPod Nano 7G с встроенным iTunes-подобным плеером, конвертацией аудио, полным пайплайном обложек и генерацией iTunesDB/SQLite/ArtworkDB.
## Features
## Возможности
- **Download** audio from YouTube videos/playlists
- **Convert** to iPod-compatible formats (AAC/M4A, MP3) with `-map 0:t?` cover preservation
- **Metadata** handling — title, artist, album, track number, genre, embedded cover art
- **Cover art pipeline** — extract from audio files (APIC/covr/FLAC pictures), encode to iPod Nano 7G formats (RGB565), write ArtworkDB + ithmb
- **iTunes-style player** — play local library with cover art display (60×60), seek, volume
- **iPod Nano 7G database** — full iTunesCDB sync, SQLite management, Locations.itdb with HASHAB checksums
- **Library management** — sortable table, source/ready track separation, bulk transfer
- **Portable AppImage** — `scripts/build-appimage.sh` builds standalone x86_64 AppImage
- **Config** stored in `~/.config/neo-pod-desktop/config.ini`
- **Встроенный плеер** — воспроизведение локальной библиотеки через QMediaPlayer, отображение обложек альбомов (60x60), перемотка, громкость, повтор (all/one), shuffle
- **Конвертация аудио** — FFmpeg конвертация в AAC/M4A (libfdk_aac с AAC-LC профилем, 160 kbps) и MP3 (libmp3lame); конвертация видео для iPod (H.264, 640x480)
- **Пайплайн ArtworkDB** — извлечение обложек из аудиофайлов (APIC/covr/FLAC pictures), кодирование в iPod-нативные форматы (RGB565, RGB555, UYVY, JPEG), запись ithmb + ArtworkDB (MHFD→MHII→MHNI структуры)
- **Генерация баз iPod** — полная регенерация iTunesDB (бинарный mhbd) и SQLite баз для Nano 7G, cross-reference валидация db_pid
- **Синхронизация статистики** — двухсторонняя синхронизация play count, rating, skip count, last played между локальным кэшем (SQLite) и iPod; автосинк при подключении; validation gate против потери данных
- **Локальная библиотека** — сортируемая таблица с поддержкой drag-and-drop, фильтрация по альбомам/плейлистам, поиск, разделение на source-треки (FLAC/WAV) и готовые (MP3/M4A)
- **Плейлисты** — создание, переименование, удаление, drag-and-drop треков; перенос плейлистов на iPod с двухпроходным сопоставлением (content hash → metadata fallback)
- **Редактор метаданных** — редактирование тегов и обложек для MP3/M4A/FLAC/OGG/OPUS, batch-режим, интеграция с iTunes Search API для автозаполнения
- **Горячие клавиши** — полностью настраиваемые шорткаты (воспроизведение, навигация, громкость, перемотка, управление библиотекой)
- **Сессия воспроизведения** — автоматическое сохранение и восстановление последнего трека, позиции, громкости, режимов shuffle/repeat
- **Автоопределение устройств** — фоновый мониторинг iPod (Linux/macOS/Windows) через DeviceMonitor с polling
- **AppImage** — портативная сборка через PyInstaller + appimagetool
## Installation
## Установка
### Dependencies
### Зависимости
- Python 3.14+
- FFmpeg (for audio conversion)
- libusb (for USB device access on Linux)
- Python 3.10+
- FFmpeg (для конвертации аудио)
- libusb (для доступа к USB на Linux)
```bash
pip install -r requirements.txt
```
Install FFmpeg:
Установка FFmpeg:
- **macOS**: `brew install ffmpeg`
- **Linux**: `sudo apt-get install ffmpeg`
- **Windows**: Download from [ffmpeg.org](https://ffmpeg.org/download.html)
- **Windows**: Скачайте с [ffmpeg.org](https://ffmpeg.org/download.html)
## Usage
## Использование
### GUI Application
### GUI приложение
```bash
python src/main.py
python src/app.py
```
### Command Line Interface
```bash
python src/cli.py --url "https://www.youtube.com/watch?v=VIDEO_ID" --output-dir "/path/to/ipod"
```
### AppImage (portable)
### AppImage (портативная версия)
```bash
./neo-pod-desktop-x86_64.AppImage
```
Build from source:
Сборка из исходников:
```bash
./scripts/build-appimage.sh
```
## Project Structure
## Структура проекта
```
src/
main.py — GUI application (PyQt6)
cli.py — CLI interface
youtube_downloader.py — YouTube audio downloader
audio_converter.py FFmpeg audio conversion
metadata_handler.py — ID3/MP4 tag reading and writing
config_loader.py — Config in ~/.config/
ipod_device.py — iPod device detection and file management
ipod_nano7_db.py — Nano 7G SQLite + iTunesCDB + ArtworkDB
hashab.py — HASHAB checksum generation
artwork/
__init__.py — Public API
presets.py — 50+ iPod artwork format definitions
types.py — ArtworkEntry, EncodedFormatPayload, IthmbLocation
codecs.py — RGB565/RGB555/UYVY/JPEG encode/decode (numpy)
extractor.py — Cover extraction from audio files (mutagen)
chunks.py — ArtworkDB binary parser/writer (MHFD→MHII→MHNI)
writer.py — ithmb file + ArtworkDB writing
cache.py — Local JPEG cover cache for player UI
scripts/
build-appimage.sh — AppImage builder (PyInstaller + appimagetool)
neo-pod-desktop/
├── src/
├── app.py -- Главное окно, плеер (QMediaPlayer), оркестрация вкладок
│ ├── worker.py -- Фоновые потоки (конвертация, передача, синхронизация)
│ ├── audio_converter.py -- FFmpeg конвертация аудио/видео для iPod
│ ├── metadata_handler.py -- Чтение и запись ID3/MP4 тегов, встраивание обложек
│ ├── track_info.py -- TrackInfo dataclass
│ ├── config_loader.py -- Конфигурация в ~/.config/ (configparser)
│ ├── session_store.py -- Состояние сессии плеера (JSON)
│ ├── library_cache.py -- SQLite кэш библиотеки с WAL и ротацией бэкапов
│ ├── playlist_manager.py -- Управление плейлистами (JSON)
│ ├── ipod_device.py -- Детекция, монтирование и передача файлов на iPod
│ ├── ipod_nano7_db.py -- Менеджер БД iPod через iOpenPod (iTunesDB + SQLite)
│ ├── device_monitor.py -- Фоновый мониторинг подключения iPod
├── hotkeys.py -- Настраиваемые горячие клавиши
├── xdg_base.py -- XDG Base Directory пути (~/.config, ~/.local/share, ~/.cache)
├── hashab.py -- HASHAB контрольные суммы
│ ├── ipod_mhbd_writer.py -- Бинарный писатель iTunesDB (mhbd)
│ ├── models/
│ │ └── __init__.py
│ ├── services/
│ │ └── itunes_search.py -- iTunes Search API для автозаполнения метаданных
│ ├── artwork/
│ │ ├── __init__.py -- Публичный API artwork пайплайна
│ │ ├── presets.py -- 50+ форматов artwork для iPod
│ │ ├── types.py -- ArtworkEntry, EncodedFormatPayload, IthmbLocation
│ │ ├── codecs.py -- RGB565/RGB555/UYVY/JPEG кодеки (numpy)
│ │ ├── extractor.py -- Извлечение обложек из аудиофайлов (mutagen)
│ │ ├── chunks.py -- ArtworkDB бинарный парсер/писатель (MHFD→MHII→MHNI)
│ │ ├── writer.py -- Запись ithmb + ArtworkDB
│ │ └── cache.py -- Кэш JPEG обложек для UI плеера
│ └── ui/
│ ├── library_tab.py -- Вкладка библиотеки (таблица, поиск, фильтры)
│ ├── ipod_tab.py -- Вкладка iPod (устройства, треки, экспорт/удаление)
│ ├── settings_tab.py -- Вкладка настроек (горячие клавиши, поведение)
│ ├── player_header.py -- Виджет плеера (кнопки, слайдер, обложка, время)
│ ├── sidebar_widget.py -- Боковая панель (навигация, плейлисты)
│ ├── metadata_editor.py -- Редактор метаданных (одиночный/batch)
│ ├── splash_overlay.py -- Экран загрузки со сканированием библиотеки
│ └── star_rating.py -- Виджет рейтинга (кликабельные звёзды)
├── scripts/
│ └── build-appimage.sh -- Сборка AppImage (PyInstaller + appimagetool)
├── tests/
│ └── test_basic.py
├── requirements.txt
├── setup.py
├── install.sh
└── install.bat
```
## Cover Art Re-embed
## Ключевые технологии
One-time script to restore cover art in already-converted files:
| Технология | Назначение |
|---|---|
| **PyQt6** | Графический интерфейс, QMediaPlayer, сигналы/слоты |
| **iOpenPod (vendored)** | Генерация iTunesDB + SQLite + ArtworkDB для iPod Nano 7G |
| **FFmpeg** | Конвертация аудио (AAC/MP3) и видео (H.264) |
| **mutagen / eyeD3** | Чтение и запись метаданных (ID3, MP4, FLAC, OGG) |
| **NumPy + Pillow** | Кодирование обложек (RGB565, масштабирование) |
| **SQLite** | Кэш библиотеки, playcounts_log, WAL режим |
## Архитектура
Приложение построено на PyQt6 с разделением на UI-слой (пакет `ui/`), бизнес-логику (корневые модули `src/`) и движок генерации баз iPod (iOpenPod и пакет `artwork/`).
- **app.py** — центральный оркестратор: создаёт плеер (QMediaPlayer + QAudioOutput), управляет вкладками (Library, iPod, Settings) и боковой панелью, обрабатывает hotkeys, сохраняет сессию
- **worker.py** — QThread для фоновых задач с сигналами progress/finished; поддерживает конвертацию, передачу, детекцию, монтирование, удаление, дедупликацию, синк метаданных, создание плейлистов
- **ipod_nano7_db.py** — делегирует генерацию iTunesDB и SQLite баз iOpenPod, управляет artwork пайплайном, мержит play stats, валидирует данные
- **artwork/** — полный пайплайн: extractor → codecs (numpy) → chunks (бинарный парсер) → writer (ithmb + ArtworkDB)
- **library_cache.py** — SQLite база с WAL журналом, автоматической ротацией бэкапов, индексами по content_hash/uuid/artist/album
## Конфигурация
- **Конфиг**: `~/.config/neo-pod-desktop/config.ini`
- **Сессия**: `~/.local/share/neo-pod-desktop/session.json`
- **Плейлисты**: `~/.local/share/neo-pod-desktop/playlists.json`
- **Кэш библиотеки**: `~/.local/share/neo-pod-desktop/library_cache.db`
- **Кэш обложек**: `~/.cache/neo-pod-desktop/`
## Сборка AppImage
```bash
python /tmp/reembed_covers.py /path/to/source_music /path/to/output_dir
./scripts/build-appimage.sh
```
Matches output files to source files by artist+album, detects embedded covers and separate `cover.jpg`/`folder.jpg` files.
Требует: Python 3.10+, PyInstaller, appimagetool, FFmpeg (включается в AppImage).
## Legal Disclaimer
## Лицензия
This tool is intended for personal use only. Users are responsible for complying with YouTube's Terms of Service and copyright laws.
## License
MIT License
MIT
+3 -5
View File
@@ -1,7 +1,7 @@
@echo off
REM Installation script for YouTube Music to iPod Nano Transfer Tool (Windows)
REM Installation script for neo-pod-desktop (Windows)
echo Installing YouTube Music to iPod Nano Transfer Tool...
echo Installing neo-pod-desktop...
REM Check if Python 3 is installed
python --version 2>NUL
@@ -32,7 +32,6 @@ pip install -r requirements.txt
REM Create necessary directories
echo Creating necessary directories...
if not exist temp mkdir temp
if not exist downloads mkdir downloads
if not exist converted mkdir converted
echo.
@@ -40,8 +39,7 @@ echo Installation complete!
echo.
echo To run the application:
echo 1. Activate the virtual environment: venv\Scripts\activate.bat
echo 2. Run the GUI application: python run.py
echo 3. Or run the CLI application: python run.py --cli --url "https://www.youtube.com/watch?v=VIDEO_ID"
echo 2. Run the application: python run.py
echo.
echo For more information, see the README.md file.
+3 -8
View File
@@ -1,9 +1,9 @@
#!/bin/bash
# Installation script for YouTube Music to iPod Nano Transfer Tool
# Installation script for neo-pod-desktop
set -e
echo "Installing YouTube Music to iPod Nano Transfer Tool..."
echo "Installing neo-pod-desktop..."
# Check if Python 3 is installed
if ! command -v python3 &> /dev/null; then
@@ -39,16 +39,11 @@ source venv/bin/activate
echo "Installing dependencies..."
pip3 install -r requirements.txt
# Create necessary directories
echo "Creating necessary directories..."
mkdir -p temp downloads converted
echo ""
echo "Installation complete!"
echo ""
echo "To run the application:"
echo " 1. Activate the virtual environment: source venv/bin/activate"
echo " 2. Run the GUI application: python run.py"
echo " 3. Or run the CLI application: python run.py --cli --url \"https://www.youtube.com/watch?v=VIDEO_ID\""
echo " 2. Run the application: python run.py"
echo ""
echo "For more information, see the README.md file."
+2 -4
View File
@@ -1,12 +1,10 @@
yt-dlp>=2023.3.4
pytube>=12.1.3
ffmpeg-python>=0.2.0
mutagen>=1.46.0
eyeD3>=0.9.7
pyusb>=1.2.1
pillow>=9.5.0
requests>=2.28.2
tqdm>=4.65.0
musicbrainzngs>=0.7.1
PyQt6>=6.5.0
numpy>=1.24.0
wasmtime>=30.0.0
pycryptodome>=3.20.0
+5 -19
View File
@@ -5,30 +5,16 @@ Launcher script for neo-pod-desktop
import os
import sys
import argparse
def main():
"""Main entry point"""
parser = argparse.ArgumentParser(description="neo-pod-desktop")
parser.add_argument("--cli", action="store_true", help="Run in command-line mode")
args = parser.parse_args()
# Add src directory to path
src_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "src")
sys.path.insert(0, src_dir)
if args.cli:
# Run CLI version
from src.cli import YouTubeToIPodCLI
cli = YouTubeToIPodCLI()
return cli.run()
else:
# Run GUI version
from src.main import QApplication, MainWindow
app = QApplication(sys.argv)
window = MainWindow()
window.show()
return app.exec()
from src.app import main as app_main
app_main()
if __name__ == "__main__":
sys.exit(main())
main()
+1 -3
View File
@@ -54,8 +54,6 @@ install_pyinstaller() {
"$VENV_DIR/bin/pip" install pyinstaller
# Install project dependencies for bundling
"$VENV_DIR/bin/pip" install -r "$PROJECT_DIR/requirements.txt"
# Install wasmtime for HASHAB signing (iPod Nano 6G/7G)
"$VENV_DIR/bin/pip" install wasmtime
}
download_appimagetool() {
@@ -100,7 +98,7 @@ build_appdir() {
--exclude-module PyQt6.QtQuick3D \
--exclude-module PyQt6.QtShaderTools \
--exclude-module PyQt6.QtSpatialAudio \
src/main.py
src/app.py
}
create_desktop_file() {
+2 -2
View File
@@ -17,7 +17,7 @@ with open('README.md', encoding='utf-8') as f:
setup(
name="neo-pod-desktop",
version="1.0.0",
description="Desktop application for downloading, converting, managing and transferring music to iPod Nano devices",
description="Desktop application for converting, managing and transferring music to iPod Nano devices",
long_description=long_description,
long_description_content_type="text/markdown",
author="neo-pod-desktop",
@@ -27,7 +27,7 @@ setup(
install_requires=requirements,
entry_points={
'console_scripts': [
'neo-pod-desktop=src.cli:main',
'neo-pod-desktop=src.app:main',
],
},
classifiers=[
+668
View File
@@ -0,0 +1,668 @@
#!/usr/bin/env python3
"""
Main Application Module for neo-pod-desktop
Provides a GUI for converting, managing and transferring music to iPod Nano devices.
"""
import os
import sys
import random
import logging
from typing import Optional
from PyQt6.QtWidgets import (
QApplication, QMainWindow, QWidget, QVBoxLayout,
QSplitter, QStackedWidget,
)
from PyQt6.QtCore import Qt, QUrl, QTimer
from PyQt6.QtGui import QPixmap
from PyQt6.QtMultimedia import QMediaPlayer, QAudioOutput
from config_loader import ConfigLoader
from device_monitor import DeviceMonitor
from hotkeys import HotkeyManager
from playlist_manager import PlaylistManager
from session_store import SessionStore
from library_cache import get_library_cache
from ui.library_tab import LibraryTab
from ui.ipod_tab import iPodTab
from ui.settings_tab import SettingsTab
from ui.sidebar_widget import SIDEBAR_DEFAULT_WIDTH
from ui.player_header import PlayerHeader
from ui.splash_overlay import SplashOverlay
from artwork.cache import cover_cache
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
logger = logging.getLogger(__name__)
class MainWindow(QMainWindow):
"""Main application window"""
def __init__(self):
super().__init__()
self.setWindowTitle("neo-pod-desktop")
self.setMinimumSize(800, 600)
self.config_loader = ConfigLoader()
self.hotkey_manager: Optional[HotkeyManager] = None
self.device_monitor: Optional[DeviceMonitor] = None
self.playlist_manager = PlaylistManager()
self.session_store = SessionStore()
self._sidebar_visible = True
self._pending_playlist_id: str = ""
self._pending_playlist_tracks: list = []
self._saved_volume: int = 80
self._saved_track_path: str = ""
self._saved_position_ms: int = 0
self._saved_playing: bool = False
self._pending_seek_ms: int = 0
self._shuffle_active: bool = False
self._repeat_mode: str = "no_repeat"
self._auto_sync_worker = None
self._setup_player()
self._setup_ui()
self.library_tab.set_playlist_manager(self.playlist_manager)
self._load_playlists_into_sidebar()
self._load_settings()
self._setup_hotkeys()
self._wire_signals()
cache = get_library_cache()
splash = SplashOverlay(self)
if len(cache) == 0:
splash.show()
QApplication.processEvents()
self.library_tab._scan_library(on_progress=splash.set_progress)
if splash.isVisible():
splash.close()
self._resume_playback_if_saved()
self._restore_library_state()
self._start_device_monitor()
def _setup_player(self):
self.player = QMediaPlayer()
self.audio_output = QAudioOutput()
self.audio_output.setVolume(self._saved_volume / 100.0)
self.player.setAudioOutput(self.audio_output)
self.player.positionChanged.connect(self._on_player_position)
self.player.durationChanged.connect(self._on_player_duration)
self.player.mediaStatusChanged.connect(self._on_media_status)
def _setup_ui(self):
from ui.sidebar_widget import SidebarWidget
main_widget = QWidget()
main_layout = QVBoxLayout(main_widget)
main_layout.setContentsMargins(0, 0, 0, 0)
main_layout.setSpacing(0)
self.player_header = PlayerHeader()
self.player_header.prev_requested.connect(self._on_prev)
self.player_header.play_pause_requested.connect(self._on_play_pause)
self.player_header.next_requested.connect(self._on_next)
self.player_header.shuffle_requested.connect(self._on_shuffle_toggle)
self.player_header.repeat_requested.connect(self._on_repeat_cycle)
self.player_header.position_changed_by_user.connect(self._on_user_seek_frac)
self.player_header.volume_changed.connect(self._on_volume_changed)
main_layout.addWidget(self.player_header)
self.stack = QStackedWidget()
self.library_tab = LibraryTab(self.config_loader)
self.library_tab.play_requested.connect(self._on_play_requested)
self.stack.addWidget(self.library_tab)
self.ipod_tab = iPodTab()
self.stack.addWidget(self.ipod_tab)
self.settings_tab = SettingsTab()
self.stack.addWidget(self.settings_tab)
self.stack.setCurrentIndex(0)
self.sidebar = SidebarWidget()
self.sidebar.item_selected.connect(self._on_sidebar_item_selected)
self.sidebar.playlist_selected.connect(self._on_playlist_selected)
self.splitter = QSplitter(Qt.Orientation.Horizontal)
self.splitter.addWidget(self.sidebar)
self.splitter.addWidget(self.stack)
self.splitter.setStretchFactor(0, 0)
self.splitter.setStretchFactor(1, 1)
self.splitter.setSizes([SIDEBAR_DEFAULT_WIDTH, 600])
self.splitter.setHandleWidth(1)
mid = self.palette().color(self.palette().ColorRole.Midlight)
self.splitter.setStyleSheet(f"QSplitter::handle {{ background: {mid.name()}; }}")
main_layout.addWidget(self.splitter, stretch=1)
self.statusBar().showMessage("Ready")
self.setCentralWidget(main_widget)
def _wire_signals(self):
self.ipod_tab.device_mounted.connect(self._on_device_mounted)
self.ipod_tab.device_unmounted.connect(self._on_device_unmounted)
self.settings_tab.hide_library_buttons_changed.connect(
self.library_tab.set_buttons_visible
)
self.settings_tab.auto_detect_changed.connect(self._on_auto_detect_toggled)
self.sidebar.tracks_dropped_on_playlist.connect(self._on_tracks_dropped_on_playlist)
self.sidebar.tracks_dropped_for_nav_transfer.connect(self._on_tracks_dropped_for_nav_transfer)
self.sidebar.playlist_dropped_for_nav_transfer.connect(self._on_playlist_dropped_for_nav_transfer)
self.ipod_tab.tracks_dropped_for_transfer.connect(self._on_tracks_dropped_for_transfer)
self.library_tab.transfer_finished.connect(self._on_transfer_finished_for_playlist)
def _on_sidebar_item_selected(self, kind: str, value: str):
if kind == "tab":
tab_index = {"library": 0, "ipod": 1, "settings": 2}.get(value, 0)
self.stack.setCurrentIndex(tab_index)
if tab_index == 0:
self.library_tab.set_view_mode("table")
elif kind == "filter":
self.stack.setCurrentIndex(0)
self.library_tab.set_view_mode(value)
elif kind == "playlist_create":
self._create_playlist(value)
elif kind == "playlist_rename":
playlist_id, new_name = value.split("|", 1)
self._rename_playlist(playlist_id, new_name)
elif kind == "playlist_delete":
self._delete_playlist(playlist_id=value)
elif kind == "playlist_reorder":
ids = value.split("|") if value else []
self.playlist_manager.reorder(ids)
def _on_playlist_selected(self, playlist_id: str):
self.stack.setCurrentIndex(0)
playlist = self.playlist_manager.get_by_id(playlist_id)
if playlist:
self.library_tab.set_current_playlist_id(playlist_id)
self.library_tab.apply_filter("playlist", playlist.track_paths)
def _create_playlist(self, name: str):
playlist_id = self.playlist_manager.create(name)
self.sidebar.add_playlist_item(playlist_id, name)
def _rename_playlist(self, playlist_id: str, new_name: str):
self.playlist_manager.rename(playlist_id, new_name)
def _delete_playlist(self, playlist_id: str):
self.playlist_manager.delete(playlist_id)
self.sidebar.remove_playlist_item(playlist_id)
self.library_tab.clear_filter()
self.sidebar.select_library_music()
def _on_tracks_dropped_on_playlist(self, playlist_id: str, paths: list):
self.playlist_manager.add_tracks(playlist_id, paths)
playlist = self.playlist_manager.get_by_id(playlist_id)
if playlist:
self._on_playlist_selected(playlist_id)
self.statusBar().showMessage(f"Added {len(paths)} track(s) to playlist", 5000)
def _on_tracks_dropped_for_transfer(self, paths: list):
self.library_tab.transfer_tracks_by_paths(paths)
def _on_tracks_dropped_for_nav_transfer(self, paths: list):
self.stack.setCurrentIndex(0)
self.library_tab.transfer_tracks_by_paths(paths)
def _on_playlist_dropped_for_nav_transfer(self, playlist_id: str):
playlist = self.playlist_manager.get_by_id(playlist_id)
if not playlist or not playlist.track_paths:
return
self.stack.setCurrentIndex(0)
self._pending_playlist_id = playlist_id
self._pending_playlist_tracks = list(playlist.track_paths)
self.library_tab.transfer_tracks_by_paths(playlist.track_paths)
def _on_transfer_finished_for_playlist(self):
playlist_id = self._pending_playlist_id
track_paths = self._pending_playlist_tracks
self._pending_playlist_id = ""
self._pending_playlist_tracks = []
if not playlist_id or not track_paths:
return
mount_point = getattr(self.ipod_tab, "current_mount_point", None)
if not mount_point:
self.statusBar().showMessage("Cannot create playlist: iPod not mounted", 5000)
return
playlist = self.playlist_manager.get_by_id(playlist_id)
if not playlist:
return
self.statusBar().showMessage(f"Creating playlist '{playlist.name}' on iPod...")
from worker import WorkerThread
self._pl_worker = WorkerThread(
task_type="create_playlist",
mount_point=mount_point,
playlist_name=playlist.name,
track_paths=track_paths,
)
self._pl_worker.finished_signal.connect(self._on_playlist_creation_finished)
self._pl_worker.start()
def _on_playlist_creation_finished(self, success, message, result):
self.statusBar().showMessage(message, 8000)
try:
self._pl_worker.quit()
self._pl_worker.wait()
except Exception:
pass
self._pl_worker = None
def _load_playlists_into_sidebar(self):
for pl in self.playlist_manager.get_all():
self.sidebar.add_playlist_item(pl.id, pl.name)
def _on_device_mounted(self, mount_point: str):
self.library_tab.on_device_mounted(mount_point)
self.library_tab.refresh_play_counts_from_cache()
self.statusBar().showMessage(f"iPod mounted at {mount_point}")
QTimer.singleShot(2000, lambda: self._start_auto_sync(mount_point))
def _on_device_unmounted(self):
self.library_tab.on_device_unmounted()
self.statusBar().showMessage("iPod disconnected")
def _start_auto_sync(self, mount_point: str):
if self._auto_sync_worker is not None:
return
from worker import WorkerThread
self._auto_sync_worker = WorkerThread(
"auto_sync_playcounts", parent=self, mount_point=mount_point,
)
self._auto_sync_worker.progress_signal.connect(self._on_auto_sync_progress)
self._auto_sync_worker.finished_signal.connect(self._on_auto_sync_finished)
self._auto_sync_worker.start()
def _on_auto_sync_progress(self, pct: int, msg: str):
pass
def _on_auto_sync_finished(self, success: bool, msg: str, _result):
if success and msg.startswith("Auto-sync complete"):
self.library_tab.refresh_play_counts_from_cache()
self._auto_sync_worker = None
def _toggle_sidebar(self):
visible = not self.sidebar.isVisible()
self.sidebar.setVisible(visible)
if visible:
self.splitter.setSizes([SIDEBAR_DEFAULT_WIDTH, self.splitter.width() - SIDEBAR_DEFAULT_WIDTH])
else:
self.splitter.setSizes([0, self.splitter.width()])
self._sidebar_visible = visible
def _start_device_monitor(self):
auto_detect = self.config_loader.get_boolean("Device", "auto_detect", fallback=True)
self.device_monitor = DeviceMonitor()
if auto_detect:
self.ipod_tab.start_monitoring(self.device_monitor)
self.device_monitor.start()
def _on_auto_detect_toggled(self, enabled: bool):
if not self.device_monitor:
return
if enabled:
self.ipod_tab.start_monitoring(self.device_monitor)
self.device_monitor.start()
else:
self.device_monitor.stop()
self.ipod_tab.stop_monitoring(self.device_monitor)
def _load_settings(self):
config = self.config_loader
self.settings_tab.load_settings(config)
hide_buttons = config.get_boolean("Advanced", "hide_library_buttons", fallback=False)
self.library_tab.set_buttons_visible(not hide_buttons)
# Migrate Playback section → SessionStore if needed
self.session_store.migrate_from_config(config)
self._saved_volume = self.session_store.get_int("last_volume", 80)
self._saved_track_path = self.session_store.get_str("last_track_path", "")
self._saved_position_ms = self.session_store.get_int("last_position_ms", 0)
self._saved_playing = self.session_store.get_bool("last_playing", False)
self._shuffle_active = self.session_store.get_bool("shuffle_active", False)
self._repeat_mode = self.session_store.get_str("repeat_mode", "no_repeat")
self.player_header.set_shuffle_active(self._shuffle_active)
self.player_header.set_repeat_mode(self._repeat_mode)
self.audio_output.setVolume(self._saved_volume / 100.0)
self.player_header.set_volume(self._saved_volume)
def _save_settings(self):
config = self.config_loader
session = self.session_store
self.settings_tab.save_settings(config)
session.set("last_volume", self.player_header.volume_slider.value())
session.set("shuffle_active", self._shuffle_active)
session.set("repeat_mode", self._repeat_mode)
if self.current_track_path and os.path.exists(self.current_track_path):
session.set("last_track_path", self.current_track_path)
position = self.player.position()
duration = self.player.duration()
if duration > 0 and position > duration - 2000:
position = 0
session.set("last_position_ms", position)
session.set("last_playing", self._is_playing_now)
else:
session.set("last_track_path", "")
session.set("last_position_ms", 0)
session.save()
state = self.library_tab.get_state()
config.set("Library", "view_mode", state.get("view_mode", "table"))
config.set("Library", "filtered_album", state.get("filtered_album", ""))
config.set("Library", "filtered_playlist_id", state.get("filtered_playlist_id", ""))
config.save()
@property
def current_track_path(self) -> str:
if self.player.source().isValid():
return self.player.source().toLocalFile()
return ""
@property
def _is_playing_now(self) -> bool:
return self.player.playbackState() == QMediaPlayer.PlaybackState.PlayingState
def _setup_hotkeys(self):
self.hotkey_manager = HotkeyManager(self, self.config_loader)
self.hotkey_manager.register_all({
"play_pause": self._on_play_pause,
"prev_track": self._on_prev,
"next_track": self._on_next,
"volume_up": lambda: self._adjust_volume(5),
"volume_down": lambda: self._adjust_volume(-5),
"seek_forward": self._seek_forward,
"seek_backward": self._seek_backward,
"toggle_sidebar": self._toggle_sidebar,
"tab_library": lambda: self.stack.setCurrentIndex(0),
"tab_ipod": lambda: self.stack.setCurrentIndex(1),
"tab_settings": lambda: self.stack.setCurrentIndex(2),
"search_focus": self.library_tab._focus_search,
"select_all": self.library_tab._select_all_current,
"library_refresh": self.library_tab._on_refresh_library_clicked,
"ipod_refresh": self.ipod_tab._on_refresh_devices_clicked,
"delete_selected": self.library_tab._delete_selected_current,
"edit_metadata": self.library_tab._on_edit_metadata,
"library_add_files": self.library_tab._on_library_add_files,
})
self.settings_tab.set_hotkey_manager(self.hotkey_manager)
self.settings_tab.refresh_shortcuts_table()
# ------------------------------------------------------------------
# Playback
# ------------------------------------------------------------------
def _on_play_requested(self, track_data: dict, start_position_ms: int, auto_play: bool):
path = track_data.get("path", "")
if not path or not os.path.exists(path):
return
self._pending_seek_ms = start_position_ms
self.player.setSource(QUrl.fromLocalFile(path))
if auto_play:
self.player.play()
self.player_header.set_playing(True)
else:
self.player_header.set_playing(False)
self.player_header.set_track_info(
f"{track_data['artist']} \u2014 {track_data['title']}"
)
cover_path = track_data.get("cover_path")
if cover_path and os.path.exists(cover_path):
self.player_header.set_cover(QPixmap(cover_path), track_data.get("album", ""))
else:
artist = track_data.get("artist", "")
album = track_data.get("album", "")
cached = cover_cache.get(artist, album) if artist else None
if cached and os.path.exists(cached):
self.player_header.set_cover(QPixmap(cached), album)
else:
self.player_header.set_cover()
def _on_play_pause(self):
if self.player.playbackState() == QMediaPlayer.PlaybackState.PlayingState:
self.player.pause()
self.player_header.set_playing(False)
self.library_tab.on_playing_changed(False)
elif self.player.playbackState() == QMediaPlayer.PlaybackState.StoppedState:
self.library_tab.play_track_at_index(
max(0, self.library_tab.current_playback_index)
)
else:
self.player.play()
self.player_header.set_playing(True)
self.library_tab.on_playing_changed(True)
def _on_prev(self):
if self.player.position() > 3000:
self.player.setPosition(0)
return
idx = max(0, self.library_tab.current_playback_index - 1)
self.library_tab.play_track_at_index(idx)
def _get_next_index(self):
playable = self.library_tab.playable_row_indices()
if not playable:
return None
current = self.library_tab.current_playback_index
if self._shuffle_active:
candidates = [r for r in playable if r != current]
if not candidates:
candidates = playable
return random.choice(candidates)
try:
pos = playable.index(current)
except ValueError:
pos = -1
if pos >= 0 and pos < len(playable) - 1:
return playable[pos + 1]
if self._repeat_mode == "repeat_all" and playable:
return playable[0]
return None
def _stop_playback(self):
self.player.stop()
self.library_tab.on_playing_changed(False)
self.player_header.set_playing(False)
def _on_next(self):
idx = self._get_next_index()
if idx is not None:
self.library_tab.play_track_at_index(idx)
def _seek_forward(self):
if self.player.duration() <= 0:
return
new_pos = min(self.player.duration(), self.player.position() + 5000)
self.player.setPosition(new_pos)
def _seek_backward(self):
if self.player.duration() <= 0:
return
new_pos = max(0, self.player.position() - 5000)
self.player.setPosition(new_pos)
def _adjust_volume(self, delta: int):
slider = self.player_header.volume_slider
new_val = max(0, min(100, slider.value() + delta))
slider.setValue(new_val)
def _on_volume_changed(self, value: int):
if self.audio_output:
self.audio_output.setVolume(value / 100.0)
def _on_user_seek_frac(self, value: int):
duration = self.player.duration()
if duration <= 0:
return
new_pos = int(value / 1000.0 * duration)
new_pos = max(0, min(duration - 1, new_pos))
self.player.setPosition(new_pos)
def _on_player_position(self, position: int):
duration = self.player.duration()
self.player_header.set_position(position, duration)
self.library_tab.on_position_changed(position, duration)
def _on_player_duration(self, duration: int):
pos = self.player.position()
self.player_header.set_position(pos, duration)
def _on_shuffle_toggle(self):
self._shuffle_active = not self._shuffle_active
self.player_header.set_shuffle_active(self._shuffle_active)
def _on_repeat_cycle(self):
cycle = {"no_repeat": "repeat_all", "repeat_all": "repeat_one", "repeat_one": "no_repeat"}
self._repeat_mode = cycle[self._repeat_mode]
self.player_header.set_repeat_mode(self._repeat_mode)
def _on_media_status(self, status):
if status == QMediaPlayer.MediaStatus.EndOfMedia:
self.library_tab.on_track_ended()
if self._repeat_mode == "repeat_one":
self.player.setPosition(0)
self.player.play()
return
next_idx = self._get_next_index()
if next_idx is not None:
self.library_tab.play_track_at_index(next_idx)
else:
self._stop_playback()
elif status == QMediaPlayer.MediaStatus.NoMedia:
self.library_tab.on_playing_changed(False)
self.player_header.reset_info()
elif status == QMediaPlayer.MediaStatus.LoadedMedia:
if self._pending_seek_ms > 0:
self.player.setPosition(self._pending_seek_ms)
self._pending_seek_ms = 0
def _resume_playback_if_saved(self):
if not self._saved_track_path or not os.path.exists(self._saved_track_path):
return
for row in range(self.library_tab.library_table.rowCount()):
item = self.library_tab.library_table.item(row, 0)
if item is None:
continue
_is_ready, data = item.data(Qt.ItemDataRole.UserRole)
if data and data.get("path", "") == self._saved_track_path:
self.library_tab.play_track_at_index(
row, start_position_ms=self._saved_position_ms,
auto_play=self._saved_playing
)
return
def _restore_library_state(self):
view_mode = self.config_loader.get("Library", "view_mode", fallback="table")
filtered_album = self.config_loader.get("Library", "filtered_album", fallback="")
filtered_playlist_id = self.config_loader.get("Library", "filtered_playlist_id", fallback="")
if filtered_album:
self.library_tab.filter_by_album(filtered_album)
elif filtered_playlist_id:
pl = self.playlist_manager.get_by_id(filtered_playlist_id)
if pl:
self.library_tab.set_current_playlist_id(filtered_playlist_id)
self.library_tab.apply_filter("playlist", pl.track_paths)
else:
self.library_tab.set_view_mode(view_mode)
self.sidebar.blockSignals(True)
if filtered_playlist_id:
self.sidebar.select_playlist_silent(filtered_playlist_id)
else:
self.sidebar.select_by_view_mode(view_mode)
self.sidebar.blockSignals(False)
def closeEvent(self, event):
if self.device_monitor:
self.device_monitor.stop()
self.ipod_tab.stop_monitoring(self.device_monitor)
for tab in (self.ipod_tab, self.library_tab):
if hasattr(tab, 'worker_thread') and tab.worker_thread:
try:
if tab.worker_thread.isRunning():
tab.worker_thread.stop()
except RuntimeError:
tab.worker_thread = None
self._save_settings()
from PyQt6.QtWidgets import QApplication
QApplication.processEvents()
event.accept()
# GTK3 platform theme paths for Linux desktop integration
_GTK3_PLUGIN_PATHS = [
"/usr/lib/qt6/plugins/platformthemes/libqgtk3.so",
"/usr/lib/x86_64-linux-gnu/qt6/plugins/platformthemes/libqgtk3.so",
"/usr/lib64/qt6/plugins/platformthemes/libqgtk3.so",
"/usr/lib/aarch64-linux-gnu/qt6/plugins/platformthemes/libqgtk3.so",
]
def _setup_linux_theming():
if sys.platform != "linux":
return
if "QT_QPA_PLATFORMTHEME" in os.environ:
return
for path in _GTK3_PLUGIN_PATHS:
if os.path.exists(path):
os.environ["QT_QPA_PLATFORMTHEME"] = "gtk3"
logger.debug(f"Using GTK3 platform theme: {path}")
return
if os.path.exists("/usr/bin/qt6ct"):
os.environ["QT_QPA_PLATFORMTHEME"] = "qt6ct"
logger.debug("Using qt6ct platform theme")
return
logger.debug("No GTK platform theme plugin found, using Qt default style")
def main():
_setup_linux_theming()
app = QApplication(sys.argv)
if not os.environ.get("QT_QPA_PLATFORMTHEME"):
app.setStyle("Fusion")
window = MainWindow()
window.show()
sys.exit(app.exec())
if __name__ == "__main__":
main()
+5 -4
View File
@@ -9,7 +9,8 @@ from __future__ import annotations
import hashlib
import logging
import os
import tempfile
from xdg_base import xdg_cache_path
logger = logging.getLogger(__name__)
@@ -18,12 +19,12 @@ class CoverCache:
"""Manages a local cache of cover art images for the player UI.
The cache stores JPEG thumbnails keyed by (artist, album) hash.
Covers are stored in $XDG_CACHE_HOME/neo-pod-desktop/covers/
(default ~/.cache/neo-pod-desktop/covers/).
"""
def __init__(self, cache_dir: str | None = None):
self.cache_dir = cache_dir or os.path.join(
tempfile.gettempdir(), "neo-pod-covers"
)
self.cache_dir = cache_dir or xdg_cache_path("covers")
os.makedirs(self.cache_dir, exist_ok=True)
def _key(self, artist: str, album: str) -> str:
+42 -23
View File
@@ -10,7 +10,7 @@ import subprocess
from typing import Optional, Dict, Any
from pathlib import Path
from youtube_downloader import TrackInfo
from track_info import TrackInfo
# Configure logging
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
@@ -42,7 +42,7 @@ class AudioConverter:
def convert_to_ipod_format(self,
track_info: TrackInfo,
input_file: str,
format: str = "m4a",
format: str = "mp3",
audio_bitrate: int = 256,
overwrite: bool = False) -> str:
"""
@@ -90,31 +90,38 @@ class AudioConverter:
logger.warning(f"Existing file is too small ({file_size} bytes), re-converting")
os.remove(output_file)
# Build ffmpeg command — iPod Nano 7 is strict about AAC parameters:
# - AAC-LC profile only (HE-AAC causes playback issues/crashes)
# - 44100 Hz sample rate (firmware expects this exactly)
# - Max 160 kbps for stereo (higher bitrates may truncate playback)
codec = "aac" if format == "m4a" else "libmp3lame"
cmd = [
"ffmpeg", "-y", "-i", input_file,
"-c:a", codec,
"-b:a", f"{audio_bitrate}k",
"-ar", "44100",
"-ac", "2",
]
# For M4A/AAC: force LC profile and use libfdk_aac if available (best quality),
# otherwise fall back to built-in aac with profile flag
if format == "m4a":
# iPod Nano 7 AAC requirements:
# - AAC-LC profile only (HE-AAC causes playback issues/crashes)
# - 44100 Hz sample rate (firmware expects this exactly)
# - Max 160 kbps for stereo (higher may cause truncation/crackle)
# - +global_header: required for iPod AAC decoder init
# - +genpts: fix timestamp gaps from web sources
m4a_bitrate = min(audio_bitrate, 160)
if audio_bitrate > 160:
logger.warning(
"Bitrate %d kbps exceeds iPod Nano 7 max (160 kbps), clamping to 160",
audio_bitrate,
)
cmd = [
"ffmpeg", "-y", "-fflags", "+genpts", "-i", input_file,
"-c:a", "aac",
"-b:a", f"{m4a_bitrate}k",
"-ar", "44100",
"-ac", "2",
"-flags", "+global_header",
]
# Try libfdk_aac first (best compatibility)
fdkaac_cmd = cmd + [
"-map", "0:a",
"-map", "0:v?",
"-c:v", "copy",
"-map", "0:t?",
"-c:a", "libfdk_aac",
"-profile:a", "aac_low",
"-movflags", "+faststart",
"-map_metadata", "0",
"-loglevel", "error",
"-loglevel", "warning",
output_file,
]
try:
@@ -125,33 +132,45 @@ class AudioConverter:
if "libfdk_aac" in test.stdout:
cmd = fdkaac_cmd
else:
# Fallback: built-in aac with LC profile
cmd += [
"-map", "0:a",
"-map", "0:v?",
"-c:v", "copy",
"-map", "0:t?",
"-profile:a", "aac_low",
"-movflags", "+faststart",
"-map_metadata", "0",
"-loglevel", "error",
"-loglevel", "warning",
output_file,
]
except Exception:
cmd += [
"-map", "0:a",
"-map", "0:v?",
"-c:v", "copy",
"-map", "0:t?",
"-profile:a", "aac_low",
"-movflags", "+faststart",
"-map_metadata", "0",
"-loglevel", "error",
"-loglevel", "warning",
output_file,
]
else:
cmd += [
# MP3 for iPod Nano 7G: reliable, no special decoder flags needed.
# libmp3lame is mature and produces streams safe for hardware decoders.
# No +global_header — that flag is AAC-only and confuses MP3 playback.
cmd = [
"ffmpeg", "-y", "-i", input_file,
"-c:a", "libmp3lame",
"-b:a", f"{audio_bitrate}k",
"-ar", "44100",
"-ac", "2",
"-bitexact",
"-map", "0:a",
"-map", "0:t?",
"-id3v2_version", "3",
"-map_metadata", "0",
"-loglevel", "error",
"-loglevel", "warning",
output_file,
]
-337
View File
@@ -1,337 +0,0 @@
#!/usr/bin/env python3
"""
Command Line Interface for neo-pod-desktop
Provides a CLI for downloading, converting, managing and transferring music to iPod Nano devices
"""
import os
import sys
import logging
import argparse
import time
from typing import List, Dict, Optional, Tuple
from pathlib import Path
from tqdm import tqdm
from youtube_downloader import YouTubeDownloader, TrackInfo
from audio_converter import AudioConverter
from metadata_handler import MetadataHandler
from ipod_device import IPodDevice
# Configure logging
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
logger = logging.getLogger(__name__)
class YouTubeToIPodCLI:
"""Command Line Interface for neo-pod-desktop"""
def __init__(self):
"""Initialize the CLI"""
self.downloader = None
self.converter = None
self.metadata_handler = None
self.ipod_device = None
self.downloaded_tracks = []
self.converted_tracks = []
self.temp_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "temp")
os.makedirs(self.temp_dir, exist_ok=True)
def parse_arguments(self):
"""Parse command line arguments"""
parser = argparse.ArgumentParser(
description="neo-pod-desktop",
formatter_class=argparse.ArgumentDefaultsHelpFormatter
)
# URL argument
parser.add_argument(
"--url", "-u",
help="YouTube video or playlist URL",
required=True
)
# Output directory
parser.add_argument(
"--output-dir", "-o",
help="Output directory (iPod mount point)",
default=os.path.join(os.path.expanduser("~"), "Music", "iPod")
)
# Format
parser.add_argument(
"--format", "-f",
help="Audio format",
choices=["m4a", "mp3"],
default="m4a"
)
# Quality
parser.add_argument(
"--quality", "-q",
help="Audio quality in kbps",
type=int,
choices=[128, 192, 256, 320],
default=256
)
# Video support
parser.add_argument(
"--video",
help="Enable video download (for compatible iPod models)",
action="store_true"
)
# Video resolution
parser.add_argument(
"--resolution", "-r",
help="Video resolution (for video download)",
choices=["640x480", "480x360", "320x240"],
default="640x480"
)
# iPod device
parser.add_argument(
"--device", "-d",
help="iPod device mount point (if not specified, auto-detection will be attempted)",
default=None
)
# Skip steps
parser.add_argument(
"--skip-download",
help="Skip download step (use previously downloaded files)",
action="store_true"
)
parser.add_argument(
"--skip-convert",
help="Skip conversion step (use previously converted files)",
action="store_true"
)
parser.add_argument(
"--skip-transfer",
help="Skip transfer step (only download and convert)",
action="store_true"
)
# Clean temp files
parser.add_argument(
"--clean-temp",
help="Clean temporary files after transfer",
action="store_true"
)
return parser.parse_args()
def download(self, url: str, format: str = "m4a", quality: int = 256, video: bool = False) -> List[TrackInfo]:
"""
Download audio/video from YouTube
Args:
url: YouTube URL
format: Audio format
quality: Audio quality in kbps
video: Whether to download video
Returns:
List of TrackInfo objects
"""
logger.info(f"Downloading from URL: {url}")
# Initialize downloader
self.downloader = YouTubeDownloader(output_dir=self.temp_dir)
# Process URL
tracks = self.downloader.process_url(url, format, quality)
logger.info(f"Downloaded {len(tracks)} tracks")
return tracks
def convert(self,
tracks: List[TrackInfo],
output_dir: str,
format: str = "m4a",
quality: int = 256,
video: bool = False,
resolution: str = "640x480") -> List[Tuple[TrackInfo, str]]:
"""
Convert downloaded tracks to iPod-compatible format
Args:
tracks: List of TrackInfo objects
output_dir: Output directory
format: Audio format
quality: Audio quality in kbps
video: Whether to convert video
resolution: Video resolution
Returns:
List of (TrackInfo, output_path) tuples
"""
logger.info(f"Converting {len(tracks)} tracks to iPod-compatible format")
# Initialize converter and metadata handler
self.converter = AudioConverter(output_dir=output_dir)
self.metadata_handler = MetadataHandler(temp_dir=self.temp_dir)
# Convert each track
converted_tracks = []
for track in tqdm(tracks, desc="Converting", unit="track"):
if track.download_path and os.path.exists(track.download_path):
# Convert to iPod format
output_path = self.converter.convert_to_ipod_format(
track, track.download_path, format, quality
)
# Embed metadata
if output_path:
self.metadata_handler.process_file(output_path, track)
converted_tracks.append((track, output_path))
logger.info(f"Converted {len(converted_tracks)} tracks")
return converted_tracks
def transfer(self,
tracks: List[Tuple[TrackInfo, str]],
mount_point: Optional[str] = None) -> int:
"""
Transfer tracks to iPod
Args:
tracks: List of (TrackInfo, path) tuples
mount_point: iPod mount point (optional)
Returns:
Number of tracks transferred
"""
logger.info("Transferring tracks to iPod")
# Initialize iPod device
self.ipod_device = IPodDevice(mount_point=mount_point)
# If no mount point specified, try to detect and mount
if not mount_point:
logger.info("No mount point specified, attempting to detect iPod")
devices = self.ipod_device.detect_devices()
if not devices:
logger.error("No iPod devices found")
return 0
logger.info(f"Found iPod: {devices[0]['name']} ({devices[0]['model']})")
mount_point = self.ipod_device.mount_device(devices[0]['id'])
if not mount_point:
logger.error(f"Failed to mount iPod: {devices[0]['name']}")
return 0
logger.info(f"Mounted iPod at: {mount_point}")
# Get device info
device_info = self.ipod_device.get_device_info()
free_space = device_info.get("free_space", 0)
# Calculate total size of tracks
total_size = 0
for _, path in tracks:
if os.path.exists(path):
total_size += os.path.getsize(path)
# Check if there's enough space
if total_size > free_space:
logger.error(
f"Not enough space on iPod. Need {total_size / 1024**2:.1f} MB, " +
f"but only {free_space / 1024**2:.1f} MB available"
)
return 0
# Transfer each track
transferred_count = 0
for track, path in tqdm(tracks, desc="Transferring", unit="track"):
if os.path.exists(path):
# Transfer file
success = self.ipod_device.transfer_file(path)
if success:
transferred_count += 1
logger.info(f"Transferred {transferred_count} tracks to iPod")
return transferred_count
def clean_temp_files(self):
"""Clean temporary files"""
logger.info("Cleaning temporary files")
# Remove all files in temp directory
for file in os.listdir(self.temp_dir):
file_path = os.path.join(self.temp_dir, file)
try:
if os.path.isfile(file_path):
os.unlink(file_path)
except Exception as e:
logger.error(f"Error deleting {file_path}: {e}")
def run(self):
"""Run the CLI"""
# Parse arguments
args = self.parse_arguments()
try:
# Download step
if not args.skip_download:
self.downloaded_tracks = self.download(
args.url,
args.format,
args.quality,
args.video
)
else:
logger.info("Skipping download step")
# TODO: Load previously downloaded tracks
# Convert step
if not args.skip_convert:
self.converted_tracks = self.convert(
self.downloaded_tracks,
args.output_dir,
args.format,
args.quality,
args.video,
args.resolution
)
else:
logger.info("Skipping conversion step")
# TODO: Load previously converted tracks
# Transfer step
if not args.skip_transfer:
transferred_count = self.transfer(
self.converted_tracks,
args.device
)
if transferred_count > 0:
logger.info(f"Successfully transferred {transferred_count} tracks to iPod")
else:
logger.error("Failed to transfer tracks to iPod")
else:
logger.info("Skipping transfer step")
# Clean temp files
if args.clean_temp:
self.clean_temp_files()
return 0
except Exception as e:
logger.exception(f"Error: {e}")
return 1
if __name__ == "__main__":
cli = YouTubeToIPodCLI()
sys.exit(cli.run())
+57 -16
View File
@@ -60,11 +60,10 @@ class ConfigLoader:
if os.path.exists(self.config_file):
logger.info(f"Loading configuration from {self.config_file}")
self.config.read(self.config_file)
return True
else:
logger.warning(f"Configuration file {self.config_file} not found, using defaults")
self._set_defaults()
return False
self._set_defaults()
return True
except Exception as e:
logger.error(f"Failed to load configuration: {e}")
self._set_defaults()
@@ -95,39 +94,81 @@ class ConfigLoader:
if not self.config.has_section('General'):
self.config.add_section('General')
self.config.set('General', 'output_dir', os.path.join(os.path.expanduser("~"), "Music", "iPod"))
self.config.set('General', 'format', 'm4a')
self.config.set('General', 'quality', '256')
self.config.set('General', 'clean_temp', 'true')
if not self.config.has_option('General', 'output_dir'):
self.config.set('General', 'output_dir', os.path.join(os.path.expanduser("~"), "Music", "iPod"))
if not self.config.has_option('General', 'format'):
self.config.set('General', 'format', 'm4a')
if not self.config.has_option('General', 'quality'):
self.config.set('General', 'quality', '256')
# Video section
if not self.config.has_section('Video'):
self.config.add_section('Video')
self.config.set('Video', 'enable_video', 'false')
self.config.set('Video', 'resolution', '640x480')
if not self.config.has_option('Video', 'enable_video'):
self.config.set('Video', 'enable_video', 'false')
if not self.config.has_option('Video', 'resolution'):
self.config.set('Video', 'resolution', '640x480')
# Metadata section
if not self.config.has_section('Metadata'):
self.config.add_section('Metadata')
self.config.set('Metadata', 'embed_artwork', 'true')
self.config.set('Metadata', 'use_musicbrainz', 'false')
if not self.config.has_option('Metadata', 'embed_artwork'):
self.config.set('Metadata', 'embed_artwork', 'true')
# Device section
if not self.config.has_section('Device'):
self.config.add_section('Device')
self.config.set('Device', 'auto_detect', 'true')
self.config.set('Device', 'mount_point', '')
if not self.config.has_option('Device', 'auto_detect'):
self.config.set('Device', 'auto_detect', 'true')
if not self.config.has_option('Device', 'mount_point'):
self.config.set('Device', 'mount_point', '')
# Advanced section
if not self.config.has_section('Advanced'):
self.config.add_section('Advanced')
self.config.set('Advanced', 'temp_dir', 'temp')
self.config.set('Advanced', 'concurrent_downloads', '2')
self.config.set('Advanced', 'debug', 'false')
if not self.config.has_option('Advanced', 'temp_dir'):
self.config.set('Advanced', 'temp_dir', 'temp')
if not self.config.has_option('Advanced', 'clean_temp'):
self.config.set('Advanced', 'clean_temp', 'true')
if not self.config.has_option('Advanced', 'hide_library_buttons'):
self.config.set('Advanced', 'hide_library_buttons', 'false')
# Hotkeys section
if not self.config.has_section('Hotkeys'):
self.config.add_section('Hotkeys')
hotkey_defaults = {
'play_pause': 'Space',
'prev_track': 'Ctrl+Left',
'next_track': 'Ctrl+Right',
'volume_up': 'Ctrl+Up',
'volume_down': 'Ctrl+Down',
'seek_forward': 'Right',
'seek_backward': 'Left',
'tab_library': 'Ctrl+1',
'tab_ipod': 'Ctrl+2',
'tab_settings': 'Ctrl+3',
'search_focus': 'Ctrl+F',
'select_all': 'Ctrl+A',
'library_refresh': 'F5',
'ipod_refresh': 'Ctrl+R',
'delete_selected': 'Delete',
'library_add_files': 'Ctrl+O',
}
for key, value in hotkey_defaults.items():
if not self.config.has_option('Hotkeys', key):
self.config.set('Hotkeys', key, value)
# Playback section is DEPRECATED — session state moved to
# session.json in ~/.local/share/. The defaults below are
# kept only for backward-compatible reads during migration.
# All new code should use SessionStore instead.
if not self.config.has_section('Playback'):
self.config.add_section('Playback')
def get(self, section: str, option: str, fallback: Any = None) -> Any:
"""
+70
View File
@@ -0,0 +1,70 @@
import logging
from typing import Set
from PyQt6.QtCore import QObject, QTimer, QThread, pyqtSignal
from ipod_device import IPodDevice
logger = logging.getLogger(__name__)
class _DetectWorker(QThread):
finished = pyqtSignal(object)
def __init__(self, parent=None):
super().__init__(parent)
def run(self):
try:
devices = IPodDevice().detect_devices()
self.finished.emit(devices)
except Exception:
logger.exception("Device detection failed")
self.finished.emit([])
class DeviceMonitor(QObject):
state_changed = pyqtSignal(list)
def __init__(self, parent=None):
super().__init__(parent)
self.timer = QTimer(self)
self.timer.timeout.connect(self._poll)
self._device_ids: Set[str] = set()
self._poll_worker = None
def start(self, interval_ms: int = 3000):
self._device_ids.clear()
self._poll()
self.timer.start(interval_ms)
def stop(self):
self.timer.stop()
if self._poll_worker:
self._poll_worker.quit()
self._poll_worker.wait()
self._poll_worker = None
self._device_ids.clear()
def _poll(self):
if self._poll_worker:
try:
if self._poll_worker.isRunning():
return
except RuntimeError:
self._poll_worker = None
worker = _DetectWorker()
worker.finished.connect(self._on_devices_detected)
worker.start()
self._poll_worker = worker
def _on_devices_detected(self, devices):
if self._poll_worker:
self._poll_worker.quit()
self._poll_worker.wait()
self._poll_worker = None
current_ids = {d["id"] for d in devices}
if current_ids != self._device_ids:
self._device_ids = current_ids
self.state_changed.emit(devices)
+308
View File
@@ -0,0 +1,308 @@
#!/usr/bin/env python3
"""
Hotkey Manager for neo-pod-desktop
Provides configurable keyboard shortcuts.
"""
import logging
from typing import Callable, Dict, Optional
from PyQt6.QtCore import Qt, QObject
from PyQt6.QtGui import QKeySequence, QShortcut, QAction
from PyQt6.QtWidgets import QApplication, QWidget, QDialog, QVBoxLayout, QLabel, QPushButton
logger = logging.getLogger(__name__)
QSeq = QKeySequence
# Key names that QKeySequence doesn't expose as enum values
_KEY_MAP = {
"Space": QSeq(Qt.Key.Key_Space),
"Delete": QSeq(Qt.Key.Key_Delete),
"Up": QSeq(Qt.Key.Key_Up),
"Down": QSeq(Qt.Key.Key_Down),
"Left": QSeq(Qt.Key.Key_Left),
"Right": QSeq(Qt.Key.Key_Right),
"Enter": QSeq(Qt.Key.Key_Enter),
"Return": QSeq(Qt.Key.Key_Return),
"Esc": QSeq(Qt.Key.Key_Escape),
"Escape": QSeq(Qt.Key.Key_Escape),
"Tab": QSeq(Qt.Key.Key_Tab),
"Backspace": QSeq(Qt.Key.Key_Backspace),
"Plus": QSeq(Qt.Key.Key_Plus),
"Minus": QSeq(Qt.Key.Key_Minus),
"F1": QSeq(Qt.Key.Key_F1),
"F2": QSeq(Qt.Key.Key_F2),
"F3": QSeq(Qt.Key.Key_F3),
"F4": QSeq(Qt.Key.Key_F4),
"F5": QSeq(Qt.Key.Key_F5),
"F6": QSeq(Qt.Key.Key_F6),
"F7": QSeq(Qt.Key.Key_F7),
"F8": QSeq(Qt.Key.Key_F8),
"F9": QSeq(Qt.Key.Key_F9),
"F10": QSeq(Qt.Key.Key_F10),
"F11": QSeq(Qt.Key.Key_F11),
"F12": QSeq(Qt.Key.Key_F12),
}
def _build_seq(text: str) -> QKeySequence:
"""Build a QKeySequence from a human-readable shortcut string."""
text = text.strip()
if not text:
return QSeq()
known = _KEY_MAP.get(text)
if known is not None:
return known
return QSeq(text)
def _seq_to_string(seq: QKeySequence) -> str:
"""Convert a QKeySequence to a human-readable string."""
return seq.toString()
def _is_editable_focused() -> bool:
"""Return True if an editable widget currently has keyboard focus."""
from PyQt6.QtWidgets import QLineEdit, QTextEdit, QPlainTextEdit, QSpinBox, QDoubleSpinBox
w = QApplication.focusWidget()
if w is None:
return False
return isinstance(w, (QLineEdit, QTextEdit, QPlainTextEdit, QSpinBox, QDoubleSpinBox))
# ---------------------------------------------------------------------------
# Default shortcut definitions
# (action_name, default_key_string, display_label)
# ---------------------------------------------------------------------------
DEFAULTS = [
("play_pause", "Space", "Play / Pause"),
("prev_track", "Ctrl+Left", "Previous Track"),
("next_track", "Ctrl+Right", "Next Track"),
("volume_up", "Ctrl+Up", "Volume Up"),
("volume_down", "Ctrl+Down", "Volume Down"),
("seek_forward", "Right", "Seek Forward 5s"),
("seek_backward", "Left", "Seek Backward 5s"),
("tab_library", "Ctrl+1", "Switch to Library"),
("tab_ipod", "Ctrl+2", "Switch to iPod"),
("tab_settings", "Ctrl+3", "Switch to Settings"),
("search_focus", "Ctrl+F", "Focus Search"),
("select_all", "Ctrl+A", "Select All"),
("library_refresh", "F5", "Refresh Library"),
("ipod_refresh", "Ctrl+R", "Refresh Devices"),
("delete_selected", "Delete", "Delete Selected"),
("edit_metadata", "F2", "Edit Metadata"),
("toggle_sidebar", "Ctrl+B", "Toggle Sidebar"),
("library_add_files", "Ctrl+O", "Library: Add Files"),
]
class KeyCaptureDialog(QDialog):
"""Modal dialog that captures a single key combination."""
def __init__(self, parent=None, current: str = ""):
super().__init__(parent)
self.setWindowTitle("Press New Shortcut")
self.setMinimumWidth(320)
self.setModal(True)
self._captured_seq: QKeySequence = QSeq()
self._modifiers = Qt.KeyboardModifier.NoModifier
self._key = Qt.Key.Key_unknown
layout = QVBoxLayout(self)
self._label = QLabel(f"Press new shortcut for this action...\n(current: {current or 'none'})")
self._label.setAlignment(Qt.AlignmentFlag.AlignCenter)
self._label.setWordWrap(True)
layout.addWidget(self._label)
clear_btn = QPushButton("Clear (Disable)")
clear_btn.clicked.connect(self._on_clear)
layout.addWidget(clear_btn)
layout.addWidget(QPushButton("Cancel", clicked=self.reject))
self.setFocusPolicy(Qt.FocusPolicy.StrongFocus)
self.setFocus()
def keyPressEvent(self, event):
key = event.key()
modifiers = event.modifiers()
mod_mask = modifiers & (
Qt.KeyboardModifier.ControlModifier
| Qt.KeyboardModifier.ShiftModifier
| Qt.KeyboardModifier.AltModifier
| Qt.KeyboardModifier.MetaModifier
)
# Ignore pure modifier presses
ignore = {
Qt.Key.Key_Control, Qt.Key.Key_Shift, Qt.Key.Key_Alt, Qt.Key.Key_Meta,
Qt.Key.Key_Super_L, Qt.Key.Key_Super_R,
}
if key in ignore:
return
# Build the sequence — handle both old (int) and new (enum) PyQt6 types
def _to_int(val):
return val if isinstance(val, int) else val.value
keys = _to_int(mod_mask) | _to_int(key)
self._captured_seq = QSeq(keys)
self._label.setText(f"Captured: {self._captured_seq.toString()}\nPress Close to accept")
self.accept()
def _on_clear(self):
self._captured_seq = QSeq()
self._label.setText("Shortcut cleared (disabled)")
self.accept()
def captured_sequence(self) -> QKeySequence:
return self._captured_seq
class HotkeyManager:
"""Manages all keyboard shortcuts for the application.
Loads key bindings from config, creates QShortcut instances,
and supports live reassignment at runtime.
"""
def __init__(self, parent: QWidget, config_loader=None):
self._parent = parent
self._config_loader = config_loader
# action_name → (QShortcut, callback, default_seq)
self._shortcuts: Dict[str, QShortcut] = {}
self._callbacks: Dict[str, Callable[[], None]] = {}
self._defaults: Dict[str, QKeySequence] = {}
self._labels: Dict[str, str] = {}
for name, key_str, label in DEFAULTS:
self._defaults[name] = _build_seq(key_str)
self._labels[name] = label
# ------------------------------------------------------------------
# Registration
# ------------------------------------------------------------------
def register(self, action_name: str, callback: Callable[[], None]) -> None:
"""Register a shortcut with its callback.
If a saved key exists in config it is used; otherwise the default.
"""
saved_key = None
if self._config_loader:
saved_key = self._config_loader.get("Hotkeys", action_name, fallback=None)
seq = _build_seq(saved_key) if saved_key else self._defaults.get(action_name, QSeq())
self._callbacks[action_name] = callback
context = Qt.ShortcutContext.WindowShortcut
sc = QShortcut(seq, self._parent, context=context)
def _make_handler(name):
def handler():
self._dispatch(name)
return handler
sc.activated.connect(_make_handler(action_name))
self._shortcuts[action_name] = sc
def register_all(self, action_map: Dict[str, Callable[[], None]]) -> None:
"""Register all known shortcuts from an action_name→callback mapping."""
for name, callback in action_map.items():
self.register(name, callback)
# ------------------------------------------------------------------
# Dispatch
# ------------------------------------------------------------------
def _dispatch(self, action_name: str):
"""Check if the shortcut should fire (not blocked by editable focus),
then invoke the callback.
"""
# For seek arrows and space, don't fire when text editing
editable_blocked = {"seek_forward", "seek_backward", "play_pause"}
if action_name in editable_blocked and _is_editable_focused():
return
cb = self._callbacks.get(action_name)
if cb:
cb()
# ------------------------------------------------------------------
# Configuration persistence
# ------------------------------------------------------------------
def save_to_config(self) -> None:
"""Write all current key bindings to the config loader."""
if self._config_loader is None:
return
for name, sc in self._shortcuts.items():
key_str = sc.key().toString()
self._config_loader.set("Hotkeys", name, key_str if key_str else "")
self._config_loader.save()
def load_from_config(self) -> None:
"""Re-read key bindings from config and update all shortcuts."""
if self._config_loader is None:
return
for name, sc in self._shortcuts.items():
saved = self._config_loader.get("Hotkeys", name, fallback=None)
if saved is not None:
seq = _build_seq(saved) if saved != "" else QSeq()
else:
seq = self._defaults.get(name, QSeq())
sc.setKey(seq)
def reset_to_defaults(self) -> None:
"""Reset all shortcuts to their default keys and save."""
for name, sc in self._shortcuts.items():
default_seq = self._defaults.get(name, QSeq())
sc.setKey(default_seq)
self.save_to_config()
# ------------------------------------------------------------------
# Query API (for Settings UI)
# ------------------------------------------------------------------
def get_label(self, action_name: str) -> str:
return self._labels.get(action_name, action_name)
def get_current_key_string(self, action_name: str) -> str:
sc = self._shortcuts.get(action_name)
if sc is None:
return ""
return sc.key().toString()
def get_default_key_string(self, action_name: str) -> str:
return self._defaults.get(action_name, QSeq()).toString()
def all_action_names(self):
return list(self._shortcuts.keys())
# ------------------------------------------------------------------
# Live reassignment (called from Settings UI)
# ------------------------------------------------------------------
def set_shortcut(self, action_name: str, key_string: str) -> None:
"""Change the key binding for an action at runtime and persist."""
sc = self._shortcuts.get(action_name)
if sc is None:
return
seq = _build_seq(key_string)
sc.setKey(seq)
if self._config_loader:
self._config_loader.set("Hotkeys", action_name, key_string)
self._config_loader.save()
def set_shortcut_seq(self, action_name: str, seq: QKeySequence) -> None:
"""Set a shortcut directly from a QKeySequence (from key capture dialog)."""
sc = self._shortcuts.get(action_name)
if sc is not None:
sc.setKey(seq)
if self._config_loader:
self._config_loader.set("Hotkeys", action_name, seq.toString())
self._config_loader.save()
+18
View File
@@ -294,6 +294,24 @@ class IPodDevice:
self.logger.error(f"Device {device_id} does not exist")
return None
# Check if already mounted via findmnt (avoids udisksctl AlreadyMounted error)
try:
fm_result = subprocess.run(
["findmnt", "-n", "-o", "TARGET", device_id],
capture_output=True, check=False, text=True
)
if fm_result.returncode == 0 and fm_result.stdout.strip():
mount_point = fm_result.stdout.strip()
if os.path.exists(mount_point):
self.mount_point = mount_point
self._block_device = device_id
name = device_id[5:]
self._parent_drive = f"/dev/{name.rstrip('0123456789')}"
self.logger.info(f"Device already mounted at: {mount_point}")
return mount_point
except Exception:
self.logger.warning("findmnt check failed, falling through to udisksctl")
try:
result = subprocess.run(
["udisksctl", "mount", "-b", device_id],
+750 -35
View File
@@ -21,6 +21,7 @@ Public API preserved for main.py compatibility:
"""
import importlib
import importlib.util
import os
import sys
import hashlib
@@ -29,13 +30,25 @@ import random
import shutil
import sqlite3
import struct
import subprocess
import time
from typing import Any, Optional
from typing import Any, Callable, Optional
from mutagen import File as MutagenFile
from mutagen.easyid3 import EasyID3
_IOP_ROOT = "/tmp/iOpenPod"
from artwork import prepare_artwork_for_track, ArtworkEntry, write_artworkdb
def _find_iop_root() -> str:
"""Return the vendored iOpenPod packages path."""
vendor_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "vendor")
if not os.path.isdir(vendor_path):
raise ImportError(
"iOpenPod vendor packages not found at %s" % vendor_path
)
return vendor_path
_IOP_ROOT = _find_iop_root()
_IOP_CACHE: dict[str, Any] = {}
_IOP_READY = False
@@ -57,25 +70,40 @@ def _import_iop(module_name: str):
return _IOP_CACHE[module_name]
if not _IOP_READY:
# Evict local shadows ONCE so iOpenPod modules resolve correctly.
sys.modules.pop("ipod_device", None)
_IOP_READY = True
logger.debug("iOpenPod boot: evicted local ipod_device")
src_paths = [p for p in sys.path if p.endswith('/src') or p == 'src']
for p in src_paths:
sys.path.remove(p)
had_iop = _IOP_ROOT in sys.path
if not had_iop:
sys.path.insert(0, _IOP_ROOT)
importlib.invalidate_caches()
try:
mod = importlib.import_module(module_name)
finally:
for p in src_paths:
if p not in sys.path:
sys.path.insert(0, p)
if not had_iop:
sys.path.remove(_IOP_ROOT)
try:
sys.path.remove(_IOP_ROOT)
except ValueError:
pass
_IOP_CACHE[module_name] = mod
return mod
logger = logging.getLogger(__name__)
# iOpenPod parsers/writers log ERROR for transient I/O (USB resettling)
logging.getLogger("iTunesDB_Parser").setLevel(logging.CRITICAL)
logging.getLogger("iTunesDB_Writer").setLevel(logging.CRITICAL)
# ── constants ────────────────────────────────────────────────────────
FILETYPE_CODES = {
"mp3": 0x4D503320,
@@ -90,12 +118,27 @@ MASTER_CONTAINER_PID = 203092939621887772
MUSIC_CONTAINER_PID = 4872745547565090077
_REMOVE_ARTWORK = object()
"""Sentinel: used in artwork_overrides to remove artwork from a track."""
# ── Helpers ──────────────────────────────────────────────────────────
def _to_mac_epoch(ts: int) -> int:
return ts + MAC_EPOCH_OFFSET if ts > 0 else 0
def content_hash(filepath: str, max_bytes: int = 65536) -> str:
"""SHA1 of first max_bytes of audio data (fast content fingerprint)."""
h = hashlib.sha1()
try:
with open(filepath, "rb") as f:
h.update(f.read(max_bytes))
except OSError:
return ""
return h.hexdigest()
def _read_audio_tags(filepath: str) -> dict[str, Any]:
"""Extract title, artist, album, duration, etc. from an audio file."""
info: dict[str, Any] = {}
@@ -149,6 +192,16 @@ def _read_audio_tags(filepath: str) -> dict[str, Any]:
ds = str(val[0])
if len(ds) >= 4:
info["year"] = int(ds[:4])
# Compilation flag (TCMP frame) — only set when explicitly present
try:
id3 = ID3(filepath)
if "TCMP" in id3:
tcmp = id3["TCMP"]
if hasattr(tcmp, "text") and tcmp.text:
info["compilation"] = str(tcmp.text[0]) == "1"
except Exception:
pass
except Exception:
pass
elif ext in (".m4a", ".aac", ".mp4", ".m4p"):
@@ -171,6 +224,14 @@ def _read_audio_tags(filepath: str) -> dict[str, Any]:
tr = t["trkn"]
if isinstance(tr, list) and tr and isinstance(tr[0], tuple):
info["track_number"] = tr[0][0]
# Compilation flag (cpil atom)
if "cpil" in t:
cpil = t["cpil"]
if isinstance(cpil, bool):
info["compilation"] = cpil
elif isinstance(cpil, list) and len(cpil) > 0:
info["compilation"] = bool(cpil[0])
else:
if audio and audio.tags:
for tag_k, dest in [("title", "title"), ("artist", "artist"),
@@ -182,13 +243,55 @@ def _read_audio_tags(filepath: str) -> dict[str, Any]:
info[dest] = str(v[0]) if isinstance(v, list) and v else str(v)
except (ValueError, TypeError):
pass
# Compilation flag (vorbis comment)
try:
if "compilation" in audio.tags:
comp = audio.tags["compilation"]
comp_val = str(comp[0]) if isinstance(comp, list) else str(comp)
info["compilation"] = comp_val == "1"
except (ValueError, TypeError):
pass
except Exception:
pass
return info
def _probe_gapless(filepath: str, sample_rate: int, duration_ms: int) -> tuple[int, int, int]:
"""Probe AAC/MP3 file for encoder delay and padding samples.
Returns (pregap, postgap, sample_count).
"""
pregap = 0
postgap = 0
try:
result = subprocess.run(
["ffprobe", "-v", "error", "-select_streams", "a:0",
"-show_entries", "stream_tags=encoder_delay,padding",
"-of", "csv=p=0", filepath],
capture_output=True, text=True, timeout=10,
)
if result.returncode == 0 and result.stdout.strip():
parts = result.stdout.strip().split(",")
pregap = int(parts[0]) if parts[0] else 0
postgap = int(parts[1]) if len(parts) > 1 and parts[1] else 0
except Exception:
pass
# Fallback: typical AAC-LC encoding delay (1 frame = 1024 samples at 44100 Hz)
if pregap == 0 and sample_rate == 44100:
ext = os.path.splitext(filepath)[1].lower()
if ext in (".m4a", ".aac", ".mp4"):
pregap = 1024
total_samples = int(duration_ms * sample_rate / 1000) if duration_ms else 0
sample_count = max(0, total_samples - pregap - postgap)
return pregap, postgap, sample_count
def _scanned_tracks_to_iop(mountpoint: str,
scanned: list[dict]) -> list:
scanned: list[dict],
artwork_map: dict[str, tuple[int, int, int]] | None = None) -> list:
"""Convert our scanned track dicts to iOpenPod TrackInfo objects."""
IopTrackInfo = _import_iop("iTunesDB_Writer.mhit_writer").TrackInfo
tracks: list = []
@@ -201,14 +304,26 @@ def _scanned_tracks_to_iop(mountpoint: str,
location = f":iPod_Control:Music:{rel.replace('/', ':')}"
size = os.path.getsize(fpath)
art_count = 0
art_size = 0
mhii_link = 0
if artwork_map and fpath in artwork_map:
mhii_link, art_count, art_size = artwork_map[fpath]
sr = s.get("sample_rate", 44100)
dur_ms = s.get("duration_ms", 0)
pregap, postgap, sample_count = _probe_gapless(fpath, sr, dur_ms)
has_album = bool(s.get("album"))
t = IopTrackInfo(
title=s.get("title") or base,
location=location,
size=size,
length=s.get("duration_ms", 0),
length=dur_ms,
filetype=ft,
bitrate=s.get("bitrate", 0) // 1000,
sample_rate=s.get("sample_rate", 44100),
sample_rate=sr,
artist=s.get("artist") or "",
album=s.get("album") or "",
album_artist=s.get("album_artist") or s.get("artist") or "",
@@ -219,9 +334,20 @@ def _scanned_tracks_to_iop(mountpoint: str,
total_tracks=s.get("track_count", 0),
disc_number=s.get("disc_number", 0),
total_discs=s.get("disc_count", 0),
artwork_count=s.get("art_count", 0),
artwork_size=s.get("art_size", 0),
mhii_link=s.get("mhii_link", 0),
artwork_count=art_count,
artwork_size=art_size,
mhii_link=mhii_link,
pregap=pregap,
postgap=postgap,
sample_count=sample_count,
gapless_track_flag=1,
gapless_album_flag=1 if has_album else 0,
compilation_flag=s.get("compilation", False),
play_count=s.get("play_count", 0),
rating=s.get("rating", 0),
last_played=s.get("last_played", 0),
skip_count=s.get("skip_count", 0),
played_mark=-1,
)
tracks.append(t)
return tracks
@@ -282,6 +408,7 @@ class Nano7Database:
tags["file_name"] = fname
tags["file_size"] = os.path.getsize(fpath)
tags["pid"] = self._hash_to_pid(fpath)
tags["content_hash"] = content_hash(fpath)
results.append(tags)
return results
@@ -361,46 +488,84 @@ class Nano7Database:
return len(self._scan_ipod_files())
def get_all_tracks(self) -> list[dict]:
"""Return all tracks on iPod with full metadata."""
"""Return all tracks on iPod with full metadata including play stats."""
scanned = self._scan_ipod_files()
base_stats, delta_stats = self.read_play_stats()
tracks = []
for s in scanned:
rel = os.path.relpath(s["file_path"], self.music_base)
location = f":iPod_Control:Music:{rel.replace(os.sep, ':')}"
bs = base_stats.get(location, {})
dd = delta_stats.get(location, {})
pc = bs.get("play_count", 0)
sc = bs.get("skip_count", 0)
dd_rt = dd.get("rating", 0)
rating = dd_rt if dd_rt >= 0 else bs.get("rating", 0)
tracks.append({
"pid": s["pid"],
"title": s.get("title") or os.path.splitext(s["file_name"])[0],
"artist": s.get("artist") or "Unknown Artist",
"album": s.get("album") or "",
"album_artist": s.get("album_artist") or "",
"genre": s.get("genre", ""),
"duration_ms": s.get("duration_ms", 0),
"track_number": s.get("track_number", 0),
"file_size": s.get("file_size", 0),
"disc_number": s.get("disc_number", 0),
"physical_order": 0,
"date_modified": 0,
"year": s.get("year", 0),
"ipod_path": None,
"file_path": s["file_path"],
"content_hash": s.get("content_hash", ""),
"play_count": pc,
"rating": rating,
"last_played": bs.get("last_played", 0),
"skip_count": sc,
})
return tracks
def delete_track(self, pid: int) -> bool:
"""Delete track by pid. Scans for matching file, deletes, re-syncs."""
deleted = False
for s in self._scan_ipod_files():
if s["pid"] == pid:
if os.path.exists(s["file_path"]):
os.remove(s["file_path"])
logger.info("Deleted: %s", s["file_path"])
deleted = True
break
if deleted:
self.sync_itunescdb()
return True
# Try deleting by file path if pid lookup fails
# pid might be a negative hash; try all files
logger.warning("No file found for pid=%d", pid)
return False
"""Delete a single track by pid (scans, deletes, syncs)."""
return self.delete_tracks([pid]) > 0
def remove_duplicates(self) -> list[dict]:
def delete_tracks(self, pids: list[int], progress_callback=None) -> int:
"""Delete tracks by pid. Scans once, deletes all matching files,
then regenerates the database exactly once.
Args:
pids: List of track pids to delete.
progress_callback: Optional callable(percent, status) for progress.
Returns:
Number of files actually deleted.
"""
pid_set = set(pids)
scanned = self._scan_ipod_files()
deleted = 0
for i, s in enumerate(scanned):
if s["pid"] in pid_set and os.path.exists(s["file_path"]):
os.remove(s["file_path"])
logger.info("Deleted: %s", s["file_path"])
deleted += 1
pid_set.discard(s["pid"])
if progress_callback and i % 5 == 0:
progress_callback(
min(50, int((i + 1) / max(len(scanned), 1) * 50)),
f"Deleting files... {deleted} removed",
)
if deleted:
if progress_callback:
progress_callback(55, "Regenerating iPod database...")
self.sync_itunescdb()
if progress_callback:
progress_callback(100, f"Removed {deleted} track(s)")
return deleted
def remove_duplicates(self, progress_callback=None) -> list[dict]:
"""Find and remove duplicate tracks (same title+artist+album)."""
all_tracks = self._scan_ipod_files()
seen: dict[tuple, list[dict]] = {}
@@ -414,7 +579,6 @@ class Nano7Database:
for key, items in seen.items():
if len(items) <= 1:
continue
# Keep first, delete rest
for item in items[1:]:
fpath = item["file_path"]
if os.path.exists(fpath):
@@ -427,9 +591,19 @@ class Nano7Database:
"file_path": fpath,
})
logger.info("Removed duplicate: %s", fpath)
if progress_callback:
progress_callback(
min(50, int(len(removed) / max(len(all_tracks), 1) * 50)),
f"Removing duplicates... {len(removed)} found",
)
if removed:
if progress_callback:
progress_callback(55, "Regenerating iPod database...")
self.sync_itunescdb()
if progress_callback:
progress_callback(100, f"Removed {len(removed)} duplicate(s)")
logger.info("Removed %d duplicate(s)", len(removed))
return removed
@@ -469,28 +643,562 @@ class Nano7Database:
deleted += 1
return deleted
# ── playlist creation ────────────────────────────────────────────
def create_playlist(self, name: str, track_paths: list[str]) -> bool:
"""Create a user playlist on the iPod.
Two-pass matching: content hash first, then title+artist+album
fallback. Only tracks already present on the iPod will be
included in the playlist.
Args:
name: Playlist name.
track_paths: Local file paths of the tracks to include.
Returns:
True if the playlist was created, False otherwise.
"""
scanned = self._scan_ipod_files()
if not scanned:
logger.warning("No tracks on iPod — cannot create playlist")
return False
playlist_created = False
self._merge_play_stats(scanned)
def _build_pl_cb(scanned, tracks):
"""Build playlist using real db_track_id from TrackInfo."""
nonlocal playlist_created
try:
IopPlaylistInfo = _import_iop(
"iTunesDB_Writer.mhyp_writer"
).PlaylistInfo
except Exception as e:
logger.exception("Failed to import PlaylistInfo: %s", e)
return None
# Pass 1: content_hash → db_track_id
hash_to_db_id: dict[str, int] = {}
for i, s in enumerate(scanned):
ch = s.get("content_hash", "")
if ch:
hash_to_db_id[ch] = tracks[i].db_track_id
found_ids: list[int] = []
unmatched: list[str] = []
for path in track_paths:
if not os.path.exists(path):
continue
ch = content_hash(path)
db_id = hash_to_db_id.get(ch)
if db_id is not None:
found_ids.append(db_id)
else:
unmatched.append(path)
hash_matched_count = len(found_ids)
# Pass 2: metadata fallback
if unmatched:
scanned_meta = []
for i, s in enumerate(scanned):
scanned_meta.append((
s.get("title", "").lower(),
s.get("artist", "").lower(),
(s.get("album") or "").lower(),
tracks[i].db_track_id,
))
found_set = set(found_ids)
for path in unmatched:
tags = _read_audio_tags(path)
title = (tags.get("title") or "").lower()
artist = (tags.get("artist") or "").lower()
album = (tags.get("album") or "").lower()
if not title or not artist:
continue
for s_title, s_artist, s_album, db_id in scanned_meta:
if (db_id not in found_set
and s_title == title
and s_artist == artist
and s_album == album):
found_ids.append(db_id)
found_set.add(db_id)
break
if not found_ids:
logger.warning(
"No tracks from playlist '%s' found on iPod",
name,
)
return None
matched_by_hash = hash_matched_count
matched_by_meta = len(found_ids) - hash_matched_count
logger.info(
"Playlist '%s': %d track(s) matched (%d by hash, %d by metadata)",
name, len(found_ids), matched_by_hash, matched_by_meta,
)
playlist_created = True
return [IopPlaylistInfo(name=name, track_ids=found_ids)]
db_ok = self._write_databases_from_tracks(
scanned, build_extra_playlists=_build_pl_cb,
)
self._cleanup_play_counts_file()
return db_ok and playlist_created
# ── sync: regenerate everything via iOpenPod ──────────────────────
def sync_itunescdb(self) -> None:
"""Regenerate ALL databases using iOpenPod's writers."""
def sync_itunescdb(self) -> bool:
"""Regenerate ALL databases using iOpenPod's writers.
Returns True if the databases were written successfully.
"""
try:
self._sync_itunescdb_impl()
return True
except Exception as e:
logger.exception("iTunesDB sync failed: %s", e)
return False
def _merge_play_stats(self, scanned: list[dict]) -> None:
"""Merge iPod + local play stats into *scanned* track dicts in-place.
Uses additive model (like iTunes):
``new_value = max(ipod_base, local_cache, play_stats_store) + ipod_delta``
Includes a **validation gate** that refuses to produce all-zero
play counts when we have previously-recorded non-zero data in the
local PlayStatsStore — this prevents accidental data loss from
silent iTunesDB parse failures.
"""
base_stats, delta_stats = self.read_play_stats()
local_play_stats = self._load_local_play_stats()
# Supplement local_play_stats with durable PlayStatsStore (SQLite)
from library_cache import get_library_cache
db_store = get_library_cache()
for s in scanned:
rel = os.path.relpath(s["file_path"], self.music_base)
location = f":iPod_Control:Music:{rel.replace(os.sep, ':')}"
bs = base_stats.get(location, {})
dd = delta_stats.get(location, {})
ch = s.get("content_hash", "")
# Three-tier fallback: iTunesDB → LibraryCache (SQLite) → old JSON cache
local_pc = local_play_stats.get(ch, {})
db_entry = db_store.get_by_content_hash(ch) if ch else None
s["play_count"] = max(
bs.get("play_count", 0),
(db_entry or {}).get("play_count", 0),
local_pc.get("play_count", 0),
)
s["skip_count"] = max(
bs.get("skip_count", 0),
(db_entry or {}).get("skip_count", 0),
local_pc.get("skip_count", 0),
)
dd_rating = dd.get("rating", 0)
store_rating = (db_entry or {}).get("rating", 0)
s["rating"] = dd_rating if dd_rating >= 0 else (
bs.get("rating", 0) or store_rating or local_pc.get("rating", 0)
)
s["last_played"] = max(
bs.get("last_played", 0),
(db_entry or {}).get("last_played", 0),
local_pc.get("last_played", 0),
)
# ── validation gate ────────────────────────────────────────────
# If every scanned track would get play_count=0 but the local DB
# previously had non-zero values, restore them.
all_zero = all(s.get("play_count", 0) == 0 for s in scanned)
if all_zero:
restored = 0
for s in scanned:
ch = s.get("content_hash", "")
if not ch:
continue
db_entry = db_store.get_by_content_hash(ch) if ch else None
stored_pc = (db_entry or {}).get("play_count", 0)
if stored_pc > 0:
s["play_count"] = stored_pc
restored += 1
if restored:
logger.warning(
"Validation gate triggered — restored play_count for %d track(s) "
"from local database after merge produced all zeros",
restored,
)
# ───────────────────────────────────────────────────────────────
def _cleanup_play_counts_file(self) -> None:
"""Delete the iPod 'Play Counts' delta file after a full sync."""
pc_file = os.path.join(
self.mountpoint, "iPod_Control", "iTunes", "Play Counts",
)
if os.path.exists(pc_file):
try:
os.remove(pc_file)
logger.debug("Deleted Play Counts file after sync")
except OSError as e:
logger.debug("Could not delete Play Counts file: %s", e)
def _sync_itunescdb_impl(self) -> None:
# 1. Scan all audio files on iPod
scanned = self._scan_ipod_files()
if not scanned:
logger.warning("No audio files found on iPod — writing empty databases")
# Write minimal empty databases
self._write_empty_databases()
return
self._merge_play_stats(scanned)
if not self._write_databases_from_tracks(scanned):
logger.error("iTunesDB sync failed — play counts not cleaned up")
return
self._cleanup_play_counts_file()
def find_track_by_content_hash(self, content_hash: str) -> dict | None:
"""Find an iPod track by its content hash."""
if not content_hash:
return None
for t in self.get_all_tracks():
if t.get("content_hash") == content_hash:
return t
return None
def find_track_by_metadata(
self, title: str, artist: str, album: str = "",
) -> dict | None:
"""Find an iPod track by title+artist+album (case-insensitive, fallback matching)."""
for t in self.get_all_tracks():
if (t["title"].lower() == title.lower() and
t["artist"].lower() == artist.lower() and
(t.get("album") or "").lower() == album.lower()):
return t
return None
def read_play_stats(self) -> tuple[dict[str, dict[str, int]], dict[str, dict[str, int]]]:
"""Read iPod play stats split into base (written) and delta (since last write).
Returns ``(base_stats, delta_stats)`` keyed by iPod ``location``.
**base_stats** — values from the binary iTunesDB / Dynamic.itdb
(the last written totals). Fields: ``play_count``, ``rating``,
``last_played``, ``skip_count``.
**delta_stats** — deltas from the ``Play Counts`` file accumulated
since the last database write. Fields: ``play_count``, ``skip_count``,
``rating`` (0 if no delta activity).
Both return empty dicts when the databases cannot be read (new or
corrupted iPod state).
"""
ipod_device = _import_iop("ipod_device")
try:
itdb_path = ipod_device.resolve_itdb_path(self.mountpoint)
except Exception:
itdb_path = os.path.join(
self.mountpoint, "iPod_Control", "iTunes", "iTunesDB",
)
base_stats: dict[str, dict[str, int]] = {}
delta_stats: dict[str, dict[str, int]] = {}
try:
ipl = _import_iop("iTunesDB_Parser.ipod_library")
data = ipl.load_ipod_library(itdb_path, merge_playcounts=False)
except Exception as e:
logger.debug("Cannot read iPod play base stats: %s", e)
return base_stats, delta_stats
if not data:
return base_stats, delta_stats
tracks = data.get("mhlt", [])
for track in tracks:
loc = track.get("Location") or track.get("location", "")
if not loc:
continue
base_stats[loc] = {
"play_count": track.get("play_count_1", 0) or 0,
"rating": track.get("rating", 0) or 0,
"last_played": track.get("last_played", 0) or 0,
"skip_count": track.get("skip_count", 0) or 0,
}
try:
pc_mod = _import_iop("iTunesDB_Parser.playcounts")
pc_dir = os.path.dirname(itdb_path)
pc_path = os.path.join(pc_dir, "Play Counts")
entries = pc_mod.parse_playcounts(pc_path)
if entries:
for i, entry in enumerate(entries):
if i >= len(tracks):
break
loc = tracks[i].get("Location") or tracks[i].get("location", "")
if not loc:
continue
delta_stats[loc] = {
"play_count": entry.play_count,
"skip_count": entry.skip_count,
"rating": entry.rating,
}
except Exception as e:
logger.debug("Cannot read iPod Play Counts deltas: %s", e)
logger.debug(
"Read play stats — base: %d tracks, delta: %d tracks",
len(base_stats), len(delta_stats),
)
return base_stats, delta_stats
def read_play_counts_delta(self) -> dict[str, dict[str, int]]:
"""Read Play Counts file using filesystem scan (no iTunesDB needed).
Positional mapping uses ``_scan_ipod_files()`` order (correct after
any sync written by this app). Returns ``{location: {play_count,
skip_count, rating}}`` or empty dict on failure.
"""
pc_path = os.path.join(
self.mountpoint, "iPod_Control", "iTunes", "Play Counts",
)
if not os.path.exists(pc_path):
return {}
pc_mod = _import_iop("iTunesDB_Parser.playcounts")
try:
entries = pc_mod.parse_playcounts(pc_path)
except Exception:
return {}
if not entries:
return {}
scanned = self._scan_ipod_files()
delta: dict[str, dict[str, int]] = {}
for i, entry in enumerate(entries):
if i >= len(scanned):
break
if entry.play_count == 0 and entry.skip_count == 0 and entry.rating < 0:
continue
rel = os.path.relpath(scanned[i]["file_path"], self.music_base)
loc = f":iPod_Control:Music:{rel.replace(os.sep, ':')}"
delta[loc] = {
"play_count": entry.play_count,
"skip_count": entry.skip_count,
"rating": entry.rating,
}
return delta
@staticmethod
def _load_local_play_stats() -> dict[str, dict[str, int]]:
"""Read accumulated play stats from the local library cache.
Returns ``{content_hash: {play_count, rating, last_played,
skip_count}}`` for every cached entry that has play data.
"""
from library_cache import get_library_cache
cache = get_library_cache()
result: dict[str, dict[str, int]] = {}
for _path, entry in cache.iter_entries():
ch = entry.get("content_hash", "")
pc = entry.get("play_count", 0)
rating = entry.get("rating", 0)
lp = entry.get("last_played", 0)
sc = entry.get("skip_count", 0)
if pc or rating or lp or sc:
if ch:
result[ch] = {
"play_count": pc,
"rating": rating,
"last_played": lp,
"skip_count": sc,
}
return result
def update_track_metadata(
self,
pid: int,
new_metadata: dict,
local_source_path: str | None = None,
) -> bool:
"""Update metadata for a single iPod track without touching audio files.
Only updates the databases (iTunesDB + SQLite + ArtworkDB).
The audio file on the iPod is never modified — new metadata is
written directly into the database records.
Args:
pid: The iPod track pid (from get_all_tracks).
new_metadata: Dict with optional keys: title, artist, album,
album_artist, genre, composer, year, track_number,
disc_number, comment.
local_source_path: If provided, artwork is extracted from this
local file and encoded into iPod-native formats.
Returns:
True if the track was found and updated, False otherwise.
"""
scanned = self._scan_ipod_files()
if not scanned:
logger.warning("No tracks on iPod to update")
return False
target_idx = None
for idx, s in enumerate(scanned):
if s["pid"] == pid:
target_idx = idx
break
if target_idx is None:
logger.warning("Track with pid=%d not found on iPod", pid)
return False
target = scanned[target_idx]
text_fields = {
"title": "title",
"artist": "artist",
"album": "album",
"album_artist": "album_artist",
"genre": "genre",
"composer": "composer",
"comment": "comment",
}
for meta_key, scan_key in text_fields.items():
if meta_key in new_metadata:
target[scan_key] = str(new_metadata[meta_key]) if new_metadata[meta_key] else ""
if "year" in new_metadata:
target["year"] = int(new_metadata["year"] or 0)
if "track_number" in new_metadata:
target["track_number"] = int(new_metadata["track_number"] or 0)
if "disc_number" in new_metadata:
target["disc_number"] = int(new_metadata["disc_number"] or 0)
artwork_overrides: dict[str, object] = {}
if local_source_path and os.path.exists(local_source_path):
try:
art_result = prepare_artwork_for_track(local_source_path)
if art_result is not None:
artwork_overrides[target["file_path"]] = art_result
else:
artwork_overrides[target["file_path"]] = _REMOVE_ARTWORK
except Exception as e:
logger.warning("Failed to extract artwork from %s: %s",
local_source_path, e)
self._merge_play_stats(scanned)
self._write_databases_from_tracks(scanned, artwork_overrides=artwork_overrides)
self._cleanup_play_counts_file()
logger.info("Updated metadata for track: %s", target.get("title", "?"))
return True
def _write_databases_from_tracks(
self,
scanned: list[dict],
artwork_overrides: dict[str, object] | None = None,
extra_playlists: list | None = None,
build_extra_playlists: Callable | None = None,
) -> bool:
"""Build artwork, convert to TrackInfo, write iTunesDB + SQLite.
Args:
scanned: List of track dicts from _scan_ipod_files().
artwork_overrides: Optional dict mapping iPod file_path →
(art_hash, {fmt_id: EncodedFormatPayload}) for tracks whose
artwork should come from a local file instead of the iPod file.
Use the ``_REMOVE_ARTWORK`` sentinel to explicitly remove
artwork for a track. Omit a key to keep existing iPod artwork.
extra_playlists: Pre-built list of PlaylistInfo (ignored when
build_extra_playlists is provided).
build_extra_playlists: Optional callable(scanned, tracks) → list
of PlaylistInfo. Called after TrackInfo conversion and
db_track_id generation, so the callback can use real
db_track_id values from TrackInfo objects.
Returns:
True if databases were written successfully.
"""
artwork_overrides = artwork_overrides or {}
# 1. Prepare artwork for all tracks
artwork_map: dict[str, tuple[int, int, int]] = {}
art_entries: list[ArtworkEntry] = []
art_hash_to_img_id: dict[str, int] = {}
next_img_id = 100
for idx, s in enumerate(scanned):
fpath = s["file_path"]
override = artwork_overrides.get(fpath)
if override is None:
try:
art_result = prepare_artwork_for_track(fpath)
except Exception:
art_result = None
elif override is _REMOVE_ARTWORK:
art_result = None
else:
art_result = override
if art_result is not None:
art_hash, formats = art_result
art_count = len(formats)
total_size = sum(p.size for p in formats.values())
if art_hash not in art_hash_to_img_id:
img_id = next_img_id
next_img_id += 1
art_hash_to_img_id[art_hash] = img_id
art_entries.append(ArtworkEntry(
img_id=img_id,
db_track_id=idx,
art_hash=art_hash,
src_img_size=total_size,
formats=formats,
db_track_ids=[idx],
))
else:
img_id = art_hash_to_img_id[art_hash]
for entry in art_entries:
if entry.img_id == img_id:
if idx not in entry.db_track_ids:
entry.db_track_ids.append(idx)
break
artwork_map[fpath] = (img_id, art_count, total_size)
if art_entries:
try:
write_artworkdb(self.mountpoint, art_entries)
logger.info("ArtworkDB written: %d unique covers, %d total tracks",
len(art_entries), len(artwork_map))
except Exception as e:
logger.warning("ArtworkDB write failed: %s", e)
# 2. Convert to iOpenPod TrackInfo
tracks = _scanned_tracks_to_iop(self.mountpoint, scanned)
tracks = _scanned_tracks_to_iop(self.mountpoint, scanned, artwork_map)
logger.info("Built %d iOpenPod TrackInfo objects", len(tracks))
# 2b. Pre-generate db_track_id for all tracks so callbacks can use them
generate_db_track_id = _import_iop("iTunesDB_Writer.mhit_writer").generate_db_track_id
for track in tracks:
if track.db_track_id == 0:
track.db_track_id = generate_db_track_id()
# 2c. Build extra playlists via callback if provided
if build_extra_playlists is not None:
result = build_extra_playlists(scanned, tracks)
if result is not None:
extra_playlists = result
else:
extra_playlists = None
# 3. Determine capabilities for this device
ipod_device = _import_iop("ipod_device")
capabilities = None
@@ -519,10 +1227,11 @@ class Nano7Database:
backup=True,
capabilities=capabilities,
master_playlist_name="iPod",
playlists=extra_playlists or None,
)
if not ok:
logger.error("iTunesDB write failed")
return
return False
logger.info("iTunesDB written (%d tracks)", len(tracks))
# 5. Extract db_pid from the binary DB for SQLite cross-reference
@@ -550,13 +1259,18 @@ class Nano7Database:
capabilities=capabilities,
firewire_id=firewire_id,
backup=True,
playlists=extra_playlists or None,
)
if sqlite_ok:
logger.info("SQLite databases written")
else:
logger.error("SQLite database write failed")
return False
except Exception as e:
logger.exception("SQLite write error: %s", e)
return False
return True
def _write_empty_databases(self) -> None:
"""Write minimal empty databases when no tracks exist."""
@@ -584,3 +1298,4 @@ class Nano7Database:
db_pid=random.getrandbits(64),
capabilities=capabilities, firewire_id=fwid, backup=True,
)
self._cleanup_play_counts_file()
+496
View File
@@ -0,0 +1,496 @@
#!/usr/bin/env python3
"""
Library metadata cache for neo-pod-desktop.
Stored as SQLite-format database in ``~/.local/share/neo-pod-desktop/library_cache.db``.
Each row is keyed by its **file path** and also carries a stable **UUID** so
play-count / rating / last-played data survives file renames, moves, and
re-encodings. The database is thread-safe (WAL + busy_timeout) and
automatically rotated to a backup file on every write.
Migration from the old JSON file (``~/.cache/neo-pod-desktop/library_cache.json``)
happens automatically on the first access.
"""
import hashlib
import json
import logging
import os
import sqlite3
import threading
import time
import uuid
from typing import Any, Iterator
from xdg_base import xdg_data_path
logger = logging.getLogger(__name__)
_DB_FILE = "library_cache.db"
_BACKUP_FILE = "library_cache.db.backup"
_OLD_JSON_FILE = "library_cache.json"
_SCHEMA_VERSION = 2
_SCHEMA = """
CREATE TABLE IF NOT EXISTS library_cache (
path TEXT PRIMARY KEY,
uuid TEXT UNIQUE NOT NULL,
title TEXT DEFAULT '',
artist TEXT DEFAULT '',
album TEXT DEFAULT '',
album_artist TEXT DEFAULT '',
genre TEXT DEFAULT '',
duration_s REAL DEFAULT 0,
size INTEGER DEFAULT 0,
track_num INTEGER DEFAULT 0,
source_type TEXT DEFAULT '',
bitrate INTEGER DEFAULT 0,
sample_rate INTEGER DEFAULT 44100,
cover_path TEXT DEFAULT '',
thumbnail_url TEXT DEFAULT '',
play_count INTEGER DEFAULT 0,
rating INTEGER DEFAULT 0,
last_played INTEGER DEFAULT 0,
skip_count INTEGER DEFAULT 0,
mtime REAL NOT NULL DEFAULT 0,
content_hash TEXT NOT NULL DEFAULT '',
updated_at INTEGER DEFAULT (cast(strftime('%s','now') as int))
);
CREATE INDEX IF NOT EXISTS idx_lc_uuid ON library_cache(uuid);
CREATE INDEX IF NOT EXISTS idx_lc_hash ON library_cache(content_hash);
CREATE INDEX IF NOT EXISTS idx_lc_artist ON library_cache(artist);
CREATE INDEX IF NOT EXISTS idx_lc_album ON library_cache(album);
CREATE TABLE IF NOT EXISTS playcounts_log (
firewire_id TEXT PRIMARY KEY,
last_synced_delta TEXT NOT NULL DEFAULT '{}'
);
"""
# ---------------------------------------------------------------------------
# Fast content fingerprint
# ---------------------------------------------------------------------------
def _content_hash(filepath: str, max_bytes: int = 65536) -> str:
"""SHA1 of first *max_bytes* of audio data."""
h = hashlib.sha1()
try:
with open(filepath, "rb") as f:
h.update(f.read(max_bytes))
except OSError:
return ""
return h.hexdigest()
# ---------------------------------------------------------------------------
# Singleton helpers (kept for backward compatibility)
# ---------------------------------------------------------------------------
_library_cache: "LibraryCache | None" = None
_migrated_flag = False
def get_library_cache() -> "LibraryCache":
global _library_cache
if _library_cache is None:
_library_cache = LibraryCache()
_library_cache._migrate_old()
return _library_cache
def save_library_cache() -> None:
global _library_cache
if _library_cache is not None:
_library_cache.save()
def invalidate_cache_for_path(file_path: str) -> None:
cache = get_library_cache()
cache.remove(file_path)
# ---------------------------------------------------------------------------
# SQLite-backed cache
# ---------------------------------------------------------------------------
class LibraryCache:
"""Persistent cache backed by a single SQLite database.
Public API (unchanged from the JSON version):
get(path) → dict | None
put(path, data) → None
remove(path) → None
clear() → None
save() → None
__len__() → int
New public API:
iter_entries() → Iterator[(path, dict)]
get_by_content_hash(chash) → dict | None
get_by_uuid(uid) → dict | None
update_play_count(path) → int (increment + return)
"""
def __init__(self, db_path: str | None = None):
if db_path is None:
db_path = os.path.join(xdg_data_path(), _DB_FILE)
self._path = db_path
self._lock = threading.RLock()
self.conn = sqlite3.connect(
db_path, check_same_thread=False,
detect_types=sqlite3.PARSE_DECLTYPES,
)
self.conn.row_factory = sqlite3.Row
self.conn.execute("PRAGMA journal_mode=WAL")
self.conn.execute("PRAGMA busy_timeout=5000")
self.conn.execute("PRAGMA synchronous=NORMAL")
self.conn.execute("PRAGMA foreign_keys=OFF")
self._init_db()
def _init_db(self) -> None:
self._migrate_schema()
self.conn.executescript(_SCHEMA)
self.conn.execute(f"PRAGMA user_version = {_SCHEMA_VERSION}")
self.conn.commit()
def _migrate_schema(self) -> None:
cur_ver = self.conn.execute("PRAGMA user_version").fetchone()[0]
if cur_ver >= 2:
return
if cur_ver == 1:
self.conn.execute("DROP TABLE IF EXISTS playcounts_log")
# ── public read API ────────────────────────────────────────────────
def get(self, file_path: str) -> dict[str, Any] | None:
"""Return cached data for *file_path*, or None if missing or stale."""
with self._lock:
row = self.conn.execute(
"SELECT * FROM library_cache WHERE path=?",
(file_path,),
).fetchone()
if row is None:
return None
if not os.path.isfile(file_path):
return None
actual_mtime = os.path.getmtime(file_path)
if actual_mtime != row["mtime"]:
return None
return self._row_to_dict(row)
def get_by_content_hash(self, content_hash: str) -> dict[str, Any] | None:
"""Look up a track by its content fingerprint."""
with self._lock:
row = self.conn.execute(
"SELECT * FROM library_cache WHERE content_hash=?",
(content_hash,),
).fetchone()
if row is None:
return None
return self._row_to_dict(row)
def get_by_uuid(self, track_uuid: str) -> dict[str, Any] | None:
"""Look up a track by its stable UUID."""
with self._lock:
row = self.conn.execute(
"SELECT * FROM library_cache WHERE uuid=?",
(track_uuid,),
).fetchone()
if row is None:
return None
return self._row_to_dict(row)
def iter_entries(self) -> Iterator[tuple[str, dict[str, Any]]]:
"""Yield (file_path, data_dict) for every row in the database."""
with self._lock:
rows = self.conn.execute(
"SELECT * FROM library_cache"
).fetchall()
for row in rows:
yield (row["path"], self._row_to_dict(row))
def __len__(self) -> int:
with self._lock:
return self.conn.execute(
"SELECT COUNT(*) FROM library_cache"
).fetchone()[0]
# ── public write API ───────────────────────────────────────────────
def put(self, file_path: str, data: dict[str, Any]) -> None:
"""Insert or replace a cache entry for *file_path*."""
try:
mtime = os.path.getmtime(file_path)
except OSError:
return
track_uuid = data.get("_track_uuid") or data.get("uuid") or str(uuid.uuid4())
ch = data.get("_content_hash") or data.get("content_hash") or _content_hash(file_path)
with self._lock:
self.conn.execute(
"""INSERT OR REPLACE INTO library_cache
(path, uuid, title, artist, album, album_artist,
genre, duration_s, size, track_num, source_type,
bitrate, sample_rate, cover_path, thumbnail_url,
play_count, rating, last_played, skip_count,
mtime, content_hash, updated_at)
VALUES (?,?,?,?,?,?,
?,?,?,?,?,
?,?,?,?,
?,?,?,?,
?,?,cast(strftime('%s','now') as int))""",
(
file_path,
track_uuid,
data.get("title", ""),
data.get("artist", ""),
data.get("album", ""),
data.get("album_artist", ""),
data.get("genre", ""),
data.get("duration_s", 0) or data.get("duration", 0),
data.get("size", 0),
data.get("track_num", 0) or data.get("track_number", 0),
data.get("source_type", ""),
data.get("bitrate", 0),
data.get("sample_rate", 44100),
data.get("cover_path", ""),
data.get("thumbnail_url", ""),
data.get("play_count", 0),
data.get("rating", 0),
data.get("last_played", 0),
data.get("skip_count", 0),
mtime,
ch,
),
)
self._backup()
def update_play_stats(self, file_path: str,
play_count: int | None = None,
rating: int | None = None,
last_played: int | None = None,
skip_count: int | None = None) -> None:
"""Update only the play-stat fields for *file_path*."""
fields = []
vals: list[Any] = []
for k, v in [("play_count", play_count), ("rating", rating),
("last_played", last_played), ("skip_count", skip_count)]:
if v is not None:
fields.append(f"{k}=?")
vals.append(v)
if not fields:
return
fields.append("updated_at=cast(strftime('%s','now') as int)")
with self._lock:
self.conn.execute(
f"UPDATE library_cache SET {', '.join(fields)} WHERE path=?",
(*vals, file_path),
)
self._backup()
def remove(self, file_path: str) -> None:
with self._lock:
self.conn.execute(
"DELETE FROM library_cache WHERE path=?", (file_path,),
)
self._backup()
def clear(self) -> None:
with self._lock:
self.conn.execute("DELETE FROM library_cache")
self._backup()
def save(self) -> None:
"""Explicit commit + backup."""
with self._lock:
self.conn.commit()
self._backup()
# ── play-counts delta deduplication ──────────────────────────────────
def get_last_synced_delta(self, firewire_id: str) -> dict[str, dict[str, int]]:
"""Return the delta dict that was last committed for this iPod.
Returns an empty dict if no sync has happened yet.
"""
with self._lock:
row = self.conn.execute(
"SELECT last_synced_delta FROM playcounts_log WHERE firewire_id=?",
(firewire_id,),
).fetchone()
if row and row["last_synced_delta"]:
try:
return json.loads(row["last_synced_delta"])
except json.JSONDecodeError:
pass
return {}
def set_last_synced_delta(self, firewire_id: str,
delta: dict[str, dict[str, int]]) -> None:
"""Store the delta that was committed during the last sync."""
blob = json.dumps(delta, sort_keys=True, separators=(",", ":"))
with self._lock:
self.conn.execute(
"INSERT OR REPLACE INTO playcounts_log (firewire_id, last_synced_delta) "
"VALUES (?, ?)",
(firewire_id, blob),
)
self._backup()
# ── backup rotation ────────────────────────────────────────────────
def _backup(self) -> None:
"""Rotate database backup files after each write.
Commits the current transaction first so the backup connection
can checkpoint the WAL without blocking.
Keeps two historical copies:
library_cache.db ← current
library_cache.db.backup ← previous version
library_cache.db.bak ← version before that
"""
# Commit so the second connection can safely checkpoint.
self.conn.commit()
db = self._path
b1 = db + ".backup"
b2 = db + ".bak"
try:
bak_conn = sqlite3.connect(db)
bak_conn.execute("PRAGMA wal_checkpoint(TRUNCATE)")
bak_conn.close()
except Exception as exc:
logger.debug("Checkpoint failed: %s", exc)
if os.path.exists(b1):
try:
os.replace(b1, b2)
except OSError:
pass
try:
import shutil
if os.path.exists(db):
shutil.copy2(db, b1)
except OSError as exc:
logger.debug("Backup rotation failed: %s", exc)
# ── internal helpers ───────────────────────────────────────────────
@staticmethod
def _row_to_dict(row: sqlite3.Row) -> dict[str, Any]:
keys = row.keys()
d = dict(zip(keys, row))
d.pop("updated_at", None)
alias = {
"duration_s": "duration",
"track_num": "track_number",
}
for old_k, new_k in alias.items():
if old_k in d and new_k not in d:
d[new_k] = d[old_k]
return d
# ── JSON migration legacy→SQLite ──────────────────────────────────
def _migrate_old(self) -> None:
"""One-shot migration from the old JSON cache file."""
global _migrated_flag
if _migrated_flag:
return
old_dir = xdg_data_path()
old_path = os.path.join(
os.path.dirname(old_dir), # parent: neo-pod-desktop
".cache", "neo-pod-desktop", _OLD_JSON_FILE,
)
# also try the actual XDG cache dir
from xdg_base import xdg_cache_path
alt_path = os.path.join(xdg_cache_path(), _OLD_JSON_FILE)
for candidate in (old_path, alt_path):
if os.path.exists(candidate):
self._do_migrate_json(candidate)
break
_migrated_flag = True
def _do_migrate_json(self, json_path: str) -> None:
"""Read old JSON and insert all entries."""
try:
with open(json_path, "r", encoding="utf-8") as f:
old_data = json.load(f)
except (json.JSONDecodeError, OSError) as exc:
logger.warning("JSON migration source corrupt: %s", exc)
return
count = 0
for file_path, entry in old_data.items():
if not isinstance(entry, dict):
continue
mtime = entry.get("_mtime", 0)
if not mtime:
try:
mtime = os.path.getmtime(file_path)
except OSError:
continue
entry["_track_uuid"] = entry.get("_track_uuid") or str(uuid.uuid4())
entry["_content_hash"] = entry.get("_content_hash") or _content_hash(file_path)
with self._lock:
self.conn.execute(
"""INSERT OR IGNORE INTO library_cache
(path, uuid, title, artist, album, album_artist,
genre, duration_s, size, track_num, source_type,
bitrate, sample_rate, cover_path, thumbnail_url,
play_count, rating, last_played, skip_count,
mtime, content_hash, updated_at)
VALUES (?,?,?,?,?,?,
?,?,?,?,?,
?,?,?,?,
?,?,?,?,
?,?,cast(strftime('%s','now') as int))""",
(
file_path,
entry.get("_track_uuid", ""),
entry.get("title", ""),
entry.get("artist", ""),
entry.get("album", ""),
entry.get("album_artist", ""),
entry.get("genre", ""),
entry.get("duration_s", 0) or entry.get("duration", 0),
entry.get("size", 0),
entry.get("track_num", 0) or entry.get("track_number", 0),
entry.get("source_type", ""),
entry.get("bitrate", 0),
entry.get("sample_rate", 44100),
entry.get("cover_path", ""),
entry.get("thumbnail_url", ""),
entry.get("play_count", 0),
entry.get("rating", 0),
entry.get("last_played", 0),
entry.get("skip_count", 0),
mtime,
entry.get("_content_hash", ""),
),
)
count += 1
self.conn.commit()
self._backup()
# Rename old file so we don't re-migrate
try:
os.rename(json_path, json_path + ".migrated")
logger.info(
"Migrated %d entries from %s → SQLite",
count, json_path,
)
except OSError:
pass
-1710
View File
File diff suppressed because it is too large Load Diff
+329 -14
View File
@@ -7,7 +7,6 @@ Handles metadata injection and album art embedding
import os
import sys
import logging
import tempfile
import struct
import base64
from typing import Optional, Dict, Any
@@ -18,11 +17,11 @@ import requests
from PIL import Image
from mutagen import File as MutagenFile
from mutagen.mp4 import MP4, MP4Cover
from mutagen.id3 import ID3, APIC, TIT2, TPE1, TALB, TRCK
from mutagen.id3 import ID3, APIC, POPM, TIT2, TPE1, TALB, TRCK
from mutagen.easyid3 import EasyID3
from youtube_downloader import TrackInfo
from artwork.cache import cover_cache
from track_info import TrackInfo
from artwork.cache import cover_cache, xdg_cache_path
# Configure logging
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
@@ -39,7 +38,7 @@ class MetadataHandler:
Args:
temp_dir: Directory for temporary files (optional)
"""
self.temp_dir = temp_dir or tempfile.gettempdir()
self.temp_dir = temp_dir or xdg_cache_path("thumbnails")
os.makedirs(self.temp_dir, exist_ok=True)
def download_thumbnail(self, thumbnail_url: str) -> Optional[str]:
@@ -357,7 +356,7 @@ class MetadataHandler:
cover_path = cover_cache.put(artist or "Unknown Artist", album or "", cover_data)
return TrackInfo(
video_id=f"local_{hash(file_path)}",
source_id=f"local_{hash(file_path)}",
title=title or base_name,
artist=artist or "Unknown Artist",
album=album or "",
@@ -367,6 +366,7 @@ class MetadataHandler:
genre=genre,
download_path=file_path,
cover_path=cover_path,
album_artist=album_artist,
)
except Exception as e:
@@ -534,7 +534,7 @@ class MetadataHandler:
artist, title = filename.split(" - ", 1)
return TrackInfo(
video_id=f"local_{hash(file_path or filename)}",
source_id=f"local_{hash(file_path or filename)}",
title=title,
artist=artist,
album="",
@@ -543,19 +543,334 @@ class MetadataHandler:
download_path=file_path,
)
def enhance_metadata_with_musicbrainz(self, track_info: TrackInfo) -> TrackInfo:
def update_tags(self, file_path: str, tags: Dict[str, Any],
new_cover_path: Optional[str] = None) -> bool:
"""
Enhance track metadata using MusicBrainz API
Update metadata tags in an audio file.
Args:
track_info: TrackInfo object with track metadata
file_path: Path to audio file
tags: Dict of tag_name -> value
Supported keys: title, artist, album, album_artist, genre,
date, track_number, track_total, disc_number, disc_total,
comment, rating (05 stars)
new_cover_path: Path to new cover image, None to keep existing,
empty string '' to remove cover art
Returns:
Enhanced TrackInfo object
True if successful, False otherwise
"""
# This is a placeholder for future implementation
# MusicBrainz integration would go here
return track_info
from mutagen.flac import FLAC, Picture as FLACPicture
from mutagen.oggvorbis import OggVorbis
from mutagen.oggopus import OggOpus
file_ext = Path(file_path).suffix.lower()
try:
if file_ext in ('.m4a', '.aac', '.mp4'):
return self._update_mp4_tags(file_path, tags, new_cover_path)
elif file_ext == '.mp3':
return self._update_mp3_tags(file_path, tags, new_cover_path)
elif file_ext == '.flac':
return self._update_vorbis_tags(file_path, tags, new_cover_path, use_picture_block=True)
elif file_ext in ('.ogg', '.opus'):
return self._update_vorbis_tags(file_path, tags, new_cover_path, use_picture_block=False)
else:
logger.warning(f"Unsupported format for tag update: {file_ext}")
return False
except Exception as e:
logger.error(f"Failed to update tags in {file_path}: {e}")
return False
def _load_cover_data(self, cover_path: str) -> Optional[bytes]:
"""Load cover image from path, convert to JPEG if needed."""
try:
img = Image.open(cover_path)
if img.mode != 'RGB':
img = img.convert('RGB')
buf = BytesIO()
img.save(buf, 'JPEG', quality=90)
return buf.getvalue()
except Exception as e:
logger.error(f"Failed to load cover image {cover_path}: {e}")
return None
def _update_mp4_tags(self, file_path: str, tags: Dict[str, Any],
new_cover_path: Optional[str]) -> bool:
audio = MP4(file_path)
text_fields = {
'title': '\xa9nam',
'artist': '\xa9ART',
'album': '\xa9alb',
'album_artist': 'aART',
'genre': '\xa9gen',
'date': '\xa9day',
'comment': '\xa9cmt',
}
for key, mp4_key in text_fields.items():
if key in tags:
val = tags[key]
if val:
audio[mp4_key] = [str(val)]
elif mp4_key in audio:
del audio[mp4_key]
if 'track_number' in tags or 'track_total' in tags:
trkn_val = list(audio.get('trkn', [(0, 0)])[0]) if 'trkn' in audio else [0, 0]
if 'track_number' in tags:
trkn_val[0] = tags['track_number'] or 0
if 'track_total' in tags:
trkn_val[1] = tags['track_total'] or 0
if trkn_val[0] or trkn_val[1]:
audio['trkn'] = [tuple(trkn_val)]
elif 'trkn' in audio:
del audio['trkn']
if 'disc_number' in tags or 'disc_total' in tags:
disk_val = list(audio.get('disk', [(0, 0)])[0]) if 'disk' in audio else [0, 0]
if 'disc_number' in tags:
disk_val[0] = tags['disc_number'] or 0
if 'disc_total' in tags:
disk_val[1] = tags['disc_total'] or 0
if disk_val[0] or disk_val[1]:
audio['disk'] = [tuple(disk_val)]
elif 'disk' in audio:
del audio['disk']
if 'rating' in tags:
val = tags['rating'] * 20 # stars → 0-100
if val > 0:
audio['\xa9rt'] = [val]
elif '\xa9rt' in audio:
del audio['\xa9rt']
if new_cover_path == '':
if 'covr' in audio:
del audio['covr']
elif new_cover_path is not None:
data = self._load_cover_data(new_cover_path)
if data:
cover = MP4Cover(data, imageformat=MP4Cover.FORMAT_JPEG)
audio['covr'] = [cover]
audio.save()
return True
def _update_mp3_tags(self, file_path: str, tags: Dict[str, Any],
new_cover_path: Optional[str]) -> bool:
try:
audio = EasyID3(file_path)
except Exception:
audio = ID3()
audio.save(file_path)
audio = EasyID3(file_path)
easy_fields = {
'title': 'title',
'artist': 'artist',
'album': 'album',
'album_artist': 'albumartist',
'genre': 'genre',
'date': 'date',
}
for key, easy_key in easy_fields.items():
if key in tags:
val = tags[key]
if val:
audio[easy_key] = str(val)
elif easy_key in audio:
del audio[easy_key]
if 'comment' in tags:
try:
audio['comment'] = str(tags['comment']) if tags['comment'] else ''
except Exception:
pass
if 'track_number' in tags or 'track_total' in tags:
current = audio.get('tracknumber', [''])[0] if 'tracknumber' in audio else ''
tn, tt = self._parse_slash_value(current)
if 'track_number' in tags:
tn = tags['track_number'] or 0
if 'track_total' in tags:
tt = tags['track_total'] or 0
if tn or tt:
audio['tracknumber'] = [f"{tn}/{tt}" if tt else str(tn)]
elif 'tracknumber' in audio:
del audio['tracknumber']
if 'disc_number' in tags or 'disc_total' in tags:
current = audio.get('discnumber', [''])[0] if 'discnumber' in audio else ''
dn, dt = self._parse_slash_value(current)
if 'disc_number' in tags:
dn = tags['disc_number'] or 0
if 'disc_total' in tags:
dt = tags['disc_total'] or 0
if dn or dt:
audio['discnumber'] = [f"{dn}/{dt}" if dt else str(dn)]
elif 'discnumber' in audio:
del audio['discnumber']
if 'rating' in tags:
stars = tags['rating']
popm_val = [0, 64, 128, 192, 224, 255][min(stars, 5)]
audio = ID3(file_path)
audio.delall("POPM")
if popm_val > 0:
audio.add(POPM(email="", rating=popm_val))
audio.save(file_path)
audio = EasyID3(file_path)
audio.save()
if new_cover_path is not None:
audio = ID3(file_path)
if new_cover_path == '':
audio.delall('APIC')
else:
data = self._load_cover_data(new_cover_path)
if data:
audio.delall('APIC')
audio.add(APIC(encoding=3, mime='image/jpeg', type=3,
desc='Cover', data=data))
audio.save()
return True
def _parse_slash_value(self, current: str) -> Tuple[int, int]:
"""Parse '5/12' format into (5, 12)."""
try:
parts = str(current).split('/')
first = int(parts[0]) if parts[0].strip().isdigit() else 0
second = int(parts[1]) if len(parts) > 1 and parts[1].strip().isdigit() else 0
return first, second
except Exception:
return 0, 0
def _update_vorbis_tags(self, file_path: str, tags: Dict[str, Any],
new_cover_path: Optional[str],
use_picture_block: bool = False) -> bool:
from mutagen.flac import FLAC, Picture as FLACPicture
from mutagen.oggvorbis import OggVorbis
from mutagen.oggopus import OggOpus
ext = Path(file_path).suffix.lower()
if ext == '.flac':
audio = FLAC(file_path)
elif ext == '.opus':
audio = OggOpus(file_path)
else:
audio = OggVorbis(file_path)
if audio.tags is None:
audio.add_tags()
vorbis_fields = {
'title': 'TITLE',
'artist': 'ARTIST',
'album': 'ALBUM',
'album_artist': 'ALBUMARTIST',
'genre': 'GENRE',
'date': 'DATE',
'comment': 'COMMENT',
}
for key, vorbis_key in vorbis_fields.items():
if key in tags:
val = tags[key]
if val:
audio.tags[vorbis_key] = str(val)
elif vorbis_key in audio.tags:
try:
del audio.tags[vorbis_key]
except (KeyError, TypeError):
pass
if 'track_number' in tags or 'track_total' in tags:
current = audio.tags.get('TRACKNUMBER', [''])[0] if 'TRACKNUMBER' in audio.tags else ''
tn, tt = self._parse_slash_value(current)
if 'track_number' in tags:
tn = tags['track_number'] or 0
if 'track_total' in tags:
tt = tags['track_total'] or 0
if tn or tt:
audio.tags['TRACKNUMBER'] = f"{tn}/{tt}" if tt else str(tn)
elif 'TRACKNUMBER' in audio.tags:
try:
del audio.tags['TRACKNUMBER']
except (KeyError, TypeError):
pass
if 'disc_number' in tags or 'disc_total' in tags:
current = audio.tags.get('DISCNUMBER', [''])[0] if 'DISCNUMBER' in audio.tags else ''
dn, dt = self._parse_slash_value(current)
if 'disc_number' in tags:
dn = tags['disc_number'] or 0
if 'disc_total' in tags:
dt = tags['disc_total'] or 0
if dn or dt:
audio.tags['DISCNUMBER'] = f"{dn}/{dt}" if dt else str(dn)
elif 'DISCNUMBER' in audio.tags:
try:
del audio.tags['DISCNUMBER']
except (KeyError, TypeError):
pass
if 'rating' in tags:
val = str(tags['rating'] * 20) # stars → "0" … "100"
if int(val) > 0:
audio.tags['RATING'] = val
elif 'RATING' in audio.tags:
try:
del audio.tags['RATING']
except (KeyError, TypeError):
pass
if new_cover_path is not None:
self._clear_cover_vorbis(audio)
if new_cover_path != '':
data = self._load_cover_data(new_cover_path)
if data:
self._embed_cover_vorbis(audio, data)
elif use_picture_block:
pass
else:
pass
audio.save()
return True
def _clear_cover_vorbis(self, audio) -> None:
"""Remove all cover art from Vorbis/FLAC file."""
if audio.tags is None:
return
if 'METADATA_BLOCK_PICTURE' in audio.tags:
del audio.tags['METADATA_BLOCK_PICTURE']
if 'COVERART' in audio.tags:
del audio.tags['COVERART']
if hasattr(audio, 'clear_pictures'):
audio.clear_pictures()
def _embed_cover_vorbis(self, audio, data: bytes) -> None:
"""Embed JPEG cover art into Vorbis/FLAC file."""
from mutagen.flac import Picture as FLACPicture
pic = FLACPicture()
pic.type = 3 # Cover (front)
pic.mime = 'image/jpeg'
pic.desc = 'Cover'
pic.width = 0
pic.height = 0
pic.depth = 24
pic.data = data
pic_data = pic.write()
if hasattr(audio, 'add_picture'):
audio.clear_pictures()
audio.add_picture(pic)
else:
encoded = base64.b64encode(pic_data).decode('ascii')
audio.tags['METADATA_BLOCK_PICTURE'] = encoded
if __name__ == "__main__":
View File
+144
View File
@@ -0,0 +1,144 @@
import os
import json
import uuid
import time
import logging
from typing import List, Dict, Optional
from xdg_base import xdg_data_path
logger = logging.getLogger(__name__)
_PLAYLIST_FILE = os.path.join(xdg_data_path(), "playlists.json")
class Playlist:
def __init__(self, name: str, playlist_id: Optional[str] = None,
track_paths: Optional[List[str]] = None,
created_at: Optional[float] = None,
updated_at: Optional[float] = None):
self.id = playlist_id or str(uuid.uuid4())
self.name = name
self.track_paths = track_paths or []
self.created_at = created_at or time.time()
self.updated_at = updated_at or time.time()
def to_dict(self) -> Dict:
return {
"id": self.id,
"name": self.name,
"track_paths": self.track_paths,
"created_at": self.created_at,
"updated_at": self.updated_at,
}
@classmethod
def from_dict(cls, data: Dict) -> "Playlist":
return cls(
name=data["name"],
playlist_id=data.get("id"),
track_paths=data.get("track_paths", []),
created_at=data.get("created_at"),
updated_at=data.get("updated_at"),
)
class PlaylistManager:
def __init__(self, filepath: str = _PLAYLIST_FILE):
self._filepath = filepath
self._playlists: List[Playlist] = []
self._migrate_old()
self._load()
def _migrate_old(self):
"""Migrate from old ~/.config/ location to ~/.local/share/."""
old = os.path.join(
os.path.expanduser("~"), ".config", "neo-pod-desktop", "playlists.json",
)
if os.path.exists(old) and not os.path.exists(self._filepath):
try:
os.makedirs(os.path.dirname(self._filepath), exist_ok=True)
os.rename(old, self._filepath)
logger.info("Migrated playlists from %s%s", old, self._filepath)
except OSError as exc:
logger.warning("Playlist migration failed: %s", exc)
def _load(self):
try:
os.makedirs(os.path.dirname(self._filepath), exist_ok=True)
if os.path.exists(self._filepath):
with open(self._filepath, "r", encoding="utf-8") as f:
data = json.load(f)
self._playlists = [Playlist.from_dict(p) for p in data]
except Exception as e:
logger.error("Failed to load playlists: %s", e)
self._playlists = []
def save(self):
try:
os.makedirs(os.path.dirname(self._filepath), exist_ok=True)
with open(self._filepath, "w", encoding="utf-8") as f:
json.dump([p.to_dict() for p in self._playlists], f, indent=2)
except Exception as e:
logger.error("Failed to save playlists: %s", e)
def create(self, name: str) -> str:
playlist = Playlist(name=name)
self._playlists.append(playlist)
self.save()
return playlist.id
def rename(self, playlist_id: str, new_name: str) -> bool:
for p in self._playlists:
if p.id == playlist_id:
p.name = new_name
p.updated_at = time.time()
self.save()
return True
return False
def delete(self, playlist_id: str) -> bool:
for i, p in enumerate(self._playlists):
if p.id == playlist_id:
del self._playlists[i]
self.save()
return True
return False
def add_tracks(self, playlist_id: str, paths: List[str]) -> bool:
for p in self._playlists:
if p.id == playlist_id:
existing = set(p.track_paths)
for path in paths:
if path not in existing:
p.track_paths.append(path)
existing.add(path)
p.updated_at = time.time()
self.save()
return True
return False
def remove_tracks(self, playlist_id: str, paths: List[str]) -> bool:
for p in self._playlists:
if p.id == playlist_id:
path_set = set(paths)
p.track_paths = [t for t in p.track_paths if t not in path_set]
p.updated_at = time.time()
self.save()
return True
return False
def reorder(self, playlist_ids: List[str]) -> bool:
id_order = {pid: i for i, pid in enumerate(playlist_ids)}
self._playlists.sort(key=lambda p: id_order.get(p.id, 999))
self.save()
return True
def get_all(self) -> List[Playlist]:
return list(self._playlists)
def get_by_id(self, playlist_id: str) -> Optional[Playlist]:
for p in self._playlists:
if p.id == playlist_id:
return p
return None
View File
+63
View File
@@ -0,0 +1,63 @@
#!/usr/bin/env python3
"""
iTunes Search API service for neo-pod-desktop.
"""
import urllib.parse
from typing import Optional, Dict, Any
import requests
def search_itunes(artist: str, title: str, limit: int = 5) -> Optional[Dict[str, Any]]:
"""
Search iTunes Store for track metadata.
Args:
artist: Artist name
title: Track title
limit: Max results to fetch (default 5)
Returns:
Dict with keys: title, artist, album, album_artist, genre, year,
track_number, track_total, disc_number, disc_total, cover_url.
Returns None on failure or no results.
"""
try:
term = " ".join(p for p in (artist, title) if p)
if not term:
return None
quoted = urllib.parse.quote(term)
url = f"https://itunes.apple.com/search?term={quoted}&entity=song&limit={limit}&country=US"
resp = requests.get(url, timeout=10)
resp.raise_for_status()
data = resp.json()
results = data.get("results", [])
if not results:
return None
r = results[0]
cover_url = r.get("artworkUrl100", "")
if cover_url:
cover_url = cover_url.replace("100x100", "600x600")
year = 0
release_date = r.get("releaseDate", "")
if release_date and len(release_date) >= 4:
try:
year = int(release_date[:4])
except (ValueError, TypeError):
pass
return {
"title": r.get("trackName", ""),
"artist": r.get("artistName", ""),
"album": r.get("collectionName", ""),
"album_artist": r.get("collectionArtistName", ""),
"genre": r.get("primaryGenreName", ""),
"year": year,
"track_number": r.get("trackNumber", 0) or 0,
"track_total": r.get("trackCount", 0) or 0,
"disc_number": r.get("discNumber", 0) or 0,
"disc_total": r.get("discCount", 0) or 0,
"cover_url": cover_url,
}
except Exception:
return None
+88
View File
@@ -0,0 +1,88 @@
#!/usr/bin/env python3
"""
Session state store — persistent playback state.
Stored as JSON in ``~/.local/share/neo-pod-desktop/session.json``.
Replaces the ``[Playback]`` section of config.ini — session state
(user's last track, position, volume, playback mode) is user data,
not configuration.
"""
import json
import logging
import os
from typing import Any
from xdg_base import xdg_data_path
logger = logging.getLogger(__name__)
_SESSION_FILE = "session.json"
class SessionStore:
def __init__(self, path: str | None = None):
self._path = path or os.path.join(xdg_data_path(), _SESSION_FILE)
self._data: dict[str, Any] = {}
self._load()
# ── load / save ────────────────────────────────────────────────────
def _load(self) -> None:
if not os.path.exists(self._path):
return
try:
with open(self._path, "r", encoding="utf-8") as f:
self._data = json.load(f)
except (json.JSONDecodeError, OSError) as exc:
logger.warning("Failed to load session: %s", exc)
self._data = {}
def save(self) -> None:
tmp = self._path + ".tmp"
try:
os.makedirs(os.path.dirname(self._path), exist_ok=True)
with open(tmp, "w", encoding="utf-8") as f:
json.dump(self._data, f, ensure_ascii=False)
os.replace(tmp, self._path)
except OSError as exc:
logger.warning("Failed to save session: %s", exc)
# ── typed accessors ────────────────────────────────────────────────
def get_str(self, key: str, fallback: str = "") -> str:
return str(self._data.get(key, fallback))
def get_int(self, key: str, fallback: int = 0) -> int:
return int(self._data.get(key, fallback))
def get_bool(self, key: str, fallback: bool = False) -> bool:
v = self._data.get(key, fallback)
if isinstance(v, bool):
return v
return str(v).lower() in ("true", "1", "yes")
def set(self, key: str, value: Any) -> None:
self._data[key] = value
# ── migrations ─────────────────────────────────────────────────────
def migrate_from_config(self, config_loader: Any) -> None:
"""Import ``[Playback]`` section from the old config loader."""
keys = [
"last_track_path", "last_position_ms", "last_volume",
"last_playing", "shuffle_active", "repeat_mode",
]
changed = False
for k in keys:
if k not in self._data:
v = config_loader.get("Playback", k, fallback="")
self.set(k, v)
changed = True
if changed:
self.save()
# Wipe old Playback section so it isn't read again.
if config_loader.config.has_section("Playback"):
config_loader.config.remove_section("Playback")
config_loader.save()
logger.info("Migrated playback state from config.ini to session.json")
+28
View File
@@ -0,0 +1,28 @@
#!/usr/bin/env python3
"""
Track Info data class shared across the iPod transfer pipeline.
"""
from typing import Optional
from dataclasses import dataclass
@dataclass
class TrackInfo:
"""Data class to store track information."""
source_id: str
title: str
artist: str
album: str
thumbnail_url: str
duration: int
track_number: Optional[int] = None
playlist_title: Optional[str] = None
download_path: Optional[str] = None
genre: Optional[str] = None
album_artist: Optional[str] = None
cover_path: Optional[str] = None
play_count: int = 0
rating: int = 0
last_played: int = 0
skip_count: int = 0
View File
+1093
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+832
View File
@@ -0,0 +1,832 @@
#!/usr/bin/env python3
"""
Metadata Editor Dialog for neo-pod-desktop
Allows editing tags and cover art of audio files (MP3, M4A, FLAC, OGG, OPUS).
Supports single-track and batch (multi-track) editing.
"""
import os
import base64
import struct
from pathlib import Path
from typing import Optional, Dict, Any, List, Tuple
from io import BytesIO
from PIL import Image
from mutagen import File as MutagenFile
from mutagen.mp4 import MP4
from mutagen.id3 import ID3
from mutagen.flac import FLAC
from mutagen.oggvorbis import OggVorbis
from mutagen.oggopus import OggOpus
from PyQt6.QtWidgets import (
QDialog, QVBoxLayout, QHBoxLayout, QFormLayout,
QPushButton, QLabel, QLineEdit, QSpinBox, QCheckBox,
QFileDialog, QMessageBox, QFrame, QDialogButtonBox,
)
from PyQt6.QtCore import Qt, QSize, QTimer
from PyQt6.QtGui import QPixmap, QDragEnterEvent, QDropEvent
from metadata_handler import MetadataHandler
from artwork.cache import cover_cache
from services.itunes_search import search_itunes
from library_cache import get_library_cache, save_library_cache
from .star_rating import ClickableStarRating
COVER_SIZE = 250
SENTINEL_MULTIPLE = "__MULTIPLE__"
class MetadataEditorDialog(QDialog):
"""Dialog for viewing and editing audio file metadata and cover art."""
def __init__(self, file_paths: List[str], parent=None):
super().__init__(parent)
self.file_paths = file_paths
self._batch_mode = len(file_paths) > 1
self._current_file = file_paths[0] if file_paths else ""
self._original_tags: Dict[str, Any] = {}
self._mixed_fields: set = set()
self._new_cover_path: Optional[str] = None
self._cover_removed = False
self._existing_cover_data: Optional[bytes] = None
title = f"Edit Metadata ({len(file_paths)} files)" if self._batch_mode else "Edit Metadata"
self.setWindowTitle(title)
self.setMinimumSize(620, 440)
self.setModal(True)
self._load_tags()
self._setup_ui()
self._populate_form()
if self._batch_mode:
self._apply_batch_mode()
def _load_tags(self):
"""Load metadata from file(s). In batch mode, finds common values."""
if not self.file_paths:
return
first_tags = self._load_single_file(self.file_paths[0])
self._original_tags = dict(first_tags)
self._existing_cover_data = self._extract_cover_from_path(self.file_paths[0])
self._load_ratings_from_cache()
if self._batch_mode:
all_tags = [first_tags]
all_different = False
for path in self.file_paths[1:]:
tags = self._load_single_file(path, skip_cover=True)
all_tags.append(tags)
if all_different:
continue
for key in list(self._original_tags.keys()):
if tags.get(key) != self._original_tags.get(key):
self._mixed_fields.add(key)
def _load_ratings_from_cache(self):
"""Load rating from LibraryCache for all selected files.
Stores the rating (05 stars) in ``self._original_tags['rating']``
(only when cache has a value; filetag rating takes precedence if
cache is empty). In batch mode, marks ``rating`` as mixed when
values differ.
"""
cache = get_library_cache()
ratings = []
for p in self.file_paths:
entry = cache.get(p)
r = entry.get("rating", 0) if entry else 0
ratings.append(max(0, min(5, round(r / 20))))
cache_val = ratings[0]
if cache_val > 0 or "rating" not in self._original_tags:
self._original_tags["rating"] = cache_val
if self._batch_mode and len(set(ratings)) > 1:
self._mixed_fields.add("rating")
self._ratings = {p: r for p, r in zip(self.file_paths, ratings)}
else:
self._ratings = {}
def _load_single_file(self, file_path: str, skip_cover: bool = False) -> Dict[str, Any]:
"""Load tags from a single file. Returns dict of tag_key -> value."""
result: Dict[str, Any] = {}
try:
audio = MutagenFile(file_path)
if audio is None:
return result
tags = audio.tags
ext = Path(file_path).suffix.lower()
field_map = {
'title': ('title', '\xa9nam'),
'artist': ('artist', '\xa9ART'),
'album': ('album', '\xa9alb'),
'album_artist': ('albumartist', 'aART'),
'genre': ('genre', '\xa9gen'),
'date': ('date', '\xa9day'),
'comment': ('comment', '\xa9cmt'),
}
for our_key, (easy_key, mp4_key) in field_map.items():
val = self._safe_get(tags, easy_key, mp4_key)
if val is not None:
result[our_key] = val
result.update(self._load_track_disc(tags))
result.update(self._load_rating_from_tags(tags, ext))
except Exception:
pass
return result
@staticmethod
def _load_rating_from_tags(tags, ext: str) -> dict:
"""Read rating (05 stars) from audio file tags, if present."""
if tags is None:
return {}
try:
if ext in ('.m4a', '.aac', '.mp4') and '\xa9rt' in tags:
rt = tags['\xa9rt']
if isinstance(rt, list) and rt:
val = int(rt[0]) // 20 # 0-100 → 0-5
return {"rating": max(0, min(5, val))}
elif ext == '.mp3':
for frame in tags.getall('POPM'):
if frame.rating > 0:
pops = [0, 64, 128, 192, 224, 255]
stars = max(i for i, v in enumerate(pops) if v <= frame.rating)
return {"rating": stars}
elif ext in ('.flac', '.ogg', '.opus') and 'RATING' in tags:
raw = tags['RATING']
if isinstance(raw, list):
raw = raw[0]
val = int(str(raw)) // 20
return {"rating": max(0, min(5, val))}
except Exception:
pass
return {}
def _load_track_disc(self, tags) -> Dict[str, Any]:
result = {}
if tags is None:
return result
if 'trkn' in tags:
trkn = tags['trkn']
if isinstance(trkn, list) and trkn:
t = trkn[0]
if isinstance(t, (tuple, list)) and len(t) >= 1:
result['track_number'] = int(t[0]) if t[0] else 0
if len(t) >= 2:
result['track_total'] = int(t[1]) if t[1] else 0
elif 'TRCK' in tags:
tn, tt = self._parse_slash(str(tags['TRCK']))
if tn:
result['track_number'] = tn
if tt:
result['track_total'] = tt
elif hasattr(tags, 'getall'):
try:
frames = tags.getall('TRCK')
if frames:
tn, tt = self._parse_slash(str(frames[0]))
if tn:
result['track_number'] = tn
if tt:
result['track_total'] = tt
except Exception:
pass
for mp4_key, (vorbis_key, num_key, total_key) in (
('disk', ('DISCNUMBER', 'disc_number', 'disc_total')),
):
if mp4_key in tags:
disk_val = tags[mp4_key]
if isinstance(disk_val, list) and disk_val:
d = disk_val[0]
if isinstance(d, (tuple, list)) and len(d) >= 1:
result[num_key] = int(d[0]) if d[0] else 0
if len(d) >= 2:
result[total_key] = int(d[1]) if d[1] else 0
elif vorbis_key in tags:
dn, dt = self._parse_slash(str(tags[vorbis_key]))
if dn:
result[num_key] = dn
if dt:
result[total_key] = dt
elif hasattr(tags, 'getall'):
try:
frames = tags.getall('TPOS')
if frames:
dn, dt = self._parse_slash(str(frames[0]))
if dn:
result[num_key] = dn
if dt:
result[total_key] = dt
except Exception:
pass
return result
def _extract_cover_from_path(self, file_path: str) -> Optional[bytes]:
try:
audio = MutagenFile(file_path)
if audio is None:
return None
return self._extract_cover_raw(audio, audio.tags)
except Exception:
return None
def _safe_get(self, tags, easy_key: str, mp4_key: str) -> Optional[str]:
if tags is None:
return None
try:
if mp4_key in tags:
val = tags[mp4_key]
if isinstance(val, list) and val:
return str(val[0])
return str(val)
except (ValueError, KeyError):
pass
try:
easy_lower = easy_key.lower()
for k in tags.keys():
if k.lower() == easy_lower:
val = tags[k]
if isinstance(val, list) and val:
return str(val[0])
return str(val)
except (ValueError, AttributeError):
pass
id3_map = {
'title': 'TIT2', 'artist': 'TPE1', 'album': 'TALB',
'albumartist': 'TPE2', 'genre': 'TCON', 'date': 'TDRC', 'comment': 'COMM',
}
frame_id = id3_map.get(easy_key.lower())
if frame_id and hasattr(tags, 'getall'):
try:
frames = tags.getall(frame_id)
if frames:
return str(frames[0])
except Exception:
pass
return None
def _parse_slash(self, val: str):
try:
parts = str(val).split('/')
first = int(parts[0]) if parts[0].strip().isdigit() else 0
second = int(parts[1]) if len(parts) > 1 and parts[1].strip().isdigit() else 0
return first, second
except Exception:
return 0, 0
def _extract_cover_raw(self, audio, tags) -> Optional[bytes]:
if hasattr(tags, 'getall') and tags is not None:
try:
for frame in tags.getall('APIC'):
return frame.data
except Exception:
pass
try:
if tags and 'covr' in tags:
cover_list = tags['covr']
if isinstance(cover_list, list) and cover_list:
return bytes(cover_list[0])
except Exception:
pass
try:
if tags and 'metadata_block_picture' in tags:
pic_data = tags['metadata_block_picture']
if isinstance(pic_data, list):
pic_data = pic_data[0]
pic_data = str(pic_data)
decoded = base64.b64decode(pic_data)
pos = 0
pos += 4
mime_len = struct.unpack('>I', decoded[pos:pos + 4])[0]
pos += 4 + mime_len
desc_len = struct.unpack('>I', decoded[pos:pos + 4])[0]
pos += 4 + desc_len
pos += 20
return decoded[pos:]
except Exception:
pass
if hasattr(audio, 'pictures') and audio.pictures:
return audio.pictures[0].data
return None
def _setup_ui(self):
outer = QVBoxLayout(self)
outer.setSpacing(12)
body = QHBoxLayout()
body.setSpacing(16)
cover_layout = QVBoxLayout()
cover_layout.setAlignment(Qt.AlignmentFlag.AlignTop)
self.cover_label = QLabel()
self.cover_label.setFixedSize(COVER_SIZE, COVER_SIZE)
self.cover_label.setAlignment(Qt.AlignmentFlag.AlignCenter)
self.cover_label.setStyleSheet(
"QLabel { border: 2px dashed #888; border-radius: 6px; background: palette(window); }"
)
self.cover_label.setAcceptDrops(True)
self.cover_label.setText("Drag cover\nhere\nor click")
self.cover_label.setToolTip("Click to choose cover image, drag & drop to replace")
self.cover_label.mousePressEvent = self._on_cover_clicked
cover_layout.addWidget(self.cover_label)
self.cover_apply_check = QCheckBox("Apply cover to all selected")
self.cover_apply_check.setVisible(False)
self.cover_apply_check.setChecked(True)
cover_layout.addWidget(self.cover_apply_check)
clear_btn = QPushButton("Remove Cover")
clear_btn.clicked.connect(self._on_clear_cover)
cover_layout.addWidget(clear_btn)
cover_layout.addStretch()
body.addLayout(cover_layout)
form_frame = QFrame()
form_layout = QFormLayout(form_frame)
form_layout.setLabelAlignment(Qt.AlignmentFlag.AlignRight)
form_layout.setFieldGrowthPolicy(
QFormLayout.FieldGrowthPolicy.ExpandingFieldsGrow
)
self.title_edit = QLineEdit()
self.title_edit.setPlaceholderText("Track Title")
self.artist_edit = QLineEdit()
self.artist_edit.setPlaceholderText("Artist")
self.album_edit = QLineEdit()
self.album_edit.setPlaceholderText("Album")
self.album_artist_edit = QLineEdit()
self.album_artist_edit.setPlaceholderText("Album Artist")
self.genre_edit = QLineEdit()
self.genre_edit.setPlaceholderText("Genre")
self.comment_edit = QLineEdit()
self.comment_edit.setPlaceholderText("Comment")
self.year_spin = QSpinBox()
self.year_spin.setRange(0, 2100)
self.year_spin.setSpecialValueText("")
self.track_num_spin = QSpinBox()
self.track_num_spin.setRange(0, 999)
self.track_num_spin.setSpecialValueText("")
self.track_total_spin = QSpinBox()
self.track_total_spin.setRange(0, 999)
self.track_total_spin.setSpecialValueText("")
self.disc_num_spin = QSpinBox()
self.disc_num_spin.setRange(0, 99)
self.disc_num_spin.setSpecialValueText("")
self.disc_total_spin = QSpinBox()
self.disc_total_spin.setRange(0, 99)
self.disc_total_spin.setSpecialValueText("")
self._title_label = QLabel("Title:")
form_layout.addRow(self._title_label, self.title_edit)
self._artist_label = QLabel("Artist:")
form_layout.addRow(self._artist_label, self.artist_edit)
self._lookup_btn = QPushButton("\U0001F50D Lookup in iTunes")
self._lookup_btn.setToolTip("Search iTunes Store for metadata by artist and title")
self._lookup_btn.clicked.connect(self._on_lookup_clicked)
self._lookup_btn.setVisible(not self._batch_mode)
self._lookup_status = QLabel("")
self._lookup_status.setStyleSheet("color: #888; font-size: 11px; font-style: italic;")
self._lookup_status.setVisible(False)
lookup_row = QHBoxLayout()
lookup_row.addWidget(self._lookup_btn)
lookup_row.addWidget(self._lookup_status)
lookup_row.addStretch()
form_layout.addRow(QLabel(""), lookup_row)
self._album_label = QLabel("Album:")
form_layout.addRow(self._album_label, self.album_edit)
self._album_artist_label = QLabel("Album Artist:")
form_layout.addRow(self._album_artist_label, self.album_artist_edit)
self._genre_label = QLabel("Genre:")
form_layout.addRow(self._genre_label, self.genre_edit)
self.rating_widget = ClickableStarRating()
self.rating_widget.rating_changed.connect(self._on_rating_changed)
self._rating_label = QLabel("Rating:")
form_layout.addRow(self._rating_label, self.rating_widget)
self._year_label = QLabel("Year:")
form_layout.addRow(self._year_label, self.year_spin)
self._track_label = QLabel("Track #:")
self._track_row_layout = QHBoxLayout()
self._track_row_layout.addWidget(self.track_num_spin)
self._track_row_layout.addWidget(QLabel("/"))
self._track_row_layout.addWidget(self.track_total_spin)
self._track_row_layout.addStretch()
form_layout.addRow(self._track_label, self._track_row_layout)
self._disc_label = QLabel("Disc #:")
self._disc_row_layout = QHBoxLayout()
self._disc_row_layout.addWidget(self.disc_num_spin)
self._disc_row_layout.addWidget(QLabel("/"))
self._disc_row_layout.addWidget(self.disc_total_spin)
self._disc_row_layout.addStretch()
form_layout.addRow(self._disc_label, self._disc_row_layout)
self._comment_label = QLabel("Comment:")
form_layout.addRow(self._comment_label, self.comment_edit)
body.addWidget(form_frame, stretch=1)
outer.addLayout(body, stretch=1)
self._track_count_label = QLabel("")
self._track_count_label.setVisible(False)
self._track_count_label.setStyleSheet("color: #888; font-size: 11px;")
outer.addWidget(self._track_count_label)
buttons = QDialogButtonBox(
QDialogButtonBox.StandardButton.Save |
QDialogButtonBox.StandardButton.Cancel
)
buttons.accepted.connect(self._on_save)
buttons.rejected.connect(self.reject)
outer.addWidget(buttons)
def _apply_batch_mode(self):
title_text = f"\uD83D\uDCCB Editing {len(self.file_paths)} selected tracks"
self._track_count_label.setText(title_text)
self._track_count_label.setVisible(True)
self._title_label.setText("Title: (hidden in batch)")
self.title_edit.setVisible(False)
for w in (self.track_num_spin, self.track_total_spin):
w.setVisible(False)
self._track_label.setText("Track #: (hidden in batch)")
for widget in self.findChildren(QLabel):
if widget.text() == "/" and widget.parent() in (self._track_row_layout.parent(), None):
pass
self.cover_apply_check.setVisible(True)
self._lookup_btn.setVisible(False)
self._lookup_status.setVisible(False)
for key in self._mixed_fields:
self._set_mixed(key)
if "rating" in self._mixed_fields:
self.rating_widget.set_rating(0)
self._rating_label.setText("Rating: (mixed)")
def _set_mixed(self, key: str):
placeholders = {
'title': "(different values)",
'artist': "(different values)",
'album': "(different values)",
'album_artist': "(different values)",
'genre': "(different values)",
'date': "",
'comment': "(different values)",
'track_number': 0,
'track_total': 0,
'disc_number': 0,
'disc_total': 0,
}
edit_map = {
'title': self.title_edit,
'artist': self.artist_edit,
'album': self.album_edit,
'album_artist': self.album_artist_edit,
'genre': self.genre_edit,
'comment': self.comment_edit,
}
spin_map = {
'date': self.year_spin,
'track_number': self.track_num_spin,
'track_total': self.track_total_spin,
'disc_number': self.disc_num_spin,
'disc_total': self.disc_total_spin,
}
if key in edit_map:
edit_map[key].setPlaceholderText(placeholders.get(key, "(different values)"))
edit_map[key].setText("")
elif key in spin_map:
spin_map[key].setValue(0)
spin_map[key].setSpecialValueText("\u2014")
def _populate_form(self):
t = self._original_tags
self.title_edit.setText(t.get('title', ''))
self.artist_edit.setText(t.get('artist', ''))
self.album_edit.setText(t.get('album', ''))
self.album_artist_edit.setText(t.get('album_artist', ''))
self.genre_edit.setText(t.get('genre', ''))
self.comment_edit.setText(t.get('comment', ''))
date_str = t.get('date', '')
try:
year = int(str(date_str)[:4])
except (ValueError, TypeError):
year = 0
self.year_spin.setValue(year)
self.track_num_spin.setValue(t.get('track_number', 0))
self.track_total_spin.setValue(t.get('track_total', 0))
self.disc_num_spin.setValue(t.get('disc_number', 0))
self.disc_total_spin.setValue(t.get('disc_total', 0))
self.rating_widget.set_rating(t.get("rating", 0))
self._update_cover_display()
def _on_rating_changed(self, stars: int) -> None:
pass # placeholder for future live-preview
def _update_cover_display(self):
if self._cover_removed:
self.cover_label.setText("No cover")
self.cover_label.setPixmap(QPixmap())
return
if self._batch_mode and not self._new_cover_path and self._existing_cover_data is None:
self.cover_label.setText("Multiple\ncovers\n\nClick to replace")
self.cover_label.setPixmap(QPixmap())
return
pixmap = None
if self._new_cover_path and os.path.exists(self._new_cover_path):
pixmap = QPixmap(self._new_cover_path)
elif self._existing_cover_data:
pixmap = QPixmap()
pixmap.loadFromData(self._existing_cover_data)
if pixmap and not pixmap.isNull():
scaled = pixmap.scaled(
COVER_SIZE - 16, COVER_SIZE - 16,
Qt.AspectRatioMode.KeepAspectRatio,
Qt.TransformationMode.SmoothTransformation,
)
self.cover_label.setPixmap(scaled)
else:
self.cover_label.setText("No cover")
self.cover_label.setPixmap(QPixmap())
def _on_cover_clicked(self, event):
path, _ = QFileDialog.getOpenFileName(
self, "Select Cover Image", "",
"Images (*.png *.jpg *.jpeg *.bmp *.gif *.tiff *.webp);;All Files (*)"
)
if path:
self._new_cover_path = path
self._cover_removed = False
self._update_cover_display()
def _on_clear_cover(self):
self._new_cover_path = None
self._cover_removed = True
self._update_cover_display()
def _on_lookup_clicked(self):
artist = self.artist_edit.text().strip()
title = self.title_edit.text().strip()
if not artist and not title:
self._show_lookup_status("Enter artist or title first", is_error=True)
return
self._lookup_btn.setEnabled(False)
self._show_lookup_status("Searching iTunes\u2026")
result = search_itunes(artist=artist, title=title)
self._lookup_btn.setEnabled(True)
if not result:
self._show_lookup_status("No results found", is_error=True)
return
self.title_edit.setText(result.get("title", ""))
self.artist_edit.setText(result.get("artist", ""))
self.album_edit.setText(result.get("album", ""))
self.album_artist_edit.setText(result.get("album_artist", ""))
self.genre_edit.setText(result.get("genre", ""))
year = result.get("year", 0)
if year and year > 0:
self.year_spin.setValue(year)
track_num = result.get("track_number", 0)
if track_num:
self.track_num_spin.setValue(track_num)
track_total = result.get("track_total", 0)
if track_total:
self.track_total_spin.setValue(track_total)
disc_num = result.get("disc_number", 0)
if disc_num:
self.disc_num_spin.setValue(disc_num)
disc_total = result.get("disc_total", 0)
if disc_total:
self.disc_total_spin.setValue(disc_total)
cover_url = result.get("cover_url", "")
if cover_url and not self._existing_cover_data and not self._new_cover_path:
try:
import tempfile
import requests
resp = requests.get(cover_url, timeout=10)
resp.raise_for_status()
fd, tmp_path = tempfile.mkstemp(suffix=".jpg")
os.close(fd)
with open(tmp_path, "wb") as f:
f.write(resp.content)
self._new_cover_path = tmp_path
self._update_cover_display()
except Exception:
pass
self._show_lookup_status("Metadata loaded from iTunes")
def _show_lookup_status(self, text: str, is_error: bool = False):
self._lookup_status.setText(text)
self._lookup_status.setStyleSheet(
"color: #c0392b; font-size: 11px; font-style: italic;"
if is_error else
"color: #27ae60; font-size: 11px; font-style: italic;"
)
self._lookup_status.setVisible(True)
QTimer.singleShot(4000, lambda: self._lookup_status.setVisible(False))
def dragEnterEvent(self, event: QDragEnterEvent):
if event.mimeData().hasUrls():
event.acceptProposedAction()
def dropEvent(self, event: QDropEvent):
urls = event.mimeData().urls()
if urls:
path = urls[0].toLocalFile()
if os.path.isfile(path):
ext = Path(path).suffix.lower()
if ext in ('.png', '.jpg', '.jpeg', '.bmp', '.gif', '.tiff', '.webp'):
self._new_cover_path = path
self._cover_removed = False
self._update_cover_display()
def _collect_current(self) -> Dict[str, Any]:
return {
'title': self.title_edit.text().strip(),
'artist': self.artist_edit.text().strip(),
'album': self.album_edit.text().strip(),
'album_artist': self.album_artist_edit.text().strip(),
'genre': self.genre_edit.text().strip(),
'date': str(self.year_spin.value()) if self.year_spin.value() > 0 else '',
'track_number': self.track_num_spin.value(),
'track_total': self.track_total_spin.value(),
'disc_number': self.disc_num_spin.value(),
'disc_total': self.disc_total_spin.value(),
'comment': self.comment_edit.text().strip(),
'rating': self.rating_widget.rating(),
}
def _collect_changes(self) -> Dict[str, Any]:
if self._batch_mode:
return self._collect_batch_changes()
return self._collect_single_changes()
def _collect_single_changes(self) -> Dict[str, Any]:
changes = {}
current = self._collect_current()
for key, val in current.items():
orig = self._original_tags.get(key)
if isinstance(val, int) and isinstance(orig, int):
if val != orig:
changes[key] = val
elif isinstance(val, int) and isinstance(orig, str):
try:
if int(str(orig)[:4]) != val:
changes[key] = val
except ValueError:
if val:
changes[key] = val
elif isinstance(val, int) and orig is None:
if val != 0:
changes[key] = val
elif str(val) != str(orig or ''):
changes[key] = val
return changes
def _collect_batch_changes(self) -> Dict[str, Any]:
current = self._collect_current()
exclude = {'title', 'track_number', 'track_total'}
changes = {}
for key, val in current.items():
if key in exclude:
continue
orig = self._original_tags.get(key)
if key in self._mixed_fields:
if isinstance(val, int) and val == 0:
continue
if isinstance(val, str) and not val:
continue
changes[key] = val
else:
if isinstance(val, int) and isinstance(orig, int):
if val != orig:
changes[key] = val
elif isinstance(val, str) and orig is not None:
if str(val) != str(orig):
changes[key] = val
elif val and not orig:
changes[key] = val
return changes
def _on_save(self):
changes = self._collect_changes()
cover_arg = None
if self._cover_removed:
cover_arg = ''
elif self._new_cover_path:
cover_arg = self._new_cover_path
if not changes and cover_arg is None:
self.accept()
return
paths = self.file_paths if self._batch_mode else [self.file_paths[0]]
try:
handler = MetadataHandler()
failed = 0
for path in paths:
ok = handler.update_tags(path, changes, cover_arg)
if not ok:
failed += 1
else:
self._invalidate_cover_cache(path, changes, cover_arg)
if not failed:
self._persist_rating(paths, changes)
if failed == len(paths):
QMessageBox.warning(self, "Error", "Failed to save metadata changes.")
elif failed:
QMessageBox.warning(
self, "Partial Success",
f"Saved {len(paths) - failed}/{len(paths)} files.\n{failed} file(s) failed."
)
self.accept()
else:
self.accept()
except Exception as e:
QMessageBox.critical(self, "Error", f"Failed to save metadata:\n{e}")
def _persist_rating(self, paths: list[str], changes: dict[str, Any]) -> None:
if "rating" not in changes:
return
try:
cache = get_library_cache()
for p in paths:
entry = cache.get(p)
if entry is None:
continue
entry["rating"] = changes["rating"] * 20
cache.put(p, entry)
save_library_cache()
except Exception:
logger.exception("Failed to persist rating to LibraryCache")
def _invalidate_cover_cache(self, file_path: str, changes: Dict[str, Any],
cover_arg: Optional[str]) -> None:
"""Update cover cache after successful save."""
try:
audio = MutagenFile(file_path)
new_cover = self._extract_cover_raw(audio, audio.tags) if audio else None
artist = changes.get('artist') or self._original_tags.get('artist', 'Unknown Artist')
album = changes.get('album') or self._original_tags.get('album', '')
if new_cover:
cover_cache.put(artist, album, new_cover)
elif cover_arg == '':
key = cover_cache._key(artist, album)
cache_path = os.path.join(cover_cache.cache_dir, f"{key}.jpg")
if os.path.exists(cache_path):
os.remove(cache_path)
except Exception:
pass
+180
View File
@@ -0,0 +1,180 @@
from PyQt6.QtWidgets import (
QWidget, QHBoxLayout, QVBoxLayout, QPushButton, QLabel, QSlider, QGroupBox, QStyle, QProxyStyle,
)
from PyQt6.QtCore import Qt, pyqtSignal
from PyQt6.QtGui import QPixmap, QColor
def _fmt_time(ms: int) -> str:
total = ms // 1000
m, s = divmod(total, 60)
return f"{m}:{s:02d}"
class JumpStyle(QProxyStyle):
def styleHint(self, hint, opt=None, widget=None, returnData=None):
if hint == QStyle.StyleHint.SH_Slider_AbsoluteSetButtons:
return Qt.MouseButton.LeftButton.value
return super().styleHint(hint, opt, widget, returnData)
class PlayerHeader(QGroupBox):
prev_requested = pyqtSignal()
play_pause_requested = pyqtSignal()
next_requested = pyqtSignal()
shuffle_requested = pyqtSignal()
repeat_requested = pyqtSignal()
position_changed_by_user = pyqtSignal(int)
volume_changed = pyqtSignal(int)
def __init__(self, parent=None):
super().__init__(parent)
self.setFlat(True)
self._setup_ui()
def _setup_ui(self):
layout = QVBoxLayout(self)
layout.setContentsMargins(8, 6, 8, 6)
controls_row = QHBoxLayout()
controls_row.setSpacing(8)
self.prev_btn = QPushButton("\u23EE")
self.prev_btn.setFixedWidth(36)
self.prev_btn.setToolTip("Previous")
self.prev_btn.clicked.connect(self.prev_requested)
self.play_btn = QPushButton("\u25B6")
self.play_btn.setFixedWidth(44)
self.play_btn.setToolTip("Play / Pause")
self.play_btn.clicked.connect(self.play_pause_requested)
self.next_btn = QPushButton("\u23ED")
self.next_btn.setFixedWidth(36)
self.next_btn.setToolTip("Next")
self.next_btn.clicked.connect(self.next_requested)
self.shuffle_btn = QPushButton("\u21C4")
self.shuffle_btn.setFixedWidth(36)
self.shuffle_btn.setToolTip("Shuffle")
self.shuffle_btn.clicked.connect(self.shuffle_requested)
self.repeat_btn = QPushButton("\u21BB")
self.repeat_btn.setFixedWidth(36)
self.repeat_btn.setToolTip("Repeat: Off")
self.repeat_btn.clicked.connect(self.repeat_requested)
controls_row.addWidget(self.shuffle_btn)
controls_row.addWidget(self.prev_btn)
controls_row.addWidget(self.play_btn)
controls_row.addWidget(self.next_btn)
controls_row.addWidget(self.repeat_btn)
self.time_label_start = QLabel("0:00")
self.time_label_start.setFixedWidth(40)
self.time_label_start.setAlignment(Qt.AlignmentFlag.AlignRight | Qt.AlignmentFlag.AlignVCenter)
self.position_slider = QSlider(Qt.Orientation.Horizontal)
self.position_slider.setStyle(JumpStyle(self.position_slider.style()))
self.position_slider.setMinimum(0)
self.position_slider.setMaximum(1000)
self.position_slider.setValue(0)
self.position_slider.sliderPressed.connect(self._on_slider_user_action)
self.position_slider.sliderMoved.connect(self._on_slider_user_action)
self.position_slider.sliderReleased.connect(self._on_slider_user_action)
self.time_label_end = QLabel("0:00")
self.time_label_end.setFixedWidth(40)
self.time_label_end.setAlignment(Qt.AlignmentFlag.AlignLeft | Qt.AlignmentFlag.AlignVCenter)
controls_row.addWidget(self.time_label_start)
controls_row.addWidget(self.position_slider, stretch=1)
controls_row.addWidget(self.time_label_end)
vol_icon = QLabel("\U0001F50A")
self.volume_slider = QSlider(Qt.Orientation.Horizontal)
self.volume_slider.setFixedWidth(80)
self.volume_slider.setRange(0, 100)
self.volume_slider.setValue(80)
self.volume_slider.valueChanged.connect(self.volume_changed)
controls_row.addWidget(vol_icon)
controls_row.addWidget(self.volume_slider)
layout.addLayout(controls_row)
now_playing_layout = QHBoxLayout()
now_playing_layout.setContentsMargins(6, 0, 0, 0)
self.cover_label = QLabel()
self.cover_label.setFixedSize(60, 60)
self.cover_label.setScaledContents(True)
self.cover_label.setStyleSheet("background: palette(window); border-radius: 6px;")
now_playing_layout.addWidget(self.cover_label)
self.now_playing_label = QLabel("Select a track to play")
self.now_playing_label.setAlignment(Qt.AlignmentFlag.AlignLeft | Qt.AlignmentFlag.AlignVCenter)
self.now_playing_label.setStyleSheet("color: palette(mid); font-size: 12px;")
now_playing_layout.addWidget(self.now_playing_label, stretch=1)
layout.addLayout(now_playing_layout)
def set_shuffle_active(self, active: bool):
if active:
self.shuffle_btn.setStyleSheet("color: palette(highlight); font-weight: bold;")
self.shuffle_btn.setToolTip("Shuffle: On")
else:
self.shuffle_btn.setStyleSheet("")
self.shuffle_btn.setToolTip("Shuffle: Off")
def set_repeat_mode(self, mode: str):
if mode == "repeat_all":
self.repeat_btn.setText("\u21BB")
self.repeat_btn.setStyleSheet("color: palette(highlight); font-weight: bold;")
self.repeat_btn.setToolTip("Repeat: All")
elif mode == "repeat_one":
self.repeat_btn.setText("\u21BB\u00B9")
self.repeat_btn.setStyleSheet("color: palette(highlight); font-weight: bold;")
self.repeat_btn.setToolTip("Repeat: One")
else:
self.repeat_btn.setText("\u21BB")
self.repeat_btn.setStyleSheet("")
self.repeat_btn.setToolTip("Repeat: Off")
def _on_slider_user_action(self):
self.position_changed_by_user.emit(self.position_slider.value())
def set_cover(self, pixmap: QPixmap = None, tooltip: str = ""):
if pixmap:
self.cover_label.setPixmap(pixmap)
else:
self.cover_label.clear()
self.cover_label.setToolTip(tooltip)
def set_track_info(self, text: str):
self.now_playing_label.setText(text)
self.now_playing_label.setStyleSheet(
"color: palette(text); font-weight: bold;" if text else "color: palette(mid); font-size: 12px;"
)
def set_playing(self, playing: bool):
self.play_btn.setText("\u23F8" if playing else "\u25B6")
def set_position(self, position_ms: int, duration_ms: int):
if not self.position_slider.isSliderDown():
if duration_ms > 0:
self.position_slider.setValue(int(position_ms / duration_ms * 1000))
else:
self.position_slider.setValue(0)
self.time_label_start.setText(_fmt_time(position_ms))
self.time_label_end.setText(_fmt_time(duration_ms))
def set_volume(self, value: int):
self.volume_slider.setValue(value)
def reset_info(self):
self.cover_label.clear()
self.cover_label.setToolTip("")
self.now_playing_label.setText("Select a track to play")
self.now_playing_label.setStyleSheet("color: palette(mid); font-size: 12px;")
self.time_label_start.setText("0:00")
self.time_label_end.setText("0:00")
self.position_slider.setValue(0)
self.play_btn.setText("\u25B6")
+204
View File
@@ -0,0 +1,204 @@
#!/usr/bin/env python3
"""
Settings Tab for neo-pod-desktop.
"""
import os
from typing import Optional
from PyQt6.QtWidgets import (
QWidget, QVBoxLayout, QHBoxLayout, QPushButton, QLabel, QLineEdit,
QComboBox, QCheckBox, QSpinBox, QGroupBox, QHeaderView,
QTableWidget, QTableWidgetItem, QFileDialog, QMessageBox,
)
from PyQt6.QtCore import Qt, pyqtSignal
from hotkeys import HotkeyManager, KeyCaptureDialog
class SettingsTab(QWidget):
"""Settings tab widget."""
hide_library_buttons_changed = pyqtSignal(bool)
auto_detect_changed = pyqtSignal(bool)
def __init__(self, parent=None):
super().__init__(parent)
self.hotkey_manager: Optional[HotkeyManager] = None
self._setup_ui()
def set_hotkey_manager(self, hm: HotkeyManager):
self.hotkey_manager = hm
def _setup_ui(self):
layout = QVBoxLayout(self)
library_group = QGroupBox("Library")
library_layout = QVBoxLayout(library_group)
output_layout = QHBoxLayout()
output_label = QLabel("Output Directory:")
self.output_dir_input = QLineEdit()
self.output_dir_input.setText(os.path.join(os.path.expanduser("~"), "Music", "iPod"))
browse_btn = QPushButton("Browse...")
browse_btn.clicked.connect(self._on_browse_output_clicked)
output_layout.addWidget(output_label)
output_layout.addWidget(self.output_dir_input)
output_layout.addWidget(browse_btn)
library_layout.addLayout(output_layout)
convert_layout = QHBoxLayout()
convert_format_label = QLabel("Convert Format:")
self.format_combo = QComboBox()
self.format_combo.addItems(["mp3", "m4a"])
convert_quality_label = QLabel("Quality (kbps):")
self.quality_spin = QSpinBox()
self.quality_spin.setRange(128, 320)
self.quality_spin.setValue(256)
self.quality_spin.setSingleStep(32)
convert_layout.addWidget(convert_format_label)
convert_layout.addWidget(self.format_combo)
convert_layout.addWidget(convert_quality_label)
convert_layout.addWidget(self.quality_spin)
convert_layout.addStretch()
library_layout.addLayout(convert_layout)
layout.addWidget(library_group)
metadata_group = QGroupBox("Metadata Options")
metadata_layout = QVBoxLayout(metadata_group)
self.embed_artwork_check = QCheckBox("Embed Album Artwork")
self.embed_artwork_check.setChecked(True)
metadata_layout.addWidget(self.embed_artwork_check)
layout.addWidget(metadata_group)
advanced_group = QGroupBox("Advanced Options")
advanced_layout = QVBoxLayout(advanced_group)
self.clean_temp_check = QCheckBox("Clean Temporary Files After Transfer")
self.clean_temp_check.setChecked(True)
advanced_layout.addWidget(self.clean_temp_check)
self.auto_detect_check = QCheckBox("Auto-Detect iPod")
self.auto_detect_check.setChecked(True)
self.auto_detect_check.toggled.connect(self.auto_detect_changed.emit)
advanced_layout.addWidget(self.auto_detect_check)
self.hide_library_buttons_check = QCheckBox("Hide Library Toolbar Buttons")
self.hide_library_buttons_check.setChecked(False)
self.hide_library_buttons_check.toggled.connect(self._on_hide_buttons_toggled)
advanced_layout.addWidget(self.hide_library_buttons_check)
layout.addWidget(advanced_group)
shortcuts_group = QGroupBox("Keyboard Shortcuts")
shortcuts_layout = QVBoxLayout(shortcuts_group)
self.shortcuts_table = QTableWidget()
self.shortcuts_table.setColumnCount(2)
self.shortcuts_table.setHorizontalHeaderLabels(["Action", "Shortcut"])
self.shortcuts_table.setSelectionBehavior(QTableWidget.SelectionBehavior.SelectRows)
self.shortcuts_table.setSelectionMode(QTableWidget.SelectionMode.SingleSelection)
self.shortcuts_table.setEditTriggers(QTableWidget.EditTrigger.NoEditTriggers)
self.shortcuts_table.setAlternatingRowColors(True)
self.shortcuts_table.cellDoubleClicked.connect(self._on_shortcut_double_clicked)
shortcuts_header = self.shortcuts_table.horizontalHeader()
shortcuts_header.setSectionResizeMode(0, QHeaderView.ResizeMode.Stretch)
shortcuts_header.setSectionResizeMode(1, QHeaderView.ResizeMode.ResizeToContents)
shortcuts_layout.addWidget(self.shortcuts_table)
reset_shortcuts_btn = QPushButton("Reset to Defaults")
reset_shortcuts_btn.clicked.connect(self._on_reset_shortcuts_clicked)
shortcuts_layout.addWidget(reset_shortcuts_btn)
layout.addWidget(shortcuts_group)
layout.addStretch()
about_label = QLabel(
"neo-pod-desktop\n"
"Version 1.0.0\n\n"
"Desktop application for downloading, converting, managing and transferring music to iPod Nano devices."
)
about_label.setAlignment(Qt.AlignmentFlag.AlignCenter)
layout.addWidget(about_label)
def _on_hide_buttons_toggled(self, checked: bool):
self.hide_library_buttons_changed.emit(checked)
def _on_browse_output_clicked(self):
directory = QFileDialog.getExistingDirectory(
self, "Select Output Directory",
self.output_dir_input.text()
)
if directory:
self.output_dir_input.setText(directory)
def load_settings(self, config):
self.output_dir_input.setText(
config.get("General", "output_dir",
fallback=os.path.join(os.path.expanduser("~"), "Music", "iPod"))
)
self.format_combo.setCurrentText(config.get("General", "format", fallback="m4a"))
self.quality_spin.setValue(config.get_int("General", "quality", fallback=256))
self.embed_artwork_check.setChecked(
config.get_boolean("Metadata", "embed_artwork", fallback=True)
)
self.clean_temp_check.setChecked(
config.get_boolean("Advanced", "clean_temp", fallback=True)
)
self.auto_detect_check.setChecked(
config.get_boolean("Device", "auto_detect", fallback=True)
)
hide_library_buttons = config.get_boolean("Advanced", "hide_library_buttons", fallback=False)
self.hide_library_buttons_check.setChecked(hide_library_buttons)
def save_settings(self, config):
config.set("General", "output_dir", self.output_dir_input.text())
config.set("General", "format", self.format_combo.currentText())
config.set("General", "quality", str(self.quality_spin.value()))
config.set("Metadata", "embed_artwork", str(self.embed_artwork_check.isChecked()).lower())
config.set("Advanced", "clean_temp", str(self.clean_temp_check.isChecked()).lower())
config.set("Device", "auto_detect", str(self.auto_detect_check.isChecked()).lower())
config.set("Advanced", "hide_library_buttons", str(self.hide_library_buttons_check.isChecked()).lower())
def refresh_shortcuts_table(self):
if self.hotkey_manager is None:
return
names = self.hotkey_manager.all_action_names()
self.shortcuts_table.setRowCount(len(names))
for row, name in enumerate(names):
label = self.hotkey_manager.get_label(name)
key = self.hotkey_manager.get_current_key_string(name)
self.shortcuts_table.setItem(row, 0, QTableWidgetItem(label))
item = QTableWidgetItem(key if key else "\u2014")
self.shortcuts_table.setItem(row, 1, item)
def _on_shortcut_double_clicked(self, row: int, _col: int):
if self.hotkey_manager is None:
return
names = self.hotkey_manager.all_action_names()
if row < 0 or row >= len(names):
return
action_name = names[row]
current = self.hotkey_manager.get_current_key_string(action_name)
dialog = KeyCaptureDialog(self, current=current)
if dialog.exec() == KeyCaptureDialog.DialogCode.Accepted:
seq = dialog.captured_sequence()
self.hotkey_manager.set_shortcut_seq(action_name, seq)
self.refresh_shortcuts_table()
def _on_reset_shortcuts_clicked(self):
if self.hotkey_manager is None:
return
reply = QMessageBox.question(
self, "Reset Shortcuts",
"Reset all keyboard shortcuts to their default values?",
QMessageBox.StandardButton.Yes | QMessageBox.StandardButton.No
)
if reply == QMessageBox.StandardButton.Yes:
self.hotkey_manager.reset_to_defaults()
self.refresh_shortcuts_table()
QMessageBox.information(self, "Done", "Shortcuts reset to defaults.")
+500
View File
@@ -0,0 +1,500 @@
import logging
import os
from typing import List, Optional
from PyQt6.QtWidgets import (
QWidget, QVBoxLayout, QLabel, QPushButton,
QListWidget, QListWidgetItem, QMenu, QInputDialog, QMessageBox,
QFrame,
)
from PyQt6.QtCore import Qt, pyqtSignal, QUrl
from PyQt6.QtGui import QColor
from PyQt6.QtGui import QDragEnterEvent, QDragMoveEvent, QDragLeaveEvent, QDropEvent
logger = logging.getLogger(__name__)
SIDEBAR_MIN_WIDTH = 48
SIDEBAR_MAX_WIDTH = 240
SIDEBAR_DEFAULT_WIDTH = 200
class PlaylistDropListWidget(QListWidget):
"""QListWidget that accepts external track drops onto playlist items."""
tracks_dropped_on_playlist = pyqtSignal(str, list)
def __init__(self, parent=None):
super().__init__(parent)
self.setAcceptDrops(True)
self._hovered_item = None
def mimeData(self, items):
mime = super().mimeData(items)
if items:
playlist_id = items[0].data(Qt.ItemDataRole.UserRole)
if playlist_id:
mime.setData("application/x-neopod-playlist", playlist_id.encode())
return mime
def dragEnterEvent(self, event: QDragEnterEvent):
if event.source() is not self and event.mimeData().hasUrls():
event.acceptProposedAction()
else:
super().dragEnterEvent(event)
def dragMoveEvent(self, event: QDragMoveEvent):
if event.source() is not self and event.mimeData().hasUrls():
pos = event.position().toPoint()
item = self.itemAt(pos)
if item != self._hovered_item:
self._clear_hover()
if item:
self._hovered_item = item
bg = self.palette().color(self.palette().ColorRole.Highlight)
c = bg.name()
item.setBackground(bg)
event.acceptProposedAction()
else:
super().dragMoveEvent(event)
def dragLeaveEvent(self, event: QDragLeaveEvent):
self._clear_hover()
super().dragLeaveEvent(event)
def dropEvent(self, event: QDropEvent):
self._clear_hover()
if event.source() is not self and event.mimeData().hasUrls():
pos = event.position().toPoint()
item = self.itemAt(pos)
if item is None:
event.ignore()
return
playlist_id = item.data(Qt.ItemDataRole.UserRole)
if not playlist_id:
event.ignore()
return
paths = [url.toLocalFile() for url in event.mimeData().urls()]
paths = [p for p in paths if os.path.exists(p)]
if paths:
self.tracks_dropped_on_playlist.emit(playlist_id, paths)
event.acceptProposedAction()
else:
event.ignore()
else:
super().dropEvent(event)
def _clear_hover(self):
if self._hovered_item:
self._hovered_item.setData(Qt.ItemDataRole.BackgroundRole, None)
self._hovered_item = None
class NavDropListWidget(QListWidget):
"""QListWidget that accepts external track drops on the iPod nav item."""
tracks_dropped_for_transfer = pyqtSignal(list)
playlist_dropped_for_transfer = pyqtSignal(str)
def __init__(self, parent=None):
super().__init__(parent)
self.setAcceptDrops(True)
self._hovered_item = None
def _target_item_at(self, pos):
item = self.itemAt(pos)
if item and item.data(Qt.ItemDataRole.UserRole + 2) == "ipod":
return item
return None
def _repolish(self):
self.style().unpolish(self)
self.style().polish(self)
def dragEnterEvent(self, event: QDragEnterEvent):
if event.mimeData().hasFormat("application/x-neopod-playlist"):
self.setProperty("dndActive", True)
self._repolish()
pos = event.position().toPoint()
if self._target_item_at(pos):
event.acceptProposedAction()
return
# Let default handler accept so dragMoveEvent still fires
if event.source() is not self and event.mimeData().hasUrls():
pos = event.position().toPoint()
if self._target_item_at(pos):
event.acceptProposedAction()
return
super().dragEnterEvent(event)
def dragMoveEvent(self, event: QDragMoveEvent):
accept = False
if event.source() is not self and event.mimeData().hasUrls():
pos = event.position().toPoint()
item = self._target_item_at(pos)
if item != self._hovered_item:
self._clear_hover()
if item:
self._hovered_item = item
bg = self.palette().color(self.palette().ColorRole.Highlight)
item.setBackground(bg)
if item:
event.acceptProposedAction()
accept = True
if not accept and event.mimeData().hasFormat("application/x-neopod-playlist"):
pos = event.position().toPoint()
item = self._target_item_at(pos)
if item != self._hovered_item:
self._clear_hover()
if item:
self._hovered_item = item
bg = self.palette().color(self.palette().ColorRole.Highlight)
item.setBackground(bg)
if item:
event.acceptProposedAction()
accept = True
if not accept:
event.ignore()
def dragLeaveEvent(self, event: QDragLeaveEvent):
self._clear_hover()
if self.property("dndActive"):
self.setProperty("dndActive", False)
self._repolish()
super().dragLeaveEvent(event)
def dropEvent(self, event: QDropEvent):
self._clear_hover()
if self.property("dndActive"):
self.setProperty("dndActive", False)
self._repolish()
if event.source() is not self and event.mimeData().hasUrls():
pos = event.position().toPoint()
if self._target_item_at(pos):
paths = [url.toLocalFile() for url in event.mimeData().urls()]
paths = [p for p in paths if os.path.exists(p)]
if paths:
self.tracks_dropped_for_transfer.emit(paths)
event.acceptProposedAction()
return
if event.mimeData().hasFormat("application/x-neopod-playlist"):
pos = event.position().toPoint()
if self._target_item_at(pos):
playlist_id = bytes(event.mimeData().data("application/x-neopod-playlist")).decode()
if playlist_id:
self.playlist_dropped_for_transfer.emit(playlist_id)
event.acceptProposedAction()
return
super().dropEvent(event)
def _clear_hover(self):
if self._hovered_item:
self._hovered_item.setData(Qt.ItemDataRole.BackgroundRole, None)
self._hovered_item = None
class SidebarWidget(QWidget):
item_selected = pyqtSignal(str, str)
playlist_selected = pyqtSignal(str)
tracks_dropped_on_playlist = pyqtSignal(str, list)
tracks_dropped_for_nav_transfer = pyqtSignal(list)
playlist_dropped_for_nav_transfer = pyqtSignal(str)
def __init__(self, parent=None):
super().__init__(parent)
self.setObjectName("sidebar")
self.setMinimumWidth(SIDEBAR_MIN_WIDTH)
self.setMaximumWidth(SIDEBAR_MAX_WIDTH)
self.setFixedWidth(SIDEBAR_DEFAULT_WIDTH)
self.setAutoFillBackground(True)
p = self.palette()
self._bg = p.color(p.ColorRole.Window)
self._text = p.color(p.ColorRole.WindowText)
self._dim_text = p.color(p.ColorRole.PlaceholderText)
if not self._dim_text.isValid():
self._dim_text = self._text.darker(160)
self._border = p.color(p.ColorRole.Midlight)
if not self._border.isValid():
self._border = self._bg.darker(120)
hl = p.color(p.ColorRole.Highlight)
hl_light = QColor(hl)
hl_light.setAlpha(60)
hl_text = p.color(p.ColorRole.HighlightedText)
self._apply_palette_style(hl, hl_light, hl_text)
self._build_ui()
def _apply_palette_style(self, hl: QColor, hl_light: QColor, hl_text: QColor):
bg_name = self._bg.name()
border_name = self._border.name()
dim_name = self._dim_text.name()
text_name = self._text.name()
hl_name = hl.name()
hl_light_name = hl_light.name()
hl_text_name = hl_text.name()
self.setStyleSheet(f"""
QWidget#sidebar {{
background: {bg_name};
border-right: 1px solid {border_name};
}}
QListWidget {{
background: transparent;
border: none;
outline: none;
font-size: 13px;
color: {text_name};
}}
QListWidget::item {{
padding: 4px 12px;
border-radius: 4px;
}}
QListWidget::item:selected {{
background: {hl_name};
color: {hl_text_name};
}}
QListWidget::item:hover:!selected {{
background: {hl_light_name};
}}
#navList[dndActive="true"]::item:hover:!selected {{
background: transparent;
}}
""")
def _build_ui(self):
layout = QVBoxLayout(self)
layout.setContentsMargins(0, 8, 0, 8)
layout.setSpacing(0)
dim_color = self._dim_text.name()
section_style = f"""
color: {dim_color};
font-size: 11px;
font-weight: bold;
letter-spacing: 1px;
padding: 8px 12px 4px 12px;
background: transparent;
"""
header = QLabel("LIBRARY")
header.setStyleSheet(section_style)
layout.addWidget(header)
self._library_list = QListWidget()
items = [
("\U0001F4BF Music", "tab", "library"),
("\u23F1 Recently Added", "filter", "recent"),
("\U0001F3A4 Artists", "filter", "artists"),
("\U0001F4BF Albums", "filter", "albums"),
("\U0001F3B5 Songs", "filter", "songs"),
]
for label, kind, value in items:
item = QListWidgetItem(label)
item.setData(Qt.ItemDataRole.UserRole + 1, kind)
item.setData(Qt.ItemDataRole.UserRole + 2, value)
self._library_list.addItem(item)
self._library_list.itemClicked.connect(self._on_library_item_clicked)
self._library_list.setCurrentRow(0)
layout.addWidget(self._library_list)
line = QFrame()
line.setFrameShape(QFrame.Shape.HLine)
line.setStyleSheet(f"color: {self._border.name()}; margin: 4px 12px;")
line.setFixedHeight(1)
layout.addWidget(line)
self._playlists_header = QLabel("PLAYLISTS")
self._playlists_header.setStyleSheet(section_style)
layout.addWidget(self._playlists_header)
self._playlist_list = PlaylistDropListWidget()
self._playlist_list.setDragDropMode(QListWidget.DragDropMode.InternalMove)
self._playlist_list.setDefaultDropAction(Qt.DropAction.MoveAction)
self._playlist_list.setContextMenuPolicy(Qt.ContextMenuPolicy.CustomContextMenu)
self._playlist_list.customContextMenuRequested.connect(
self._on_playlist_context_menu
)
self._playlist_list.itemClicked.connect(self._on_playlist_clicked)
self._playlist_list.model().rowsMoved.connect(self._on_playlists_reordered)
self._playlist_list.tracks_dropped_on_playlist.connect(self._on_tracks_dropped_on_playlist)
layout.addWidget(self._playlist_list, stretch=1)
btn_style = f"""
QPushButton {{
text-align: left;
padding: 6px 12px;
border: none;
color: {dim_color};
font-size: 12px;
}}
QPushButton:hover {{
color: {self._text.name()};
background: {self._border.name()};
}}
"""
self._new_playlist_btn = QPushButton(" \u2795 New Playlist")
self._new_playlist_btn.setFlat(True)
self._new_playlist_btn.setCursor(Qt.CursorShape.PointingHandCursor)
self._new_playlist_btn.setStyleSheet(btn_style)
self._new_playlist_btn.clicked.connect(self._on_new_playlist)
layout.addWidget(self._new_playlist_btn)
line2 = QFrame()
line2.setFrameShape(QFrame.Shape.HLine)
line2.setStyleSheet(f"color: {self._border.name()}; margin: 4px 12px;")
line2.setFixedHeight(1)
layout.addWidget(line2)
self._nav_list = NavDropListWidget()
self._nav_list.setObjectName("navList")
items = [
("🎸 iPod", "ipod"),
("⚙️ Settings", "settings"),
]
for label, tab_id in items:
item = QListWidgetItem(label)
item.setData(Qt.ItemDataRole.UserRole + 1, "tab")
item.setData(Qt.ItemDataRole.UserRole + 2, tab_id)
self._nav_list.addItem(item)
self._nav_list.itemClicked.connect(self._on_nav_item_clicked)
self._nav_list.tracks_dropped_for_transfer.connect(self._on_tracks_dropped_for_nav_transfer)
self._nav_list.playlist_dropped_for_transfer.connect(self._on_playlist_dropped_for_transfer)
layout.addWidget(self._nav_list)
self._playlist_items: dict = {}
def _on_library_item_clicked(self, item: QListWidgetItem):
kind = item.data(Qt.ItemDataRole.UserRole + 1)
value = item.data(Qt.ItemDataRole.UserRole + 2)
self._library_list.setCurrentItem(item)
self._playlist_list.clearSelection()
self._nav_list.clearSelection()
self.item_selected.emit(kind, value)
def _on_playlist_clicked(self, item: QListWidgetItem):
playlist_id = item.data(Qt.ItemDataRole.UserRole)
self._library_list.clearSelection()
self._playlist_list.setCurrentItem(item)
self._nav_list.clearSelection()
self.playlist_selected.emit(playlist_id)
def _on_playlist_context_menu(self, pos):
item = self._playlist_list.itemAt(pos)
if item is None:
return
menu = QMenu(self)
rename_action = menu.addAction("Rename")
delete_action = menu.addAction("Delete")
action = menu.exec(self._playlist_list.viewport().mapToGlobal(pos))
playlist_id = item.data(Qt.ItemDataRole.UserRole)
if action == rename_action:
self._rename_playlist(item, playlist_id)
elif action == delete_action:
self._delete_playlist(item, playlist_id)
def _rename_playlist(self, item: QListWidgetItem, playlist_id: str):
current = item.text()
name, ok = QInputDialog.getText(
self, "Rename Playlist", "New name:", text=current
)
if ok and name.strip():
item.setText(name.strip())
self.item_selected.emit("playlist_rename", playlist_id + "|" + name.strip())
def _delete_playlist(self, item: QListWidgetItem, playlist_id: str):
reply = QMessageBox.question(
self, "Delete Playlist",
f'Delete playlist "{item.text()}"?\nThis cannot be undone.',
QMessageBox.StandardButton.Yes | QMessageBox.StandardButton.No,
)
if reply == QMessageBox.StandardButton.Yes:
row = self._playlist_list.row(item)
self._playlist_list.takeItem(row)
self.item_selected.emit("playlist_delete", playlist_id)
def _on_new_playlist(self):
name, ok = QInputDialog.getText(self, "New Playlist", "Playlist name:")
if ok and name.strip():
self.item_selected.emit("playlist_create", name.strip())
def add_playlist_item(self, playlist_id: str, name: str):
item = QListWidgetItem(f"\U0001F4CB {name}")
item.setData(Qt.ItemDataRole.UserRole, playlist_id)
self._playlist_list.addItem(item)
self._playlist_items[playlist_id] = item
def remove_playlist_item(self, playlist_id: str):
item = self._playlist_items.pop(playlist_id, None)
if item:
row = self._playlist_list.row(item)
self._playlist_list.takeItem(row)
def rename_playlist_item(self, playlist_id: str, new_name: str):
item = self._playlist_items.get(playlist_id)
if item:
item.setText(f"\U0001F4CB {new_name}")
def clear_playlist_items(self):
self._playlist_list.clear()
self._playlist_items.clear()
def get_playlist_order_ids(self) -> List[str]:
ids = []
for i in range(self._playlist_list.count()):
item = self._playlist_list.item(i)
ids.append(item.data(Qt.ItemDataRole.UserRole))
return ids
def _on_tracks_dropped_on_playlist(self, playlist_id: str, paths: list):
self.tracks_dropped_on_playlist.emit(playlist_id, paths)
def _on_tracks_dropped_for_nav_transfer(self, paths: list):
self.tracks_dropped_for_nav_transfer.emit(paths)
def _on_playlist_dropped_for_transfer(self, playlist_id: str):
self.playlist_dropped_for_nav_transfer.emit(playlist_id)
def _on_playlists_reordered(self):
order = self.get_playlist_order_ids()
self.item_selected.emit("playlist_reorder", "|".join(order))
def _on_nav_item_clicked(self, item: QListWidgetItem):
kind = item.data(Qt.ItemDataRole.UserRole + 1)
value = item.data(Qt.ItemDataRole.UserRole + 2)
self._library_list.clearSelection()
self._playlist_list.clearSelection()
self._nav_list.setCurrentItem(item)
self.item_selected.emit(kind, value)
def select_library_music(self):
if self._library_list.count() > 0:
self._library_list.setCurrentRow(0)
self._on_library_item_clicked(self._library_list.item(0))
def select_playlist(self, playlist_id: str):
for i in range(self._playlist_list.count()):
item = self._playlist_list.item(i)
if item.data(Qt.ItemDataRole.UserRole) == playlist_id:
self._playlist_list.setCurrentItem(item)
self._on_playlist_clicked(item)
return
def select_playlist_silent(self, playlist_id: str):
for i in range(self._playlist_list.count()):
item = self._playlist_list.item(i)
if item.data(Qt.ItemDataRole.UserRole) == playlist_id:
self._playlist_list.setCurrentItem(item)
self._library_list.clearSelection()
self._nav_list.clearSelection()
return
def select_by_view_mode(self, mode: str):
mapping = {"songs": 4, "albums": 3, "artists": 2, "recent": 1}
row = mapping.get(mode, 0)
if row < self._library_list.count():
self._library_list.setCurrentRow(row)
self._playlist_list.clearSelection()
self._nav_list.clearSelection()
def width(self) -> int:
return self.geometry().width()
+118
View File
@@ -0,0 +1,118 @@
#!/usr/bin/env python3
"""Splash overlay shown while scanning the music library for the first time."""
from PyQt6.QtWidgets import QDialog, QVBoxLayout, QLabel, QWidget, QApplication
from PyQt6.QtCore import Qt, QTimer
from PyQt6.QtGui import QPainter, QPen, QColor, QFont
class _Spinner(QWidget):
"""A spinning arc — like a circular progress indicator."""
def __init__(self, parent=None):
super().__init__(parent)
self.setFixedSize(64, 64)
self._angle = 0.0
self._timer = QTimer(self)
self._timer.timeout.connect(self._tick)
self._timer.start(30)
def _tick(self):
self._angle = (self._angle + 6) % 360
self.update()
def paintEvent(self, event):
p = QPainter(self)
p.setRenderHint(QPainter.RenderHint.Antialiasing)
r = self.rect().adjusted(4, 4, -4, -4)
p.setPen(QPen(QColor("#e0e0e0"), 3, Qt.PenStyle.SolidLine, Qt.PenCapStyle.RoundCap))
p.drawEllipse(r)
p.setPen(QPen(QColor("#3b82f6"), 4, Qt.PenStyle.SolidLine, Qt.PenCapStyle.RoundCap))
p.drawArc(r, int(self._angle * 16), 60 * 16)
class SplashOverlay(QDialog):
"""Modal overlay with a spinning animation + progress counter.
Usage::
splash = SplashOverlay(window)
splash.show()
# … scan …
splash.set_progress(current, total)
splash.close()
"""
def __init__(self, parent=None):
super().__init__(parent)
self.setWindowFlags(
Qt.WindowType.FramelessWindowHint | Qt.WindowType.Dialog
)
self.setAttribute(Qt.WidgetAttribute.WA_TranslucentBackground)
self.setAttribute(Qt.WidgetAttribute.WA_ShowWithoutActivating)
self.setModal(True)
self._card = QWidget(self)
self._card.setObjectName("card")
self._card.setFixedSize(340, 240)
layout = QVBoxLayout(self._card)
layout.setAlignment(Qt.AlignmentFlag.AlignCenter)
layout.setSpacing(16)
self._spinner = _Spinner()
layout.addWidget(self._spinner, alignment=Qt.AlignmentFlag.AlignCenter)
self._title = QLabel("Создаём музыкальную\nмедиатеку…")
self._title.setAlignment(Qt.AlignmentFlag.AlignCenter)
f = self._title.font()
f.setPointSize(16)
f.setBold(True)
self._title.setFont(f)
layout.addWidget(self._title)
self._count = QLabel("")
self._count.setAlignment(Qt.AlignmentFlag.AlignCenter)
f2 = self._count.font()
f2.setPointSize(12)
self._count.setFont(f2)
layout.addWidget(self._count)
self._card.setStyleSheet("""
#card {
background: white;
border-radius: 16px;
}
QLabel {
color: #333;
background: transparent;
}
""")
def _recenter(self):
if not self.parent():
return
pr = self.parent().rect()
self.setGeometry(pr)
cx = (pr.width() - self._card.width()) // 2
cy = (pr.height() - self._card.height()) // 2
self._card.move(cx, cy)
def showEvent(self, event):
self._recenter()
super().showEvent(event)
QApplication.processEvents()
def resizeEvent(self, event):
self._recenter()
super().resizeEvent(event)
def set_progress(self, current: int, total: int) -> None:
"""Update the file counter shown on the overlay."""
if total > 0:
self._count.setText(f"Файл {current} из {total}")
else:
self._count.setText(f"Файл {current}")
QApplication.processEvents()
+107
View File
@@ -0,0 +1,107 @@
#!/usr/bin/env python3
"""
Clickable 5star rating widget.
Scale: 0 (unrated) 5 (). Stored internally as stars;
callers convert to/from the iPod 0100 scale as needed.
"""
import math
from PyQt6.QtCore import Qt, QRectF, pyqtSignal
from PyQt6.QtGui import QPainter, QColor, QPainterPath, QBrush, QPen
from PyQt6.QtWidgets import QWidget
class ClickableStarRating(QWidget):
"""Five clickable stars with hover preview and toggle-to-unrate."""
rating_changed = pyqtSignal(int)
STAR_COUNT = 5
STAR_SIZE = 22 # diameter of each star
PADDING = 4
FILLED_COLOR = QColor("#ffb347") # warm amber
EMPTY_COLOR = QColor("#555555")
HOVER_COLOR = QColor("#ffd700") # gold
def __init__(self, parent=None):
super().__init__(parent)
self._rating = 0
self._hover_idx = -1
w = self.STAR_COUNT * self.STAR_SIZE + (self.STAR_COUNT - 1) * self.PADDING
self.setFixedSize(w, self.STAR_SIZE + 8)
self.setMouseTracking(True)
# ── public API ────────────────────────────────────────────────
def set_rating(self, stars: int) -> None:
self._rating = max(0, min(self.STAR_COUNT, stars))
self.update()
def rating(self) -> int:
return self._rating
# ── painting ───────────────────────────────────────────────────
def paintEvent(self, event): # noqa: N802
painter = QPainter(self)
painter.setRenderHint(QPainter.RenderHint.Antialiasing)
painter.setPen(Qt.PenStyle.NoPen)
hovering = self._hover_idx >= 0
for i in range(self.STAR_COUNT):
if hovering:
filled = i <= self._hover_idx
else:
filled = i < self._rating
color = (
self.HOVER_COLOR if hovering and filled
else self.FILLED_COLOR if filled
else self.EMPTY_COLOR
)
painter.setBrush(QBrush(color))
cx = self.STAR_SIZE // 2 + i * (self.STAR_SIZE + self.PADDING)
cy = self.height() // 2
self._draw_star(painter, cx, cy, self.STAR_SIZE // 2 - 1)
@staticmethod
def _draw_star(painter: QPainter, cx: int, cy: int, r: int) -> None:
"""Draw a five-pointed star centred at (cx, cy) with outer radius *r*."""
path = QPainterPath()
inner_r = r * 0.4
points = []
for i in range(10):
angle = math.pi / 2 + i * math.pi / 5
radius = r if i % 2 == 0 else inner_r
x = cx + radius * math.cos(angle)
y = cy - radius * math.sin(angle)
points.append((x, y))
path.moveTo(points[0][0], points[0][1])
for x, y in points[1:]:
path.lineTo(x, y)
path.closeSubpath()
painter.drawPath(path)
# ── mouse interaction ──────────────────────────────────────────
def _star_at(self, x: int) -> int:
idx = x // (self.STAR_SIZE + self.PADDING)
return min(max(idx, 0), self.STAR_COUNT - 1)
def mousePressEvent(self, event): # noqa: N802
star = self._star_at(int(event.position().x()))
new_val = star + 1 if self._rating != star + 1 else 0
self._rating = new_val
self.rating_changed.emit(new_val)
self.update()
def mouseMoveEvent(self, event): # noqa: N802
self._hover_idx = self._star_at(int(event.position().x()))
self.update()
def leaveEvent(self, event): # noqa: N802
self._hover_idx = -1
self.update()
+21
View File
@@ -0,0 +1,21 @@
"""SQLiteDB_Writer — Write SQLite databases for iPod Nano 6G/7G.
These iPods ignore the traditional binary iTunesDB (or iTunesCDB) and read
music metadata from SQLite databases located in:
/iPod_Control/iTunes/iTunes Library.itlp/
The directory contains:
Library.itdb tracks, albums, artists, composers, playlists, genres
Locations.itdb iPod file paths for each track
Dynamic.itdb play counts, ratings, bookmarks
Extras.itdb lyrics, chapters (optional, can be empty)
Genius.itdb genius data (optional, can be empty)
Locations.itdb.cbk HASHAB-signed block checksums of Locations.itdb
Reference implementation: libgpod itdb_sqlite.c
"""
from .sqlite_writer import write_sqlite_databases
__all__ = ["write_sqlite_databases"]
+64
View File
@@ -0,0 +1,64 @@
"""Shared helpers for SQLiteDB_Writer modules.
Centralises utilities that were previously duplicated across
library_writer, locations_writer, dynamic_writer, extras_writer,
and genius_writer.
"""
import os
import sqlite3
# ── Timestamp helpers ──────────────────────────────────────────────────
# SQLite databases use Core Data timestamps: seconds since 2001-01-01 UTC
# (the Cocoa/Core Foundation reference date).
CORE_DATA_EPOCH = 978307200 # Unix timestamp of 2001-01-01 00:00:00 UTC
def unix_to_coredata(unix_ts: int, tz_offset: int = 0) -> int:
"""Convert Unix timestamp to Core Data timestamp.
Args:
unix_ts: Unix timestamp (seconds since 1970-01-01)
tz_offset: Timezone offset in seconds (positive = east of UTC)
Returns:
Core Data timestamp (seconds since 2001-01-01) adjusted for timezone.
Returns 0 if input is 0.
"""
if unix_ts == 0:
return 0
return unix_ts - CORE_DATA_EPOCH - tz_offset
def s64(val: int) -> int:
"""Convert unsigned 64-bit int to signed for SQLite INTEGER storage.
SQLite INTEGER is signed 64-bit (max 2^63-1). iPod db_ids and PIDs
are unsigned 64-bit values that may exceed this limit.
"""
if val >= (1 << 63):
return val - (1 << 64)
return val
def open_db(path: str, extra_pragmas: list[str] | None = None) -> tuple[sqlite3.Connection, sqlite3.Cursor]:
"""Create a fresh SQLite database at *path*.
Deletes any existing file, opens a new connection with performance
PRAGMAs (journal_mode=OFF, synchronous=OFF), and returns (conn, cursor).
Args:
path: Output file path.
extra_pragmas: Additional PRAGMA statements to execute
(e.g. ``["encoding='UTF-8'"]``).
"""
if os.path.exists(path):
os.remove(path)
conn = sqlite3.connect(path)
conn.execute("PRAGMA journal_mode=OFF")
conn.execute("PRAGMA synchronous=OFF")
for pragma in extra_pragmas or []:
conn.execute(f"PRAGMA {pragma}")
return conn, conn.cursor()
+171
View File
@@ -0,0 +1,171 @@
"""Locations.itdb.cbk writer — HASHAB-signed block checksums.
The cbk (checksum book) file contains SHA1 checksums of 1024-byte blocks
of Locations.itdb, plus a final SHA1 of all those checksums, signed with
HASHAB.
File format:
[57 bytes] HASHAB signature of final_sha1 (or 20 bytes for HASH58/72)
[20 bytes] final_sha1 = SHA1(all_block_sha1s concatenated)
[Nx20 bytes] SHA1 of each 1024-byte block of Locations.itdb
For HASHAB (Nano 6G/7G), the header is 57 bytes.
For HASH58, the header would be 20 bytes.
For HASH72, the header would be 46 bytes.
Reference: libgpod itdb_sqlite.c mk_Locations_cbk()
"""
import hashlib
import logging
import os
from ipod_device import ChecksumType
logger = logging.getLogger(__name__)
# Block size for checksumming
BLOCK_SIZE = 1024
def _compute_block_sha1s(data: bytes) -> list[bytes]:
"""Compute SHA1 hash of each 1024-byte block.
The last block may be smaller than 1024 bytes; it's still hashed.
Args:
data: Raw file contents.
Returns:
List of 20-byte SHA1 digests, one per block.
"""
block_hashes = []
offset = 0
while offset < len(data):
block = data[offset:offset + BLOCK_SIZE]
block_hashes.append(hashlib.sha1(block).digest())
offset += BLOCK_SIZE
return block_hashes
def write_locations_cbk(
cbk_path: str,
locations_itdb_path: str,
checksum_type: ChecksumType,
firewire_id: bytes | None = None,
ipod_path: str | None = None,
) -> None:
"""Generate and write the Locations.itdb.cbk checksum file.
Args:
cbk_path: Output path for the .cbk file.
locations_itdb_path: Path to the Locations.itdb file to checksum.
checksum_type: The device's checksum algorithm (HASHAB, HASH58, etc.).
firewire_id: 8-byte FireWire GUID (required for HASHAB and HASH58).
ipod_path: Mount point of iPod (used for HASH72 HashInfo fallback).
Raises:
ValueError: If firewire_id is missing when needed.
FileNotFoundError: If Locations.itdb doesn't exist.
"""
with open(locations_itdb_path, 'rb') as f:
locations_data = f.read()
# Compute block SHA1s
block_sha1s = _compute_block_sha1s(locations_data)
# Compute final SHA1 = SHA1(concatenation of all block SHA1s)
all_sha1s = b''.join(block_sha1s)
final_sha1 = hashlib.sha1(all_sha1s).digest()
logger.debug("Locations.itdb: %d bytes, %d blocks, final SHA1: %s",
len(locations_data), len(block_sha1s), final_sha1.hex())
# Generate header signature based on checksum type
if checksum_type == ChecksumType.HASHAB:
if not firewire_id or len(firewire_id) < 8:
raise ValueError("FireWire ID required for HASHAB cbk signature")
from iTunesDB_Writer.hashab import compute_hashab
header = compute_hashab(final_sha1, firewire_id[:8])
if len(header) != 57:
raise RuntimeError(f"HASHAB returned {len(header)} bytes, expected 57")
logger.debug("CBK header: HASHAB signature (%d bytes)", len(header))
elif checksum_type == ChecksumType.HASH58:
if not firewire_id or len(firewire_id) < 8:
raise ValueError("FireWire ID required for HASH58 cbk signature")
from iTunesDB_Writer.hash58 import compute_hash58
header = compute_hash58(firewire_id, final_sha1)
logger.debug("CBK header: HASH58 signature (%d bytes)", len(header))
elif checksum_type == ChecksumType.HASH72:
from iTunesDB_Writer.hash72 import (
read_hash_info, extract_hash_info_to_dict,
_hash_generate, HashInfo,
)
# Try centralized store first
hash_info = None
try:
from ipod_device import get_current_device
dev = get_current_device()
if dev and dev.hash_info_iv and dev.hash_info_rndpart:
hash_info = HashInfo(
uuid=b'\x00' * 20,
rndpart=dev.hash_info_rndpart,
iv=dev.hash_info_iv,
)
except Exception:
pass
if hash_info is None and ipod_path:
try:
hash_info = read_hash_info(ipod_path)
except Exception:
pass
# Fallback: extract from existing iTunesCDB on device
if hash_info is None and ipod_path:
try:
from ipod_device import resolve_itdb_path
itdb_path = resolve_itdb_path(ipod_path)
if itdb_path:
with open(itdb_path, "rb") as f:
itdb_data = f.read()
hd = extract_hash_info_to_dict(itdb_data)
if hd:
hash_info = HashInfo(
uuid=b'\x00' * 20,
rndpart=hd['rndpart'],
iv=hd['iv'],
)
logger.debug("CBK: extracted HashInfo from existing %s",
os.path.basename(itdb_path))
except Exception:
pass
if hash_info:
header = _hash_generate(final_sha1, hash_info.iv, hash_info.rndpart)
logger.debug("CBK header: HASH72 signature (%d bytes)", len(header))
else:
logger.warning("No HashInfo available for HASH72 cbk — writing final SHA1 only")
header = final_sha1
else:
# No checksum needed — older devices or NONE
# Just write the SHA1 as header (20 bytes)
header = final_sha1
# Write the cbk file: header + final_sha1 + block_sha1s
with open(cbk_path, 'wb') as f:
f.write(header)
f.write(final_sha1)
for bsha1 in block_sha1s:
f.write(bsha1)
total_size = len(header) + 20 + len(block_sha1s) * 20
logger.info("Wrote Locations.itdb.cbk: %d bytes "
"(%d-byte header + 20-byte final SHA1 + %d×20 block SHA1s)",
total_size, len(header), len(block_sha1s))
+124
View File
@@ -0,0 +1,124 @@
"""Dynamic.itdb writer — play counts, ratings, and bookmark data.
Contains item_stats (per-track play/skip counts, ratings, bookmarks)
and container_ui (playlist UI state like play order, repeat, shuffle).
Reference: libgpod itdb_sqlite.c mk_Dynamic()
"""
import logging
from typing import Optional
from iTunesDB_Writer.mhit_writer import TrackInfo
from ._helpers import s64 as _s64, unix_to_coredata, open_db
logger = logging.getLogger(__name__)
_DYNAMIC_SCHEMA = """
CREATE TABLE IF NOT EXISTS item_stats (
item_pid INTEGER NOT NULL,
has_been_played INTEGER DEFAULT 0,
date_played INTEGER DEFAULT 0,
play_count_user INTEGER DEFAULT 0,
play_count_recent INTEGER DEFAULT 0,
date_skipped INTEGER DEFAULT 0,
skip_count_user INTEGER DEFAULT 0,
skip_count_recent INTEGER DEFAULT 0,
bookmark_time_ms REAL,
bookmark_time_ms_common REAL,
user_rating INTEGER DEFAULT 0,
user_rating_common INTEGER DEFAULT 0,
rental_expired INTEGER DEFAULT 0,
play_count_user_original INTEGER DEFAULT 0,
skip_count_user_original INTEGER DEFAULT 0,
genius_id INTEGER DEFAULT 0,
PRIMARY KEY (item_pid)
);
CREATE TABLE IF NOT EXISTS container_ui (
container_pid INTEGER NOT NULL,
play_order INTEGER DEFAULT 0,
is_reversed INTEGER DEFAULT 0,
album_field_order INTEGER DEFAULT 0,
repeat_mode INTEGER DEFAULT 0,
shuffle_items INTEGER DEFAULT 0,
has_been_shuffled INTEGER DEFAULT 0,
PRIMARY KEY (container_pid)
);
CREATE TABLE IF NOT EXISTS rental_info (
item_pid INTEGER NOT NULL,
rental_date_started INTEGER DEFAULT 0,
rental_duration INTEGER DEFAULT 0,
rental_playback_date_started INTEGER DEFAULT 0,
rental_playback_duration INTEGER DEFAULT 0,
is_demo INTEGER DEFAULT 0,
PRIMARY KEY (item_pid)
);
"""
def write_dynamic_itdb(
path: str,
tracks: list[TrackInfo],
playlist_pids: Optional[list[int]] = None,
tz_offset: int = 0,
) -> None:
"""Write Dynamic.itdb SQLite database.
Args:
path: Output file path.
tracks: List of TrackInfo objects.
playlist_pids: All playlist PIDs (master + user + smart), as returned
by ``write_library_itdb()``. One ``container_ui`` row
is written per PID.
tz_offset: Timezone offset in seconds.
"""
conn, cur = open_db(path)
cur.executescript(_DYNAMIC_SCHEMA)
# ── item_stats ─────────────────────────────────────────────────────
for track in tracks:
has_been_played = 1 if track.play_count > 0 else 0
date_played = unix_to_coredata(track.last_played or 0, tz_offset)
date_skipped = unix_to_coredata(track.last_skipped or 0, tz_offset)
cur.execute(
"""INSERT INTO item_stats (
item_pid, has_been_played, date_played,
play_count_user, play_count_recent,
date_skipped, skip_count_user, skip_count_recent,
bookmark_time_ms, bookmark_time_ms_common,
user_rating, user_rating_common,
rental_expired,
play_count_user_original, skip_count_user_original,
genius_id
) VALUES (?, ?, ?, ?, 0, ?, ?, 0, ?, ?, ?, ?, 0, ?, ?, 0)""",
(
_s64(track.db_track_id), has_been_played, date_played,
track.play_count,
date_skipped, track.skip_count,
float(track.bookmark_time), float(track.bookmark_time),
track.rating, track.app_rating,
track.play_count, track.skip_count,
)
)
# ── container_ui ───────────────────────────────────────────────────
# One row per playlist PID (master + user + smart)
for pid in (playlist_pids or []):
cur.execute(
"INSERT INTO container_ui (container_pid, play_order, is_reversed, "
"album_field_order, repeat_mode, shuffle_items, has_been_shuffled) "
"VALUES (?, 0, 0, 1, 0, 0, 0)",
(_s64(pid),)
)
conn.commit()
conn.close()
logger.info("Wrote Dynamic.itdb: %d item_stats, %d container_ui",
len(tracks), len(playlist_pids or []))
+82
View File
@@ -0,0 +1,82 @@
"""Extras.itdb writer — lyrics and chapter data.
Creates the Extras.itdb with empty tables. Lyrics from tracks are
inserted if available.
Reference: libgpod itdb_sqlite.c mk_Extras()
"""
import logging
from iTunesDB_Writer.mhit_writer import TrackInfo
from ._helpers import s64 as _s64, open_db
logger = logging.getLogger(__name__)
_EXTRAS_SCHEMA = """
CREATE TABLE IF NOT EXISTS chapter (
item_pid INTEGER NOT NULL,
data BLOB,
PRIMARY KEY (item_pid)
);
CREATE TABLE IF NOT EXISTS lyrics (
item_pid INTEGER NOT NULL,
checksum INTEGER,
lyrics TEXT,
PRIMARY KEY (item_pid)
);
"""
def write_extras_itdb(
path: str,
tracks: list[TrackInfo],
) -> None:
"""Write Extras.itdb SQLite database.
Args:
path: Output file path.
tracks: List of TrackInfo objects.
"""
conn, cur = open_db(path)
cur.executescript(_EXTRAS_SCHEMA)
# Insert lyrics for tracks that have them
lyrics_count = 0
for track in tracks:
if track.lyrics:
# Simple checksum: sum of bytes mod 2^32
checksum = sum(track.lyrics.encode('utf-8')) & 0xFFFFFFFF
cur.execute(
"INSERT INTO lyrics (item_pid, checksum, lyrics) VALUES (?, ?, ?)",
(_s64(track.db_track_id), checksum, track.lyrics)
)
lyrics_count += 1
# Insert chapter data for tracks that have chapters
chapter_count = 0
for track in tracks:
cd = track.chapter_data or {}
chapters = cd.get("chapters")
if chapters:
from iTunesDB_Writer.mhod_writer import build_chapter_blob
blob = build_chapter_blob(
chapters,
unk024=cd.get("unk024", 0),
unk028=cd.get("unk028", 0),
unk032=cd.get("unk032", 0),
)
if blob:
cur.execute(
"INSERT INTO chapter (item_pid, data) VALUES (?, ?)",
(_s64(track.db_track_id), blob)
)
chapter_count += 1
conn.commit()
conn.close()
logger.info("Wrote Extras.itdb: %d lyrics entries, %d chapter entries", lyrics_count, chapter_count)
+56
View File
@@ -0,0 +1,56 @@
"""Genius.itdb writer — Genius playlists and similarity data.
Creates a minimal Genius.itdb with empty tables. We don't support Genius
features, but the database file must exist for the firmware.
Reference: libgpod itdb_sqlite.c mk_Genius()
"""
import logging
from ._helpers import open_db
logger = logging.getLogger(__name__)
_GENIUS_SCHEMA = """
CREATE TABLE IF NOT EXISTS genius_config (
id INTEGER NOT NULL,
version INTEGER,
default_num_results INTEGER DEFAULT 0,
min_num_results INTEGER DEFAULT 0,
data BLOB,
PRIMARY KEY (id),
UNIQUE (version)
);
CREATE TABLE IF NOT EXISTS genius_metadata (
genius_id INTEGER NOT NULL,
version INTEGER,
data BLOB,
PRIMARY KEY (genius_id)
);
CREATE TABLE IF NOT EXISTS genius_similarities (
genius_id INTEGER NOT NULL,
version INTEGER,
data BLOB,
PRIMARY KEY (genius_id)
);
"""
def write_genius_itdb(path: str) -> None:
"""Write Genius.itdb SQLite database (empty tables).
Args:
path: Output file path.
"""
conn, cur = open_db(path)
cur.executescript(_GENIUS_SCHEMA)
conn.commit()
conn.close()
logger.info("Wrote Genius.itdb (empty)")
File diff suppressed because it is too large Load Diff
+145
View File
@@ -0,0 +1,145 @@
"""Locations.itdb writer — iPod file path mapping database.
Maps track PIDs to their physical file locations on the iPod filesystem.
Schema:
base_location: single row with root path "iPod_Control/Music"
location: one row per track, mapping item_pid "Fxx/ABCD.mp3"
Reference: libgpod itdb_sqlite.c mk_Locations()
"""
import time
import logging
from iTunesDB_Writer.mhit_writer import TrackInfo
from iTunesDB_Shared.constants import FILETYPE_CODES
from ._helpers import s64 as _s64, unix_to_coredata, open_db
logger = logging.getLogger(__name__)
# location_type = 0x46494C45 = "FILE" as big-endian int
LOCATION_TYPE_FILE = 0x46494C45
# Extension codes — same as FILETYPE_CODES (big-endian 4-byte ASCII)
_EXTENSION_CODES = FILETYPE_CODES
# kind_id mapping — matches location_kind_map in Library.itdb
# IDs from real iTunes-written databases on Nano 6G
_KIND_ID = {
'mp3': 1, # "MPEG audio file"
'aac': 3, # "AAC audio file"
'm4a': 3, # "AAC audio file" (or ALAC in M4A container)
'm4p': 2, # "Purchased AAC audio file"
'm4b': 3, # "AAC audio file" (audiobook)
'm4v': 3, #
'mp4': 3, #
'wav': 1, #
'aif': 1, #
'aiff': 1, #
'alac': 3, # ALAC is in M4A container
}
def _ipod_path_to_location(ipod_path: str) -> str:
"""Convert iPod colon-separated path to slash-based location.
Input: ":iPod_Control:Music:F04:ZEUN.mp3"
Output: "F04/ZEUN.mp3"
The location field stores the path relative to the base_location
("iPod_Control/Music"), using forward slashes.
"""
# Strip leading colon and split
parts = ipod_path.strip(':').split(':')
# Skip "iPod_Control" and "Music" prefix
# The path format is :iPod_Control:Music:Fxx:filename
# We want: Fxx/filename
if len(parts) >= 4 and parts[0] == 'iPod_Control' and parts[1] == 'Music':
return '/'.join(parts[2:])
elif len(parts) >= 2:
# Fallback: just take the last two parts
return '/'.join(parts[-2:])
else:
return ipod_path.strip(':').replace(':', '/')
_LOCATIONS_SCHEMA = """
CREATE TABLE IF NOT EXISTS base_location (
id INTEGER NOT NULL,
path TEXT,
PRIMARY KEY (id)
);
CREATE TABLE IF NOT EXISTS location (
item_pid INTEGER NOT NULL,
sub_id INTEGER NOT NULL DEFAULT 0,
base_location_id INTEGER DEFAULT 0,
location_type INTEGER,
location TEXT,
extension INTEGER,
kind_id INTEGER DEFAULT 0,
date_created INTEGER DEFAULT 0,
file_size INTEGER DEFAULT 0,
file_creator INTEGER,
file_type INTEGER,
num_dir_levels_file INTEGER,
num_dir_levels_lib INTEGER,
PRIMARY KEY (item_pid, sub_id)
);
"""
def write_locations_itdb(
path: str,
tracks: list[TrackInfo],
tz_offset: int = 0,
) -> None:
"""Write Locations.itdb SQLite database.
Args:
path: Output file path.
tracks: List of TrackInfo objects (with db_track_id and location set).
tz_offset: Timezone offset in seconds (positive = east of UTC).
"""
conn, cur = open_db(path)
cur.executescript(_LOCATIONS_SCHEMA)
# Single base_location entry
cur.execute(
"INSERT INTO base_location (id, path) VALUES (1, 'iPod_Control/Music')"
)
# One location per track
now = int(time.time())
for track in tracks:
location = _ipod_path_to_location(track.location)
ft = track.filetype.lower()
extension = _EXTENSION_CODES.get(ft, _EXTENSION_CODES.get('mp3', 0x4D503320))
kind_id = _KIND_ID.get(ft, 0)
# date_created: Core Data timestamp of when the file was added
date_added = track.date_added or now
date_cd = unix_to_coredata(date_added, tz_offset)
cur.execute(
"""INSERT INTO location (
item_pid, sub_id, base_location_id, location_type,
location, extension, kind_id, date_created, file_size,
file_creator, file_type,
num_dir_levels_file, num_dir_levels_lib
) VALUES (?, 0, 1, ?, ?, ?, ?, ?, ?, NULL, NULL, NULL, NULL)""",
(
_s64(track.db_track_id), LOCATION_TYPE_FILE,
location, extension, kind_id,
date_cd, track.size,
)
)
conn.commit()
conn.close()
logger.info("Wrote Locations.itdb: %d locations", len(tracks))
+211
View File
@@ -0,0 +1,211 @@
"""SQLite database writer — orchestrates writing all SQLite databases.
This is the main entry point for the SQLiteDB_Writer module. It
coordinates writing all five databases plus the checksum file for
iPod Nano 6G/7G.
The databases are written to:
/iPod_Control/iTunes/iTunes Library.itlp/
Usage:
from SQLiteDB_Writer import write_sqlite_databases
write_sqlite_databases(
ipod_path="/media/ipod",
tracks=tracks,
playlists=playlists,
smart_playlists=smart_playlists,
master_playlist_name="iPod",
)
"""
import os
import random
import shutil
import time
import logging
import tempfile
from typing import Optional
from iTunesDB_Writer.mhit_writer import TrackInfo
from iTunesDB_Writer.mhyp_writer import PlaylistInfo
from ipod_device import ChecksumType, DeviceCapabilities
from ipod_device import detect_checksum_type, get_firewire_id
from .library_writer import write_library_itdb
from .locations_writer import write_locations_itdb
from .dynamic_writer import write_dynamic_itdb
from .extras_writer import write_extras_itdb
from .genius_writer import write_genius_itdb
from .cbk_writer import write_locations_cbk
logger = logging.getLogger(__name__)
# Directory within iPod where SQLite databases live
ITLP_DIR = os.path.join("iPod_Control", "iTunes", "iTunes Library.itlp")
def write_sqlite_databases(
ipod_path: str,
tracks: list[TrackInfo],
playlists: Optional[list[PlaylistInfo]] = None,
smart_playlists: Optional[list[PlaylistInfo]] = None,
master_playlist_name: str = "iPod",
db_pid: int = 0,
capabilities: Optional[DeviceCapabilities] = None,
firewire_id: Optional[bytes] = None,
backup: bool = True,
) -> bool:
"""Write all SQLite databases for iPod Nano 6G/7G.
Writes the databases to a temp directory first, then atomically
replaces the files in the iTunes Library.itlp directory.
Args:
ipod_path: Mount point of iPod (e.g. "E:\\")
tracks: List of TrackInfo objects (db_track_id must already be assigned).
playlists: User playlists (master is auto-generated).
smart_playlists: Smart playlists.
master_playlist_name: Name for the master playlist.
db_pid: Database persistent ID (from mhbd db_id).
capabilities: Device capabilities.
firewire_id: 8-byte FireWire GUID for signing.
backup: Whether to backup existing databases.
Returns:
True if all databases were written successfully.
"""
itlp_path = os.path.join(ipod_path, ITLP_DIR)
# Ensure the directory exists
os.makedirs(itlp_path, exist_ok=True)
# Determine timezone offset
if time.daylight:
tz_offset = -time.altzone
else:
tz_offset = -time.timezone
# Determine checksum type
checksum_type = ChecksumType.NONE
if capabilities:
checksum_type = capabilities.checksum
else:
checksum_type = detect_checksum_type(ipod_path)
# Get FireWire ID if needed and not provided
if firewire_id is None and checksum_type in (
ChecksumType.HASHAB, ChecksumType.HASH58
):
try:
firewire_id = get_firewire_id(ipod_path)
except Exception as e:
logger.warning("Could not get FireWire ID for cbk signing: %s", e)
# Generate db_pid if not provided
if not db_pid:
db_pid = random.getrandbits(64)
# Backup existing databases
if backup:
for fname in ("Library.itdb", "Locations.itdb", "Dynamic.itdb",
"Extras.itdb", "Genius.itdb", "Locations.itdb.cbk"):
fpath = os.path.join(itlp_path, fname)
if os.path.exists(fpath):
try:
shutil.copy2(fpath, fpath + ".backup")
except Exception as e:
logger.warning("Could not backup %s: %s", fname, e)
# Write all databases to temp directory first, then move
# This gives us atomicity — if any write fails, the originals are intact.
with tempfile.TemporaryDirectory(prefix="iOpenPod_sqlite_", ignore_cleanup_errors=True) as tmp_dir:
try:
# 1. Library.itdb (tracks, albums, artists, playlists, …)
lib_path = os.path.join(tmp_dir, "Library.itdb")
playlist_pids = write_library_itdb(
path=lib_path,
tracks=tracks,
playlists=playlists,
smart_playlists=smart_playlists,
master_playlist_name=master_playlist_name,
db_pid=db_pid,
tz_offset=tz_offset,
)
# 2. Locations.itdb (file path mappings)
loc_path = os.path.join(tmp_dir, "Locations.itdb")
write_locations_itdb(
path=loc_path,
tracks=tracks,
tz_offset=tz_offset,
)
# 3. Dynamic.itdb (play counts, ratings, bookmarks)
dyn_path = os.path.join(tmp_dir, "Dynamic.itdb")
write_dynamic_itdb(
path=dyn_path,
tracks=tracks,
playlist_pids=playlist_pids,
tz_offset=tz_offset,
)
# 4. Extras.itdb (lyrics, chapters)
extras_path = os.path.join(tmp_dir, "Extras.itdb")
write_extras_itdb(
path=extras_path,
tracks=tracks,
)
# 5. Genius.itdb (empty tables)
genius_path = os.path.join(tmp_dir, "Genius.itdb")
write_genius_itdb(path=genius_path)
# 6. Locations.itdb.cbk (HASHAB-signed block checksums)
cbk_path = os.path.join(tmp_dir, "Locations.itdb.cbk")
try:
write_locations_cbk(
cbk_path=cbk_path,
locations_itdb_path=loc_path,
checksum_type=checksum_type,
firewire_id=firewire_id,
ipod_path=ipod_path,
)
except Exception as e:
logger.error("Failed to write Locations.itdb.cbk: %s", e)
# CBK is critical for signed devices — fail the whole write
if checksum_type in (ChecksumType.HASHAB, ChecksumType.HASH72):
raise
# For other devices, continue without it
cbk_path = None
# Move all files to the target directory
files_to_move = [
("Library.itdb", lib_path),
("Locations.itdb", loc_path),
("Dynamic.itdb", dyn_path),
("Extras.itdb", extras_path),
("Genius.itdb", genius_path),
]
if cbk_path and os.path.exists(cbk_path):
files_to_move.append(("Locations.itdb.cbk", cbk_path))
for fname, src_path in files_to_move:
dst_path = os.path.join(itlp_path, fname)
try:
shutil.copyfile(src_path, dst_path)
except Exception as e:
logger.error("Failed to copy %s to iPod: %s", fname, e)
raise
logger.info("SQLite databases written to %s "
"(%d tracks, %d playlists, %d smart playlists)",
itlp_path, len(tracks),
len(playlists or []),
len(smart_playlists or []))
return True
except Exception as e:
logger.error("Failed to write SQLite databases: %s", e,
exc_info=True)
return False
+22
View File
@@ -0,0 +1,22 @@
from .exceptions import (
CorruptHeaderError,
InsufficientDataError,
ITunesDBParseError,
UnknownChunkTypeError,
)
from .parser import decompress_itunescdb, parse_itunesdb
from .playcounts import PlayCountEntry, merge_playcounts, parse_playcounts
__all__ = [
# Public parsing API
"parse_itunesdb",
"decompress_itunescdb",
"parse_playcounts",
"merge_playcounts",
"PlayCountEntry",
# Exceptions
"ITunesDBParseError",
"CorruptHeaderError",
"UnknownChunkTypeError",
"InsufficientDataError",
]
+69
View File
@@ -0,0 +1,69 @@
"""
Internal parsing helpers shared across iTunesDB chunk parsers.
Provides:
- Pre-compiled ``struct.Struct`` objects for common binary field widths.
- :func:`read_generic_header` reads the 12-byte generic chunk header.
Child-iteration helpers live in :mod:`chunk_parser` to avoid circular
imports (they need ``parse_chunk``, which dispatches back to the typed
parsers that use this module's struct helpers).
"""
from __future__ import annotations
import struct
from typing import Any
from .exceptions import CorruptHeaderError, InsufficientDataError
# ── Pre-compiled struct objects ──────────────────────────────────────
# Used by callers (e.g. mhod_parser) that do inline struct reads.
# The Shared defs module still uses ad-hoc struct.unpack calls; these
# are for Parser-local code.
UINT16_LE = struct.Struct("<H")
UINT32_LE = struct.Struct("<I")
UINT64_LE = struct.Struct("<Q")
INT32_LE = struct.Struct("<i")
FLOAT32_LE = struct.Struct("<f")
# The generic chunk header shared by every iTunesDB chunk:
# +0x00 chunk_type (4 bytes ASCII)
# +0x04 header_len (u32 LE)
# +0x08 length_or_child_count (u32 LE)
_GENERIC_HEADER = struct.Struct("<4sII")
GENERIC_HEADER_SIZE = _GENERIC_HEADER.size # 12 bytes
ParseResult = dict[str, Any]
"""Return type of every chunk parser: ``{"next_offset": int, "data": ...}``."""
def read_generic_header(
data: bytes | bytearray,
offset: int,
) -> tuple[str, int, int]:
"""Read the 12-byte generic chunk header at *offset*.
Returns:
Tuple of ``(chunk_type, header_length, length_or_child_count)``.
Raises:
InsufficientDataError: If fewer than 12 bytes remain at *offset*.
CorruptHeaderError: If the chunk type bytes are not valid ASCII.
"""
end = offset + GENERIC_HEADER_SIZE
if end > len(data):
raise InsufficientDataError(offset, GENERIC_HEADER_SIZE, len(data) - offset)
raw_type, header_length, length_or_children = _GENERIC_HEADER.unpack_from(data, offset)
try:
chunk_type = raw_type.decode("ascii")
except UnicodeDecodeError as exc:
raise CorruptHeaderError(
offset,
f"chunk type bytes are not valid ASCII: {raw_type!r}",
) from exc
return chunk_type, header_length, length_or_children
+88
View File
@@ -0,0 +1,88 @@
"""Helpers for reconciling track artwork links with ArtworkDB.
Older iPod database versions can omit the MHIT ``artwork_id_ref`` field even
when the track has album art. In those databases the reliable link lives in
ArtworkDB's MHII ``songId`` field, which equals the track ``db_track_id``.
"""
from __future__ import annotations
import logging
from pathlib import Path
from typing import Any
logger = logging.getLogger(__name__)
def _artworkdb_path_from_itunesdb(itunesdb_path: str | Path) -> Path:
itunes_path = Path(itunesdb_path)
ipod_control = itunes_path.parent.parent
return ipod_control / "Artwork" / "ArtworkDB"
def _build_song_to_artwork_id(artworkdb_path: Path) -> dict[int, int]:
if not artworkdb_path.exists():
return {}
try:
from ArtworkDB_Parser.parser import parse_artworkdb
artworkdb = parse_artworkdb(str(artworkdb_path))
except Exception as exc:
logger.debug("Could not parse ArtworkDB for artwork links: %s", exc)
return {}
links: dict[int, int] = {}
for entry in artworkdb.get("mhli", []):
if not isinstance(entry, dict):
continue
try:
song_id = int(entry.get("songId") or entry.get("song_id") or 0)
img_id = int(entry.get("img_id") or 0)
except (TypeError, ValueError):
continue
if song_id and img_id:
links.setdefault(song_id, img_id)
return links
def hydrate_track_artwork_refs(
tracks: list[dict[str, Any]],
itunesdb_path: str | Path,
) -> int:
"""Fill missing ``artwork_id_ref`` values from ArtworkDB ``songId`` links.
Returns the number of tracks updated.
"""
if not tracks:
return 0
song_to_artwork_id = _build_song_to_artwork_id(
_artworkdb_path_from_itunesdb(itunesdb_path)
)
if not song_to_artwork_id:
return 0
hydrated = 0
for track in tracks:
if not isinstance(track, dict):
continue
if track.get("artwork_id_ref") not in (None, "", 0):
continue
try:
db_track_id = int(track.get("db_track_id") or track.get("db_id") or 0)
except (TypeError, ValueError):
continue
artwork_id = song_to_artwork_id.get(db_track_id)
if not artwork_id:
continue
track["artwork_id_ref"] = artwork_id
if track.get("mhii_link") in (None, "", 0):
track["mhii_link"] = artwork_id
if not track.get("artwork_count"):
track["artwork_count"] = 1
hydrated += 1
if hydrated:
logger.info("Hydrated %d track artwork refs from ArtworkDB song links", hydrated)
return hydrated
+143
View File
@@ -0,0 +1,143 @@
"""
Generic chunk dispatcher for iTunesDB chunks.
Every chunk in the iTunesDB starts with the same 12-byte generic header::
+0x00 chunk_type (4 bytes ASCII) e.g. ``mhbd``, ``mhit``
+0x04 header_length (u32 LE) bytes to end of header
+0x08 length_or_children (u32 LE) total length *or* child count
This module reads the generic header via :func:`_parsing.read_generic_header`,
then dispatches to the appropriate ``mh*_parser`` module.
Every parser returns::
{"next_offset": int, "data": dict | list}
Child-iteration helpers (:func:`parse_children`, :func:`parse_child_list`)
also live here so that the recursive ``parse_chunk parser parse_children
parse_chunk`` loop stays within one module, eliminating the circular import
that previously existed between ``_parsing`` and ``chunk_parser``.
"""
from __future__ import annotations
import logging
from typing import Any
from ._parsing import ParseResult, read_generic_header
logger = logging.getLogger(__name__)
# ── Child-iteration helpers ──────────────────────────────────────────
def parse_children(
data: bytes | bytearray,
offset: int,
child_count: int,
) -> tuple[list[dict[str, Any]], int]:
"""Parse *child_count* consecutive child chunks starting at *offset*.
Returns:
Tuple of ``(children_list, next_offset)`` where each child is
``{"chunk_type": str, "data": <parsed>}``.
"""
children: list[dict[str, Any]] = []
current = offset
for _ in range(child_count):
parsed, chunk_type = parse_chunk(data, current)
current = parsed["next_offset"]
children.append({"chunk_type": chunk_type, "data": parsed["data"]})
return children, current
def _parse_child_list(
data: bytes | bytearray,
offset: int,
header_length: int,
child_count: int,
) -> ParseResult:
"""Parse a pure-list container (mhlt, mhla, mhli, mhlp).
These chunks consist solely of a thin header followed by *child_count*
sub-chunks with no additional header fields.
Returns:
``{"next_offset": int, "data": list[...]}``
"""
children, next_offset = parse_children(data, offset + header_length, child_count)
return {"next_offset": next_offset, "data": children}
# ── Top-level dispatcher ────────────────────────────────────────────
def parse_chunk(
data: bytes | bytearray,
offset: int,
) -> tuple[dict[str, Any], str]:
"""Read the generic header at *offset* and delegate to the typed parser.
Args:
data: Full iTunesDB byte buffer.
offset: Byte position of the chunk to parse.
Returns:
Tuple of ``(result_dict, chunk_type)`` where *result_dict* contains
``"next_offset"`` and ``"data"`` keys.
"""
chunk_type, header_length, length_or_children = read_generic_header(data, offset)
match chunk_type:
case "mhbd":
from .mhbd_parser import parse_db
result = parse_db(data, offset, header_length, length_or_children)
case "mhsd":
from .mhsd_parser import parse_dataset
result = parse_dataset(data, offset, header_length, length_or_children)
# Pure-list containers — no dedicated parser needed.
case "mhlt" | "mhla" | "mhli" | "mhlp":
result = _parse_child_list(data, offset, header_length, length_or_children)
case "mhit":
from .mhit_parser import parse_track_item
result = parse_track_item(data, offset, header_length, length_or_children)
case "mhyp":
from .mhyp_parser import parse_playlist
result = parse_playlist(data, offset, header_length, length_or_children)
case "mhip":
from .mhip_parser import parse_playlist_item
result = parse_playlist_item(data, offset, header_length, length_or_children)
case "mhod":
from .mhod_parser import parse_mhod
result = parse_mhod(data, offset, header_length, length_or_children)
case "mhia":
from .mhia_parser import parse_album_item
result = parse_album_item(data, offset, header_length, length_or_children)
case "mhii":
# NOTE: shares the 'mhii' magic with ArtworkDB image items,
# but in iTunesDB context this is an artist item.
from .mhii_parser import parse_artist_item
result = parse_artist_item(data, offset, header_length, length_or_children)
case _:
logger.warning(
"Skipping unknown iTunesDB chunk type %r at offset 0x%X",
chunk_type, offset,
)
# NOTE: length_or_children may be a child count rather than
# a byte length. For unknown types we naively treat it as a
# length — the worst case is skipping too little, which the
# parent's child loop will catch on the next iteration.
return {
"next_offset": offset + length_or_children,
"data": {
"chunk_type": chunk_type,
"header": bytes(data[offset:offset + header_length]),
"body": bytes(data[offset + header_length:offset + length_or_children]),
},
}, chunk_type
return result, chunk_type
+59
View File
@@ -0,0 +1,59 @@
"""
Custom exception hierarchy for iTunesDB parsing.
All exceptions inherit from :class:`ITunesDBParseError` so callers can catch
a single base class for any parsing failure.
"""
class ITunesDBParseError(Exception):
"""Base exception for all iTunesDB parsing failures."""
class CorruptHeaderError(ITunesDBParseError):
"""Raised when a chunk header contains invalid or unrecognizable data.
Attributes:
offset: Byte offset in the data buffer where the header was found.
detail: Human-readable description of what went wrong.
"""
def __init__(self, offset: int, detail: str) -> None:
self.offset = offset
self.detail = detail
super().__init__(f"Corrupt header at offset 0x{offset:X}: {detail}")
class UnknownChunkTypeError(ITunesDBParseError):
"""Raised when an unrecognized 4-byte chunk identifier is encountered.
Attributes:
offset: Byte offset where the unknown chunk starts.
chunk_type: The 4-byte ASCII identifier that was not recognized.
"""
def __init__(self, offset: int, chunk_type: str) -> None:
self.offset = offset
self.chunk_type = chunk_type
super().__init__(
f"Unknown chunk type {chunk_type!r} at offset 0x{offset:X}"
)
class InsufficientDataError(ITunesDBParseError):
"""Raised when the data buffer is too short for the expected read.
Attributes:
offset: Byte offset where the read was attempted.
needed: Number of bytes required.
available: Number of bytes actually available.
"""
def __init__(self, offset: int, needed: int, available: int) -> None:
self.offset = offset
self.needed = needed
self.available = available
super().__init__(
f"Insufficient data at offset 0x{offset:X}: "
f"need {needed} bytes, only {available} available"
)
+140
View File
@@ -0,0 +1,140 @@
"""
iPod Library Loader standalone parser service, no GUI dependency.
Parses iTunesDB + Play Counts, inlines MHOD strings, converts timestamps
and field values, and returns a flat dict ready for consumption by any
layer (GUI, CLI, sync engine, tests).
Usage::
from iTunesDB_Parser.ipod_library import load_ipod_library
data = load_ipod_library("/Volumes/IPOD/iPod_Control/iTunes/iTunesDB")
tracks = data["mhlt"] # list[dict]
albums = data["mhla"] # list[dict]
playlists = data["mhlp"] # list[dict]
"""
import logging
import os
from iTunesDB_Shared.extraction import (
extract_datasets,
extract_mhod_strings,
extract_playlist_extras,
extract_track_extras,
)
from iTunesDB_Shared.field_base import filetype_to_string
from .parser import parse_itunesdb
logger = logging.getLogger(__name__)
def load_ipod_library(itunesdb_path: str,
merge_playcounts: bool = True) -> dict | None:
"""Parse an iTunesDB file and return normalised data.
Args:
itunesdb_path: Absolute path to the iTunesDB binary file.
merge_playcounts: If True (default), also read the sibling
``Play Counts`` file and merge deltas into the track dicts.
Returns:
A dict with keys ``mhlt``, ``mhla``, ``mhlp``, ``mhlp_podcast``,
``mhlp_smart``, ``mhsd_type_8``, etc. Returns ``None`` when the
file does not exist or cannot be parsed.
"""
if not itunesdb_path or not os.path.exists(itunesdb_path):
return None
try:
raw = parse_itunesdb(itunesdb_path)
data = extract_datasets(raw)
_inline_track_strings(data)
from .artwork_links import hydrate_track_artwork_refs
hydrate_track_artwork_refs(data.get("mhlt", []), itunesdb_path)
_inline_album_strings(data)
_inline_playlist_strings(data)
_inline_artist_strings(data)
if merge_playcounts:
_merge_play_counts(data, itunesdb_path)
# Import On-The-Go playlists from OTGPlaylistInfo files.
# These are device-created playlists stored outside the iTunesDB.
from .otg import load_otg_playlists
itunes_dir = os.path.dirname(itunesdb_path)
otg = load_otg_playlists(itunes_dir, data.get("mhlt", []))
if otg:
data.setdefault("mhlp", []).extend(otg)
return data
except Exception:
logger.error("Error parsing iTunesDB", exc_info=True)
return None
# ── Internal helpers ────────────────────────────────────────────────────────
def _inline_track_strings(data: dict) -> None:
for track in data.get("mhlt", []):
children = track.pop("children", [])
strings = extract_mhod_strings(children)
track.update(strings)
track.update(extract_track_extras(children))
# filetype u32 → ASCII
ft = track.get("filetype")
if isinstance(ft, int) and ft > 0:
track["filetype"] = filetype_to_string(ft)
# sample_rate_1 is already converted from 16.16 fixed-point to Hz
# by the read_transform (fixed_to_sample_rate) in mhit_defs.py
# sort_mhod_indicators raw bytes → list for JSON serialization
raw = track.get("sort_mhod_indicators", b"")
if isinstance(raw, (bytes, bytearray)):
track["sort_mhod_indicators"] = list(raw)
def _inline_album_strings(data: dict) -> None:
for album in data.get("mhla", []):
strings = extract_mhod_strings(album.pop("children", []))
album.update(strings)
def _inline_playlist_strings(data: dict) -> None:
for key in ("mhlp", "mhlp_podcast", "mhlp_smart"):
for pl in data.get(key, []):
mhod_children = pl.pop("mhod_children", [])
strings = extract_mhod_strings(mhod_children)
pl.update(strings)
extras = extract_playlist_extras(mhod_children)
pl.update(extras)
# Flatten MHIP children → items list.
# parse_children always returns {"chunk_type": ..., "data": {...}}.
items = []
for mhip in pl.pop("mhip_children", []):
items.append({"track_id": mhip["data"].get("track_id", 0)})
pl["items"] = items
def _inline_artist_strings(data: dict) -> None:
for artist in data.get("mhsd_type_8", []):
strings = extract_mhod_strings(artist.pop("children", []))
artist.update(strings)
def _merge_play_counts(data: dict, itunesdb_path: str) -> None:
try:
from .playcounts import merge_playcounts as _merge
from .playcounts import parse_playcounts
pc_path = os.path.join(os.path.dirname(itunesdb_path), "Play Counts")
entries = parse_playcounts(pc_path)
if entries is not None:
tracks = data.get("mhlt", [])
_merge(tracks, entries)
except Exception:
logger.debug("Play Counts merge skipped", exc_info=True)
+39
View File
@@ -0,0 +1,39 @@
"""MHBD (Database Header) parser.
The MHBD chunk is the root of the iTunesDB file. It contains global
metadata (version, hashing scheme, persistent IDs, cryptographic hashes)
followed by one or more MHSD (DataSet) children.
Binary layout (offsets relative to chunk start)::
+0x00 'mhbd' magic
+0x04 header_length
+0x08 total_length (entire file size)
+0x0C compressed flag
+0x10 database version
+0x14 child_count (number of MHSD datasets)
+0x18 db_id (u64)
+0x20 platform (u16) -- 1=Mac, 2=Windows
... (see mhbd_defs.py for complete field map)
"""
from __future__ import annotations
import iTunesDB_Shared as idb
from ._parsing import ParseResult
from .chunk_parser import parse_children
def parse_db(
data: bytes | bytearray,
offset: int,
header_length: int,
chunk_length: int,
) -> ParseResult:
"""Parse an MHBD (Database) chunk and its MHSD children."""
mhbd = idb.read_fields(data, offset, "mhbd", header_length)
mhbd["children"], _ = parse_children(
data, offset + header_length, mhbd["child_count"],
)
return {"next_offset": offset + chunk_length, "data": mhbd}
+27
View File
@@ -0,0 +1,27 @@
"""MHIA (Album Item) parser.
Each MHIA lives inside an MHLA (album list) and contains album-level
metadata (album_id, SQL ID, compilation flag) plus MHOD string children
(types 200-204) with album name, artist, etc.
"""
from __future__ import annotations
import iTunesDB_Shared as idb
from ._parsing import ParseResult
from .chunk_parser import parse_children
def parse_album_item(
data: bytes | bytearray,
offset: int,
header_length: int,
chunk_length: int,
) -> ParseResult:
"""Parse an MHIA (Album Item) chunk and its MHOD children."""
mhia = idb.read_fields(data, offset, "mhia", header_length)
mhia["children"], _ = parse_children(
data, offset + header_length, mhia["child_count"],
)
return {"next_offset": offset + chunk_length, "data": mhia}
+30
View File
@@ -0,0 +1,30 @@
"""MHII (Artist Item) parser for iTunesDB.
Each MHII lives inside an MHLI (artist list, MHSD type 8) and contains
artist-level metadata (artist_id, SQL ID) plus MHOD type-300 children
with the artist name string.
NOTE: This chunk shares the ``mhii`` magic with ArtworkDB image items,
but in the iTunesDB context it represents an artist record.
"""
from __future__ import annotations # noqa: I001
import iTunesDB_Shared as idb
from ._parsing import ParseResult
from .chunk_parser import parse_children
def parse_artist_item(
data: bytes | bytearray,
offset: int,
header_length: int,
chunk_length: int,
) -> ParseResult:
"""Parse an MHII (Artist Item) chunk and its MHOD children."""
mhii = idb.read_fields(data, offset, "mhii", header_length)
mhii["children"], _ = parse_children(
data, offset + header_length, mhii["child_count"],
)
return {"next_offset": offset + chunk_length, "data": mhii}
+27
View File
@@ -0,0 +1,27 @@
"""MHIP (Playlist Item) parser.
Each MHIP lives inside an MHYP (playlist) and references a track by
``track_id``. It may also carry MHOD type-100 children for position
information.
"""
from __future__ import annotations
import iTunesDB_Shared as idb
from ._parsing import ParseResult
from .chunk_parser import parse_children
def parse_playlist_item(
data: bytes | bytearray,
offset: int,
header_length: int,
chunk_length: int,
) -> ParseResult:
"""Parse an MHIP (Playlist Item) chunk and its MHOD children."""
mhip = idb.read_fields(data, offset, "mhip", header_length)
mhip["children"], _ = parse_children(
data, offset + header_length, mhip["child_count"],
)
return {"next_offset": offset + chunk_length, "data": mhip}
+30
View File
@@ -0,0 +1,30 @@
"""MHIT (Track Item) parser.
Parses a single track record and its MHOD string children. The MHIT
header is the largest in the iTunesDB (up to ~500 bytes in newer
database versions) and contains all numeric track metadata.
The third generic-header field is ``total_length`` (header + body).
Child count is stored inside the header at offset 0x0C.
"""
from __future__ import annotations
import iTunesDB_Shared as idb
from ._parsing import ParseResult
from .chunk_parser import parse_children
def parse_track_item(
data: bytes | bytearray,
offset: int,
header_length: int,
chunk_length: int,
) -> ParseResult:
"""Parse an MHIT (Track Item) chunk and its MHOD children."""
mhit = idb.read_fields(data, offset, "mhit", header_length)
mhit["children"], _ = parse_children(
data, offset + header_length, mhit["child_count"],
)
return {"next_offset": offset + chunk_length, "data": mhit}
+613
View File
@@ -0,0 +1,613 @@
"""MHOD (Data Object) parser.
MHODs are the most varied chunk type in the iTunesDB. They are always
leaf nodes (no sub-chunks) but have a ``type`` field at offset 0x0C that
determines how the body should be decoded.
Body layouts by type family:
- **String types** (1-14, 18-31, 33-44, 200-204, 300):
Standard sub-header at +0x18 with encoding + string_length, then
UTF-16LE or UTF-8 string data starting at +0x28.
- **Podcast URL types** (15, 16):
UTF-8 string directly after the 24-byte MHOD header (no sub-header).
- **Chapter data** (17):
Big-endian atom tree (sean chap name) after 12-byte preamble.
Contains chapter titles and start positions for audiobooks/podcasts.
- **Binary blob types** (32=video track data):
Raw binary stored as hex string for JSON round-tripping.
- **Smart playlist types** (50=SPLPref, 51=SLst rules):
Dedicated binary formats. SLst is the **only** big-endian section
in the entire iTunesDB (besides chapter data atoms).
- **Index types** (52=sorted index, 53=jump table):
Library playlist indexing data.
- **Playlist settings** (100=position/preferences, 102=binary prefs):
Context-dependent layout based on parent chunk (MHIP vs MHYP).
"""
from __future__ import annotations
import logging
import struct
from typing import Any
import iTunesDB_Shared as idb
from ._parsing import UINT16_LE, UINT32_LE, ParseResult
logger = logging.getLogger(__name__)
# ── Local layout constants ─────────────────────────────────────────
# Binary offsets not already defined in iTunesDB_Shared.mhod_defs.
# Each is derived from the layout documented in the per-type docstrings.
# MHOD52 (sorted index): sort_type(4) + count(4) + padding(40) = 48
_MHOD52_INDICES_OFFSET = 48
# MHOD53 (jump table): sort_type(4) + count(4) + padding(8) = 16
_MHOD53_ENTRIES_OFFSET = 16
_MHOD53_ENTRY_SIZE = 12 # letter(2) + pad(2) + start(4) + count(4)
# MHOD100: bodies ≤ 20 bytes are MHIP-context (position);
# larger bodies are MHYP-context (playlist display preferences).
_MHOD100_MHIP_MAX_BODY = 20
# ────────────────────────────────────────────────────────────────────
# Top-level dispatcher
# ────────────────────────────────────────────────────────────────────
def parse_mhod(
data: bytes | bytearray,
offset: int,
header_length: int,
chunk_length: int,
) -> ParseResult:
"""Parse a complete MHOD chunk, dispatching by type to the appropriate decoder."""
mhod: dict[str, Any] = idb.read_fields(data, offset, "mhod")
mhod_type: int = mhod["mhod_type"]
if mhod_type in idb.mhod_defs.NON_STRING_MHOD_TYPES:
body_offset = offset + header_length
body_length = chunk_length - header_length
mhod["data"] = _parse_nonstring_mhod(data, body_offset, body_length, mhod_type)
elif mhod_type in idb.mhod_defs.PODCAST_URL_MHOD_TYPES:
# Podcast URL types (15, 16): UTF-8/ASCII string directly after
# header, with NO sub-header.
url_length = chunk_length - header_length
if url_length > 0:
raw = data[offset + header_length:offset + header_length + url_length]
mhod["string"] = raw.decode("utf-8", errors="replace").rstrip("\x00")
else:
mhod["string"] = ""
elif mhod_type in idb.mhod_defs.CHAPTER_DATA_MHOD_TYPES:
body_offset = offset + header_length
body_length = chunk_length - header_length
mhod["data"] = _parse_chapter_data(data, body_offset, body_length)
elif mhod_type in idb.mhod_defs.BINARY_BLOB_MHOD_TYPES:
# Binary blob types (32=video track data).
blob_length = chunk_length - header_length
blob = data[offset + header_length:offset + header_length + blob_length]
mhod["string"] = blob.hex()
elif mhod_type in idb.mhod_defs.STRING_MHOD_TYPES:
_parse_string_mhod(data, offset, mhod)
else:
# Unknown MHOD type — return stub.
mhod["string"] = ""
return {"next_offset": offset + chunk_length, "data": mhod}
# ────────────────────────────────────────────────────────────────────
# String MHOD decoder
# ────────────────────────────────────────────────────────────────────
def _parse_string_mhod(
data: bytes | bytearray,
offset: int,
mhod: dict[str, Any],
) -> None:
"""Decode a standard string MHOD (sub-header at +0x18) into *mhod* in-place."""
encoding = idb.mhod_defs.mhod_string_encoding(data, offset)
string_length = idb.mhod_defs.mhod_string_length(data, offset)
mhod["unk_0x20"] = idb.mhod_defs.mhod_string_unk0x20(data, offset)
mhod["unk_0x24"] = idb.mhod_defs.mhod_string_unk0x24(data, offset)
# String data starts after 24-byte header + 16-byte sub-header.
string_start = offset + idb.mhod_defs.MHOD_STRING_DATA_OFFSET
string_data = data[string_start:string_start + string_length]
if encoding == 2:
mhod["string"] = string_data.decode("utf-8", errors="replace")
else:
# encoding 0 or 1 = UTF-16LE (most common on iPod).
mhod["string"] = string_data.decode("utf-16-le", errors="replace")
# ────────────────────────────────────────────────────────────────────
# Non-string MHOD dispatcher
# ────────────────────────────────────────────────────────────────────
def _parse_nonstring_mhod(
data: bytes | bytearray,
body_offset: int,
body_length: int,
mhod_type: int,
) -> dict[str, Any]:
"""Route non-string MHODs to their specific decoders."""
match mhod_type:
case 50:
return _parse_mhod50(data, body_offset, body_length)
case 51:
return _parse_mhod51(data, body_offset, body_length)
case 52:
return _parse_mhod52(data, body_offset, body_length)
case 53:
return _parse_mhod53(data, body_offset, body_length)
case 100:
return _parse_mhod100(data, body_offset, body_length)
case 102:
return _parse_mhod102(data, body_offset, body_length)
case _:
return {}
# ────────────────────────────────────────────────────────────────────
# MHOD Type 50 — Smart Playlist Preferences (SPLPref)
# ────────────────────────────────────────────────────────────────────
def _parse_mhod50(
data: bytes | bytearray,
body_offset: int,
body_length: int,
) -> dict[str, Any]:
"""Parse SPLPref (Smart Playlist Preferences) from MHOD type 50.
Binary layout (relative to body_offset)::
+0x00 liveUpdate (u8)
+0x01 checkRules (u8)
+0x02 checkLimits (u8)
+0x03 limitType (u8)
+0x04 limitSort (u8) + 3 padding bytes
+0x08 limitValue (u32 LE)
+0x0C matchCheckedOnly (u8) optional
+0x0D reverseSort (u8) optional
"""
if body_length < 12:
logger.warning("MHOD50 (SPLPref) body too short: %d bytes", body_length)
return {}
defs = idb.mhod_defs
result: dict[str, Any] = {
"live_update": defs.mhod_spl_live_update(data, body_offset),
"check_rules": defs.mhod_spl_check_rules(data, body_offset),
"check_limits": defs.mhod_spl_check_limits(data, body_offset),
"limit_type": defs.mhod_spl_limit_type(data, body_offset),
"limit_sort": defs.mhod_spl_limit_sort_raw(data, body_offset),
"limit_value": defs.mhod_spl_limit_value(data, body_offset),
}
if body_length >= 13:
result["match_checked_only"] = defs.mhod_spl_match_checked_only(data, body_offset)
if body_length >= 14:
result["reverse_sort"] = defs.mhod_spl_reverse_sort(data, body_offset)
return result
# ────────────────────────────────────────────────────────────────────
# MHOD Type 51 — Smart Playlist Rules (SLst)
# ────────────────────────────────────────────────────────────────────
def _parse_mhod51(
data: bytes | bytearray,
body_offset: int,
body_length: int,
) -> dict[str, Any]:
"""Parse SPLRules (Smart Playlist Rules) from MHOD type 51.
CRITICAL: The SLst blob uses BIG-ENDIAN encoding for ALL multi-byte
integers the only part of the iTunesDB that does so.
"""
if body_length < 16:
logger.warning("MHOD51 (SPLRules) body too short: %d bytes", body_length)
return {}
slst_magic = idb.mhod_defs.mhod_slst_magic(data, body_offset)
if slst_magic != b'SLst':
logger.warning("MHOD51: expected SLst magic, got %r", slst_magic)
return {}
defs = idb.mhod_defs
rule_count = defs.mhod_slst_rule_count(data, body_offset)
result: dict[str, Any] = {
"unk004": defs.mhod_slst_unk004(data, body_offset),
"rule_count": rule_count,
"conjunction": defs.mhod_slst_conjunction(data, body_offset),
}
# Parse individual rules (start after 136-byte SLst header).
rules: list[dict[str, Any]] = []
rule_offset = body_offset + defs.SLST_HEADER_SIZE
for _ in range(rule_count):
if rule_offset + defs.SPL_RULE_HEADER_SIZE > body_offset + body_length:
break
rule, rule_total_size = _parse_spl_rule(data, rule_offset)
rules.append(rule)
rule_offset += rule_total_size
result["rules"] = rules
return result
def _parse_spl_rule(
data: bytes | bytearray,
rule_offset: int,
) -> tuple[dict[str, Any], int]:
"""Parse a single SPL rule starting at *rule_offset*.
All multi-byte integers within SLst rules are BIG-ENDIAN.
Returns:
Tuple of ``(rule_dict, total_rule_size_in_bytes)``.
"""
defs = idb.mhod_defs
rule: dict[str, Any] = {}
field_id = defs.mhod_spl_rule_field(data, rule_offset)
rule["field_id"] = field_id
rule["action_id"] = defs.mhod_spl_rule_action(data, rule_offset)
data_length = defs.mhod_spl_rule_data_length(data, rule_offset)
rule["data_length"] = data_length
data_offset = rule_offset + defs.SPL_RULE_HEADER_SIZE
field_type = defs.spl_get_field_type(field_id)
if field_type == defs.SPLFT_STRING:
# SLst strings are UTF-16 BIG-endian.
if data_length > 0:
raw = data[data_offset:data_offset + data_length]
rule["string_value"] = raw.decode("utf-16-be", errors="replace")
else:
rule["string_value"] = ""
else:
# Numeric rule data (INT, DATE, BOOLEAN, PLAYLIST, BINARY_AND).
rule["from_value"] = defs.mhod_spl_rule_from_value(data, data_offset)
rule["from_date"] = defs.mhod_spl_rule_from_date(data, data_offset)
rule["from_units"] = defs.mhod_spl_rule_from_units(data, data_offset)
rule["to_value"] = defs.mhod_spl_rule_to_value(data, data_offset)
rule["to_date"] = defs.mhod_spl_rule_to_date(data, data_offset)
rule["to_units"] = defs.mhod_spl_rule_to_units(data, data_offset)
rule["unk052"] = defs.mhod_spl_rule_unk052(data, data_offset)
rule["unk056"] = defs.mhod_spl_rule_unk056(data, data_offset)
rule["unk060"] = defs.mhod_spl_rule_unk060(data, data_offset)
rule["unk064"] = defs.mhod_spl_rule_unk064(data, data_offset)
rule["unk068"] = defs.mhod_spl_rule_unk068(data, data_offset)
total_size = defs.SPL_RULE_HEADER_SIZE + data_length
return rule, total_size
# ────────────────────────────────────────────────────────────────────
# MHOD Type 52 — Library Playlist Index
# ────────────────────────────────────────────────────────────────────
def _parse_mhod52(
data: bytes | bytearray,
body_offset: int,
body_length: int,
) -> dict[str, Any]:
"""Parse library playlist sorted index from MHOD type 52.
Layout::
+0x00 sort_type (u32 LE)
+0x04 count (u32 LE)
+0x08 padding (40 bytes)
+0x30 indices (count x u32 LE) sorted track positions
"""
if body_length < 8:
logger.warning("MHOD52 (sorted index) body too short: %d bytes", body_length)
return {}
defs = idb.mhod_defs
count = defs.mhod52_count(data, body_offset)
result: dict[str, Any] = {
"sort_type": defs.mhod52_sort_type(data, body_offset),
"count": count,
}
indices_start = body_offset + _MHOD52_INDICES_OFFSET
indices: list[int] = []
for i in range(count):
pos = indices_start + i * 4
if pos + 4 <= body_offset + body_length:
indices.append(UINT32_LE.unpack_from(data, pos)[0])
result["indices"] = indices
return result
# ────────────────────────────────────────────────────────────────────
# MHOD Type 53 — Library Playlist Jump Table
# ────────────────────────────────────────────────────────────────────
def _parse_mhod53(
data: bytes | bytearray,
body_offset: int,
body_length: int,
) -> dict[str, Any]:
"""Parse library playlist jump table from MHOD type 53.
Layout::
+0x00 sort_type (u32 LE)
+0x04 count (u32 LE)
+0x08 padding (8 bytes)
+0x10 entries (count x 12 bytes each):
letter (u16 LE) + pad(2) + start(u32 LE) + count(u32 LE)
"""
if body_length < 8:
logger.warning("MHOD53 (jump table) body too short: %d bytes", body_length)
return {}
defs = idb.mhod_defs
count = defs.mhod53_count(data, body_offset)
result: dict[str, Any] = {
"sort_type": defs.mhod53_sort_type(data, body_offset),
"count": count,
}
entries_start = body_offset + _MHOD53_ENTRIES_OFFSET
entries: list[dict[str, int]] = []
for i in range(count):
pos = entries_start + i * _MHOD53_ENTRY_SIZE
if pos + _MHOD53_ENTRY_SIZE <= body_offset + body_length:
letter_code = UINT16_LE.unpack_from(data, pos)[0]
start = UINT32_LE.unpack_from(data, pos + 4)[0]
entry_count = UINT32_LE.unpack_from(data, pos + 8)[0]
entries.append({
"letter_code": letter_code,
"start": start,
"count": entry_count,
})
result["entries"] = entries
return result
# ────────────────────────────────────────────────────────────────────
# MHOD Type 100 — Playlist Position / Preferences
# ────────────────────────────────────────────────────────────────────
#
# Type 100 appears in two contexts:
# 1. As a child of MHIP: contains track position (small, <=20-byte body)
# 2. As a child of MHYP: contains playlist display preferences (large, ~624-byte body)
def _parse_mhod100(
data: bytes | bytearray,
body_offset: int,
body_length: int,
) -> dict[str, Any]:
"""Parse playlist position or preferences from MHOD type 100."""
result: dict[str, Any] = {}
if body_length <= _MHOD100_MHIP_MAX_BODY:
# MHIP context: simple position field.
if body_length >= 4:
result["position"] = idb.mhod_defs.mhod100_position(data, body_offset)
else:
# MHYP context: playlist display preferences.
result["fields"] = _scan_nonzero_fields(data, body_offset, body_length)
# Preserve raw bytes for round-trip fidelity.
result["raw_body"] = bytes(data[body_offset:body_offset + body_length])
return result
def _scan_nonzero_fields(
data: bytes | bytearray,
body_offset: int,
body_length: int,
) -> dict[str, int]:
"""Scan a binary body for all nonzero bytes, grouped into u32 values.
Returns a dict mapping hex-offset strings to integer values.
Contiguous nonzero bytes within the same 4-byte-aligned u32 are
merged into a single LE u32 entry. Isolated single bytes are
returned as-is.
"""
fields: dict[str, int] = {}
body = data[body_offset:body_offset + body_length]
visited: set[int] = set()
for i in range(len(body)):
if body[i] != 0 and i not in visited:
# Try to read as aligned u32 if within bounds.
aligned = (i // 4) * 4
if aligned + 4 <= len(body):
val = UINT32_LE.unpack_from(body, aligned)[0]
if val != 0:
fields[f"0x{aligned:03X}"] = val
visited.update(range(aligned, aligned + 4))
continue
# Fallback: single byte.
fields[f"0x{i:03X}"] = body[i]
visited.add(i)
return fields
# ────────────────────────────────────────────────────────────────────
# MHOD Type 102 — Playlist Settings (binary, post-iTunes 7)
# ────────────────────────────────────────────────────────────────────
def _parse_mhod102(
data: bytes | bytearray,
body_offset: int,
body_length: int,
) -> dict[str, Any]:
"""Parse MHOD type 102 — playlist settings (opaque binary blob)."""
return {
"fields": _scan_nonzero_fields(data, body_offset, body_length),
# Preserve raw bytes for round-trip fidelity.
"raw_body": bytes(data[body_offset:body_offset + body_length]),
}
# ────────────────────────────────────────────────────────────────────
# MHOD Type 17 — Chapter Data (big-endian atom tree)
# ────────────────────────────────────────────────────────────────────
#
# Chapter data for audiobooks and enhanced podcasts. The body
# contains a 12-byte preamble (3 × u32 LE unknown fields) followed by
# a big-endian atom tree: ``sean`` → ``chap`` × N → ``name`` + ``hedr``.
#
# This is the ONLY part of the iTunesDB (besides the SLst smart
# playlist rules) that uses big-endian encoding for its atoms.
#
# Layout (from libgpod itdb_itunesdb.c and iPodLinux wiki):
#
# Preamble (LE):
# +0x00 unk024 (u32)
# +0x04 unk028 (u32)
# +0x08 unk032 (u32)
#
# sean atom (BE):
# +0x00 total_size (u32 BE)
# +0x04 "sean" (4 bytes)
# +0x08 unknown (u32 BE, always 1)
# +0x0C child_count (u32 BE, = num_chapters + 1 for hedr)
# +0x10 unknown (u32 BE, always 0)
#
# chap atom (BE), repeated per chapter:
# +0x00 total_size (u32 BE)
# +0x04 "chap" (4 bytes)
# +0x08 startpos (u32 BE, milliseconds)
# +0x0C child_count (u32 BE, = 1 for name)
# +0x10 unknown (u32 BE, always 0)
# +0x14 name atom...
#
# name atom (BE):
# +0x00 total_size (u32 BE)
# +0x04 "name" (4 bytes)
# +0x08 unknown (u32 BE, always 1)
# +0x0C unknown (u32 BE, always 0)
# +0x10 unknown (u32 BE, always 0)
# +0x14 string_length (u16 BE, in UTF-16BE code units)
# +0x16 title (string_length × 2 bytes, UTF-16BE)
#
# hedr atom (BE, 28 bytes):
# +0x00 size=28 (u32 BE)
# +0x04 "hedr" (4 bytes)
# +0x08 unknown (u32 BE, always 1)
# +0x0C child_count=0 (u32 BE)
# +0x10 unknown (u32 BE, always 0)
# +0x14 unknown (u32 BE, always 0)
# +0x18 unknown (u32 BE, always 1)
_UINT32_BE = struct.Struct(">I")
_UINT16_BE = struct.Struct(">H")
def _parse_chapter_data(
data: bytes | bytearray,
body_offset: int,
body_length: int,
) -> dict[str, Any]:
"""Parse chapter data atom tree from MHOD type 17.
Returns a dict with:
- ``unk024``, ``unk028``, ``unk032``: preamble unknowns
- ``chapters``: list of {``startpos``: int, ``title``: str}
"""
defs = idb.mhod_defs
result: dict[str, Any] = {}
if body_length < defs.CHAPTER_PREAMBLE_SIZE:
logger.warning("MHOD17 (chapter data) too short for preamble: %d bytes", body_length)
result["chapters"] = []
return result
# Read 12-byte preamble (little-endian, like the rest of iTunesDB).
result["unk024"] = UINT32_LE.unpack_from(data, body_offset)[0]
result["unk028"] = UINT32_LE.unpack_from(data, body_offset + 4)[0]
result["unk032"] = UINT32_LE.unpack_from(data, body_offset + 8)[0]
seek = body_offset + defs.CHAPTER_PREAMBLE_SIZE
end = body_offset + body_length
# Check for "sean" atom.
if seek + 20 > end:
result["chapters"] = []
return result
sean_size = _UINT32_BE.unpack_from(data, seek)[0]
if sean_size < 20 or seek + sean_size > end:
logger.warning("Chapter data: invalid 'sean' atom size: %d", sean_size)
result["chapters"] = []
return result
sean_magic = data[seek + 4:seek + 8]
if sean_magic != defs.SEAN_ATOM:
logger.warning("Chapter data: expected 'sean' atom, got %r", sean_magic)
result["chapters"] = []
return result
num_children = _UINT32_BE.unpack_from(data, seek + 12)[0]
num_chapters = max(0, num_children - 1) # subtract 1 for hedr
seek += 20 # skip sean header
chapters: list[dict[str, Any]] = []
for _ in range(num_chapters):
if seek + 20 > end:
break
chap_magic = data[seek + 4:seek + 8]
if chap_magic != defs.CHAP_ATOM:
break # unexpected atom, stop parsing
chap_size = _UINT32_BE.unpack_from(data, seek)[0]
startpos = _UINT32_BE.unpack_from(data, seek + 8)[0]
children = _UINT32_BE.unpack_from(data, seek + 12)[0]
child_seek = seek + 20
title = ""
for _ in range(children):
if child_seek + 22 > end:
break
child_size = _UINT32_BE.unpack_from(data, child_seek)[0]
child_magic = data[child_seek + 4:child_seek + 8]
if child_magic == defs.NAME_ATOM:
str_len = _UINT16_BE.unpack_from(data, child_seek + 20)[0]
str_start = child_seek + 22
str_end = str_start + str_len * 2
if str_end <= end:
title = data[str_start:str_end].decode("utf-16-be", errors="replace")
child_seek += child_size
chapters.append({"startpos": startpos, "title": title})
seek += chap_size
# Skip hedr atom if present.
if seek + 8 <= end:
hedr_magic = data[seek + 4:seek + 8]
if hedr_magic == defs.HEDR_ATOM:
hedr_size = _UINT32_BE.unpack_from(data, seek)[0]
seek += hedr_size
result["chapters"] = chapters
return result
+30
View File
@@ -0,0 +1,30 @@
"""MHSD (DataSet) parser.
An MHSD contains exactly one child chunk whose type is determined by the
dataset type field at offset 0x0C (see ``constants.chunk_type_map``).
Dataset types: 1=TrackList, 2=PlaylistList, 3=PodcastList, 4=AlbumList,
5=SmartPlaylistList, 6/10=empty stubs, 8=ArtistList, 9=Genius CUID.
"""
from __future__ import annotations
from typing import Any
import iTunesDB_Shared as idb
from ._parsing import ParseResult
from .chunk_parser import parse_children
def parse_dataset(
data: bytes | bytearray,
offset: int,
header_length: int,
chunk_length: int,
) -> ParseResult:
"""Parse an MHSD (DataSet) chunk and its single child."""
mhsd: dict[str, Any] = idb.read_fields(data, offset, "mhsd", header_length)
# MHSD always has exactly one child.
mhsd["children"], _ = parse_children(data, offset + header_length, 1)
return {"next_offset": offset + chunk_length, "data": mhsd}
+34
View File
@@ -0,0 +1,34 @@
"""MHYP (Playlist) parser.
An MHYP represents a single playlist. Its children are split into two
groups parsed sequentially: MHOD metadata objects first, then MHIP
playlist-item entries. The counts are stored separately in the header.
"""
from __future__ import annotations
import iTunesDB_Shared as idb
from ._parsing import ParseResult
from .chunk_parser import parse_children
def parse_playlist(
data: bytes | bytearray,
offset: int,
header_length: int,
chunk_length: int,
) -> ParseResult:
"""Parse an MHYP (Playlist) chunk with MHOD + MHIP child groups."""
mhyp = idb.read_fields(data, offset, "mhyp", header_length)
# MHODs come first, then MHIPs — parsed sequentially with shared offset.
body_start = offset + header_length
mhyp["mhod_children"], mhip_start = parse_children(
data, body_start, mhyp["mhod_child_count"],
)
mhyp["mhip_children"], _ = parse_children(
data, mhip_start, mhyp["mhip_child_count"],
)
return {"next_offset": offset + chunk_length, "data": mhyp}
+187
View File
@@ -0,0 +1,187 @@
"""On-The-Go (OTG) playlist parser for iPod devices.
The iPod firmware stores device-created playlists in a separate MHPO binary
file (``OTGPlaylistInfo``) rather than in the iTunesDB. iTunes and libgpod
both read these files on connect, import the playlists into the database as
regular MHYP entries, and then delete the source file.
Reference: libgpod ``process_OTG_file`` / ``read_OTG_playlists`` in ``src/itdb_itunesdb.c``.
MHPO binary layout (little-endian; big-endian devices use magic ``ohpm``):
+0x00 magic 4 B "mhpo" (LE) or "ohpm" (BE)
+0x04 header_len u32 size of the header block always 0x14 (20 B)
+0x08 entry_len u32 size of each track entry always 0x04 (4 B)
+0x0C entry_num u32 number of track entries
+0x10 (reserved) 4 B unknown; ignored
+0x14 entries entry_num x entry_len bytes
each entry: u32 = 0-based index into the iTunesDB track list (mhlt)
File naming (all in ``iPod_Control/iTunes/``):
OTGPlaylistInfo first saved OTG playlist
OTGPlaylistInfo_1 second saved OTG playlist
OTGPlaylistInfo_2 third, etc.
The PC sync manager deletes only the base ``OTGPlaylistInfo`` file after
writing the database. The iPod firmware removes the numbered variants itself
once it has processed the updated database.
(libgpod comment: "the iPod will remove the remaining files".)
"""
from __future__ import annotations
import hashlib
import logging
import os
import struct
logger = logging.getLogger(__name__)
def load_otg_playlists(itunes_dir: str, track_list: list) -> list[dict]:
"""Parse OTGPlaylistInfo files and return them as regular playlist dicts.
Each returned dict has the same shape as a playlist produced by
``_inline_playlist_strings`` in ``ipod_library.py``:
``Title`` "On-The-Go 1", "On-The-Go 2",
``items`` list of ``{"track_id": <sequential-id>}`` dicts
``playlist_id`` stable 64-bit ID derived from the file's MD5 so that
repeated rescans without an intervening write don't
introduce duplicates (same file same ID)
Args:
itunes_dir: Path to the ``iPod_Control/iTunes`` directory.
track_list: Ordered list of track dicts as returned by the iTunesDB
parser (``data["mhlt"]``). Entry indices in the MHPO
file are 0-based positions into this list.
"""
paths = _collect_otg_paths(itunes_dir)
result: list[dict] = []
for pl_num, path_str in enumerate(paths, 1):
playlist = _parse_one_otg_file(path_str, pl_num, track_list)
if playlist is not None:
result.append(playlist)
return result
def delete_otg_files(itunes_dir: str) -> None:
"""Delete the base OTGPlaylistInfo file after a successful database write.
Matches libgpod's ``itdb_rename_files()`` behaviour: only the base
``OTGPlaylistInfo`` file is removed by the PC sync manager. The iPod
firmware removes the numbered variants (``OTGPlaylistInfo_1``, ``_2``, )
after it processes the new database.
"""
base = os.path.join(itunes_dir, "OTGPlaylistInfo")
if not os.path.exists(base):
return
try:
os.unlink(base)
logger.info("Deleted OTGPlaylistInfo")
except OSError as exc:
logger.warning("Could not delete OTGPlaylistInfo: %s", exc)
# ── Internal helpers ─────────────────────────────────────────────────────────
def _collect_otg_paths(itunes_dir: str) -> list[str]:
"""Return the ordered list of OTGPlaylistInfo paths that exist.
Mirrors libgpod's iteration: start with the base file; if it exists,
also collect ``_1``, ``_2``, stopping at the first missing numbered
file. If the base file is absent, return an empty list (libgpod comment:
"only parse if OTGPlaylistInfo exists").
"""
base = os.path.join(itunes_dir, "OTGPlaylistInfo")
if not os.path.exists(base):
return []
paths = [base]
for i in range(1, 20):
p = os.path.join(itunes_dir, f"OTGPlaylistInfo_{i}")
if not os.path.exists(p):
break
paths.append(p)
return paths
def _parse_one_otg_file(
path_str: str,
pl_num: int,
track_list: list,
) -> dict | None:
"""Parse a single MHPO file and return a playlist dict, or None on failure."""
try:
with open(path_str, "rb") as fh:
raw = fh.read()
except OSError as exc:
logger.warning("OTG: could not read %s: %s", path_str, exc)
return None
if len(raw) < 0x14:
logger.warning("OTG: %s too short (%d B)", os.path.basename(path_str), len(raw))
return None
magic = raw[0:4]
if magic == b"mhpo":
fmt = "<I"
elif magic == b"ohpm":
fmt = ">I"
else:
logger.warning("OTG: %s has unrecognised magic %r — skipping",
os.path.basename(path_str), magic)
return None
header_len = struct.unpack_from(fmt, raw, 4)[0]
entry_len = struct.unpack_from(fmt, raw, 8)[0]
entry_num = struct.unpack_from(fmt, raw, 12)[0]
if header_len < 0x14:
logger.warning("OTG: %s header_len %d < 20 — skipping",
os.path.basename(path_str), header_len)
return None
if entry_len < 4:
logger.warning("OTG: %s entry_len %d < 4 — skipping",
os.path.basename(path_str), entry_len)
return None
items: list[dict] = []
for i in range(entry_num):
offset = header_len + entry_len * i
if offset + 4 > len(raw):
logger.warning("OTG: %s entry %d extends past EOF — truncating",
os.path.basename(path_str), i)
break
track_index = struct.unpack_from(fmt, raw, offset)[0]
if track_index >= len(track_list):
logger.warning(
"OTG: %s entry %d references track index %d but track list "
"has only %d entries — skipping entry",
os.path.basename(path_str), i, track_index, len(track_list),
)
continue
tid = track_list[track_index].get("track_id", 0)
if tid:
items.append({"track_id": tid})
# Consistent with libgpod: don't create a playlist for an empty file.
if not items:
return None
# Stable playlist_id derived from file content so that re-scanning the
# same OTG file before a sync produces the same ID each time, preventing
# duplicates in the deduplication step of read_existing_database.
playlist_id = int.from_bytes(hashlib.md5(raw).digest()[:8], "little")
name = f"On-The-Go {pl_num}"
logger.info("OTG: imported '%s' (%d tracks) from %s",
name, len(items), os.path.basename(path_str))
return {
"Title": name,
"items": items,
"playlist_id": playlist_id,
}
+112
View File
@@ -0,0 +1,112 @@
"""
iTunesDB / iTunesCDB entry-point parser.
This module provides the public parsing API for Apple's proprietary
iTunesDB binary database format (and its zlib-compressed variant,
iTunesCDB). It accepts either a file path or a file-like object and
returns a nested dict tree representing the full database hierarchy.
Typical usage::
from iTunesDB_Parser import parse_itunesdb
db = parse_itunesdb("/media/ipod/iPod_Control/iTunes/iTunesDB")
"""
from __future__ import annotations
import logging
import os
import zlib
from typing import Any, BinaryIO
from ._parsing import UINT32_LE
from .exceptions import CorruptHeaderError
logger = logging.getLogger(__name__)
# Recognized mhbd magic at file offset 0.
_MHBD_MAGIC = b"mhbd"
# Minimum length to contain the mhbd generic header (magic + header_len + total_len + compressed).
_MIN_MHBD_HEADER = 16
# iTunesCDB compressed-flag value (mhbd offset 0x0C).
_COMPRESSED_DB_FLAG = 0x02
def decompress_itunescdb(data: bytes | bytearray) -> bytes | bytearray:
"""Transparently decompress an iTunesCDB into a standard iTunesDB stream.
If *data* is already an uncompressed iTunesDB (or is too short / has the
wrong magic), it is returned as-is.
Detection logic: the mhbd ``compressed`` field at offset 0x0C is ``2`` for
compressed-DB-capable devices, and the payload after the header is a zlib
stream.
Args:
data: Raw bytes of an iTunesDB or iTunesCDB file.
Returns:
Decompressed iTunesDB byte stream (original header preserved).
"""
if len(data) < _MIN_MHBD_HEADER or data[:4] != _MHBD_MAGIC:
return data
header_length = UINT32_LE.unpack_from(data, 0x04)[0]
compressed_flag = UINT32_LE.unpack_from(data, 0x0C)[0]
if compressed_flag != _COMPRESSED_DB_FLAG:
return data
try:
decompressed = zlib.decompress(data[header_length:])
except zlib.error:
return data # not actually compressed — return as-is
# Reconstruct: original (unmodified) header + decompressed children.
# Header's total_length (offset 8) and compression flag are preserved
# as-is. MHBD children are parsed by child_count so the stale
# total_length is harmless.
logger.debug("iTunesCDB decompressed: %d -> %d payload bytes",
len(data) - header_length, len(decompressed))
return data[:header_length] + decompressed
def parse_itunesdb(file: str | os.PathLike[str] | BinaryIO) -> dict[str, Any]:
"""Parse an iTunesDB (or iTunesCDB) file into a nested dict tree.
Args:
file: A filesystem path (``str`` or ``os.PathLike``) or an open
binary file-like object positioned at the start of the data.
Returns:
Dict representation of the mhbd root chunk and all children.
Raises:
TypeError: If *file* is not a path or file-like object.
ITunesDBParseError: If the binary data cannot be parsed.
OSError: If a file path cannot be read.
"""
from .chunk_parser import parse_chunk
if isinstance(file, (str, os.PathLike)):
with open(file, "rb") as fh:
data: bytes | bytearray = fh.read()
elif hasattr(file, "read"):
data = file.read()
else:
raise TypeError(
f"file must be a path (str/PathLike) or a file-like object, "
f"got {type(file).__name__}"
)
if not data:
raise CorruptHeaderError(0, "empty file")
# Transparently handle iTunesCDB (compressed database)
data = decompress_itunescdb(data)
parsed, _chunk_type = parse_chunk(data, 0)
return parsed["data"]
+260
View File
@@ -0,0 +1,260 @@
"""
Play Counts file parser for iPod.
The iPod firmware does NOT modify the iTunesDB directly. Instead it creates
a separate binary file at ``/iPod_Control/iTunes/Play Counts`` that records
per-track deltas (play count, skip count, rating, timestamps) accumulated
since the last sync.
File layout (iTunes 7+ / entry_length 0x1C):
Header (``mhdp``)
0x00 4B magic "mhdp"
0x04 4B header_length
0x08 4B entry_length
0x0C 4B entry_count
0x10 padding header_length
Per-entry (28 bytes for entry_length == 0x1C)
0x00 4B play_count
0x04 4B last_played (Mac)
0x08 4B bookmark_time
0x0C 4B rating (0-100)
0x10 4B unk16 / podcast
0x14 4B skip_count
0x18 4B last_skipped (Mac)
Entries are ordered 1:1 with tracks in the mhlt (matched by index, **not**
by track ID). After a sync tool reads this file and folds the deltas into
the iTunesDB, the file must be **deleted** so the iPod creates a fresh one.
Reference: libgpod ``itdb_itunesdb.c`` ``playcounts_read()``.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from pathlib import Path
from iTunesDB_Shared.field_base import MAC_EPOCH_OFFSET
from ._parsing import UINT32_LE
logger = logging.getLogger(__name__)
@dataclass(slots=True)
class PlayCountEntry:
"""Delta values for a single track from the Play Counts file."""
play_count: int = 0
last_played_mac: int = 0 # Mac epoch timestamp (0 = not played)
bookmark_time: int = 0
rating: int = -1 # -1 = no change; 0-100 = new rating
skip_count: int = 0
last_skipped_mac: int = 0 # Mac epoch timestamp (0 = not skipped)
# Convenience: is there any delta data in this entry?
@property
def has_data(self) -> bool:
return (
self.play_count > 0
or self.skip_count > 0
or self.rating >= 0
)
@property
def last_played_unix(self) -> int:
"""Last-played as Unix timestamp (0 if never played)."""
if self.last_played_mac == 0:
return 0
return self.last_played_mac - MAC_EPOCH_OFFSET
@property
def last_skipped_unix(self) -> int:
"""Last-skipped as Unix timestamp (0 if never skipped)."""
if self.last_skipped_mac == 0:
return 0
return self.last_skipped_mac - MAC_EPOCH_OFFSET
def parse_playcounts(path: str | Path) -> list[PlayCountEntry] | None:
"""
Parse an iPod Play Counts file.
Args:
path: Path to the ``Play Counts`` file.
Returns:
List of :class:`PlayCountEntry` (one per track, ordered by mhlt
index), or ``None`` if the file doesn't exist or can't be parsed.
"""
path = Path(path)
if not path.exists():
logger.debug("No Play Counts file at %s", path)
return None
try:
data = path.read_bytes()
except OSError as exc:
logger.warning("Could not read Play Counts file: %s", exc)
return None
if len(data) < 16:
logger.warning("Play Counts file too small (%d bytes)", len(data))
return None
magic = data[0:4]
if magic != b"mhdp":
logger.warning("Play Counts file bad magic: %r (expected b'mhdp')", magic)
return None
header_len = UINT32_LE.unpack_from(data, 4)[0]
entry_len = UINT32_LE.unpack_from(data, 8)[0]
entry_count = UINT32_LE.unpack_from(data, 12)[0]
expected_size = header_len + entry_len * entry_count
if len(data) < expected_size:
logger.warning(
"Play Counts file truncated: %d bytes < expected %d",
len(data), expected_size,
)
return None
logger.info(
"Play Counts: header=%d, entry_len=%d, entries=%d",
header_len, entry_len, entry_count,
)
entries: list[PlayCountEntry] = []
for i in range(entry_count):
offset = header_len + i * entry_len
entry = PlayCountEntry()
# Minimum fields (always present)
entry.play_count = UINT32_LE.unpack_from(data, offset)[0]
if entry_len >= 8:
entry.last_played_mac = UINT32_LE.unpack_from(data, offset + 4)[0]
if entry_len >= 12:
entry.bookmark_time = UINT32_LE.unpack_from(data, offset + 8)[0]
if entry_len >= 16:
raw_rating = UINT32_LE.unpack_from(data, offset + 12)[0]
# Convention: rating=0 in the Play Counts file means "no change"
# when the track had no user interaction. The iPod firmware
# initialises all entries to zero. We treat 0 as "unchanged"
# to avoid accidentally clearing ratings set on the PC.
#
# Ratings 20-100 (1-5 stars) are genuine user-set values.
# A user *removing* a rating on the iPod is indistinguishable
# from "no interaction" — this is a known limitation shared
# with libgpod (which checks ``rating != NO_PLAYCOUNT (-1)``
# but the firmware never writes -1).
if raw_rating > 0:
entry.rating = raw_rating
# else: stays -1 (no change)
# entry_len >= 20: unk16 / podcast flag — skipped
if entry_len >= 24:
entry.skip_count = UINT32_LE.unpack_from(data, offset + 20)[0]
if entry_len >= 28:
entry.last_skipped_mac = UINT32_LE.unpack_from(data, offset + 24)[0]
entries.append(entry)
active = sum(1 for e in entries if e.has_data)
logger.info("Play Counts: %d / %d entries have activity", active, entry_count)
return entries
def merge_playcounts(
tracks: list[dict],
entries: list[PlayCountEntry],
) -> None:
"""
Fold Play Counts deltas into parsed track dicts **in place**.
After calling this:
- ``track["play_count_1"]`` is the **new cumulative** play count
- ``track["skip_count"]`` is the **new cumulative** skip count
- ``track["recent_playcount"]`` is the delta from this session
- ``track["recent_skipcount"]`` is the delta from this session
- ``track["rating"]`` may be updated if the user rated on the iPod
- ``track["last_played"]`` / ``track["last_skipped"]`` may be updated
This mirrors libgpod's ``get_mhit()`` merge logic.
"""
count = min(len(tracks), len(entries))
if len(tracks) != len(entries):
logger.warning(
"Track count (%d) != Play Counts entry count (%d); "
"merging first %d",
len(tracks), len(entries), count,
)
merged_plays = 0
merged_skips = 0
merged_ratings = 0
for i in range(count):
track = tracks[i]
entry = entries[i]
# --- Play count (additive) ---
track["recent_playcount"] = entry.play_count
track["play_count_1"] = track.get("play_count_1", 0) + entry.play_count
if entry.play_count > 0:
merged_plays += 1
# --- Skip count (additive) ---
track["recent_skipcount"] = entry.skip_count
track["skip_count"] = track.get("skip_count", 0) + entry.skip_count
if entry.skip_count > 0:
merged_skips += 1
# --- Rating (override if changed) ---
if entry.rating >= 0: # -1 = no change
old_rating = track.get("rating", 0)
if old_rating != entry.rating:
track["app_rating"] = old_rating # backup (libgpod convention)
track["rating"] = entry.rating
merged_ratings += 1
# --- Bookmark (override — iPod always has the latest position) ---
if entry.bookmark_time > 0:
track["bookmark_time"] = entry.bookmark_time
# --- Timestamps (use more-recent value) ---
# track["last_played"] is a Unix timestamp (converted from Mac
# epoch during iTunesDB parsing). entry.last_played_mac is raw
# Mac epoch. Use the .last_played_unix property to compare in
# the same unit and avoid double-conversion downstream.
if entry.last_played_mac > 0:
unix_ts = entry.last_played_unix
if unix_ts > track.get("last_played", 0):
track["last_played"] = unix_ts
if entry.last_skipped_mac > 0:
unix_ts = entry.last_skipped_unix
if unix_ts > track.get("last_skipped", 0):
track["last_skipped"] = unix_ts
# Tracks beyond the Play Counts entries get zero deltas
for i in range(count, len(tracks)):
tracks[i]["recent_playcount"] = 0
tracks[i]["recent_skipcount"] = 0
logger.info(
"Merged Play Counts: %d plays, %d skips, %d ratings across %d tracks",
merged_plays, merged_skips, merged_ratings, count,
)
+32
View File
@@ -0,0 +1,32 @@
from . import field_base as _fb
from .constants import * # noqa: F401, F403
from .extraction import * # noqa: F401, F403
from .field_base import * # noqa: F401, F403
from .mhbd_defs import * # noqa: F401, F403
from .mhbd_defs import MHBD_FIELDS as _mhbd
from .mhia_defs import * # noqa: F401, F403
from .mhia_defs import MHIA_FIELDS as _mhia
from .mhii_defs import * # noqa: F401, F403
from .mhii_defs import MHII_FIELDS as _mhii
from .mhip_defs import * # noqa: F401, F403
from .mhip_defs import MHIP_FIELDS as _mhip
from .mhit_defs import * # noqa: F401, F403
from .mhit_defs import MHIT_FIELDS as _mhit
from .mhod_defs import * # noqa: F401, F403
from .mhod_defs import MHOD_FIELDS as _mhod
from .mhsd_defs import * # noqa: F401, F403
from .mhsd_defs import MHSD_FIELDS as _mhsd
from .mhyp_defs import * # noqa: F401, F403
from .mhyp_defs import MHYP_FIELDS as _mhyp
# ── Build FIELD_REGISTRY from per-chunk defs ────────────────────────
_fb.FIELD_REGISTRY.update({
"mhbd": _mhbd,
"mhit": _mhit,
"mhsd": _mhsd,
"mhia": _mhia,
"mhii": _mhii,
"mhip": _mhip,
"mhyp": _mhyp,
"mhod": _mhod,
})
+93
View File
@@ -0,0 +1,93 @@
from __future__ import annotations
from collections.abc import Callable, Iterable, Mapping
from dataclasses import dataclass
from typing import Generic, TypeVar
T = TypeVar("T")
def _clean_text(value: object | None) -> str | None:
if value is None:
return None
text = str(value).strip()
return text or None
@dataclass(frozen=True)
class AlbumIdentity:
album: str | None
album_artist: str | None
artist: str | None
show_name: str | None
def album_identity_from_track(track: object) -> AlbumIdentity:
return AlbumIdentity(
album=_clean_text(getattr(track, "album", None)),
album_artist=_clean_text(getattr(track, "album_artist", None)),
artist=_clean_text(getattr(track, "artist", None)),
show_name=_clean_text(getattr(track, "show_name", None)),
)
def album_identity_from_mapping(track: Mapping[str, object]) -> AlbumIdentity:
return AlbumIdentity(
album=_clean_text(track.get("Album") or track.get("album")),
album_artist=_clean_text(
track.get("Album Artist") or track.get("album_artist")
),
artist=_clean_text(track.get("Artist") or track.get("artist")),
show_name=_clean_text(
track.get("Show")
or track.get("Show Name")
or track.get("TV Show")
or track.get("show_name")
),
)
def albums_match(left: AlbumIdentity, right: AlbumIdentity) -> bool:
"""Match albums using libgpod's album equality rules."""
if left.show_name != right.show_name:
return False
if left.album != right.album:
return False
if left.album_artist and right.album_artist:
return left.album_artist == right.album_artist
return left.artist == right.artist
@dataclass
class AlbumGroup(Generic[T]):
identity: AlbumIdentity
tracks: list[T]
def group_tracks_by_album_identity(
tracks: Iterable[T],
identity_fn: Callable[[T], AlbumIdentity],
) -> list[AlbumGroup]:
"""Group tracks into albums using libgpod-compatible matching."""
groups: list[AlbumGroup] = []
buckets: dict[tuple[str, str], list[int]] = {}
for track in tracks:
identity = identity_fn(track)
bucket = (identity.album or "", identity.show_name or "")
candidate_idxs = buckets.get(bucket, [])
match_idx = None
for idx in candidate_idxs:
if albums_match(identity, groups[idx].identity):
match_idx = idx
break
if match_idx is None:
match_idx = len(groups)
groups.append(AlbumGroup(identity=identity, tracks=[]))
buckets.setdefault(bucket, []).append(match_idx)
groups[match_idx].tracks.append(track)
return groups
+378
View File
@@ -0,0 +1,378 @@
"""
iTunesDB constants chunk identifiers, version maps, MHOD type definitions,
media type bitmask, and playlist sort order values.
Cross-referenced against:
- iPodLinux wiki: https://web.archive.org/web/20081006030946/http://ipodlinux.org/wiki/ITunesDB
- libgpod itdb_itunesdb.c, itdb.h
"""
# maps the id used in mhsd to the proper header marker
chunk_type_map = {
1: "mhlt", # Track list (contains MHIT children)
2: "mhlp", # Playlist list (contains MHYP children) — regular playlists
3: "mhlp_podcast", # Podcast list (same MHLP format, different dataset)
# NOTE: Type 3 MHSD MUST come between type 1 and type 2
# for the iPod to list podcasts correctly.
4: "mhla", # Album list (iTunes 7.1+; contains MHIA children)
5: "mhlp_smart", # Smart playlist list (iTunes 7.3+; contains MHYP children)
# Types 610 were added in iTunes 9+ for Genius and other features.
# Their child chunk reuses the 'mhli' magic (same as ArtworkDB's image
# list, but here it is a generic item list — different semantics).
# We skip their contents but must recognise them to avoid crashing.
6: "mhsd_type_6", # Empty mhlt stub (purpose unknown, written by libgpod/iTunes)
7: "mhsd_type_7", # (reserved, rarely seen)
8: "mhsd_type_8", # Artist list (mhli with mhii children, MHOD type 300)
9: "mhsd_type_9", # Genius Chill list
10: "mhsd_type_10", # Empty mhlt stub (purpose unknown, written by libgpod/iTunes)
}
# maps the database version to an iTunes version
version_map = {
0x01: "iTunes 1.0",
0x02: "iTunes 2.0",
0x03: "iTunes 3.0",
0x04: "iTunes 4.0",
0x05: "iTunes 4.0.1",
0x06: "iTunes 4.1",
0x07: "iTunes 4.1.1",
0x08: "iTunes 4.1.2",
0x09: "iTunes 4.2",
0x0a: "iTunes 4.5",
0x0b: "iTunes 4.7",
0x0c: "iTunes 4.71/4.8",
0x0d: "iTunes 4.9",
0x0e: "iTunes 5",
0x0f: "iTunes 6",
0x10: "iTunes 6.0.1",
0x11: "iTunes 6.0.2-6.0.4",
0x12: "iTunes 6.0.5",
0x13: "iTunes 7.0",
0x14: "iTunes 7.1",
0x15: "iTunes 7.2",
0x16: "Unknown (0x16)",
0x17: "iTunes 7.3.0",
0x18: "iTunes 7.3.1-7.3.2",
0x19: "iTunes 7.4",
0x1a: "iTunes 7.4.1",
0x1b: "iTunes 7.4.2",
0x1c: "iTunes 7.5",
0x1d: "iTunes 7.6",
0x1e: "iTunes 7.7",
0x1f: "iTunes 8.0",
0x20: "iTunes 8.0.1",
0x21: "iTunes 8.0.2",
0x22: "iTunes 8.1",
0x23: "iTunes 8.1.1",
0x24: "iTunes 8.2",
0x25: "iTunes 8.2.1",
0x26: "iTunes 9.0",
0x27: "iTunes 9.0.1",
0x28: "iTunes 9.0.2",
0x29: "iTunes 9.0.3",
0x2a: "iTunes 9.1",
0x2b: "iTunes 9.1.1",
0x2c: "iTunes 9.2",
0x2d: "iTunes 9.2.1",
0x30: "iTunes 9.2+",
# Extended versions for newer databases
0x40: "iTunes 10.x",
0x50: "iTunes 11.x",
0x60: "iTunes 12.x",
0x70: "iTunes 12.5+",
0x75: "iTunes 12.9+",
}
def get_version_name(version_hex: int | str) -> str:
"""
Get iTunes version name from database version number.
Args:
version_hex: Version as int (0x19) or hex string ('0x19')
Returns:
Human-readable version string
"""
if isinstance(version_hex, str):
# Remove '0x' prefix if present and convert
version_hex = int(version_hex, 16) if version_hex.startswith('0x') else int(version_hex)
if version_hex in version_map:
return version_map[version_hex]
# If not exact match, find closest lower version
lower_versions = [v for v in version_map if v <= version_hex]
if lower_versions:
closest = max(lower_versions)
return f"{version_map[closest]} (or newer)"
return f"Unknown (version {hex(version_hex)})"
# maps the chunk header marker to a readable name
# The identifiers read backwards conceptually — the convention is:
# mhbd = DataBase Header Marker mhsd = DataSet Header Marker
# mhlt = Track List Header Marker mhit = Track Item Header Marker
# mhlp = Playlist List Header Marker mhla = Album List Header Marker
# mhyp = plaYlist Header Marker mhip = playlist Item Header Marker
# mhia = album Item Header Marker mhod = Data Object Header Marker
identifier_readable_map = {
"mhbd": "Database",
"mhsd": "Dataset",
"mhlt": "Track List",
"mhlp": "Playlist or Podcast List",
"mhla": "Album List",
"mhli": "Artist List",
"mhlp_smart": "Smart Playlist List",
"mhia": "Album Item",
"mhii": "Artist Item",
"mhit": "Track Item",
"mhyp": "Playlist",
"mhod": "Data Object",
"mhip": "Playlist Item",
}
# maps the mhod type to its readable name
#
# Types 1-14: Track string MHODs (standard sub-header at offset 24)
# Types 15-16: Podcast URL MHODs (UTF-8 string at offset 24, NO sub-header)
# Type 17: Chapter data (big-endian atom-based binary blob)
# Types 18-31: Track string MHODs (standard sub-header)
# Type 32: Unknown binary data for video tracks (not a string!)
# Types 33-44: Track string MHODs (standard sub-header)
# Type 50: Smart playlist preferences (SPLPref binary)
# Type 51: Smart playlist rules (SLst — BIG-endian binary)
# Type 52: Library playlist sorted index (binary)
# Type 53: Library playlist jump table (binary)
# Type 100: Playlist column prefs (MHYP child) or position (MHIP child)
# Type 102: Playlist settings (post-iTunes 7, binary blob)
# Types 200-204: Album item string MHODs (standard sub-header)
mhod_type_map = {
1: "Title",
2: "Location",
3: "Album",
4: "Artist",
5: "Genre",
6: "Filetype",
7: "eq_setting",
8: "Comment",
9: "Category",
10: "Lyrics",
12: "Composer",
13: "Grouping",
14: "Description Text",
15: "Podcast Enclosure URL",
16: "Podcast RSS URL",
17: "Chapter Data",
18: "Subtitle",
19: "Show",
20: "Episode",
21: "TV Network",
22: "Album Artist",
23: "Sort Artist",
24: "Track Keywords",
25: "Show Locale",
26: "iTunes Store Asset Info",
27: "Sort Title",
28: "Sort Album",
29: "Sort Album Artist",
30: "Sort Composer",
31: "Sort Show",
32: "Unknown for Video Track",
33: "Unknown (33)",
34: "Unknown (34)",
35: "Unknown (35)",
36: "Unknown (36)",
37: "Content Provider",
38: "Unknown (38)",
39: "Copyright",
40: "Unknown (40)",
41: "Unknown (41)",
42: "Encoding Quality Descriptor",
43: "Purchase Account",
44: "Purchaser Name",
50: "Smart Playlist Data",
51: "Smart Playlist Rules",
52: "Library Playlist Index",
53: "Library Playlist Jump Table",
100: "Column Size or Playlist Order",
102: "Playlist Settings (binary)",
200: "Album (Used by Album Item)",
201: "Artist (Used by Album Item)",
202: "Sort Artist (Used by Album Item)",
203: "Podcast URL (Used by Album Item)",
204: "Show (Used by Album Item)",
# Types 300+: Artist item string MHODs (MHSD type 8)
300: "Artist (Used by Artist Item)",
}
# ============================================================
# Media Type bitmask values (MHIT offset 208 / 0xD0)
# From libgpod ItdbMediatype enum and iPodLinux wiki.
# ============================================================
MEDIA_TYPE_MAP = {
0x00000000: "Audio/Video", # shows in both audio and video menus
0x00000001: "Audio",
0x00000002: "Video", # Movie
0x00000004: "Podcast",
0x00000006: "Video Podcast",
0x00000008: "Audiobook",
0x00000020: "Music Video",
0x00000040: "TV Show",
0x00000060: "TV Show (alt)",
0x00000100: "Ringtone", # libgpod ITDB_MEDIATYPE_RINGTONE (1 << 8)
0x00000200: "Rental", # iTunes rental movie
0x00004000: "Ringtone (alt)", # Alternate ringtone value (some firmware)
0x00040000: "iTunes Pass",
0x00060000: "Memo / Voice Memo",
}
# ============================================================
# Playlist Sort Order (MHYP offset 44 / 0x2C)
# From iPodLinux wiki "List Sort Order" and libgpod ItdbPlaylistSortOrder.
# ============================================================
PLAYLIST_SORT_ORDER_MAP = {
0: "default (unset)",
1: "playlist order (manual)",
# 2: unknown
3: "title",
4: "album",
5: "artist",
6: "bitrate",
7: "genre",
8: "kind",
9: "date modified",
10: "track number",
11: "size",
12: "time",
13: "year",
14: "sample rate",
15: "comment",
16: "date added",
17: "equalizer",
18: "composer",
# 19: unknown
20: "play count",
21: "last played",
22: "disc number",
23: "my rating",
24: "release date", # used for Podcasts list
25: "BPM",
26: "grouping",
27: "category",
28: "description",
29: "show",
30: "season",
31: "episode number",
}
# ============================================================
# Explicit / content advisory flag values (MHIT offset 146 / 0x92)
# ============================================================
EXPLICIT_FLAG_MAP = {
0: "none",
1: "explicit",
2: "clean",
}
# ============================================================
# MHOD Type Integer Constants
# Shared by MHOD parser/writer modules.
# ============================================================
MHOD_TYPE_TITLE = 1
MHOD_TYPE_LOCATION = 2
MHOD_TYPE_ALBUM = 3
MHOD_TYPE_ARTIST = 4
MHOD_TYPE_GENRE = 5
MHOD_TYPE_FILETYPE = 6
MHOD_TYPE_EQ_SETTING = 7
MHOD_TYPE_COMMENT = 8
MHOD_TYPE_CATEGORY = 9
MHOD_TYPE_LYRICS = 10
MHOD_TYPE_COMPOSER = 12
MHOD_TYPE_GROUPING = 13
MHOD_TYPE_DESCRIPTION = 14
MHOD_TYPE_PODCAST_ENCLOSURE_URL = 15
MHOD_TYPE_PODCAST_RSS_URL = 16
MHOD_TYPE_CHAPTER_DATA = 17
MHOD_TYPE_SUBTITLE = 18
MHOD_TYPE_SHOW_NAME = 19
MHOD_TYPE_EPISODE_ID = 20
MHOD_TYPE_NETWORK_NAME = 21
MHOD_TYPE_ALBUM_ARTIST = 22
MHOD_TYPE_SORT_ARTIST = 23
MHOD_TYPE_KEYWORDS = 24
MHOD_TYPE_SHOW_LOCALE = 25
MHOD_TYPE_SORT_NAME = 27
MHOD_TYPE_SORT_ALBUM = 28
MHOD_TYPE_SORT_ALBUM_ARTIST = 29
MHOD_TYPE_SORT_COMPOSER = 30
MHOD_TYPE_SORT_SHOW = 31
MHOD_TYPE_SMART_PLAYLIST_DATA = 50
MHOD_TYPE_SMART_PLAYLIST_RULES = 51
MHOD_TYPE_LIBRARY_PLAYLIST_INDEX = 52
MHOD_TYPE_LIBRARY_PLAYLIST_JUMP_TABLE = 53
MHOD_TYPE_COLUMN_SIZE_OR_ORDER = 100
MHOD_TYPE_PLAYLIST_SETTINGS = 102
# Album item string types
MHOD_TYPE_ALBUM_ALBUM = 200
MHOD_TYPE_ALBUM_ARTIST_ITEM = 201
MHOD_TYPE_ALBUM_SORT_ARTIST = 202
MHOD_TYPE_ALBUM_PODCAST_URL = 203
MHOD_TYPE_ALBUM_SHOW = 204
# Artist item string type
MHOD_TYPE_ARTIST_NAME = 300
# ============================================================
# File Format Codes (big-endian ASCII stored as LE u32)
# Shared by track and locations writers.
# ============================================================
FILETYPE_CODES: dict[str, int] = {
'mp3': 0x4D503320, # "MP3 "
'm4a': 0x4D344120, # "M4A "
'm4p': 0x4D345020, # "M4P "
'm4b': 0x4D344220, # "M4B "
'm4v': 0x4D345620, # "M4V "
'mp4': 0x4D503420, # "MP4 "
'wav': 0x57415620, # "WAV "
'aif': 0x41494646, # "AIFF"
'aiff': 0x41494646, # "AIFF"
'aac': 0x41414320, # "AAC "
}
# ============================================================
# Media Type Integer Constants (from libgpod Itdb_Mediatype)
# Shared by track conversion and writer code.
# ============================================================
MEDIA_TYPE_AUDIO = 0x01
MEDIA_TYPE_VIDEO = 0x02
MEDIA_TYPE_PODCAST = 0x04
MEDIA_TYPE_VIDEO_PODCAST = 0x06
MEDIA_TYPE_AUDIOBOOK = 0x08
MEDIA_TYPE_MUSIC_VIDEO = 0x20
MEDIA_TYPE_TV_SHOW = 0x40
MEDIA_TYPE_RINGTONE = 0x4000
MEDIA_TYPE_VIDEO_MASK = MEDIA_TYPE_VIDEO | MEDIA_TYPE_MUSIC_VIDEO | MEDIA_TYPE_TV_SHOW
# ============================================================
# Audio Format Flag map (MHIT offset 0x7E)
# Maps filetype → codec hint value for the audio_format_flag field.
# 0xFFFF = default (MP3/AAC/ALAC), 0x0000 = lossless (WAV/AIFF),
# 0x0001 = Audible (M4B audiobooks).
# ============================================================
AUDIO_FORMAT_FLAG_MAP: dict[str, int] = {
'wav': 0x0000,
'aif': 0x0000,
'aiff': 0x0000,
'm4b': 0x0001,
}
AUDIO_FORMAT_FLAG_DEFAULT: int = 0xFFFF
+120
View File
@@ -0,0 +1,120 @@
"""Parser post-processing helpers for flattening parsed iTunesDB dicts.
These functions walk the nested chunk structures produced by the iTunesDB
parser and extract them into flat, easy-to-consume dictionaries. They are
used by ``iTunesDB_Parser.ipod_library`` and ``SyncEngine.sync_executor``.
"""
from .constants import chunk_type_map, mhod_type_map
def extract_datasets(mhbd: dict) -> dict:
"""Walk the MHBD children and extract datasets into a flat dict.
Returns a dict with:
- All MHBD header fields (excluding 'children')
- "mhlt", "mhlp", "mhlp_podcast", "mhla", "mhlp_smart", etc.
mapped from MHSD dataset_type via chunk_type_map
- Each value is the list of item dicts from the list chunk
"""
result = {}
for key, value in mhbd.items():
if key != "children":
result[key] = value
for mhsd_wrapper in mhbd.get("children", []):
mhsd_data = mhsd_wrapper.get("data", {})
dataset_type = mhsd_data.get("dataset_type")
result_key = chunk_type_map.get(dataset_type)
if result_key is None:
continue
mhsd_children = mhsd_data.get("children", [])
if not mhsd_children:
result[result_key] = []
continue
# The MHSD has one child: the list chunk (mhlt, mhlp, mhla, mhli)
list_chunk = mhsd_children[0]
items = list_chunk.get("data", [])
# Extract items from their wrapper dicts
flat_items = []
for item in items:
if isinstance(item, dict) and "data" in item:
flat_items.append(item["data"])
else:
flat_items.append(item)
result[result_key] = flat_items
return result
def extract_mhod_strings(children: list) -> dict:
"""Extract MHOD string values from a chunk's children list.
Args:
children: The 'children' list from a parsed track/album/artist/playlist.
Returns:
dict mapping mhod_type_map field keys to string values,
e.g. {"Title": "My Song", "Artist": "Foo"}
"""
strings = {}
for wrapper in children:
mhod_data = wrapper.get("data", {})
mhod_type = mhod_data.get("mhod_type")
if mhod_type is None:
continue
field_name = mhod_type_map.get(mhod_type)
if field_name and "string" in mhod_data:
strings[field_name] = mhod_data["string"]
return strings
def extract_track_extras(mhod_children: list) -> dict:
"""Extract non-string MHOD data from track children.
Returns dict with optional keys:
- "chapter_data": parsed MHOD type 17 data
"""
extras = {}
for wrapper in mhod_children:
mhod_data = wrapper.get("data", {})
if mhod_data.get("mhod_type") != 17 or "data" not in mhod_data:
continue
raw_chapter_data = mhod_data["data"]
if not isinstance(raw_chapter_data, dict):
continue
extras["chapter_data"] = raw_chapter_data
return extras
def extract_playlist_extras(mhod_children: list) -> dict:
"""Extract non-string MHOD data from playlist children.
Returns dict with optional keys:
- "smart_playlist_data": SPL prefs dict (from MHOD type 50)
- "smart_playlist_rules": SPL rules dict (from MHOD type 51)
- "library_indices": sorted index data (from MHOD type 52)
- "playlist_prefs": column prefs (from MHOD type 100)
- "playlist_settings": settings blob (from MHOD type 102)
"""
extras = {}
for wrapper in mhod_children:
mhod_data = wrapper.get("data", {})
mhod_type = mhod_data.get("mhod_type")
if mhod_type == 50 and "data" in mhod_data:
extras["smart_playlist_data"] = mhod_data["data"]
elif mhod_type == 51 and "data" in mhod_data:
extras["smart_playlist_rules"] = mhod_data["data"]
elif mhod_type == 52 and "data" in mhod_data:
extras.setdefault("library_indices", []).append(mhod_data["data"])
elif mhod_type == 100 and "data" in mhod_data:
extras["playlist_prefs"] = mhod_data["data"]
elif mhod_type == 102 and "data" in mhod_data:
extras["playlist_settings"] = mhod_data["data"]
return extras
+415
View File
@@ -0,0 +1,415 @@
"""Shared infrastructure for bidirectional iTunesDB field definitions.
This module provides the :class:`FieldDef` dataclass, factory helpers,
transform / validator functions, exception hierarchy, and the read/write
helpers that all per-chunk ``*_defs.py`` modules build on.
Per-chunk field lists (``MHBD_FIELDS``, ``MHIT_FIELDS``, ) live in their
own ``*_defs.py`` modules. The :data:`FIELD_REGISTRY` is assembled at
import time by :mod:`iTunesDB_Shared.__init__` from those modules.
"""
from __future__ import annotations
import struct
from collections.abc import Callable
from dataclasses import dataclass
from typing import Any
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# 1. Exception Hierarchy
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
class WriteError(Exception):
"""Base exception for iTunesDB write-time errors."""
class MissingRequiredFieldError(WriteError):
"""A field marked ``required=True`` was absent from the values dict."""
def __init__(self, section_type: str, field_name: str) -> None:
super().__init__(
f"Required field '{field_name}' missing for section '{section_type}'"
)
self.section_type = section_type
self.field_name = field_name
class InvalidFieldValueError(WriteError):
"""A field validator rejected the value."""
def __init__(self, section_type: str, field_name: str, detail: str) -> None:
super().__init__(
f"Invalid value for '{field_name}' in section '{section_type}': {detail}"
)
self.section_type = section_type
self.field_name = field_name
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# 2. Transform & Validator Functions
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# Mac HFS+ epoch offset (seconds between 1904-01-01 and 1970-01-01).
MAC_EPOCH_OFFSET: int = 2082844800
def mac_to_unix(mac_ts: int) -> int:
"""Convert Mac HFS+ timestamp to Unix epoch."""
return mac_ts - MAC_EPOCH_OFFSET if mac_ts > 0 else 0
def unix_to_mac(unix_ts: int) -> int:
"""Convert Unix epoch timestamp to Mac HFS+ timestamp."""
return unix_ts + MAC_EPOCH_OFFSET if unix_ts > 0 else 0
def sample_rate_to_fixed(hz: int) -> int:
"""Encode sample rate as 16.16 fixed-point for MHIT offset 0x3C."""
return (hz << 16) & 0xFFFFFFFF
def fixed_to_sample_rate(raw: int) -> int:
"""Decode 16.16 fixed-point sample rate to integer Hz."""
return raw >> 16
def validate_rating(value: int) -> None:
"""Raise if rating is outside 0-100."""
if not (0 <= value <= 100):
raise ValueError(f"rating {value} outside 0-100")
def clamp_rating(value: int) -> int:
"""Clamp rating to 0-100."""
return max(0, min(100, value))
def validate_volume(value: int) -> None:
"""Raise if volume adjustment is outside -255..+255."""
if not (-255 <= value <= 255):
raise ValueError(f"volume {value} outside -255..+255")
def filetype_to_string(val: int) -> str:
"""Convert a u32 filetype code to its ASCII representation.
e.g. 0x4D503320 "MP3", 0x4D344120 "M4A"
"""
if not isinstance(val, int) or val <= 0:
return ""
try:
return val.to_bytes(4, "big").decode("ascii").rstrip("\x00").strip()
except (OverflowError, UnicodeDecodeError):
return str(val)
def strip_article(name: str) -> str:
"""Strip leading English articles (A, An, The) for sort field generation.
iTunes auto-generates sort_title/sort_album/etc. by stripping common
English leading articles. Used by both the binary iTunesDB writer
(MHOD type 52 jump tables) and the SQLite writer (sort key columns).
"""
if not name:
return name
lower = name.lower()
for article in ('the ', 'a ', 'an '):
if lower.startswith(article):
return name[len(article):]
return name
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# 3. FieldDef Dataclass
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
@dataclass(frozen=True, slots=True)
class FieldDef:
"""Complete, bidirectional contract for one binary field.
Attributes:
name: Canonical snake_case identifier shared by parser and writer.
offset: Byte offset within the section header (relative to chunk start).
size: Byte width of the field.
struct_format: :mod:`struct` format string (e.g. ``'<I'``, ``'B'``, ``'<8s'``).
read_transform: Callable applied AFTER unpacking on parse.
write_transform: Callable applied BEFORE packing on write.
default: Value used when the field is absent from the data dict.
validator: Callable that raises :class:`ValueError` if the value is invalid.
min_header_length: Minimum ``header_length`` for this field to exist.
``None`` means the field is always present.
required: If ``True``, :func:`write_fields` raises
:class:`MissingRequiredFieldError` when the field is absent.
section_type: ASCII tag of the parent section (e.g. ``'mhit'``).
"""
name: str
offset: int
size: int
struct_format: str
read_transform: Callable[..., Any] | None = None
write_transform: Callable[..., Any] | None = None
default: Any = 0
validator: Callable[..., None] | None = None
min_header_length: int | None = None
required: bool = False
section_type: str = ""
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# 3b. List-container header sizes
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# Simple list chunks (mhlt, mhla, mhli, mhlp) have only the 12-byte
# generic header padded to 92 bytes. They don't carry FieldDef lists.
MHLT_HEADER_SIZE: int = 92
MHLA_HEADER_SIZE: int = 92
MHLI_HEADER_SIZE: int = 92
MHLP_HEADER_SIZE: int = 92
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# 4. Factory Helpers
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
def _u32(name: str, offset: int, **kw: Any) -> FieldDef:
return FieldDef(name=name, offset=offset, size=4, struct_format="<I", **kw)
def _i32(name: str, offset: int, **kw: Any) -> FieldDef:
return FieldDef(name=name, offset=offset, size=4, struct_format="<i", **kw)
def _u16(name: str, offset: int, **kw: Any) -> FieldDef:
return FieldDef(name=name, offset=offset, size=2, struct_format="<H", **kw)
def _u64(name: str, offset: int, **kw: Any) -> FieldDef:
return FieldDef(name=name, offset=offset, size=8, struct_format="<Q", **kw)
def _u8(name: str, offset: int, **kw: Any) -> FieldDef:
return FieldDef(name=name, offset=offset, size=1, struct_format="B", **kw)
def _f32(name: str, offset: int, **kw: Any) -> FieldDef:
return FieldDef(name=name, offset=offset, size=4, struct_format="<f", **kw)
def _raw(name: str, offset: int, size: int, **kw: Any) -> FieldDef:
kw.setdefault("default", b"\x00" * size)
return FieldDef(name=name, offset=offset, size=size,
struct_format=f"<{size}s", **kw)
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# 5. FIELD_REGISTRY & lookup helpers
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# Populated by __init__.py after all *_defs modules have been imported.
FIELD_REGISTRY: dict[str, list[FieldDef]] = {}
def get_fields(
section_type: str,
header_length: int | None = None,
) -> list[FieldDef]:
"""Return field definitions for *section_type*, optionally filtered.
Args:
section_type: ASCII chunk tag (e.g. ``'mhit'``, ``'mhbd'``).
header_length: If provided, fields whose ``min_header_length``
exceeds this value are excluded.
Returns:
List of :class:`FieldDef` in offset order.
"""
fields = FIELD_REGISTRY.get(section_type, [])
if header_length is not None:
return [
f for f in fields
if f.min_header_length is None or header_length >= f.min_header_length
]
return list(fields)
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# 6. Read / Write Helpers
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
def read_field(
data: bytes | bytearray,
base_offset: int,
field: FieldDef,
header_length: int | None = None,
) -> Any:
"""Read a single field from *data* at *base_offset*.
Args:
data: Raw byte buffer.
base_offset: Start of the chunk in *data*.
field: Field definition.
header_length: Chunk header length (for version guard).
Returns:
The unpacked (and optionally transformed) value, or
``field.default`` if the header is too small.
"""
if field.min_header_length is not None:
if header_length is None or header_length < field.min_header_length:
return field.default
abs_offset = base_offset + field.offset
raw = struct.unpack_from(field.struct_format, data, abs_offset)[0]
if field.read_transform is not None:
return field.read_transform(raw)
return raw
def read_fields(
data: bytes | bytearray,
base_offset: int,
section_type: str,
header_length: int | None = None,
) -> dict[str, Any]:
"""Read all fields for *section_type* into a dict.
This is the read-side counterpart to :func:`write_fields`.
Args:
data: Raw byte buffer.
base_offset: Start of the chunk in *data*.
section_type: Chunk tag (e.g. ``'mhit'``).
header_length: Actual header length from the chunk. Extended
fields not covered by this length are set to their defaults.
Returns:
Dict mapping field name value.
"""
result: dict[str, Any] = {}
for field in FIELD_REGISTRY.get(section_type, []):
result[field.name] = read_field(data, base_offset, field, header_length)
return result
def write_field(
buffer: bytearray,
base_offset: int,
field: FieldDef,
value: Any,
section_type: str = "",
) -> None:
"""Pack a single field value into *buffer*.
Applies ``write_transform`` then ``validator`` before packing.
Args:
buffer: Mutable byte buffer to write into.
base_offset: Start of the chunk header in *buffer*.
field: Field definition.
value: The logical value to write.
section_type: For error messages only.
"""
if field.write_transform is not None:
value = field.write_transform(value)
if field.validator is not None:
try:
field.validator(value)
except (ValueError, TypeError) as exc:
raise InvalidFieldValueError(
section_type or field.section_type, field.name, str(exc),
) from exc
# Coerce to the type required by the format code so that floats or other
# numeric types from metadata sources never reach struct.pack_into as the
# wrong type (e.g. float for an integer field, or int for a float field).
format_code = field.struct_format[-1]
if format_code in 'IiHhQqBbNnP':
if not isinstance(value, int):
value = int(value)
# Clamp to the valid range for the format so bad metadata (e.g. negative
# BPM, oversized play counts) never crashes the packer.
int_ranges = {
'B': (0, 0xFF),
'H': (0, 0xFFFF),
'I': (0, 0xFFFF_FFFF),
'Q': (0, 0xFFFF_FFFF_FFFF_FFFF),
'b': (-0x80, 0x7F),
'h': (-0x8000, 0x7FFF),
'i': (-0x8000_0000, 0x7FFF_FFFF),
'q': (-0x8000_0000_0000_0000, 0x7FFF_FFFF_FFFF_FFFF),
}
if format_code in int_ranges:
lower, upper = int_ranges[format_code]
value = max(lower, min(upper, value))
elif format_code in 'fd':
if not isinstance(value, float):
value = float(value)
try:
struct.pack_into(field.struct_format, buffer, base_offset + field.offset, value)
except struct.error as exc:
raise struct.error(
f"Failed to pack field '{field.name}' (format={field.struct_format!r}, "
f"value={value!r} type={type(value).__name__}): {exc}"
) from exc
def write_fields(
buffer: bytearray,
base_offset: int,
section_type: str,
values: dict[str, Any],
header_length: int,
) -> None:
"""Write all applicable fields from *values* into *buffer*.
This is the single serialization entrypoint used by the writer.
Fields are written in offset order. Fields whose
``min_header_length`` exceeds *header_length* are skipped.
Args:
buffer: Pre-allocated mutable buffer (must be header_length).
base_offset: Chunk start position in *buffer*.
section_type: Chunk tag (e.g. ``'mhit'``).
values: Field name logical value mapping.
header_length: Target header size. Fields beyond this are skipped.
Raises:
MissingRequiredFieldError: A ``required`` field is missing from *values*.
InvalidFieldValueError: A validator rejected a value.
"""
for field in FIELD_REGISTRY.get(section_type, []):
# Skip fields outside the target header.
if field.min_header_length is not None and header_length < field.min_header_length:
continue
if field.name in values:
value = values[field.name]
elif field.required:
raise MissingRequiredFieldError(section_type, field.name)
else:
value = field.default
write_field(buffer, base_offset, field, value, section_type)
def write_generic_header(
buffer: bytearray,
offset: int,
tag: bytes,
header_length: int,
total_length_or_count: int,
) -> None:
"""Write the 12-byte generic iTunesDB chunk header.
Args:
buffer: Target buffer.
offset: Position in buffer.
tag: 4-byte ASCII tag (e.g. ``b'mhit'``).
header_length: Header size to write at +0x04.
total_length_or_count: Value for +0x08 (total_length for item
chunks, child_count for list chunks).
"""
buffer[offset:offset + 4] = tag
struct.pack_into("<I", buffer, offset + 4, header_length)
struct.pack_into("<I", buffer, offset + 8, total_length_or_count)
+48
View File
@@ -0,0 +1,48 @@
"""MHBD (Database Header) field definitions.
Declarative :class:`FieldDef` list for the MHBD chunk the root of
the iTunesDB file. Both the parser and writer derive their behaviour
from these definitions.
"""
from .field_base import FieldDef, _u32, _i32, _u16, _u64, _raw
_S = "mhbd"
# Writer header size (matching iTunes / libgpod default).
MHBD_HEADER_SIZE: int = 244 # 0xF4
# Named offsets used by hash modules for zeroing before signing.
MHBD_OFFSET_DB_ID: int = 0x18 # 8 bytes (u64)
MHBD_OFFSET_HASHING_SCHEME: int = 0x30 # 2 bytes (u16)
MHBD_OFFSET_UNK_0x32: int = 0x32 # 20 bytes (raw)
MHBD_OFFSET_HASH58: int = 0x58 # 20 bytes (raw)
MHBD_OFFSET_HASH72: int = 0x72 # 46 bytes (raw)
MHBD_OFFSET_HASHAB: int = 0xAB # 57 bytes (raw)
MHBD_FIELDS: list[FieldDef] = [
_u32("compressed", 0x0C, section_type=_S, default=1),
_u32("version", 0x10, section_type=_S, required=True),
_u32("child_count", 0x14, section_type=_S),
_u64("db_id", 0x18, section_type=_S, required=True),
_u16("platform", 0x20, section_type=_S, default=2),
_u16("unk0x22", 0x22, section_type=_S, default=0),
_u64("db_id_2", 0x24, section_type=_S),
_u32("unk0x2c", 0x2C, section_type=_S),
_u16("hashing_scheme", 0x30, section_type=_S),
_raw("unk0x32", 0x32, 20, section_type=_S),
_raw("language", 0x46, 2, section_type=_S, default=b"en"),
_u64("db_persistent_id", 0x48, section_type=_S),
_u32("unk0x50", 0x50, section_type=_S, default=1),
_u32("unk0x54", 0x54, section_type=_S, default=15),
_raw("hash58", 0x58, 20, section_type=_S),
_i32("timezone_offset", 0x6C, section_type=_S),
_u16("hash_type_indicator", 0x70, section_type=_S),
_raw("hash72", 0x72, 46, section_type=_S),
# Extended fields — only in newer database headers.
_u16("audio_language", 0xA0, section_type=_S, min_header_length=0xA2),
_u16("subtitle_language", 0xA2, section_type=_S, min_header_length=0xA4),
_u16("unk0xa4", 0xA4, section_type=_S, min_header_length=0xA6),
_u16("unk0xa6", 0xA6, section_type=_S, min_header_length=0xA8),
_u16("cdb_flag", 0xA8, section_type=_S, min_header_length=0xAA),
]
+21
View File
@@ -0,0 +1,21 @@
"""MHIA (Album Item) field definitions.
Declarative :class:`FieldDef` list for the MHIA chunk an album
record inside an MHLA (album list).
"""
from .field_base import FieldDef, _u32, _u16, _u64
_S = "mhia"
MHIA_HEADER_SIZE: int = 88
MHIA_FIELDS: list[FieldDef] = [
_u32("child_count", 0x0C, section_type=_S),
_u32("album_id", 0x10, section_type=_S, required=True),
_u64("sql_id", 0x14, section_type=_S),
_u16("platform_flag", 0x1C, section_type=_S, default=2),
_u16("album_compilation_flag", 0x1E, section_type=_S),
# Representative track db_track_id — only populated by some iTunes versions
_u64("album_track_db_id", 0x20, section_type=_S, min_header_length=0x28),
]
+18
View File
@@ -0,0 +1,18 @@
"""MHII (Artist Item) field definitions.
Declarative :class:`FieldDef` list for the MHII chunk an artist
record inside an MHLI (artist list, MHSD type 8).
"""
from .field_base import FieldDef, _u32, _u64
_S = "mhii"
MHII_HEADER_SIZE: int = 80
MHII_FIELDS: list[FieldDef] = [
_u32("child_count", 0x0C, section_type=_S),
_u32("artist_id", 0x10, section_type=_S, required=True),
_u64("sql_id", 0x14, section_type=_S),
_u32("platform_flag", 0x1C, section_type=_S, default=2),
]
+26
View File
@@ -0,0 +1,26 @@
"""MHIP (Playlist Item) field definitions.
Declarative :class:`FieldDef` list for the MHIP chunk a playlist
entry that references a track by ``track_id``.
"""
from .field_base import FieldDef, _u32, _u16, _u64, mac_to_unix, unix_to_mac
_S = "mhip"
MHIP_HEADER_SIZE: int = 76
MHIP_FIELDS: list[FieldDef] = [
_u32("child_count", 0x0C, section_type=_S),
_u16("podcast_group_flag", 0x10, section_type=_S),
_u16("unk0x12", 0x12, section_type=_S),
_u32("group_id", 0x14, section_type=_S),
_u32("track_id", 0x18, section_type=_S, required=True),
_u32("timestamp", 0x1C, section_type=_S,
read_transform=mac_to_unix, write_transform=unix_to_mac),
_u32("group_id_ref", 0x20, section_type=_S),
# Extended (header_length >= 0x34)
_u64("track_persistent_id", 0x2C, section_type=_S, min_header_length=0x34),
# Extended (header_length >= 0x44) — per-track persistent ID, absent in older iTunes
_u64("mhip_persistent_id", 0x3C, section_type=_S, min_header_length=0x44),
]
+141
View File
@@ -0,0 +1,141 @@
"""MHIT (Track Item) field definitions.
Declarative :class:`FieldDef` list for the MHIT chunk a single track
record. Both the parser and writer derive their behaviour from these
definitions.
"""
from .field_base import (
FieldDef,
_u32, _i32, _u16, _u64, _u8, _f32, _raw,
clamp_rating, validate_rating, validate_volume,
mac_to_unix, unix_to_mac,
fixed_to_sample_rate, sample_rate_to_fixed,
)
_S = "mhit"
# Writer header size (iTunes 8+ default).
MHIT_HEADER_SIZE: int = 0x270 # 624 bytes
def mhit_header_size_for_version(db_version: int) -> int:
"""Return the MHIT header size appropriate for *db_version*.
Older iPod firmware uses smaller MHIT headers. The writer must pad
to the correct boundary so the firmware can locate child MHODs.
"""
if db_version <= 0x12:
return 0x9C # 156 — pre-iTunes 7 minimum
if db_version <= 0x19:
return 0x148 # 328 — iTunes 7.x
if db_version <= 0x2D:
return 0x1F8 # 504 — iTunes 9.x
return 0x270 # 624 — iTunes 10+ / modern
MHIT_FIELDS: list[FieldDef] = [
# ── Core fields (always present, minimum header ≥ 0x9C) ──────
_u32("child_count", 0x0C, section_type=_S),
_u32("track_id", 0x10, section_type=_S, required=True),
_u32("visible", 0x14, section_type=_S, default=1),
_u32("filetype", 0x18, section_type=_S),
_u8("vbr_flag", 0x1C, section_type=_S),
_u8("mp3_flag", 0x1D, section_type=_S),
_u8("compilation_flag", 0x1E, section_type=_S),
_u8("rating", 0x1F, section_type=_S,
write_transform=clamp_rating, validator=validate_rating),
_u32("last_modified", 0x20, section_type=_S,
read_transform=mac_to_unix, write_transform=unix_to_mac),
_u32("size", 0x24, section_type=_S),
_u32("length", 0x28, section_type=_S),
_u32("track_number", 0x2C, section_type=_S),
_u32("total_tracks", 0x30, section_type=_S),
_u32("year", 0x34, section_type=_S),
_u32("bitrate", 0x38, section_type=_S),
_u32("sample_rate_1", 0x3C, section_type=_S,
read_transform=fixed_to_sample_rate,
write_transform=sample_rate_to_fixed),
_i32("volume", 0x40, section_type=_S, validator=validate_volume),
_u32("start_time", 0x44, section_type=_S),
_u32("stop_time", 0x48, section_type=_S),
_u32("sound_check", 0x4C, section_type=_S),
_u32("play_count_1", 0x50, section_type=_S),
_u32("play_count_2", 0x54, section_type=_S),
_u32("last_played", 0x58, section_type=_S,
read_transform=mac_to_unix, write_transform=unix_to_mac),
_u32("disc_number", 0x5C, section_type=_S),
_u32("total_discs", 0x60, section_type=_S),
_u32("user_id", 0x64, section_type=_S),
_u32("date_added", 0x68, section_type=_S,
read_transform=mac_to_unix, write_transform=unix_to_mac),
_u32("bookmark_time", 0x6C, section_type=_S),
_u64("db_track_id", 0x70, section_type=_S, required=True),
_u8("checked_flag", 0x78, section_type=_S),
_u8("app_rating", 0x79, section_type=_S),
_u16("bpm", 0x7A, section_type=_S),
_u16("artwork_count", 0x7C, section_type=_S),
_u16("audio_format_flag", 0x7E, section_type=_S, default=0xFFFF),
_u32("artwork_size", 0x80, section_type=_S),
_u32("unk0x84", 0x84, section_type=_S),
_f32("sample_rate_2", 0x88, section_type=_S),
_u32("date_released", 0x8C, section_type=_S,
read_transform=mac_to_unix, write_transform=unix_to_mac),
_u16("mpeg_audio_type", 0x90, section_type=_S),
_u8("explicit_flag", 0x92, section_type=_S),
_u8("purchased_aac_flag", 0x93, section_type=_S),
_u32("unk0x94", 0x94, section_type=_S),
_u32("genius_category_id", 0x98, section_type=_S),
# ── Extended fields (guarded by header_length) ───────────────
_u32("skip_count", 0x9C, section_type=_S, min_header_length=0xA0),
_u32("last_skipped", 0xA0, section_type=_S, min_header_length=0xA4,
read_transform=mac_to_unix, write_transform=unix_to_mac),
_u8("has_artwork", 0xA4, section_type=_S, min_header_length=0xA5),
_u8("skip_when_shuffling", 0xA5, section_type=_S, min_header_length=0xA6),
_u8("remember_position", 0xA6, section_type=_S, min_header_length=0xA7),
_u8("use_podcast_now_playing_flag", 0xA7, section_type=_S, min_header_length=0xA8),
_u64("db_track_id_2", 0xA8, section_type=_S, min_header_length=0xB0),
_u8("lyrics_flag", 0xB0, section_type=_S, min_header_length=0xB1),
_u8("movie_flag", 0xB1, section_type=_S, min_header_length=0xB2),
_u8("not_played_flag", 0xB2, section_type=_S, min_header_length=0xB3),
_u8("unk0xB3", 0xB3, section_type=_S, min_header_length=0xB4),
_u32("unk0xB4", 0xB4, section_type=_S, min_header_length=0xB8),
_u32("pregap", 0xB8, section_type=_S, min_header_length=0xBC),
_u64("sample_count", 0xBC, section_type=_S, min_header_length=0xC4),
_u32("unk0xC4", 0xC4, section_type=_S, min_header_length=0xC8),
_u32("postgap", 0xC8, section_type=_S, min_header_length=0xCC),
_u32("encoder", 0xCC, section_type=_S, min_header_length=0xD0),
_u32("media_type", 0xD0, section_type=_S, min_header_length=0xD4,
default=1),
_u32("season_number", 0xD4, section_type=_S, min_header_length=0xD8),
_u32("episode_number", 0xD8, section_type=_S, min_header_length=0xDC),
_u32("date_added_to_itunes", 0xDC, section_type=_S, min_header_length=0xE0,
read_transform=mac_to_unix, write_transform=unix_to_mac),
_u32("store_track_id", 0xE0, section_type=_S, min_header_length=0xE4),
_u32("store_encoder_version", 0xE4, section_type=_S, min_header_length=0xE8),
_u32("store_artist_id", 0xE8, section_type=_S, min_header_length=0xEC),
_u32("unk0xEC", 0xEC, section_type=_S, min_header_length=0xF0),
_u32("store_album_id", 0xF0, section_type=_S, min_header_length=0xF4),
_u32("store_content_flag", 0xF4, section_type=_S, min_header_length=0xF8),
_u32("gapless_audio_payload_size", 0xF8, section_type=_S, min_header_length=0xFC),
_u32("unk0xFC", 0xFC, section_type=_S, min_header_length=0x100),
_u16("gapless_track_flag", 0x100, section_type=_S, min_header_length=0x102),
_u16("gapless_album_flag", 0x102, section_type=_S, min_header_length=0x104),
_raw("hash_0x104", 0x104, 20, section_type=_S, min_header_length=0x118),
_u32("unk0x118", 0x118, section_type=_S, min_header_length=0x11C),
_u32("unk0x11C", 0x11C, section_type=_S, min_header_length=0x120),
_u32("album_id", 0x120, section_type=_S, min_header_length=0x124),
_u64("db_id_2_ref", 0x124, section_type=_S, min_header_length=0x12C),
_u32("size_2", 0x12C, section_type=_S, min_header_length=0x130),
_u32("unk0x130", 0x130, section_type=_S, min_header_length=0x134),
_raw("sort_mhod_indicators", 0x134, 8, section_type=_S, min_header_length=0x13C),
# Gap: 0x13C..0x15F (zero padding / unknown fields)
_u32("artwork_id_ref", 0x160, section_type=_S, min_header_length=0x164),
# 0x168: unknown, libgpod always writes 1
_u32("unk0x168", 0x168, section_type=_S, min_header_length=0x16C, default=1),
# Gap: 0x16C..0x1DF (zero padding / unknown fields)
_u32("artist_id_ref", 0x1E0, section_type=_S, min_header_length=0x1E4),
# Gap: 0x1E4..0x1F3 (zero padding / unknown fields)
_u32("composer_id", 0x1F4, section_type=_S, min_header_length=0x1F8),
]
+586
View File
@@ -0,0 +1,586 @@
"""MHOD (Data Object) field definitions and helpers.
Contains the :class:`FieldDef` list for the 24-byte common MHOD header,
plus type classification sets, string sub-header accessors, and
SPL / SLst / MHOD-52 / MHOD-53 / MHOD-100 / MHOD-102 parsing helpers.
The body-level layouts vary widely by MHOD type and do NOT fit the
simple ``FieldDef`` pattern, so they remain as hand-written helpers here.
"""
import struct
from .field_base import FieldDef, _u32
_S = "mhod"
MHOD_HEADER_SIZE: int = 24 # common header (body varies by type)
# ── String MHOD sub-header layout ──────────────────────────
MHOD_STRING_SUBHEADER_OFFSET = 0x18 # sub-header start (relative to chunk start)
MHOD_STRING_SUBHEADER_SIZE = 16 # encoding(4) + length(4) + unk0x20(4) + unk0x24(4)
MHOD_STRING_DATA_OFFSET = 0x28 # string data start (header + sub-header)
# ── SPLPref body (MHOD type 50) ───────────────────────────
SPLPREF_BODY_SIZE = 132
# ── SLst rule data (MHOD type 51) non-string body size ────
SPL_RULE_DATA_SIZE = 0x44 # 68 bytes
# ── MHOD type 52/53 body layout sizes ─────────────────────
MHOD52_BODY_HEADER_SIZE = 48 # sort_type(4) + count(4) + padding(40)
MHOD53_BODY_HEADER_SIZE = 16 # sort_type(4) + count(4) + padding(8)
MHOD53_ENTRY_SIZE = 12 # letter(2) + pad(2) + start(4) + count(4)
# ── MHOD type 100 body layout ─────────────────────────────
MHOD100_POSITION_BODY_SIZE = 20 # position(4) + padding(16)
# ── MHOD 52/53 sort type constants (from libgpod MHOD52_SORTTYPE) ──
SORT_TITLE = 0x03
SORT_ALBUM = 0x04
SORT_ARTIST = 0x05
SORT_GENRE = 0x07
SORT_COMPOSER = 0x12
SORT_SHOW = 0x1D
SORT_SEASON = 0x1E
SORT_EPISODE = 0x1F
SORT_ALBUM_ARTIST = 0x23
# ── Chapter Data atom constants (MHOD type 17, all big-endian) ──
# The chapter data body starts with 12 bytes of unknown data
# (unk024, unk028, unk032 per libgpod), followed by a "sean" atom tree.
CHAPTER_PREAMBLE_SIZE = 12 # 3 × u32 LE before the atom tree
SEAN_ATOM = b'sean'
CHAP_ATOM = b'chap'
NAME_ATOM = b'name'
HEDR_ATOM = b'hedr'
HEDR_SIZE = 28 # hedr atom is always 28 bytes
MHOD_FIELDS: list[FieldDef] = [
_u32("mhod_type", 0x0C, section_type=_S, required=True),
_u32("unk0x10", 0x10, section_type=_S),
_u32("unk0x14", 0x14, section_type=_S),
]
# ============================================================
# Type Classification Sets
# ============================================================
# String MHOD types that use the standard sub-header at offset 0x18.
# Types 1-14, 18-31, 33-44 are track/item string metadata.
# Types 200-204 are album item strings.
#
# EXCLUDED from this set (handled separately):
# 15-16: Podcast URLs — UTF-8 string with NO sub-header
# 17: Chapter data — big-endian atom blob
# 32: Video track data — binary, not a string
STRING_MHOD_TYPES = (
set(range(1, 15)) # 1..14
| set(range(18, 32)) # 18..31
| set(range(33, 45)) # 33..44
| set(range(200, 205)) # 200..204
| {300} # artist item name (MHSD type 8)
)
# Podcast URL types — UTF-8/ASCII string directly at body start, no sub-header.
PODCAST_URL_MHOD_TYPES = {15, 16}
# Chapter data MHOD type — big-endian atom tree (sean/chap/name/hedr).
CHAPTER_DATA_MHOD_TYPES = {17}
# Binary / opaque MHOD types — stored as raw hex for round-tripping.
BINARY_BLOB_MHOD_TYPES = {32}
# Non-string MHOD types with dedicated binary formats.
NON_STRING_MHOD_TYPES = {50, 51, 52, 53, 100, 102}
# ============================================================
# Common MHOD Header (24 bytes — always 0x18)
# ============================================================
# The 3 common header fields (type, unk0x10, unk0x14) are defined in
# MHOD_FIELDS above and read via read_fields(). The per-type body
# helpers below still use struct directly since they don't fit FieldDef.
def write_mhod_header(mhod_type: int, total_length: int,
unk0x10: int = 0, unk0x14: int = 0) -> bytes:
"""Build the 24-byte MHOD common header.
This is the shared pattern used by every MHOD writer string,
SPL, index, position, etc.
Args:
mhod_type: MHOD type ID (e.g. 1, 50, 51, 52, 53, 100, 102).
total_length: Total length of the complete MHOD chunk
(header + body).
unk0x10: Unknown field at offset 0x10 (preserved from parser).
unk0x14: Unknown field at offset 0x14 (preserved from parser).
Returns:
24-byte packed header.
"""
return struct.pack(
'<4sIIIII',
b'mhod',
MHOD_HEADER_SIZE,
total_length,
mhod_type,
unk0x10,
unk0x14,
)
# ============================================================
# String MHOD Sub-Header (starts at 0x18, 16 bytes)
# ============================================================
# Present on STRING_MHOD_TYPES only. NOT present on podcast URLs (15/16).
# All functions take offset = start of the MHOD chunk.
#
# +0x18: encoding (4 bytes) — 1=UTF-16LE, 2=UTF-8
# +0x1C: string_length (4 bytes) — byte count of string data
# +0x20: unk0x20 (4 bytes)
# +0x24: unk0x24 (4 bytes)
# +0x28: string data begins (string_length bytes)
def mhod_string_encoding(data, offset) -> int:
"""Position/encoding indicator at 0x18.
1 (or 0) = UTF-16LE (standard iPod, little-endian strings).
2 = UTF-8 (mobile-phone iTunesDBs, inversed endian).
libgpod checks this same field to decide encoding."""
return struct.unpack("<I", data[offset + 0x18:offset + 0x1C])[0]
def mhod_string_length(data, offset) -> int:
"""Byte length of string data at 0x1C."""
return struct.unpack("<I", data[offset + 0x1C:offset + 0x20])[0]
def mhod_string_unk0x20(data, offset) -> int:
return struct.unpack("<I", data[offset + 0x20:offset + 0x24])[0]
def mhod_string_unk0x24(data, offset) -> int:
return struct.unpack("<I", data[offset + 0x24:offset + 0x28])[0]
# ============================================================
# SPLPref — Smart Playlist Preferences (MHOD type 50)
# ============================================================
# All functions take body_offset = start of SPLPref data (MHOD chunk + header_length).
#
# Based on libgpod's SPLPref struct (itdb_spl.c) and the iPodLinux wiki.
#
# +0x00: liveUpdate (1 byte) — 1 = auto-update when library changes
# +0x01: checkRules (1 byte) — 1 = limit by rules (match checked items)
# +0x02: checkLimits (1 byte) — 1 = limit by size/count/time
# +0x03: limitType (1 byte) — what the limit applies to (see SPL_LIMIT_TYPE_MAP)
# +0x04: limitSort (1 byte) — how to choose items when limited (see SPL_LIMIT_SORT_MAP)
# +0x05: pad (3 bytes)
# +0x08: limitValue (4 bytes LE) — the limit value
# +0x0C: matchCheckedOnly (1 byte) — 1 = only match checked items
# +0x0D: reverseSort (1 byte) — if set, limitsort |= 0x80000000
def mhod_spl_live_update(data, body_offset) -> int:
return data[body_offset]
def mhod_spl_check_rules(data, body_offset) -> int:
return data[body_offset + 1]
def mhod_spl_check_limits(data, body_offset) -> int:
return data[body_offset + 2]
def mhod_spl_limit_type(data, body_offset) -> int:
return data[body_offset + 3]
def mhod_spl_limit_sort_raw(data, body_offset) -> int:
"""Raw limit sort byte at +0x04 (before reverse flag is applied)."""
return data[body_offset + 4]
def mhod_spl_limit_value(data, body_offset) -> int:
return struct.unpack("<I", data[body_offset + 8:body_offset + 12])[0]
def mhod_spl_match_checked_only(data, body_offset) -> int:
return data[body_offset + 12]
def mhod_spl_reverse_sort(data, body_offset) -> int:
"""Reverse flag at +0x0D. If set, limitsort |= 0x80000000."""
return data[body_offset + 13]
# Limit type names (from libgpod ItdbLimitType)
SPL_LIMIT_TYPE_MAP = {
0x01: "minutes",
0x02: "MB",
0x03: "songs",
0x04: "hours",
0x05: "GB",
}
# Limit sort names (from libgpod ItdbLimitSort).
# The 0x80000000 bit is the "reverse" flag, stored separately at SPLPref +13.
SPL_LIMIT_SORT_MAP = {
0x02: "random",
0x03: "song_name",
0x04: "album",
0x05: "artist",
0x07: "genre",
0x10: "most_recently_added",
0x80000010: "least_recently_added",
0x14: "most_often_played",
0x80000014: "least_often_played",
0x15: "most_recently_played",
0x80000015: "least_recently_played",
0x17: "highest_rating",
0x80000017: "lowest_rating",
}
# ============================================================
# SLst — Smart Playlist Rules (MHOD type 51)
# ============================================================
# CRITICAL: The SLst blob is the ONLY part of the iTunesDB that uses
# big-endian encoding. All multi-byte integers within SLst use big-endian.
#
# SLst header (136 bytes):
# +0x00: 'SLst' magic (4 bytes)
# +0x04: unk004 (4 bytes BE) — usually 0
# +0x08: rule_count (4 bytes BE)
# +0x0C: conjunction (4 bytes BE) — 0=AND, 1=OR
# +0x10: padding (120 bytes)
#
# All SLst header functions take body_offset = start of SLst data.
SLST_HEADER_SIZE = 136
def mhod_slst_magic(data, body_offset) -> bytes:
return data[body_offset:body_offset + 4]
def mhod_slst_unk004(data, body_offset) -> int:
return struct.unpack(">I", data[body_offset + 4:body_offset + 8])[0]
def mhod_slst_rule_count(data, body_offset) -> int:
return struct.unpack(">I", data[body_offset + 8:body_offset + 12])[0]
def mhod_slst_conjunction(data, body_offset) -> int:
"""0=AND (match all), 1=OR (match any)."""
return struct.unpack(">I", data[body_offset + 12:body_offset + 16])[0]
# SPL Rule header fields.
# Each rule starts at a variable offset within the SLst body.
# Functions take rule_offset = start of the individual rule.
#
# Rule layout:
# +0x00: field (4 bytes BE) — what field to match (see SPL_FIELD_MAP)
# +0x04: action (4 bytes BE) — comparison operator (see SPL_ACTION_MAP)
# +0x08: padding (44 bytes)
# +0x34: data_length (4 bytes BE) — byte length of following data
# +0x38: data (data_length bytes)
#
# Total rule size = 56 + data_length.
SPL_RULE_HEADER_SIZE = 56
def mhod_spl_rule_field(data, rule_offset) -> int:
return struct.unpack(">I", data[rule_offset:rule_offset + 4])[0]
def mhod_spl_rule_action(data, rule_offset) -> int:
return struct.unpack(">I", data[rule_offset + 4:rule_offset + 8])[0]
def mhod_spl_rule_data_length(data, rule_offset) -> int:
return struct.unpack(">I", data[rule_offset + 0x34:rule_offset + 0x38])[0]
# SPL Rule non-string data fields (0x44 = 68 bytes).
# Functions take data_offset = rule_offset + 0x38.
#
# +0x00: fromValue (8 bytes BE, guint64)
# +0x08: fromDate (8 bytes BE, gint64 — signed)
# +0x10: fromUnits (8 bytes BE, guint64)
# +0x18: toValue (8 bytes BE, guint64)
# +0x20: toDate (8 bytes BE, gint64 — signed)
# +0x28: toUnits (8 bytes BE, guint64)
# +0x30: unk052 (4 bytes BE)
# +0x34: unk056 (4 bytes BE)
# +0x38: unk060 (4 bytes BE)
# +0x3C: unk064 (4 bytes BE)
# +0x40: unk068 (4 bytes BE)
def mhod_spl_rule_from_value(data, data_offset) -> int:
return struct.unpack(">Q", data[data_offset:data_offset + 8])[0]
def mhod_spl_rule_from_date(data, data_offset) -> int:
"""Signed 64-bit big-endian."""
return struct.unpack(">q", data[data_offset + 8:data_offset + 16])[0]
def mhod_spl_rule_from_units(data, data_offset) -> int:
return struct.unpack(">Q", data[data_offset + 16:data_offset + 24])[0]
def mhod_spl_rule_to_value(data, data_offset) -> int:
return struct.unpack(">Q", data[data_offset + 24:data_offset + 32])[0]
def mhod_spl_rule_to_date(data, data_offset) -> int:
"""Signed 64-bit big-endian."""
return struct.unpack(">q", data[data_offset + 32:data_offset + 40])[0]
def mhod_spl_rule_to_units(data, data_offset) -> int:
return struct.unpack(">Q", data[data_offset + 40:data_offset + 48])[0]
def mhod_spl_rule_unk052(data, data_offset) -> int:
return struct.unpack(">I", data[data_offset + 48:data_offset + 52])[0]
def mhod_spl_rule_unk056(data, data_offset) -> int:
return struct.unpack(">I", data[data_offset + 52:data_offset + 56])[0]
def mhod_spl_rule_unk060(data, data_offset) -> int:
return struct.unpack(">I", data[data_offset + 56:data_offset + 60])[0]
def mhod_spl_rule_unk064(data, data_offset) -> int:
return struct.unpack(">I", data[data_offset + 60:data_offset + 64])[0]
def mhod_spl_rule_unk068(data, data_offset) -> int:
return struct.unpack(">I", data[data_offset + 64:data_offset + 68])[0]
# Field ID → human-readable name (from libgpod ItdbSPLField enum in itdb.h)
SPL_FIELD_MAP = {
0x02: "Song Name",
0x03: "Album",
0x04: "Artist",
0x05: "Bitrate",
0x06: "Sample Rate",
0x07: "Year",
0x08: "Genre",
0x09: "Kind",
0x0A: "Date Modified",
0x0B: "Track Number",
0x0C: "Size",
0x0D: "Time",
0x0E: "Comment",
0x10: "Date Added",
0x12: "Composer",
0x16: "Play Count",
0x17: "Last Played",
0x18: "Disc Number",
0x19: "Rating",
0x1F: "Compilation",
0x23: "BPM",
0x27: "Grouping",
0x28: "Playlist",
0x29: "Purchased",
0x36: "Description",
0x37: "Category",
0x39: "Podcast",
0x3C: "Media Type",
0x3E: "TV Show",
0x3F: "Season Number",
0x44: "Skip Count",
0x45: "Last Skipped",
0x47: "Album Artist",
0x4E: "Sort Song Name",
0x4F: "Sort Album",
0x50: "Sort Artist",
0x51: "Sort Album Artist",
0x52: "Sort Composer",
0x53: "Sort TV Show",
0x5A: "Album Rating",
}
# Action ID → human-readable name (from libgpod ItdbSPLAction enum in itdb.h;
# confirmed against iPodLinux wiki).
# Actions are 32-bit bitmapped values, NOT small sequential integers.
# Byte layout:
# Bits 24-25: 0x00=int/date, 0x01=string, 0x02=negated int, 0x03=negated string
# Bits 0-10: comparison operator flags
SPL_ACTION_MAP = {
# Integer / date comparisons (0x00xxxxxx)
0x00000001: "is",
0x00000010: "is greater than",
0x00000020: "is greater than or equal to", # not in iTunes UI
0x00000040: "is less than",
0x00000080: "is less than or equal to", # not in iTunes UI
0x00000100: "is in the range",
0x00000200: "is in the last",
0x00000400: "binary AND", # used for Media Type / Video Kind
0x00000800: "binary unknown1",
# String comparisons (0x01xxxxxx)
0x01000001: "is (string)",
0x01000002: "contains",
0x01000004: "starts with",
0x01000008: "ends with",
# Negated integer / date (0x02xxxxxx)
0x02000001: "is not",
0x02000010: "is not greater than", # not in iTunes UI
0x02000020: "is not greater than or equal to", # not in iTunes UI
0x02000040: "is not less than", # not in iTunes UI
0x02000080: "is not less than or equal to", # not in iTunes UI
0x02000100: "is not in the range", # not in iTunes UI
0x02000200: "is not in the last",
0x02000400: "not binary AND",
0x02000800: "binary unknown2",
# Negated string (0x03xxxxxx)
0x03000001: "is not (string)",
0x03000002: "does not contain",
0x03000004: "does not start with", # not in iTunes UI
0x03000008: "does not end with", # not in iTunes UI
}
# Field type enum (from libgpod ItdbSPLFieldType — values start at 1)
SPLFT_STRING = 1
SPLFT_INT = 2
SPLFT_BOOLEAN = 3
SPLFT_DATE = 4
SPLFT_PLAYLIST = 5
SPLFT_UNKNOWN = 6
SPLFT_BINARY_AND = 7
# Map field ID → field type (equivalent to libgpod's itdb_splr_get_field_type).
# This is how libgpod determines how to parse the rule data — NOT from a binary field.
SPL_FIELD_TYPE_MAP = {
# String fields
0x02: SPLFT_STRING, # Song Name
0x03: SPLFT_STRING, # Album
0x04: SPLFT_STRING, # Artist
0x08: SPLFT_STRING, # Genre
0x09: SPLFT_STRING, # Kind
0x0E: SPLFT_STRING, # Comment
0x12: SPLFT_STRING, # Composer
0x27: SPLFT_STRING, # Grouping
0x36: SPLFT_STRING, # Description
0x37: SPLFT_STRING, # Category
0x3E: SPLFT_STRING, # TV Show
0x47: SPLFT_STRING, # Album Artist
0x4E: SPLFT_STRING, # Sort Song Name
0x4F: SPLFT_STRING, # Sort Album
0x50: SPLFT_STRING, # Sort Artist
0x51: SPLFT_STRING, # Sort Album Artist
0x52: SPLFT_STRING, # Sort Composer
0x53: SPLFT_STRING, # Sort TV Show
# Integer fields
0x05: SPLFT_INT, # Bitrate
0x06: SPLFT_INT, # Sample Rate
0x07: SPLFT_INT, # Year
0x0B: SPLFT_INT, # Track Number
0x0C: SPLFT_INT, # Size
0x0D: SPLFT_INT, # Time
0x16: SPLFT_INT, # Play Count
0x18: SPLFT_INT, # Disc Number
0x19: SPLFT_INT, # Rating
0x23: SPLFT_INT, # BPM
0x3F: SPLFT_INT, # Season Number
0x44: SPLFT_INT, # Skip Count
0x5A: SPLFT_INT, # Album Rating
# Date fields
0x0A: SPLFT_DATE, # Date Modified
0x10: SPLFT_DATE, # Date Added
0x17: SPLFT_DATE, # Last Played
0x45: SPLFT_DATE, # Last Skipped
# Boolean fields
0x1F: SPLFT_BOOLEAN, # Compilation
0x29: SPLFT_BOOLEAN, # Purchased
0x39: SPLFT_INT, # Podcast
# Playlist field
0x28: SPLFT_PLAYLIST, # Playlist
# Binary AND
0x3C: SPLFT_BINARY_AND, # Video Kind
}
# Date units for relative date rules
SPL_DATE_UNITS_MAP = {
1: "seconds",
60: "minutes",
3600: "hours",
86400: "days",
604800: "weeks",
2628000: "months", # ~30.4 days
}
def spl_get_field_type(field_id: int) -> int:
"""Determine SPL field type from field ID (equivalent to libgpod's itdb_splr_get_field_type)."""
return SPL_FIELD_TYPE_MAP.get(field_id, SPLFT_UNKNOWN)
# ============================================================
# MHOD Type 52/53 — Library Playlist Index / Jump Table
# ============================================================
# Both types share header structure (sort_type + count).
# Functions take body_offset = start of body data (MHOD chunk + header_length).
#
# Type 52 layout:
# +0x00: sort_type (4 bytes LE) — 3=title, 4=album, 5=artist, 7=genre, 18=composer
# +0x04: count (4 bytes LE) — number of index entries
# +0x08: padding (40 bytes)
# +0x30: indices (count × 4 bytes LE) — sorted track positions
#
# Type 53 layout:
# +0x00: sort_type (4 bytes LE) — must match corresponding type 52
# +0x04: count (4 bytes LE) — number of jump entries
# +0x08: padding (8 bytes)
# +0x10: entries (count × 12 bytes):
# letter (2 bytes UTF-16 LE) + pad (2 bytes) + start (4 bytes) + count (4 bytes)
SORT_TYPE_MAP = {
0x03: "title",
0x04: "album", # then disc/track number, then title
0x05: "artist", # then album, then disc/track number, then title
0x07: "genre", # then artist, then album, then disc/track number, then title
0x12: "composer", # then title
0x1D: "show", # iTunes 7.2+; secondary sort TBD
0x1E: "season_number", # iTunes 7.2+; secondary sort TBD
0x1F: "episode_number", # iTunes 7.2+; secondary sort TBD
0x23: "album_artist", # then artist (ignoring sort-artist), then album, disc/track, title
0x24: "artist_nosort", # artist (ignoring sort-artist), then album, disc/track, title
}
def mhod52_sort_type(data, body_offset) -> int:
return struct.unpack("<I", data[body_offset:body_offset + 4])[0]
def mhod52_count(data, body_offset) -> int:
return struct.unpack("<I", data[body_offset + 4:body_offset + 8])[0]
def mhod53_sort_type(data, body_offset) -> int:
return struct.unpack("<I", data[body_offset:body_offset + 4])[0]
def mhod53_count(data, body_offset) -> int:
return struct.unpack("<I", data[body_offset + 4:body_offset + 8])[0]
# ============================================================
# MHOD Type 100 — Playlist Position (MHIP context)
# ============================================================
# In MHIP context (body ≤ 20 bytes):
# +0x00: position (4 bytes LE) — 0-based track position in playlist
def mhod100_position(data, body_offset) -> int:
return struct.unpack("<I", data[body_offset:body_offset + 4])[0]
+15
View File
@@ -0,0 +1,15 @@
"""MHSD (DataSet) field definitions.
Declarative :class:`FieldDef` list for the MHSD chunk a dataset
container with a type field that determines its child.
"""
from .field_base import FieldDef, _u32
_S = "mhsd"
MHSD_HEADER_SIZE: int = 96
MHSD_FIELDS: list[FieldDef] = [
_u32("dataset_type", 0x0C, section_type=_S, required=True),
]
+35
View File
@@ -0,0 +1,35 @@
"""MHYP (Playlist) field definitions.
Declarative :class:`FieldDef` list for the MHYP chunk a single
playlist record.
"""
from .field_base import FieldDef, _u8, _u16, _u32, _u64, mac_to_unix, unix_to_mac
_S = "mhyp"
MHYP_HEADER_SIZE: int = 184
MHYP_FIELDS: list[FieldDef] = [
_u32("mhod_child_count", 0x0C, section_type=_S),
_u32("mhip_child_count", 0x10, section_type=_S),
_u8("master_flag", 0x14, section_type=_S),
_u8("flag1", 0x15, section_type=_S),
_u8("flag2", 0x16, section_type=_S),
_u8("flag3", 0x17, section_type=_S),
_u32("timestamp", 0x18, section_type=_S,
read_transform=mac_to_unix, write_transform=unix_to_mac),
_u64("playlist_id", 0x1C, section_type=_S),
_u32("unk0x24", 0x24, section_type=_S),
_u16("string_mhod_child_count", 0x28, section_type=_S),
_u16("podcast_flag", 0x2A, section_type=_S),
_u32("sort_order", 0x2C, section_type=_S),
# Extended
_u64("db_id_2", 0x3C, section_type=_S, min_header_length=0x44),
_u64("playlist_id_2", 0x44, section_type=_S, min_header_length=0x4C),
_u16("mhsd5_type", 0x50, section_type=_S, min_header_length=0x52),
_u16("mhsd5_type_2", 0x52, section_type=_S, min_header_length=0x54),
_u32("mhsd5_special_flag", 0x54, section_type=_S, min_header_length=0x58),
_u32("timestamp_2", 0x58, section_type=_S, min_header_length=0x5C,
read_transform=mac_to_unix, write_transform=unix_to_mac),
]
+148
View File
@@ -0,0 +1,148 @@
"""
iTunesDB Writer module for iOpenPod.
This module provides write support for iTunesDB (and iTunesCDB) files.
Supported devices:
- Pre-2007 iPods (1G-5G, Mini, Photo, Nano 1G-2G): No hash required
- iPod Classic (all gens), Nano 3G, Nano 4G: HASH58 (needs FireWire ID)
- iPod Nano 5G: HASH72 (requires HashInfo file from an iTunes sync)
- iPod Nano 6G/7G: HASHAB (needs FireWire ID + WASM runtime)
Usage:
from iTunesDB_Writer import write_checksum, detect_checksum_type
checksum_type = detect_checksum_type(ipod_path)
with open(itunesdb_path, 'rb') as f:
itdb_data = bytearray(f.read())
success = write_checksum(itdb_data, ipod_path)
"""
from ipod_device import ChecksumType
from ipod_device import detect_checksum_type, get_firewire_id
from .hash58 import (
compute_hash58,
write_hash58,
)
from .hash72 import (
compute_hash72,
write_hash72,
read_hash_info,
extract_hash_info,
extract_hash_info_to_dict,
)
from .hashab import (
compute_hashab,
write_hashab,
)
from .mhit_writer import TrackInfo, write_mhit
from iTunesDB_Shared.constants import (
MEDIA_TYPE_AUDIO,
MEDIA_TYPE_VIDEO,
MEDIA_TYPE_PODCAST,
MEDIA_TYPE_VIDEO_PODCAST,
MEDIA_TYPE_AUDIOBOOK,
MEDIA_TYPE_MUSIC_VIDEO,
MEDIA_TYPE_TV_SHOW,
MEDIA_TYPE_RINGTONE,
)
from .mhyp_writer import PlaylistInfo, write_playlist, write_mhyp
from .mhli_writer import write_mhli, write_mhii_artist, write_mhli_empty
from .mhod_spl_writer import (
SmartPlaylistPrefs,
SmartPlaylistRules,
SmartPlaylistRule,
prefs_from_parsed,
rules_from_parsed,
)
from .mhbd_writer import write_itunesdb, write_mhbd, extract_db_info
def write_checksum(itdb_data: bytearray, ipod_path: str) -> bool:
"""
Write appropriate checksum to iTunesDB based on device type.
Args:
itdb_data: Mutable bytearray of complete iTunesDB file
ipod_path: Mount point of iPod
Returns:
True if checksum was written successfully
Raises:
ValueError: For unsupported devices
"""
checksum_type = detect_checksum_type(ipod_path)
if checksum_type == ChecksumType.NONE:
# No hash needed
return True
elif checksum_type == ChecksumType.HASH58:
firewire_id = get_firewire_id(ipod_path)
write_hash58(itdb_data, firewire_id)
return True
elif checksum_type == ChecksumType.HASH72:
write_hash72(itdb_data, ipod_path)
return True
elif checksum_type == ChecksumType.HASHAB:
firewire_id = get_firewire_id(ipod_path)
write_hashab(itdb_data, firewire_id)
return True
else:
raise ValueError(
f"Unsupported checksum type: {checksum_type}."
)
__all__ = [
'ChecksumType',
'detect_checksum_type',
'get_firewire_id',
'compute_hash58',
'write_hash58',
'compute_hash72',
'write_hash72',
'read_hash_info',
'extract_hash_info',
'extract_hash_info_to_dict',
'compute_hashab',
'write_hashab',
'write_checksum',
# Writer
'TrackInfo',
'write_mhit',
# Media type constants
'MEDIA_TYPE_AUDIO',
'MEDIA_TYPE_VIDEO',
'MEDIA_TYPE_PODCAST',
'MEDIA_TYPE_VIDEO_PODCAST',
'MEDIA_TYPE_AUDIOBOOK',
'MEDIA_TYPE_MUSIC_VIDEO',
'MEDIA_TYPE_TV_SHOW',
'MEDIA_TYPE_RINGTONE',
'write_mhbd',
'write_itunesdb',
'extract_db_info',
# Artist list
'write_mhli',
'write_mhii_artist',
'write_mhli_empty',
# Playlists
'PlaylistInfo',
'write_playlist',
'write_mhyp',
'SmartPlaylistPrefs',
'SmartPlaylistRules',
'SmartPlaylistRule',
'prefs_from_parsed',
'rules_from_parsed',
]
+280
View File
@@ -0,0 +1,280 @@
"""
HASH58 implementation for iPod Classic (all gens), Nano 3G, and Nano 4G.
Ported from libgpod's itdb_hash58.c.
This is the complete, working implementation for signing iTunesDB files
for devices that use hashing_scheme=1 (HASH58).
Usage:
from hash58 import write_hash58
with open("iTunesDB", "rb") as f:
itdb_data = bytearray(f.read())
firewire_id = bytes.fromhex("0011223344556677") # From SysInfo
write_hash58(itdb_data, firewire_id)
with open("iTunesDB", "wb") as f:
f.write(itdb_data)
"""
import hashlib
from math import gcd
from iTunesDB_Shared.mhbd_defs import (
MHBD_OFFSET_DB_ID as OFFSET_DB_ID,
MHBD_OFFSET_HASHING_SCHEME as OFFSET_HASHING_SCHEME,
MHBD_OFFSET_UNK_0x32 as OFFSET_UNK_0x32,
MHBD_OFFSET_HASH58 as OFFSET_HASH58,
)
# AES S-Box (from libgpod itdb_hash58.c lines 45-76)
TABLE1 = bytes([
0x63, 0x7C, 0x77, 0x7B, 0xF2, 0x6B, 0x6F, 0xC5,
0x30, 0x01, 0x67, 0x2B, 0xFE, 0xD7, 0xAB, 0x76,
0xCA, 0x82, 0xC9, 0x7D, 0xFA, 0x59, 0x47, 0xF0,
0xAD, 0xD4, 0xA2, 0xAF, 0x9C, 0xA4, 0x72, 0xC0,
0xB7, 0xFD, 0x93, 0x26, 0x36, 0x3F, 0xF7, 0xCC,
0x34, 0xA5, 0xE5, 0xF1, 0x71, 0xD8, 0x31, 0x15,
0x04, 0xC7, 0x23, 0xC3, 0x18, 0x96, 0x05, 0x9A,
0x07, 0x12, 0x80, 0xE2, 0xEB, 0x27, 0xB2, 0x75,
0x09, 0x83, 0x2C, 0x1A, 0x1B, 0x6E, 0x5A, 0xA0,
0x52, 0x3B, 0xD6, 0xB3, 0x29, 0xE3, 0x2F, 0x84,
0x53, 0xD1, 0x00, 0xED, 0x20, 0xFC, 0xB1, 0x5B,
0x6A, 0xCB, 0xBE, 0x39, 0x4A, 0x4C, 0x58, 0xCF,
0xD0, 0xEF, 0xAA, 0xFB, 0x43, 0x4D, 0x33, 0x85,
0x45, 0xF9, 0x02, 0x7F, 0x50, 0x3C, 0x9F, 0xA8,
0x51, 0xA3, 0x40, 0x8F, 0x92, 0x9D, 0x38, 0xF5,
0xBC, 0xB6, 0xDA, 0x21, 0x10, 0xFF, 0xF3, 0xD2,
0xCD, 0x0C, 0x13, 0xEC, 0x5F, 0x97, 0x44, 0x17,
0xC4, 0xA7, 0x7E, 0x3D, 0x64, 0x5D, 0x19, 0x73,
0x60, 0x81, 0x4F, 0xDC, 0x22, 0x2A, 0x90, 0x88,
0x46, 0xEE, 0xB8, 0x14, 0xDE, 0x5E, 0x0B, 0xDB,
0xE0, 0x32, 0x3A, 0x0A, 0x49, 0x06, 0x24, 0x5C,
0xC2, 0xD3, 0xAC, 0x62, 0x91, 0x95, 0xE4, 0x79,
0xE7, 0xC8, 0x37, 0x6D, 0x8D, 0xD5, 0x4E, 0xA9,
0x6C, 0x56, 0xF4, 0xEA, 0x65, 0x7A, 0xAE, 0x08,
0xBA, 0x78, 0x25, 0x2E, 0x1C, 0xA6, 0xB4, 0xC6,
0xE8, 0xDD, 0x74, 0x1F, 0x4B, 0xBD, 0x8B, 0x8A,
0x70, 0x3E, 0xB5, 0x66, 0x48, 0x03, 0xF6, 0x0E,
0x61, 0x35, 0x57, 0xB9, 0x86, 0xC1, 0x1D, 0x9E,
0xE1, 0xF8, 0x98, 0x11, 0x69, 0xD9, 0x8E, 0x94,
0x9B, 0x1E, 0x87, 0xE9, 0xCE, 0x55, 0x28, 0xDF,
0x8C, 0xA1, 0x89, 0x0D, 0xBF, 0xE6, 0x42, 0x68,
0x41, 0x99, 0x2D, 0x0F, 0xB0, 0x54, 0xBB, 0x16
])
# AES Inverse S-Box (from libgpod itdb_hash58.c lines 78-115)
TABLE2 = bytes([
0x52, 0x09, 0x6A, 0xD5, 0x30, 0x36, 0xA5, 0x38,
0xBF, 0x40, 0xA3, 0x9E, 0x81, 0xF3, 0xD7, 0xFB,
0x7C, 0xE3, 0x39, 0x82, 0x9B, 0x2F, 0xFF, 0x87,
0x34, 0x8E, 0x43, 0x44, 0xC4, 0xDE, 0xE9, 0xCB,
0x54, 0x7B, 0x94, 0x32, 0xA6, 0xC2, 0x23, 0x3D,
0xEE, 0x4C, 0x95, 0x0B, 0x42, 0xFA, 0xC3, 0x4E,
0x08, 0x2E, 0xA1, 0x66, 0x28, 0xD9, 0x24, 0xB2,
0x76, 0x5B, 0xA2, 0x49, 0x6D, 0x8B, 0xD1, 0x25,
0x72, 0xF8, 0xF6, 0x64, 0x86, 0x68, 0x98, 0x16,
0xD4, 0xA4, 0x5C, 0xCC, 0x5D, 0x65, 0xB6, 0x92,
0x6C, 0x70, 0x48, 0x50, 0xFD, 0xED, 0xB9, 0xDA,
0x5E, 0x15, 0x46, 0x57, 0xA7, 0x8D, 0x9D, 0x84,
0x90, 0xD8, 0xAB, 0x00, 0x8C, 0xBC, 0xD3, 0x0A,
0xF7, 0xE4, 0x58, 0x05, 0xB8, 0xB3, 0x45, 0x06,
0xD0, 0x2C, 0x1E, 0x8F, 0xCA, 0x3F, 0x0F, 0x02,
0xC1, 0xAF, 0xBD, 0x03, 0x01, 0x13, 0x8A, 0x6B,
0x3A, 0x91, 0x11, 0x41, 0x4F, 0x67, 0xDC, 0xEA,
0x97, 0xF2, 0xCF, 0xCE, 0xF0, 0xB4, 0xE6, 0x73,
0x96, 0xAC, 0x74, 0x22, 0xE7, 0xAD, 0x35, 0x85,
0xE2, 0xF9, 0x37, 0xE8, 0x1C, 0x75, 0xDF, 0x6E,
0x47, 0xF1, 0x1A, 0x71, 0x1D, 0x29, 0xC5, 0x89,
0x6F, 0xB7, 0x62, 0x0E, 0xAA, 0x18, 0xBE, 0x1B,
0xFC, 0x56, 0x3E, 0x4B, 0xC6, 0xD2, 0x79, 0x20,
0x9A, 0xDB, 0xC0, 0xFE, 0x78, 0xCD, 0x5A, 0xF4,
0x1F, 0xDD, 0xA8, 0x33, 0x88, 0x07, 0xC7, 0x31,
0xB1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xEC, 0x5F,
0x60, 0x51, 0x7F, 0xA9, 0x19, 0xB5, 0x4A, 0x0D,
0x2D, 0xE5, 0x7A, 0x9F, 0x93, 0xC9, 0x9C, 0xEF,
0xA0, 0xE0, 0x3B, 0x4D, 0xAE, 0x2A, 0xF5, 0xB0,
0xC8, 0xEB, 0xBB, 0x3C, 0x83, 0x53, 0x99, 0x61,
0x17, 0x2B, 0x04, 0x7E, 0xBA, 0x77, 0xD6, 0x26,
0xE1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0C, 0x7D
])
# Fixed bytes for key derivation (from libgpod itdb_hash58.c lines 113-115)
FIXED = bytes([
0x67, 0x23, 0xFE, 0x30, 0x45, 0x33, 0xF8, 0x90, 0x99,
0x21, 0x07, 0xC1, 0xD0, 0x12, 0xB2, 0xA1, 0x07, 0x81
])
# Hash scheme identifier for HASH58
ITDB_CHECKSUM_HASH58 = 1
def _lcm(a: int, b: int) -> int:
"""Least common multiple."""
if a == 0 or b == 0:
return 1
return (a * b) // gcd(a, b)
def _generate_key(firewire_id: bytes) -> bytes:
"""
Generate a 64-byte HMAC key from the 8-byte FireWire ID.
Algorithm:
1. Take LCM of each pair of consecutive bytes in the FireWire ID
2. Use the high and low bytes of each LCM to index into TABLE1 and TABLE2
3. SHA1 hash the FIXED bytes + derived bytes
4. Pad to 64 bytes
"""
if len(firewire_id) < 8:
raise ValueError(f"FireWire ID must be at least 8 bytes, got {len(firewire_id)}")
y = bytearray(16)
for i in range(4):
a = firewire_id[i * 2]
b = firewire_id[i * 2 + 1]
cur_lcm = _lcm(a, b)
hi = (cur_lcm >> 8) & 0xFF
lo = cur_lcm & 0xFF
y[i * 4] = TABLE1[hi]
y[i * 4 + 1] = TABLE2[hi]
y[i * 4 + 2] = TABLE1[lo]
y[i * 4 + 3] = TABLE2[lo]
# SHA1(FIXED + y), then pad to 64 bytes
h = hashlib.sha1(FIXED + y).digest()
key = bytearray(64)
key[:len(h)] = h
return bytes(key)
def compute_hash58(firewire_id: bytes, itdb_data: bytes) -> bytes:
"""
Compute HMAC-SHA1 of iTunesDB data using FireWire ID-derived key.
This is a standard HMAC-SHA1 implementation with a custom key derivation.
Args:
firewire_id: 8-20 byte FireWire GUID from SysInfo
itdb_data: Complete iTunesDB file contents (with hash fields zeroed)
Returns:
20-byte SHA1 hash to write at offset 0x58
"""
key = _generate_key(firewire_id)
# HMAC-SHA1: H(K XOR opad, H(K XOR ipad, message))
# Inner hash
inner_key = bytes(b ^ 0x36 for b in key)
inner_hash = hashlib.sha1(inner_key + itdb_data).digest()
# Outer hash
outer_key = bytes(b ^ 0x5c for b in key)
return hashlib.sha1(outer_key + inner_hash).digest()
def write_hash58(itdb_data: bytearray, firewire_id: bytes) -> None:
"""
Compute and write HASH58 checksum to iTunesDB data in-place.
From libgpod itdb_hash58_write_hash():
1. Backs up db_id and unk_0x32 fields
2. Zeros out db_id, unk_0x32, and hash58 fields (required for hash computation)
3. Sets hashing_scheme to HASH58 (1)
4. Computes HMAC-SHA1 hash
5. Writes hash to offset 0x58
6. Restores backed up fields
NOTE: Unlike HASH72, HASH58 zeros unk_0x32 before computing the hash!
This is a key difference between the two algorithms.
Args:
itdb_data: Mutable bytearray of complete iTunesDB file
firewire_id: 8-20 byte FireWire GUID from /iPod_Control/Device/SysInfo
Raises:
ValueError: If iTunesDB is too small or FireWire ID is invalid
"""
if len(itdb_data) < 0x6C:
raise ValueError(f"iTunesDB file too small ({len(itdb_data)} bytes), need at least 0x6C")
# Verify this is an mhbd header
if itdb_data[:4] != b'mhbd':
raise ValueError("Invalid iTunesDB: expected 'mhbd' header")
# Backup fields that will be zeroed
backup_db_id = bytes(itdb_data[OFFSET_DB_ID:OFFSET_DB_ID + 8])
backup_unk32 = bytes(itdb_data[OFFSET_UNK_0x32:OFFSET_UNK_0x32 + 20])
# Zero out fields for hash computation
itdb_data[OFFSET_DB_ID:OFFSET_DB_ID + 8] = b'\x00' * 8
itdb_data[OFFSET_UNK_0x32:OFFSET_UNK_0x32 + 20] = b'\x00' * 20
itdb_data[OFFSET_HASH58:OFFSET_HASH58 + 20] = b'\x00' * 20
# Set hashing scheme to HASH58
itdb_data[OFFSET_HASHING_SCHEME:OFFSET_HASHING_SCHEME + 2] = \
ITDB_CHECKSUM_HASH58.to_bytes(2, 'little')
# Compute and write hash
hash_val = compute_hash58(firewire_id, bytes(itdb_data))
if len(hash_val) != 20:
raise RuntimeError(f"Hash computation failed: expected 20 bytes, got {len(hash_val)}")
itdb_data[OFFSET_HASH58:OFFSET_HASH58 + 20] = hash_val
# Restore backed up fields
itdb_data[OFFSET_DB_ID:OFFSET_DB_ID + 8] = backup_db_id
itdb_data[OFFSET_UNK_0x32:OFFSET_UNK_0x32 + 20] = backup_unk32
def read_firewire_id(ipod_path: str) -> bytes:
"""Return the FireWire GUID for the connected iPod.
Reads from the centralised DeviceInfo store. Raises if not available.
"""
from ipod_device import get_current_device
device = get_current_device()
if device is not None:
fwid = device.firewire_id_bytes
if fwid:
return fwid
raise RuntimeError(
"FireWire GUID not available. Device info was not populated "
"by the device scanner."
)
if __name__ == "__main__":
# Example usage
import sys
if len(sys.argv) < 3:
print("Usage: python hash58.py <ipod_path> <itunesdb_path>")
print("Example: python hash58.py E: E:/iPod_Control/iTunes/iTunesDB")
sys.exit(1)
ipod_path = sys.argv[1]
itunesdb_path = sys.argv[2]
try:
firewire_id = read_firewire_id(ipod_path)
print(f"FireWire ID: {firewire_id.hex()}")
with open(itunesdb_path, 'rb') as f:
itdb_data = bytearray(f.read())
print(f"Read {len(itdb_data)} bytes from iTunesDB")
write_hash58(itdb_data, firewire_id)
print("Hash computed successfully!")
# Write back (uncomment to actually write)
# with open(itunesdb_path, 'wb') as f:
# f.write(itdb_data)
# print("iTunesDB updated!")
except Exception as e:
print(f"Error: {e}")
sys.exit(1)
+476
View File
@@ -0,0 +1,476 @@
"""
HASH72 implementation for iPod Nano 5G.
Ported from libgpod's itdb_hash72.c.
Note: iTunes also writes a HASH72 signature on iPod Classic devices, but
the Classic firmware only checks HASH58 (scheme=1). We preserve HASH72
from a reference database when available but do not require it for Classic.
IMPORTANT: This requires a HashInfo file that must be extracted from a valid
iTunes sync. The HashInfo file contains the IV and random bytes needed to
generate signatures.
If you don't have a HashInfo file:
1. Sync once with iTunes (creates /iPod_Control/Device/HashInfo)
2. OR use extract_hash_info() with a known-good iTunesDB from iTunes
Usage:
from hash72 import write_hash72
with open("iTunesDB", "rb") as f:
itdb_data = bytearray(f.read())
# Requires HashInfo file to exist at /iPod_Control/Device/HashInfo
write_hash72(itdb_data, ipod_path="/media/ipod")
with open("iTunesDB", "wb") as f:
f.write(itdb_data)
"""
import hashlib
import os
from typing import Optional
from iTunesDB_Shared.mhbd_defs import (
MHBD_OFFSET_DB_ID as OFFSET_DB_ID,
MHBD_OFFSET_HASHING_SCHEME as OFFSET_HASHING_SCHEME,
MHBD_OFFSET_HASH58 as OFFSET_HASH58,
MHBD_OFFSET_HASH72 as OFFSET_HASH72,
)
# AES-128 key (from libgpod itdb_hash72.c line 40)
AES_KEY = bytes([
0x61, 0x8c, 0xa1, 0x0d, 0xc7, 0xf5, 0x7f, 0xd3,
0xb4, 0x72, 0x3e, 0x08, 0x15, 0x74, 0x63, 0xd7
])
# Hash scheme identifier for HASH72
ITDB_CHECKSUM_HASH72 = 2
# HashInfo file structure
HASHINFO_HEADER = b"HASHv0"
HASHINFO_HEADER_LEN = 6
HASHINFO_UUID_LEN = 20
HASHINFO_RNDPART_LEN = 12
HASHINFO_IV_LEN = 16
class HashInfo:
"""Parsed HashInfo file data."""
def __init__(self, uuid: bytes, rndpart: bytes, iv: bytes):
self.uuid = uuid
self.rndpart = rndpart
self.iv = iv
def _get_hash_info_path(ipod_path: str) -> str:
"""Get path to HashInfo file."""
return os.path.join(ipod_path, "iPod_Control", "Device", "HashInfo")
def read_hash_info(ipod_path: str) -> Optional[HashInfo]:
"""
Read and parse HashInfo file from iPod.
HashInfo structure (54 bytes total):
- header[6]: "HASHv0"
- uuid[20]: Device UUID (should match FirewireGuid)
- rndpart[12]: Random bytes for signature
- iv[16]: AES initialization vector
Args:
ipod_path: Mount point of iPod
Returns:
HashInfo object or None if file doesn't exist
"""
# Check centralized device_info store first
try:
from ipod_device import get_current_device
dev = get_current_device()
if dev and dev.hash_info_iv and dev.hash_info_rndpart:
return HashInfo(uuid=b'\x00' * 20, rndpart=dev.hash_info_rndpart, iv=dev.hash_info_iv)
except Exception:
pass
# Fallback: read from disk
path = _get_hash_info_path(ipod_path)
if not os.path.exists(path):
return None
with open(path, 'rb') as f:
data = f.read()
if len(data) < 54:
return None
if data[:6] != HASHINFO_HEADER:
return None
# Parse structure
uuid = data[6:26]
rndpart = data[26:38]
iv = data[38:54]
return HashInfo(uuid, rndpart, iv)
def write_hash_info(ipod_path: str, uuid: bytes, iv: bytes, rndpart: bytes) -> bool:
"""
Write HashInfo file to iPod.
Args:
ipod_path: Mount point of iPod
uuid: 20-byte device UUID
iv: 16-byte AES IV
rndpart: 12-byte random bytes
Returns:
True if successful
"""
if len(uuid) != 20 or len(iv) != 16 or len(rndpart) != 12:
return False
data = HASHINFO_HEADER + uuid + rndpart + iv
path = _get_hash_info_path(ipod_path)
device_dir = os.path.dirname(path)
os.makedirs(device_dir, exist_ok=True)
with open(path, 'wb') as f:
f.write(data)
return True
def _compute_itunesdb_sha1(itdb_data: bytearray) -> bytes:
"""
Compute SHA1 of iTunesDB with hash fields zeroed.
From libgpod itdb_hash72_compute_itunesdb_sha1():
- db_id (offset 0x18, 8 bytes) is zeroed
- hash58 (offset 0x58, 20 bytes) is zeroed
- hash72 (offset 0x72, 46 bytes) is zeroed
NOTE: Unlike HASH58, unk_0x32 is NOT zeroed for HASH72!
libgpod backs it up and restores it, but since it's never zeroed,
we don't need to do anything with it.
"""
# Work on a copy to avoid modifying original
data = bytearray(itdb_data)
# Zero fields for hash computation (same as libgpod)
# hash58 lives at offset 0x58 (20 bytes), hash72 at 0x72 (46 bytes)
data[OFFSET_DB_ID:OFFSET_DB_ID + 8] = b'\x00' * 8
data[OFFSET_HASH58:OFFSET_HASH58 + 20] = b'\x00' * 20
data[OFFSET_HASH72:OFFSET_HASH72 + 46] = b'\x00' * 46
return hashlib.sha1(bytes(data)).digest()
def _hash_generate(sha1: bytes, iv: bytes, rndpart: bytes) -> bytes:
"""
Generate 46-byte signature using AES encryption.
Signature format:
- bytes 0-1: 0x01 0x00 (prefix)
- bytes 2-13: rndpart (12 bytes)
- bytes 14-45: AES-CBC encrypted (sha1 + rndpart) (32 bytes)
Args:
sha1: 20-byte SHA1 of iTunesDB
iv: 16-byte initialization vector
rndpart: 12-byte random bytes
Returns:
46-byte signature
"""
try:
from Crypto.Cipher import AES
except ImportError:
try:
from Cryptodome.Cipher import AES # type: ignore[import-not-found]
except ImportError:
raise ImportError(
"PyCryptodome is required for HASH72. "
"Install with: pip install pycryptodome"
)
# Plaintext: sha1 (20 bytes) + rndpart (12 bytes) = 32 bytes
plaintext = sha1 + rndpart
# AES-CBC encrypt
cipher = AES.new(AES_KEY, AES.MODE_CBC, iv)
encrypted = cipher.encrypt(plaintext)
# Build signature
signature = bytearray(46)
signature[0] = 0x01
signature[1] = 0x00
signature[2:14] = rndpart
signature[14:46] = encrypted
return bytes(signature)
def _hash_extract(signature: bytes, sha1: bytes) -> Optional[tuple]:
"""
Extract IV and random bytes from a valid signature.
This can be used to create a HashInfo file from a known-good
iTunes-generated iTunesDB.
Algorithm from libgpod itdb_hash72.c hash_extract():
The signature was created by:
C = AES_encrypt_CBC(plaintext, IV) where plaintext = sha1 + rndpart
In CBC mode, the first block is:
C_0 = AES_encrypt(P_0 XOR IV) where P_0 = sha1[:16]
To recover IV, we decrypt C_0 using sha1[:16] as a fake IV:
output = AES_decrypt(C_0) XOR sha1[:16]
= (P_0 XOR IV) XOR sha1[:16]
= (sha1[:16] XOR IV) XOR sha1[:16]
= IV
The libgpod code also does a sanity check comparing plaintext[16:32]
to output[16:32], but since only the first 16 bytes are decrypted,
output[16:32] is always equal to plaintext[16:32]. We keep this check
for compatibility.
Args:
signature: 46-byte signature from valid iTunesDB
sha1: 20-byte SHA1 that was used to generate the signature
Returns:
(iv, rndpart) tuple or None if invalid
"""
try:
from Crypto.Cipher import AES
except ImportError:
try:
from Cryptodome.Cipher import AES # type: ignore[import-not-found]
except ImportError:
raise ImportError(
"PyCryptodome is required for HASH72. "
"Install with: pip install pycryptodome"
)
if len(signature) < 46 or signature[0] != 0x01 or signature[1] != 0x00:
return None
# Build plaintext = sha1 + rndpart (matches libgpod)
rndpart = signature[2:14]
plaintext = bytearray(32)
plaintext[:20] = sha1
plaintext[20:32] = rndpart
# Initialize output as copy of plaintext (matches libgpod: memcpy(output, plaintext, 32))
output = bytearray(plaintext)
# AES-CBC decrypt first 16 bytes only, using sha1[:16] as IV
# This recovers the real IV through the XOR cancellation described above
cipher = AES.new(AES_KEY, AES.MODE_CBC, bytes(plaintext[:16]))
decrypted_block = cipher.decrypt(bytes(signature[14:30]))
output[:16] = decrypted_block
# Sanity check from libgpod - always passes since output[16:32] was
# copied from plaintext[16:32] and never modified
if bytes(plaintext[16:32]) != bytes(output[16:32]):
return None
# The IV is now in output[:16]
iv = bytes(output[:16])
return (iv, bytes(rndpart))
def extract_hash_info(ipod_path: str, valid_itdb_data: bytes) -> bool:
"""
Extract HashInfo from a valid iTunes-generated iTunesDB.
Use this when you have an iTunesDB that was created by iTunes
but you don't have a HashInfo file.
Args:
ipod_path: Mount point of iPod
valid_itdb_data: Contents of valid iTunes-generated iTunesDB
Returns:
True if HashInfo was successfully extracted and saved
"""
if len(valid_itdb_data) < 0xA0:
return False
if valid_itdb_data[:4] != b'mhbd':
return False
# Get existing hash72 from CORRECT offset (0x72)
hash72 = bytes(valid_itdb_data[OFFSET_HASH72:OFFSET_HASH72 + 46])
# Check for valid signature marker
if hash72[0:2] != bytes([0x01, 0x00]):
# Not a valid hash72 signature
return False
# Compute SHA1
itdb_copy = bytearray(valid_itdb_data)
sha1 = _compute_itunesdb_sha1(itdb_copy)
# Extract IV and rndpart
result = _hash_extract(hash72, sha1)
if result is None:
return False
iv, rndpart = result
# Get UUID from device (or use zeros if not available)
try:
from .hash58 import read_firewire_id
fw_id = read_firewire_id(ipod_path)
uuid = bytearray(20)
uuid[:len(fw_id)] = fw_id
except Exception:
uuid = bytes(20)
return write_hash_info(ipod_path, bytes(uuid), iv, rndpart)
def extract_hash_info_to_dict(valid_itdb_data: bytes) -> dict | None:
"""
Extract HashInfo from a valid iTunes-generated iTunesDB.
Returns the extracted info as a dict instead of writing to disk.
Args:
valid_itdb_data: Contents of valid iTunes-generated iTunesDB
Returns:
Dict with 'iv' and 'rndpart' keys, or None if extraction failed
"""
if len(valid_itdb_data) < 0xA0:
return None
if valid_itdb_data[:4] != b'mhbd':
return None
# Get existing hash72 from CORRECT offset (0x72)
hash72 = bytes(valid_itdb_data[OFFSET_HASH72:OFFSET_HASH72 + 46])
# Check for valid signature marker
if hash72[0:2] != bytes([0x01, 0x00]):
return None
# Compute SHA1
itdb_copy = bytearray(valid_itdb_data)
sha1 = _compute_itunesdb_sha1(itdb_copy)
# Extract IV and rndpart
result = _hash_extract(hash72, sha1)
if result is None:
return None
iv, rndpart = result
return {'iv': iv, 'rndpart': rndpart}
def compute_hash72(ipod_path: str, itdb_data: bytes) -> bytes:
"""
Compute HASH72 signature for iTunesDB data.
Args:
ipod_path: Mount point of iPod (for reading HashInfo)
itdb_data: Complete iTunesDB file contents
Returns:
46-byte signature
Raises:
FileNotFoundError: If HashInfo file doesn't exist
"""
hash_info = read_hash_info(ipod_path)
if hash_info is None:
raise FileNotFoundError(
f"HashInfo file not found at {_get_hash_info_path(ipod_path)}. "
"Sync once with iTunes to create it, or use extract_hash_info() "
"with a valid iTunes-generated iTunesDB."
)
sha1 = _compute_itunesdb_sha1(bytearray(itdb_data))
return _hash_generate(sha1, hash_info.iv, hash_info.rndpart)
def write_hash72(itdb_data: bytearray, ipod_path: str) -> None:
"""
Compute and write HASH72 checksum to iTunesDB data in-place.
Args:
itdb_data: Mutable bytearray of complete iTunesDB file
ipod_path: Mount point of iPod (for reading HashInfo)
Raises:
ValueError: If iTunesDB is too small
FileNotFoundError: If HashInfo file doesn't exist
"""
if len(itdb_data) < 0x6C:
raise ValueError(f"iTunesDB file too small ({len(itdb_data)} bytes)")
if itdb_data[:4] != b'mhbd':
raise ValueError("Invalid iTunesDB: expected 'mhbd' header")
# Set hashing scheme
itdb_data[OFFSET_HASHING_SCHEME:OFFSET_HASHING_SCHEME + 2] = \
ITDB_CHECKSUM_HASH72.to_bytes(2, 'little')
# Compute and write signature
signature = compute_hash72(ipod_path, bytes(itdb_data))
itdb_data[OFFSET_HASH72:OFFSET_HASH72 + 46] = signature
if __name__ == "__main__":
import sys
if len(sys.argv) < 3:
print("Usage: python hash72.py <ipod_path> <itunesdb_path>")
print("Example: python hash72.py /media/ipod /media/ipod/iPod_Control/iTunes/iTunesDB")
sys.exit(1)
ipod_path = sys.argv[1]
itunesdb_path = sys.argv[2]
try:
# Check for HashInfo
hash_info = read_hash_info(ipod_path)
if hash_info:
print(f"HashInfo found: IV={hash_info.iv.hex()[:16]}...")
else:
print("HashInfo not found. Attempting to extract from iTunesDB...")
with open(itunesdb_path, 'rb') as f:
itdb_data = f.read()
if extract_hash_info(ipod_path, itdb_data):
print("HashInfo extracted and saved successfully!")
else:
print("Failed to extract HashInfo. Sync with iTunes first.")
sys.exit(1)
with open(itunesdb_path, 'rb') as f:
itdb_data = bytearray(f.read())
print(f"Read {len(itdb_data)} bytes from iTunesDB")
write_hash72(itdb_data, ipod_path)
print("Hash computed successfully!")
# Write back (uncomment to actually write)
# with open(itunesdb_path, 'wb') as f:
# f.write(itdb_data)
# print("iTunesDB updated!")
except Exception as e:
print(f"Error: {e}")
import traceback
traceback.print_exc()
sys.exit(1)
+287
View File
@@ -0,0 +1,287 @@
"""
HASHAB implementation for iPod Nano 6G and 7G.
Uses a WebAssembly module (calcHashAB.wasm) from dstaley/hashab a clean-room
reimplementation of Apple's white-box AES signing algorithm. The WASM binary
is executed via wasmtime-py, giving cross-platform support without compiling
native code.
Algorithm overview (4 phases inside the WASM module):
1. CBC-MAC compression of UUID with AES
2. Key material expansion (44 190 bytes)
3. Initial buffer generation (190 16 bytes)
4. White-box AES-128 encryption
The output is a 57-byte signature written at mhbd offset 0xAB. This is
analogous to HASH58 (20 bytes at 0x58) and HASH72 (46 bytes at 0x72).
Source: https://github.com/dstaley/hashab (The Unlicense)
WASM release: https://github.com/dstaley/hashab/releases/tag/2025-01-04
Usage:
from hashab import write_hashab
with open("iTunesDB", "rb") as f:
itdb_data = bytearray(f.read())
firewire_id = bytes.fromhex("0011223344556677") # From SysInfo
write_hashab(itdb_data, firewire_id)
with open("iTunesDB", "wb") as f:
f.write(itdb_data)
"""
import hashlib
import logging
from pathlib import Path
from iTunesDB_Shared.mhbd_defs import (
MHBD_OFFSET_DB_ID as OFFSET_DB_ID,
MHBD_OFFSET_HASHING_SCHEME as OFFSET_HASHING_SCHEME,
MHBD_OFFSET_UNK_0x32 as OFFSET_UNK_0x32,
MHBD_OFFSET_HASH58 as OFFSET_HASH58,
MHBD_OFFSET_HASH72 as OFFSET_HASH72,
MHBD_OFFSET_HASHAB as OFFSET_HASHAB,
)
logger = logging.getLogger(__name__)
HASHAB_SIZE = 57
ITDB_CHECKSUM_HASHAB = 4 # hashing_scheme value for HASHAB
# Path to the WASM module (shipped alongside this file)
_WASM_DIR = Path(__file__).parent / "wasm"
_WASM_PATH = _WASM_DIR / "calcHashAB.wasm"
# Lazy-loaded WASM engine (expensive to create — reuse across calls)
_wasm_instance = None
_wasm_store = None
def _get_wasm_instance():
"""Load the WASM module and return (store, instance).
The module exports:
memory linear memory
getInputSha1() returns pointer to 20-byte SHA1 input buffer
getInputUuid() returns pointer to 8-byte UUID input buffer
getOutput() returns pointer to 57-byte output buffer
calculateHash() run the hash computation
"""
global _wasm_instance, _wasm_store
if _wasm_instance is not None:
return _wasm_store, _wasm_instance
try:
import wasmtime
except ImportError:
raise ImportError(
"wasmtime is required for HASHAB (iPod Nano 6G/7G). "
"Install with: uv add wasmtime or pip install wasmtime"
)
if not _WASM_PATH.exists():
raise FileNotFoundError(
f"WASM module not found at {_WASM_PATH}. "
"Download calcHashAB.wasm from "
"https://github.com/dstaley/hashab/releases/tag/2025-01-04"
)
engine = wasmtime.Engine()
store = wasmtime.Store(engine)
module = wasmtime.Module.from_file(engine, str(_WASM_PATH))
instance = wasmtime.Instance(store, module, [])
_wasm_store = store
_wasm_instance = instance
logger.debug("HASHAB WASM module loaded from %s", _WASM_PATH)
return store, instance
def compute_hashab(sha1_digest: bytes, uuid: bytes) -> bytes:
"""
Compute 57-byte HASHAB signature using the WASM module.
Args:
sha1_digest: 20-byte SHA1 hash of the iTunesDB (with hash fields zeroed)
uuid: 8-byte FireWire GUID / UUID from SysInfo
Returns:
57-byte signature to write at mhbd offset 0xAB
"""
if len(sha1_digest) != 20:
raise ValueError(f"SHA1 must be 20 bytes, got {len(sha1_digest)}")
if len(uuid) < 8:
raise ValueError(f"UUID must be at least 8 bytes, got {len(uuid)}")
store, instance = _get_wasm_instance()
# Get exported functions and memory
# wasmtime stubs type exports() return as a union; runtime types are correct
exports = instance.exports(store) # type: ignore[arg-type]
memory = exports["memory"]
get_input_sha1 = exports["getInputSha1"]
get_input_uuid = exports["getInputUuid"]
get_output = exports["getOutput"]
calculate_hash = exports["calculateHash"]
# Get pointers into WASM linear memory
sha1_ptr = get_input_sha1(store) # type: ignore[misc]
uuid_ptr = get_input_uuid(store) # type: ignore[misc]
output_ptr = get_output(store) # type: ignore[misc]
# Write inputs into WASM memory
mem_data = memory.data_ptr(store) # type: ignore[union-attr]
# Write SHA1 (20 bytes)
for i in range(20):
mem_data[sha1_ptr + i] = sha1_digest[i]
# Write UUID (8 bytes)
for i in range(8):
mem_data[uuid_ptr + i] = uuid[i]
# Execute the hash computation
calculate_hash(store) # type: ignore[misc]
# Read 57-byte output
result = bytes(mem_data[output_ptr + i] for i in range(HASHAB_SIZE))
logger.debug("HASHAB computed: %s", result[:4].hex())
return result
def _compute_itunesdb_sha1_for_hashab(itdb_data: bytearray) -> bytes:
"""
Compute SHA1 of iTunesDB with all hash fields zeroed for HASHAB.
Zeroed fields before hashing:
- db_id (offset 0x18, 8 bytes)
- unk_0x32 (offset 0x32, 20 bytes)
- hash58 (offset 0x58, 20 bytes)
- hash72 (offset 0x72, 46 bytes)
- hashAB (offset 0xAB, 57 bytes)
We zero unk_0x32 (matching HASH58 behavior) because HASHAB devices
(Nano 6G/7G) also maintain hash58 compatibility fields.
"""
data = bytearray(itdb_data)
data[OFFSET_DB_ID:OFFSET_DB_ID + 8] = b'\x00' * 8
data[OFFSET_UNK_0x32:OFFSET_UNK_0x32 + 20] = b'\x00' * 20
data[OFFSET_HASH58:OFFSET_HASH58 + 20] = b'\x00' * 20
data[OFFSET_HASH72:OFFSET_HASH72 + 46] = b'\x00' * 46
data[OFFSET_HASHAB:OFFSET_HASHAB + HASHAB_SIZE] = b'\x00' * HASHAB_SIZE
return hashlib.sha1(bytes(data)).digest()
def write_hashab(itdb_data: bytearray, firewire_id: bytes) -> None:
"""
Compute and write HASHAB signature to iTunesDB data in-place.
Steps:
1. Zero db_id, unk_0x32, hash58, hash72, hashAB
2. Set hashing_scheme to 4 (HASHAB)
3. Compute SHA1 of entire database
4. Call WASM module with SHA1 + UUID
5. Write 57-byte result at offset 0xAB
6. Restore backed-up fields
Args:
itdb_data: Mutable bytearray of complete iTunesDB file
firewire_id: 8+ byte FireWire GUID from /iPod_Control/Device/SysInfo
Raises:
ValueError: If iTunesDB is too small or FireWire ID is invalid
"""
min_size = OFFSET_HASHAB + HASHAB_SIZE # 0xAB + 57 = 0xE4 = 228
if len(itdb_data) < min_size:
raise ValueError(
f"iTunesDB file too small ({len(itdb_data)} bytes), "
f"need at least {min_size} (0x{min_size:X})"
)
if itdb_data[:4] != b'mhbd':
raise ValueError("Invalid iTunesDB: expected 'mhbd' header")
if len(firewire_id) < 8:
raise ValueError(
f"FireWire ID must be at least 8 bytes, got {len(firewire_id)}"
)
# Backup fields that will be zeroed for SHA1 computation
backup_db_id = bytes(itdb_data[OFFSET_DB_ID:OFFSET_DB_ID + 8])
backup_unk32 = bytes(itdb_data[OFFSET_UNK_0x32:OFFSET_UNK_0x32 + 20])
# Set hashing scheme to HASHAB (4)
itdb_data[OFFSET_HASHING_SCHEME:OFFSET_HASHING_SCHEME + 2] = \
ITDB_CHECKSUM_HASHAB.to_bytes(2, 'little')
# Compute SHA1 with hash fields zeroed
sha1_digest = _compute_itunesdb_sha1_for_hashab(itdb_data)
# Compute HASHAB via WASM
signature = compute_hashab(sha1_digest, firewire_id[:8])
if len(signature) != HASHAB_SIZE:
raise RuntimeError(
f"WASM returned {len(signature)} bytes, expected {HASHAB_SIZE}"
)
# Write signature to mhbd header
itdb_data[OFFSET_HASHAB:OFFSET_HASHAB + HASHAB_SIZE] = signature
# Restore backed-up fields
itdb_data[OFFSET_DB_ID:OFFSET_DB_ID + 8] = backup_db_id
itdb_data[OFFSET_UNK_0x32:OFFSET_UNK_0x32 + 20] = backup_unk32
logger.info("HASHAB signature written at offset 0x%X (%d bytes)",
OFFSET_HASHAB, HASHAB_SIZE)
def read_firewire_id(ipod_path: str) -> bytes:
"""Return the FireWire GUID for the connected iPod.
Reads from the centralised DeviceInfo store. Raises if not available.
"""
from ipod_device import get_current_device
device = get_current_device()
if device is not None:
fwid = device.firewire_id_bytes
if fwid:
return fwid
raise RuntimeError(
"FireWire GUID not available. Device info was not populated "
"by the device scanner."
)
if __name__ == "__main__":
import sys
if len(sys.argv) < 3:
print("Usage: python hashab.py <ipod_path> <itunesdb_path>")
print("Example: python hashab.py /media/ipod /media/ipod/iPod_Control/iTunes/iTunesDB")
sys.exit(1)
ipod_path = sys.argv[1]
itunesdb_path = sys.argv[2]
try:
firewire_id = read_firewire_id(ipod_path)
print(f"FireWire ID: {firewire_id.hex()}")
with open(itunesdb_path, 'rb') as f:
itdb_data = bytearray(f.read())
print(f"Read {len(itdb_data)} bytes from iTunesDB")
write_hashab(itdb_data, firewire_id)
print("HASHAB computed successfully!")
except Exception as e:
print(f"Error: {e}")
sys.exit(1)
File diff suppressed because it is too large Load Diff
+138
View File
@@ -0,0 +1,138 @@
"""MHIP Writer — Write playlist item chunks for iTunesDB.
MHIP chunks are playlist entries that reference tracks by their ID.
Each playlist (MHYP) contains MHIP entries for each track in the playlist.
The binary layout of the header is defined declaratively in
``iTunesDB_Shared.field_defs.MHIP_FIELDS``.
Cross-referenced against:
- iTunesDB_Shared/field_defs.py (single source of truth for offsets)
- iTunesDB_Parser/mhip_parser.py parse_playlistItem()
- libgpod itdb_itunesdb.c: mk_mhip(), write_podcast_mhips()
"""
import struct
from iTunesDB_Shared.field_base import write_fields, write_generic_header
from iTunesDB_Shared.mhip_defs import MHIP_HEADER_SIZE
from iTunesDB_Shared.mhod_defs import (
MHOD_HEADER_SIZE as _MHOD_HEADER_SIZE,
MHOD100_POSITION_BODY_SIZE,
write_mhod_header,
)
from iTunesDB_Shared.constants import MHOD_TYPE_TITLE
def write_mhip(
track_id: int,
position: int = 0,
mhip_id: int = 0,
timestamp: int = 0,
podcast_group_flag: int = 0,
podcast_group_ref: int = 0,
track_persistent_id: int = 0,
mhip_persistent_id: int = 0,
) -> bytes:
"""
Write an MHIP (playlist item) chunk.
MHIP entries link tracks to playlists by referencing the track ID.
Each entry also includes an MHOD type 100 with the position.
Args:
track_id: The track's ID (from MHIT)
position: Position in playlist (0-based)
mhip_id: Unique ID for this MHIP entry (written at offset 0x14)
In libgpod this is called "podcastgroupid" but it's used
for ALL playlists as a unique entry identifier.
timestamp: Mac timestamp (usually 0)
podcast_group_flag: For podcast grouping (usually 0)
podcast_group_ref: For podcast grouping (usually 0)
track_persistent_id: The track's db_track_id (persistent identifier)
mhip_persistent_id: Per-track persistent ID for this playlist item
Returns:
Complete MHIP chunk bytes
"""
mhod_position = write_mhod_position(position)
total_length = MHIP_HEADER_SIZE + len(mhod_position)
header = bytearray(MHIP_HEADER_SIZE)
write_generic_header(header, 0, b'mhip', MHIP_HEADER_SIZE, total_length)
write_fields(header, 0, 'mhip', {
'child_count': 1,
'podcast_group_flag': podcast_group_flag,
'group_id': mhip_id,
'track_id': track_id,
'timestamp': timestamp,
'group_id_ref': podcast_group_ref,
'track_persistent_id': track_persistent_id,
'mhip_persistent_id': mhip_persistent_id,
}, MHIP_HEADER_SIZE)
return bytes(header) + mhod_position
def write_mhod_position(position: int) -> bytes:
"""
Write an MHOD type 100 (playlist position).
This MHOD is attached to each MHIP and indicates the track's
position within the playlist.
Args:
position: Track position in playlist (0-based)
Returns:
MHOD chunk bytes
"""
total_len = _MHOD_HEADER_SIZE + MHOD100_POSITION_BODY_SIZE
header = write_mhod_header(100, total_len)
# Data section: position(4) + padding(16)
data = struct.pack('<I', position) + (b'\x00' * 16)
return header + data
def write_mhip_podcast_group(album_name: str, group_id: int) -> bytes:
"""Write a podcast group header MHIP.
In the type 3 (podcast) MHSD dataset, episodes are grouped under
their podcast show. Each show gets a "group header" MHIP that
serves as a parent node. Child episode MHIPs reference it via
``group_id_ref``.
Group header MHIPs differ from regular MHIPs:
- ``podcast_group_flag`` = 256 (0x100)
- ``track_id`` = 0 (no track reference)
- Contains an MHOD type 1 (title) with the album/show name
instead of an MHOD type 100 (position)
This matches libgpod's ``write_one_podcast_group()`` in
``itdb_itunesdb.c``.
Args:
album_name: Podcast show / album name for the group header
group_id: Unique identifier for this group (child MHIPs
reference this value in their ``group_id_ref`` field)
Returns:
Complete MHIP chunk bytes (header + MHOD title)
"""
from .mhod_writer import write_mhod_string
mhod_title = write_mhod_string(MHOD_TYPE_TITLE, album_name)
total_length = MHIP_HEADER_SIZE + len(mhod_title)
header = bytearray(MHIP_HEADER_SIZE)
write_generic_header(header, 0, b'mhip', MHIP_HEADER_SIZE, total_length)
write_fields(header, 0, 'mhip', {
'child_count': 1,
'podcast_group_flag': 256, # 0x100 = podcast group header
'group_id': group_id,
'track_id': 0, # group headers don't reference a track
}, MHIP_HEADER_SIZE)
return bytes(header) + mhod_title
+394
View File
@@ -0,0 +1,394 @@
"""
MHIT Writer Write track item chunks for iTunesDB.
MHIT chunks contain all metadata for a single track, plus child MHOD
chunks for strings (title, artist, path, etc.).
The binary layout of the header is defined declaratively in
``iTunesDB_Shared.field_defs.MHIT_FIELDS``. This writer builds a
values dict and delegates serialization to ``write_fields()``,
guaranteeing that field offsets / sizes / transforms stay in sync
with the parser.
Cross-referenced against:
- iTunesDB_Shared/field_defs.py (single source of truth for offsets)
- iTunesDB_Parser/mhit_parser.py parse_trackItem()
- libgpod itdb_itunesdb.c: mk_mhit()
- iPodLinux wiki MHIT documentation
"""
import random
import time
from dataclasses import dataclass
from iTunesDB_Shared.constants import (
AUDIO_FORMAT_FLAG_DEFAULT,
AUDIO_FORMAT_FLAG_MAP,
FILETYPE_CODES,
MEDIA_TYPE_AUDIO,
MEDIA_TYPE_MUSIC_VIDEO,
MEDIA_TYPE_PODCAST,
MEDIA_TYPE_TV_SHOW,
MEDIA_TYPE_VIDEO,
MEDIA_TYPE_VIDEO_PODCAST,
)
from iTunesDB_Shared.field_base import write_fields, write_generic_header
from iTunesDB_Shared.mhit_defs import MHIT_HEADER_SIZE, mhit_header_size_for_version
from .mhod_writer import write_track_mhods
def generate_db_track_id() -> int:
"""Generate a random 64-bit persistent ID for a track."""
return random.getrandbits(64)
generate_db_id = generate_db_track_id
@dataclass
class TrackInfo:
"""Track metadata for writing to iTunesDB."""
# Required
title: str
location: str # iPod path like ":iPod_Control:Music:F00:ABCD.mp3"
# File info
size: int = 0 # File size in bytes
length: int = 0 # Duration in milliseconds
filetype: str = 'mp3' # mp3, m4a, m4p, etc.
bitrate: int = 0 # kbps
sample_rate: int = 44100 # Hz
vbr: bool = False
# Metadata
artist: str | None = None
album: str | None = None
album_artist: str | None = None
genre: str | None = None
composer: str | None = None
comment: str | None = None
year: int = 0
track_number: int = 0
total_tracks: int = 0
disc_number: int = 1
total_discs: int = 1
bpm: int = 0
compilation_flag: bool = False
# Playback
rating: int = 0 # 0-100 (stars × 20)
play_count: int = 0
skip_count: int = 0
volume: int = 0 # -255 to +255
start_time: int = 0 # ms
stop_time: int = 0 # ms
sound_check: int = 0 # Volume normalization value (from ReplayGain)
bookmark_time: int = 0 # Resume position in ms (audiobooks/podcasts)
checked_flag: int = 0 # 0 = checked/enabled, 1 = unchecked/disabled
# Gapless playback
gapless_data: int = 0 # Gapless playback encoder delay data
gapless_track_flag: int = 0 # 1 = track has gapless info
gapless_album_flag: int = 0 # 1 = album is gapless
pregap: int = 0 # Encoder pregap samples
postgap: int = 0 # Encoder postgap/padding samples (0xC8)
sample_count: int = 0 # Total decoded sample count (64-bit)
encoder_flag: int = 0 # 0xCC: 0x01=MP3 encoder, 0x00=other
# Track flags
skip_when_shuffling: bool = False # 1 = skip in shuffle mode
remember_position: bool = False # 1 = resume from bookmark (audiobooks)
podcast_flag: int = 0 # 0xA7: 0x00=normal, 0x01/0x02=podcast
movie_file_flag: int = 0 # 0xB1: 0x01=video/movie file, 0x00=audio
played_mark: int = -1 # 0xB2: -1=auto (derive from play_count), 0x01=played, 0x02=unplayed
explicit_flag: int = 0 # 0=none, 1=explicit, 2=clean
purchased_aac_flag: int = 0 # 0x93: 1 for M4A/iTunes purchases, 0 for most MP3s
has_lyrics: bool = False # True if track has embedded lyrics
lyrics: str | None = None # Full lyrics text (MHOD type 10)
eq_setting: str | None = None # EQ preset name (MHOD type 7), e.g. "Bass Booster"
# Timestamps (Unix)
date_added: int = 0 # Will be set to now if 0
date_released: int = 0
last_modified: int = 0 # 0x20: file modification time (0 = use date_added)
last_played: int = 0
last_skipped: int = 0
# iPod-specific
track_id: int = 0 # Will be assigned during write
db_track_id: int = 0 # Will be generated if 0
media_type: int = MEDIA_TYPE_AUDIO
season_number: int = 0 # 0xD4: TV show season number
episode_number: int = 0 # 0xD8: TV show episode number
artwork_count: int = 0
artwork_size: int = 0
mhii_link: int = 0 # Link to ArtworkDB
album_id: int = 0 # Links to MHIA album entry
source_path: str | None = None # PC source path; internal write-time helper context
source_relative_path: str | None = None # PC path relative to library root, if known
# Sorting
sort_artist: str | None = None
sort_name: str | None = None
sort_album: str | None = None
sort_album_artist: str | None = None
sort_composer: str | None = None
# Extra string metadata
grouping: str | None = None
keywords: str | None = None # MHOD type 24 (track keywords)
# Podcast string metadata (written as MHODs)
podcast_enclosure_url: str | None = None # MHOD type 15
podcast_rss_url: str | None = None # MHOD type 16
category: str | None = None # MHOD type 9
# Video string metadata (written as MHODs)
description: str | None = None # MHOD type 14
subtitle: str | None = None # MHOD type 18
show_name: str | None = None # MHOD type 19 (TV show name)
episode_id: str | None = None # MHOD type 20 (e.g. "S01E05")
network_name: str | None = None # MHOD type 21 (TV network)
sort_show: str | None = None # MHOD type 31
show_locale: str | None = None # MHOD type 25 (show locale, e.g. "en_US")
# Filetype description
filetype_desc: str | None = None # e.g., "MPEG audio file"
# Round-trip fields (preserved from existing iPod database)
user_id: int = 0 # 0x64: DRM user ID (preserved for round-trip)
app_rating: int = 0 # 0x79: Application-computed rating (preserved for round-trip)
mpeg_audio_type: int = 0 # 0x90: MPEG Audio Object Type (12=MP3, 51=AAC, 41=Audible)
# iTunes Store metadata (round-trip, only for Store purchases)
date_added_to_itunes: int = 0 # 0xDC: Unix ts, original iTunes library add date
store_track_id: int = 0 # 0xE0: iTunes Store per-track content ID
store_encoder_version: int = 0 # 0xE4: iTunes version that encoded the file
store_artist_id: int = 0 # 0xE8: iTunes Store artist/collection ID
store_album_id: int = 0 # 0xF0: iTunes Store album ID
store_content_flag: int = 0 # 0xF4: iTunes Store content type flag
# Internal IDs (assigned during database write, NOT user-provided)
artist_id: int = 0 # Links to artist entry (assigned by writer)
composer_id: int = 0 # Links to composer entry (assigned by writer)
# Chapter data (MHOD type 17) lives in iTunesDB, independent of filetype.
chapter_data: dict | None = None
# Internal sync hint (transient, not written to database)
# Used by ArtworkDB writer to determine preservation strategy:
# "preserve_existing" = use existing art without re-encoding
# "clear_art" = remove art for this track
# "" (empty) = normal processing
_iop_artwork_sync_hint: str = ""
@property
def db_id(self) -> int:
"""Backward-compatible alias for the track persistent ID."""
return self.db_track_id
@db_id.setter
def db_id(self, value: int) -> None:
self.db_track_id = value
def _compute_sort_indicators(track: TrackInfo) -> bytes:
"""Build the 8-byte sort_mhod_indicators field from sort field presence.
Byte layout (verified via exhaustive bit-correlation across 9 databases):
[0] = sort_title (MHOD 27)
[1] = sort_album (MHOD 28)
[2] = sort_artist (MHOD 23)
[3] = sort_album_artist (MHOD 29)
[4] = sort_composer (MHOD 30)
[5] = sort_show (MHOD 31)
[6..7] = unused (always 0)
bit 0 = has corresponding sort MHOD override
bit 7 = collation flag (0x80), always set for compatibility
"""
ind = bytearray(8)
ind[0] = 0x81 if track.sort_name else 0x80
ind[1] = 0x81 if track.sort_album else 0x80
ind[2] = 0x81 if track.sort_artist else 0x80
ind[3] = 0x81 if track.sort_album_artist else 0x80
ind[4] = 0x81 if track.sort_composer else 0x80
ind[5] = 0x81 if track.sort_show else 0x80
return bytes(ind)
def _resolve_media_type(track: TrackInfo, capabilities) -> int:
"""Downgrade media type when the device lacks required capability."""
media_type = track.media_type
if capabilities is None:
return media_type
if not capabilities.supports_video:
if media_type in (MEDIA_TYPE_VIDEO, MEDIA_TYPE_MUSIC_VIDEO, MEDIA_TYPE_TV_SHOW):
return MEDIA_TYPE_AUDIO
if media_type == MEDIA_TYPE_VIDEO_PODCAST:
return MEDIA_TYPE_PODCAST
if not capabilities.supports_podcast:
if media_type in (MEDIA_TYPE_PODCAST, MEDIA_TYPE_VIDEO_PODCAST):
return MEDIA_TYPE_AUDIO
return media_type
def _resolve_movie_flag(track: TrackInfo, media_type: int) -> int:
"""Derive movie_flag from media_type when not explicitly set."""
if track.movie_file_flag != 0:
return track.movie_file_flag
if media_type in (MEDIA_TYPE_VIDEO, MEDIA_TYPE_MUSIC_VIDEO,
MEDIA_TYPE_TV_SHOW, MEDIA_TYPE_VIDEO_PODCAST):
return 1
return 0
def _resolve_not_played(track: TrackInfo) -> int:
"""Resolve the not_played_flag: auto-derive from play_count when -1."""
if track.played_mark >= 0:
return track.played_mark
return 0x01 if track.play_count > 0 else 0x02
def _gapless_or_zero(value: int, capabilities) -> int:
"""Return *value* when the device supports gapless, else 0."""
if capabilities is not None and not capabilities.supports_gapless:
return 0
return value
def write_mhit(track: TrackInfo, track_id: int, db_id_2: int = 0,
capabilities=None, db_version: int = 0) -> bytes:
"""Write a complete MHIT chunk with all child MHODs.
Args:
track: TrackInfo dataclass with all track metadata.
track_id: Unique track ID within this database.
db_id_2: Database-wide ID from MHBD offset 0x24 (written into every track).
capabilities: Optional DeviceCapabilities for gapless/video filtering.
db_version: Database version controls the MHIT header size.
Older iPods require smaller headers (e.g. 0x148 for db_version 0x19).
Returns:
Complete MHIT chunk bytes (header + MHODs).
"""
if track.db_track_id == 0:
track.db_track_id = generate_db_track_id()
if track.date_added == 0:
track.date_added = int(time.time())
ft = track.filetype.lower()
filetype_code = FILETYPE_CODES.get(ft, FILETYPE_CODES['mp3'])
media_type = _resolve_media_type(track, capabilities)
has_lyrics = track.has_lyrics or bool(track.lyrics)
# Build child MHODs first to know count + size.
mhod_data, mhod_count = write_track_mhods(
title=track.title, location=track.location,
artist=track.artist, album=track.album, genre=track.genre,
album_artist=track.album_artist, composer=track.composer,
comment=track.comment, filetype_desc=track.filetype_desc,
sort_artist=track.sort_artist, sort_name=track.sort_name,
sort_album=track.sort_album, sort_album_artist=track.sort_album_artist,
sort_composer=track.sort_composer, grouping=track.grouping,
keywords=track.keywords, description=track.description,
subtitle=track.subtitle, show_name=track.show_name,
episode_id=track.episode_id, network_name=track.network_name,
sort_show=track.sort_show, show_locale=track.show_locale,
podcast_enclosure_url=track.podcast_enclosure_url,
podcast_rss_url=track.podcast_rss_url, category=track.category,
lyrics=track.lyrics, eq_setting=track.eq_setting,
chapter_data=track.chapter_data,
)
# Use device-appropriate header size. Older iPod firmware expects smaller
# MHIT headers; fields beyond the header boundary are automatically skipped
# by write_fields() via each field's min_header_length attribute.
header_size = mhit_header_size_for_version(db_version) if db_version else MHIT_HEADER_SIZE
total_length = header_size + len(mhod_data)
# Assemble the values dict — write_fields handles transforms & packing.
values: dict = {
'child_count': mhod_count,
'track_id': track_id,
'visible': 1,
'filetype': filetype_code,
'vbr_flag': 1 if track.vbr else 0,
'mp3_flag': 1 if ft == 'mp3' else 0,
'compilation_flag': 1 if track.compilation_flag else 0,
'rating': track.rating,
'last_modified': track.last_modified or track.date_added,
'size': track.size,
'length': track.length,
'track_number': track.track_number,
'total_tracks': track.total_tracks,
'year': track.year,
'bitrate': track.bitrate,
'sample_rate_1': track.sample_rate,
'volume': track.volume,
'start_time': track.start_time,
'stop_time': track.stop_time,
'sound_check': track.sound_check,
'play_count_1': track.play_count,
'play_count_2': 0, # reset after sync
'last_played': track.last_played,
'disc_number': track.disc_number,
'total_discs': track.total_discs,
'user_id': track.user_id,
'date_added': track.date_added,
'bookmark_time': track.bookmark_time,
'db_track_id': track.db_track_id,
'checked_flag': track.checked_flag,
'app_rating': track.app_rating,
'bpm': max(0, track.bpm) if track.bpm is not None else 0,
'artwork_count': track.artwork_count,
'audio_format_flag': AUDIO_FORMAT_FLAG_MAP.get(ft, AUDIO_FORMAT_FLAG_DEFAULT),
'artwork_size': track.artwork_size,
'sample_rate_2': float(track.sample_rate),
'date_released': track.date_released,
'mpeg_audio_type': track.mpeg_audio_type,
'explicit_flag': track.explicit_flag,
'purchased_aac_flag': track.purchased_aac_flag,
# Extended fields
'skip_count': track.skip_count,
'last_skipped': track.last_skipped,
'has_artwork': 1 if track.artwork_count > 0 else 2,
'skip_when_shuffling': 1 if track.skip_when_shuffling else 0,
'remember_position': 1 if track.remember_position else 0,
'use_podcast_now_playing_flag': track.podcast_flag,
'db_track_id_2': track.db_track_id,
'lyrics_flag': 1 if has_lyrics else 0,
'movie_flag': _resolve_movie_flag(track, media_type),
'not_played_flag': _resolve_not_played(track),
'pregap': _gapless_or_zero(track.pregap, capabilities),
'sample_count': _gapless_or_zero(track.sample_count, capabilities),
'postgap': _gapless_or_zero(track.postgap, capabilities),
'encoder': track.encoder_flag,
'media_type': media_type,
'season_number': track.season_number,
'episode_number': track.episode_number,
'date_added_to_itunes': track.date_added_to_itunes,
'store_track_id': track.store_track_id,
'store_encoder_version': track.store_encoder_version,
'store_artist_id': track.store_artist_id,
'store_album_id': track.store_album_id,
'store_content_flag': track.store_content_flag,
'gapless_audio_payload_size': _gapless_or_zero(track.gapless_data, capabilities),
'gapless_track_flag': _gapless_or_zero(track.gapless_track_flag, capabilities),
'gapless_album_flag': _gapless_or_zero(track.gapless_album_flag, capabilities),
'album_id': track.album_id,
'db_id_2_ref': db_id_2,
'size_2': track.size,
'sort_mhod_indicators': _compute_sort_indicators(track),
'artwork_id_ref': track.mhii_link,
'artist_id_ref': track.artist_id,
'composer_id': track.composer_id,
}
header = bytearray(header_size)
write_generic_header(header, 0, b'mhit', header_size, total_length)
write_fields(header, 0, 'mhit', values, header_size)
return bytes(header) + mhod_data
+199
View File
@@ -0,0 +1,199 @@
"""MHLA Writer — Write album list chunks for iTunesDB.
MHLA (album list) contains MHIA (album item) entries that group tracks.
Introduced in iTunes 7.1 (dbversion >= 0x14).
MHLA header layout (MHLA_HEADER_SIZE = 92 bytes):
+0x00: 'mhla' magic (4B)
+0x04: header_length (4B)
+0x08: album_count (4B)
MHIA header layout (MHIA_HEADER_SIZE = 88 bytes):
+0x00: 'mhia' magic (4B)
+0x04: header_length (4B)
+0x08: total_length (4B) header + child MHODs
+0x0C: child_count (4B)
+0x10: album_id (4B) links to MHIT.albumID
+0x14: sql_id (8B) internal iPod DB id (must be non-zero)
+0x1C: platform_flag (2B, always 2) + album_compilation_flag (2B, 0=normal, 1=compilation)
Children: MHOD types 200 (album name), 201 (artist), 202 (sort artist)
Cross-referenced against:
- iTunesDB_Parser/mhia_parser.py parse_albumItem()
- libgpod itdb_itunesdb.c: mk_mhia()
"""
import random
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from .mhit_writer import TrackInfo
from iTunesDB_Shared.album_identity import (
album_identity_from_track,
group_tracks_by_album_identity,
)
from iTunesDB_Shared.constants import (
MHOD_TYPE_ALBUM_ALBUM,
MHOD_TYPE_ALBUM_ARTIST_ITEM,
MHOD_TYPE_ALBUM_PODCAST_URL,
MHOD_TYPE_ALBUM_SHOW,
MHOD_TYPE_ALBUM_SORT_ARTIST,
)
from iTunesDB_Shared.field_base import (
MHLA_HEADER_SIZE,
write_fields,
write_generic_header,
)
from iTunesDB_Shared.mhia_defs import MHIA_HEADER_SIZE
from .mhod_writer import write_mhod_string
def write_mhia(album_id: int, album_name: str, album_artist: str,
sort_album_artist: str = "",
podcast_url: str = "", show_name: str = "",
is_compilation: bool = False,
album_track_db_id: int = 0) -> bytes:
"""
Write an MHIA (album item) chunk.
Args:
album_id: Unique album ID (used to link tracks to albums)
album_name: Album name
album_artist: Album artist
sort_album_artist: Sort album artist (for proper alphabetical sorting)
podcast_url: Podcast RSS URL (MHOD type 203)
show_name: Show/series name (MHOD type 204)
is_compilation: True for Various Artists / compilation albums
album_track_db_id: db_track_id of a representative track in this album
Returns:
Complete MHIA chunk with MHODs
"""
# Build child MHODs
children = bytearray()
child_count = 0
if album_name:
children.extend(write_mhod_string(MHOD_TYPE_ALBUM_ALBUM, album_name))
child_count += 1
if album_artist:
children.extend(write_mhod_string(MHOD_TYPE_ALBUM_ARTIST_ITEM, album_artist))
child_count += 1
if sort_album_artist:
children.extend(write_mhod_string(MHOD_TYPE_ALBUM_SORT_ARTIST, sort_album_artist))
child_count += 1
if podcast_url:
children.extend(write_mhod_string(MHOD_TYPE_ALBUM_PODCAST_URL, podcast_url))
child_count += 1
if show_name:
children.extend(write_mhod_string(MHOD_TYPE_ALBUM_SHOW, show_name))
child_count += 1
# Total chunk length
total_length = MHIA_HEADER_SIZE + len(children)
# Build header
header = bytearray(MHIA_HEADER_SIZE)
write_generic_header(header, 0, b'mhia', MHIA_HEADER_SIZE, total_length)
# CRITICAL: sql_id must be non-zero! Clean iTunes DBs have random u64 values here.
sql_id = random.getrandbits(64)
write_fields(header, 0, 'mhia', {
'child_count': child_count,
'album_id': album_id,
'sql_id': sql_id,
'platform_flag': 2,
'album_compilation_flag': 1 if is_compilation else 0,
'album_track_db_id': album_track_db_id,
}, MHIA_HEADER_SIZE)
return bytes(header) + bytes(children)
def _pick_first(tracks: list["TrackInfo"], attr: str) -> str:
for track in tracks:
value = getattr(track, attr, None) or ""
if value:
return value
return ""
def write_mhla(
tracks: list["TrackInfo"],
starting_index_for_album_id,
) -> tuple[bytes, dict[tuple[str, str], int], int]:
"""
Write an MHLA (album list) chunk with albums derived from tracks.
Args:
tracks: List of TrackInfo objects
Returns:
Tuple of (MHLA chunk bytes, album_map dict mapping (album, artist) to album_id)
"""
groups = group_tracks_by_album_identity(tracks, album_identity_from_track)
# Build album items
album_items = bytearray()
album_map: dict[tuple[str, str], int] = {} # (album, artist) -> album_id
def _album_sort_key(group):
identity = group.identity
album_name = identity.album or ""
album_artist = identity.album_artist or identity.artist or ""
show_name = identity.show_name or ""
return (album_name, album_artist, show_name)
album_id = starting_index_for_album_id
for group in sorted(groups, key=_album_sort_key):
identity = group.identity
album_name = identity.album or ""
album_artist = identity.album_artist or identity.artist or ""
album_map[(album_name, album_artist)] = album_id
# Use sort_albumartist from track first, fall back to sort_artist (per libgpod mk_mhia)
sort_artist = _pick_first(group.tracks, "sort_album_artist")
if not sort_artist:
sort_artist = _pick_first(group.tracks, "sort_artist")
podcast_url = _pick_first(group.tracks, "podcast_rss_url")
show_name = identity.show_name or _pick_first(group.tracks, "show_name")
# Album is a compilation if any track in it has compilation_flag=True
is_compilation = any(t.compilation_flag for t in group.tracks)
# Use first track's db_track_id as the representative track for this album
rep_db_track_id = group.tracks[0].db_track_id if group.tracks else 0
for track in group.tracks:
track.album_id = album_id
album_items.extend(write_mhia(
album_id, album_name, album_artist, sort_artist,
podcast_url=podcast_url, show_name=show_name,
is_compilation=is_compilation,
album_track_db_id=rep_db_track_id,
))
album_id += 1
album_count = len(album_map)
# Build header
header = bytearray(MHLA_HEADER_SIZE)
write_generic_header(header, 0, b'mhla', MHLA_HEADER_SIZE, album_count)
return bytes(header) + bytes(album_items), album_map, album_id
def write_mhla_empty() -> bytes:
"""
Write an empty MHLA (album list) chunk.
Returns:
MHLA header with 0 albums
"""
header = bytearray(MHLA_HEADER_SIZE)
write_generic_header(header, 0, b'mhla', MHLA_HEADER_SIZE, 0)
return bytes(header)
+136
View File
@@ -0,0 +1,136 @@
"""MHLI Writer — Write artist list chunks for iTunesDB.
MHSD type 8 contains an artist list using 'mhli' as the list header and
'mhii' as individual artist items. Despite sharing the 'mhii' magic with
ArtworkDB image items, these are structurally different chunks.
MHLI header layout (MHLI_HEADER_SIZE = 92 bytes):
+0x00: 'mhli' magic (4B)
+0x04: header_length (4B)
+0x08: artist_count (4B)
MHII header layout (MHII_HEADER_SIZE = 80 bytes, per libgpod mk_mhii):
+0x00: 'mhii' magic (4B)
+0x04: header_length (4B)
+0x08: total_length (4B) header + child MHODs
+0x0C: child_count (4B) always 1 (the artist-name MHOD)
+0x10: artist_id (4B) links to MHIT.artist_id
+0x14: sql_id (8B) internal iPod DB id (must be non-zero)
+0x1C: platform_flag (4B) always 2
Children: MHOD type 300 (artist name / album-artist name)
Cross-referenced against:
- libgpod itdb_itunesdb.c: mk_mhii() (artist variant)
- docs/iTunesCDB-internals.md §Type 8
"""
import random
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from .mhit_writer import TrackInfo
from iTunesDB_Shared.field_base import (
MHLI_HEADER_SIZE,
write_fields,
write_generic_header,
)
from iTunesDB_Shared.mhii_defs import MHII_HEADER_SIZE
from iTunesDB_Shared.constants import MHOD_TYPE_ARTIST_NAME
from .mhod_writer import write_mhod_string
def write_mhii_artist(artist_id: int, artist_name: str) -> bytes:
"""
Write an MHII (artist item) chunk for the artist list.
Args:
artist_id: Unique artist ID (used to link tracks to artists)
artist_name: Artist name string
Returns:
Complete MHII chunk with MHOD type 300
"""
# Build child MHOD (always exactly 1: the artist name)
children = bytearray()
child_count = 0
if artist_name:
children.extend(write_mhod_string(MHOD_TYPE_ARTIST_NAME, artist_name))
child_count += 1
# Total chunk length
total_length = MHII_HEADER_SIZE + len(children)
# Build header
header = bytearray(MHII_HEADER_SIZE)
write_generic_header(header, 0, b'mhii', MHII_HEADER_SIZE, total_length)
# CRITICAL: sql_id must be non-zero! Clean iTunes DBs have random u64 values here.
sql_id = random.getrandbits(64)
write_fields(header, 0, 'mhii', {
'child_count': child_count,
'artist_id': artist_id,
'sql_id': sql_id,
'platform_flag': 2,
}, MHII_HEADER_SIZE)
return bytes(header) + bytes(children)
def write_mhli(tracks: list["TrackInfo"], starting_index_for_artist_id: int) -> tuple[bytes, dict[str, int], int]:
"""
Write an MHLI (artist list) chunk with artists derived from tracks.
Deduplicates artists using case-insensitive matching (same as album
deduplication in mhla_writer.py).
Args:
tracks: List of TrackInfo objects
Returns:
Tuple of (MHLI chunk bytes, artist_map dict mapping artist_name_lower to artist_id)
"""
# Collect unique artists: lowercase artist name → display name
# Use the first occurrence's casing as the canonical display name
artist_display: dict[str, str] = {}
for track in tracks:
artist_name = track.artist or ""
if not artist_name:
continue
key = artist_name.lower()
if key not in artist_display:
artist_display[key] = artist_name
# Build artist items
artist_items = bytearray()
artist_map: dict[str, int] = {} # lowercase artist → artist_id
artist_id = starting_index_for_artist_id
for key in sorted(artist_display.keys()):
display_name = artist_display[key]
artist_map[key] = artist_id
artist_items.extend(write_mhii_artist(artist_id, display_name))
artist_id += 1
artist_count = len(artist_map)
# Build header
header = bytearray(MHLI_HEADER_SIZE)
write_generic_header(header, 0, b'mhli', MHLI_HEADER_SIZE, artist_count)
return bytes(header) + bytes(artist_items), artist_map, artist_id
def write_mhli_empty() -> bytes:
"""
Write an empty MHLI (artist list) chunk.
Returns:
MHLI header with 0 artists
"""
header = bytearray(MHLI_HEADER_SIZE)
write_generic_header(header, 0, b'mhli', MHLI_HEADER_SIZE, 0)
return bytes(header)
+238
View File
@@ -0,0 +1,238 @@
"""MHLP Writer — Write playlist list chunks for iTunesDB.
MHLP (playlist list) wraps all MHYP (playlist) chunks and provides
the playlist count in its header. Every iTunesDB needs at least a
"master playlist" referencing all tracks.
Header layout (MHLP_HEADER_SIZE = 92 bytes):
+0x00: 'mhlp' magic (4B)
+0x04: header_length (4B)
+0x08: playlist_count (4B)
Supports:
- Master + user playlists (write_mhlp_with_playlists)
- Dataset 3 podcast playlists (podcast clone of dataset 2)
- Dataset 5 smart playlists (write_mhlp_smart)
Cross-referenced against:
- iTunesDB_Parser/mhlp_parser.py
- libgpod itdb_itunesdb.c: mk_mhlp()
"""
from __future__ import annotations
import logging
from typing import List, Optional, TYPE_CHECKING
if TYPE_CHECKING:
from .mhit_writer import TrackInfo
from .mhyp_writer import PlaylistInfo
from iTunesDB_Shared.field_base import MHLP_HEADER_SIZE, write_generic_header
from .mhyp_writer import write_master_playlist, write_playlist
logger = logging.getLogger(__name__)
def write_mhlp_empty() -> bytes:
"""
Write an empty MHLP (playlist list) chunk.
Note: An empty MHLP means NO playlists, which may cause issues
on some iPods. Use write_mhlp_with_playlists() for a valid database.
Returns:
MHLP header with 0 playlists
"""
header = bytearray(MHLP_HEADER_SIZE)
write_generic_header(header, 0, b'mhlp', MHLP_HEADER_SIZE, 0)
return bytes(header)
def write_mhlp(playlist_chunks: List[bytes]) -> bytes:
"""
Write a MHLP chunk with playlists.
Args:
playlist_chunks: List of MHYP (playlist) chunks
Returns:
Complete MHLP chunk
"""
# Concatenate all playlist data
playlists_data = b''.join(playlist_chunks)
header = bytearray(MHLP_HEADER_SIZE)
write_generic_header(header, 0, b'mhlp', MHLP_HEADER_SIZE, len(playlist_chunks))
return bytes(header) + playlists_data
def write_mhlp_with_playlists(
track_ids: List[int],
playlists: List[PlaylistInfo],
db_id_2,
tracks: Optional[List[TrackInfo]] = None,
capabilities=None,
master_playlist_name: str = "iPod",
master_playlist_id: Optional[int] = None,
) -> bytes:
"""
Write an MHLP chunk with the master playlist + user playlists.
The master playlist is always first, followed by regular/smart playlists.
This is used for MHSD type 2 (playlists dataset).
The master playlist is auto-generated from the full track list; its
display name is controlled by *master_playlist_name*. The *playlists*
list should contain only user playlists (no master).
Args:
track_ids: List of ALL track IDs in the database (for master playlist)
playlists: List of user PlaylistInfo objects (master is NOT included)
tracks: List of ALL TrackInfo objects (needed for library indices)
db_id_2: Database-wide ID from MHBD offset 0x24
capabilities: Optional DeviceCapabilities for video sort indices.
master_playlist_name: Display name for the auto-generated master playlist.
Returns:
Complete MHLP chunk
"""
chunks = []
# Master playlist MUST be first
master = write_master_playlist(
track_ids, tracks=tracks, db_id_2=db_id_2,
capabilities=capabilities, name=master_playlist_name,
playlist_id=master_playlist_id,
)
chunks.append(master)
# Sanity: strip rogue master flags from dataset-2 user playlists.
# Only the auto-generated master above should have master=True.
# Dataset 5 built-in categories (mhsd5_type != 0) legitimately
# need master=True, so we never touch those — even if they end up
# here by accident.
for p in playlists:
if p.master and not p.mhsd5_type:
logger.warning(
"Stripped master flag from user playlist '%s'"
"master is auto-generated for dataset 2",
p.name,
)
p.master = False
# Write all user playlists (regular and smart).
for pl in playlists:
chunks.append(write_playlist(pl, db_id_2=db_id_2))
return write_mhlp(chunks)
def write_mhlp_with_playlists_type3(
track_ids: List[int],
playlists: List["PlaylistInfo"],
db_id_2: int,
track_album_map: dict[int, str],
tracks: Optional[List["TrackInfo"]] = None,
capabilities=None,
master_playlist_name: str = "iPod",
next_mhip_id_start: int = 1,
master_playlist_id: Optional[int] = None,
) -> bytes:
"""Write an MHLP for MHSD type 3 with podcast grouping.
Identical to :func:`write_mhlp_with_playlists` **except** that playlist
entries marked as podcast (``podcast_flag == 1``) use the grouped
MHIP structure described by libgpod's ``write_podcast_mhips()``.
In the grouped structure, podcast episodes are nested under their
podcast show (album). Each show gets a group-header MHIP
(``podcast_group_flag=256``, MHOD title = album name) followed by
child episode MHIPs whose ``group_id_ref`` points back to the header.
Non-podcast playlists are written with the standard flat MHIP layout,
identical to type 2.
Args:
track_ids: ALL track IDs in the database (for the master playlist)
playlists: User playlist list (same objects as type 2; master is
auto-generated)
db_id_2: Database-wide ID from MHBD offset 0x24
track_album_map: track_id album name for podcast grouping
tracks: TrackInfo list (needed for master playlist library indices)
capabilities: DeviceCapabilities (for video sort indices etc.)
master_playlist_name: Display name for the master playlist.
next_mhip_id_start: Starting ID for generated MHIP identifiers.
Returns:
Complete MHLP chunk bytes.
"""
chunks = []
# Master playlist — identical to type 2
master = write_master_playlist(
track_ids, tracks=tracks, db_id_2=db_id_2,
capabilities=capabilities, name=master_playlist_name,
playlist_id=master_playlist_id,
)
chunks.append(master)
for p in playlists:
if p.master and not p.mhsd5_type:
logger.warning(
"Stripped master flag from user playlist '%s'"
"master is auto-generated for dataset 3",
p.name,
)
p.master = False
for pl in playlists:
chunks.append(write_playlist(
pl, db_id_2=db_id_2,
podcast_grouping=True,
track_album_map=track_album_map,
next_mhip_id_start=next_mhip_id_start,
))
return write_mhlp(chunks)
def write_mhlp_smart(
playlists: List[PlaylistInfo],
db_id_2: int = 0,
) -> bytes:
"""
Write an MHLP chunk for dataset type 5 (smart playlist list).
These playlists define iPod built-in browse categories (Music, Movies,
TV Shows, Audiobooks, Podcasts, Rentals). Each has a mhsd5_type value
and smart rules that filter by media type.
**Master flag semantics for dataset 5:**
All built-in categories legitimately have ``master=True`` which writes
``type=1`` at MHYP offset +0x14. This is the SAME byte used by the
master playlist in dataset 2, but the meaning differs:
- Dataset 2 ``type=1``: "this is the master playlist" (exactly one)
- Dataset 5 ``type=1``: "this is a built-in system category" (all of them)
No single-master constraint is enforced here every ds5 category
needs ``master=True`` for the iPod firmware to recognise it.
Args:
playlists: List of PlaylistInfo objects (smart playlists only)
db_id_2: Database-wide ID from MHBD offset 0x24
Returns:
Complete MHLP chunk, or empty MHLP if no smart playlists
"""
if not playlists:
return write_mhlp_empty()
chunks = []
for pl in playlists:
chunks.append(write_playlist(pl, db_id_2=db_id_2))
return write_mhlp(chunks)
+59
View File
@@ -0,0 +1,59 @@
"""MHLT Writer — Write track list chunks for iTunesDB.
MHLT (track list) wraps all MHIT (track) chunks and provides
the track count in its header.
Header layout (MHLT_HEADER_SIZE = 92 bytes):
+0x00: 'mhlt' magic (4B)
+0x04: header_length (4B)
+0x08: track_count (4B)
Cross-referenced against:
- iTunesDB_Parser/mhlt_parser.py
- libgpod itdb_itunesdb.c: mk_mhlt()
"""
from typing import List
from iTunesDB_Shared.field_base import MHLT_HEADER_SIZE, write_generic_header
from .mhit_writer import write_mhit, TrackInfo
def write_mhlt(tracks: List[TrackInfo], start_track_id: int, db_id_2: int,
capabilities=None, db_version: int = 0) -> tuple[bytes, int]:
"""
Write a complete MHLT chunk with all tracks.
Args:
tracks: List of TrackInfo objects
start_track_id: Starting track ID (increments for each track)
db_id_2: Database-wide ID from MHBD (written into every MHIT db_id_2_ref at offset 0x124)
capabilities: Optional DeviceCapabilities for gapless/video filtering
db_version: Database version forwarded to write_mhit for header sizing
Returns:
Tuple of (complete MHLT chunk bytes, next available track ID)
"""
# Build all track chunks first
track_chunks = []
track_id = start_track_id
for track in tracks:
try:
mhit_data = write_mhit(track, track_id, db_id_2, capabilities=capabilities,
db_version=db_version)
except Exception as exc:
raise type(exc)(
f"{exc} (track #{track_id}: {track.artist!r} {track.title!r})"
) from exc
track_chunks.append(mhit_data)
track_id += 1
# Concatenate all track data
all_tracks_data = b''.join(track_chunks)
header = bytearray(MHLT_HEADER_SIZE)
write_generic_header(header, 0, b'mhlt', MHLT_HEADER_SIZE, len(tracks))
return bytes(header) + all_tracks_data, track_id
+334
View File
@@ -0,0 +1,334 @@
"""
MHOD Type 52/53 Writer - Library Playlist Index for iTunesDB.
These MHODs are written ONLY for the Master Playlist and provide
pre-sorted track indices that the iPod uses to build its browsing
views (Songs, Artists, Albums, Genres, Composers).
Without these indices, the iPod Classic shows "no songs, no albums"
even if tracks exist in the database.
Based on libgpod's mk_mhod52(), mk_mhod53(), and write_playlist()
in itdb_itunesdb.c.
Type 52 (MHOD_ID_LIBPLAYLISTINDEX):
Pre-sorted track position arrays for each sort category.
Format: header(24) + sort_type(4) + count(4) + padding(40) + indices(count*4)
Total = 4*count + 72
Type 53 (MHOD_ID_LIBPLAYLISTJUMPTABLE):
Letter-jump table for quick scrolling in each category.
Format: header(24) + sort_type(4) + count(4) + padding(8) + entries(count*12)
Total = 12*count + 40
"""
import struct
import unicodedata
from typing import TYPE_CHECKING
from iTunesDB_Shared.field_base import strip_article
from iTunesDB_Shared.mhod_defs import (
MHOD52_BODY_HEADER_SIZE,
MHOD53_BODY_HEADER_SIZE,
MHOD53_ENTRY_SIZE,
MHOD_HEADER_SIZE,
SORT_ALBUM,
SORT_ALBUM_ARTIST,
SORT_ARTIST,
SORT_COMPOSER,
SORT_EPISODE,
SORT_GENRE,
SORT_SEASON,
SORT_SHOW,
SORT_TITLE,
write_mhod_header,
)
if TYPE_CHECKING:
from .mhit_writer import TrackInfo
# Base sort types — always written
BASE_SORT_TYPES = [SORT_TITLE, SORT_ALBUM, SORT_ARTIST, SORT_GENRE, SORT_COMPOSER]
# Video sort types — only for devices with supports_video
VIDEO_SORT_TYPES = [SORT_SHOW, SORT_SEASON, SORT_EPISODE]
# Legacy alias for backward compatibility
ALL_SORT_TYPES = BASE_SORT_TYPES
def _sort_key(s: str) -> str:
"""
Create a case-insensitive sort key for a string.
Strips leading articles (A, An, The) for sorting (matching iTunes behavior),
normalizes unicode, and lowercases.
"""
if not s:
return ""
s = strip_article(s)
# Normalize unicode for consistent comparison
return unicodedata.normalize('NFKD', s).casefold()
def _jump_table_letter(s: str) -> int:
"""
Get the first alphanumeric character for jump table grouping.
Returns uppercase letter (A-Z) as Unicode codepoint, or ord('0')
for strings starting with digits.
Based on libgpod's jump_table_letter().
"""
if not s:
return ord('0')
for ch in s:
if ch.isalnum():
if ch.isdigit():
return ord('0')
upper = ord(ch.upper()[0])
if upper > 0xFFFF:
continue # non-BMP char can't fit in UTF-16 jump table
return upper
return ord('0')
def _get_sort_fields(track: "TrackInfo", sort_type: int) -> tuple:
"""
Get sort key fields for a track based on sort type.
Returns a tuple used for sorting. Multi-field sorts match
libgpod's mhod52_sort_* comparison functions.
IMPORTANT: For every field, prefer the sort_* variant over the
display variant (e.g. sort_album over album). This matches
libgpod's ``sort_compare(track->sort_X ? track->sort_X : track->X, ...)``
pattern used in mhod52_sort_album(), mhod52_sort_artist(), etc.
"""
title = _sort_key(track.sort_name or track.title or "")
album = _sort_key(track.sort_album or track.album or "")
artist = _sort_key(track.sort_artist or track.artist or "")
genre = _sort_key(track.genre or "")
composer = _sort_key(track.sort_composer or track.composer or "")
track_nr = track.track_number or 0
cd_nr = track.disc_number or 0
if sort_type == SORT_TITLE:
return (title,)
elif sort_type == SORT_ALBUM:
return (album, cd_nr, track_nr, title)
elif sort_type == SORT_ARTIST:
return (artist, album, cd_nr, track_nr, title)
elif sort_type == SORT_GENRE:
return (genre, artist, album, cd_nr, track_nr, title)
elif sort_type == SORT_COMPOSER:
return (composer, album, cd_nr, track_nr, title)
elif sort_type == SORT_SHOW:
show = _sort_key(track.sort_show or track.show_name or "")
season = track.season_number or 0
episode = track.episode_number or 0
return (show, season, episode, title)
elif sort_type == SORT_SEASON:
season = track.season_number or 0
episode = track.episode_number or 0
show = _sort_key(track.sort_show or track.show_name or "")
return (season, episode, show, title)
elif sort_type == SORT_EPISODE:
episode = track.episode_number or 0
season = track.season_number or 0
show = _sort_key(track.sort_show or track.show_name or "")
return (episode, season, show, title)
elif sort_type == SORT_ALBUM_ARTIST:
album_artist = _sort_key(
track.sort_album_artist
or track.album_artist
or track.sort_artist or track.artist or ""
)
return (album_artist, album, cd_nr, track_nr, title)
else:
return (title,)
def _get_jump_letter(track: "TrackInfo", sort_type: int) -> int:
"""Get the letter for jump table grouping based on sort type.
Uses sort_* field variants for consistency with ``_get_sort_fields``.
"""
if sort_type == SORT_TITLE:
return _jump_table_letter(track.sort_name or track.title or "")
elif sort_type == SORT_ALBUM:
return _jump_table_letter(track.sort_album or track.album or "")
elif sort_type == SORT_ARTIST:
s = track.sort_artist or track.artist or ""
return _jump_table_letter(s)
elif sort_type == SORT_GENRE:
return _jump_table_letter(track.genre or "")
elif sort_type == SORT_COMPOSER:
return _jump_table_letter(track.sort_composer or track.composer or "")
elif sort_type == SORT_SHOW:
return _jump_table_letter(track.sort_show or track.show_name or "")
elif sort_type == SORT_SEASON:
n = track.season_number or 0
return _jump_table_letter(str(n)) if n else ord('0')
elif sort_type == SORT_EPISODE:
n = track.episode_number or 0
return _jump_table_letter(str(n)) if n else ord('0')
elif sort_type == SORT_ALBUM_ARTIST:
s = (
track.sort_album_artist
or track.album_artist
or track.sort_artist or track.artist or ""
)
return _jump_table_letter(s)
else:
return _jump_table_letter(track.sort_name or track.title or "")
def write_mhod_type52(tracks: list["TrackInfo"], sort_type: int) -> tuple[bytes, list[tuple[int, int, int]]]:
"""
Write a Type 52 MHOD (library playlist index) for one sort category.
Args:
tracks: List of all TrackInfo objects (in original order)
sort_type: Sort category (SORT_TITLE, SORT_ALBUM, etc.)
Returns:
Tuple of (MHOD bytes, jump_table_entries) where jump_table_entries
is a list of (letter, start, count) tuples for the corresponding
Type 53 MHOD.
"""
num_tracks = len(tracks)
# Create indexed list: (sort_key, original_index, track)
indexed = []
for i, track in enumerate(tracks):
sort_key = _get_sort_fields(track, sort_type)
indexed.append((sort_key, i, track))
# Sort by the sort key
indexed.sort(key=lambda x: x[0])
# Build sorted track indices (original position in track list)
sorted_indices = [idx for _, idx, _ in indexed]
# Build jump table entries: group by first letter
jump_entries: list[tuple[int, int, int]] = []
last_letter = -1
current_entry = None
for pos, (_, _, track) in enumerate(indexed):
letter = _get_jump_letter(track, sort_type)
if letter != last_letter:
current_entry = (letter, pos, 0)
jump_entries.append(current_entry)
last_letter = letter
# Increment count for current entry
letter_val, start, count = jump_entries[-1]
jump_entries[-1] = (letter_val, start, count + 1)
# Build MHOD type 52 binary data
# Body: sort_type(4) + count(4) + padding(40) + indices(count*4)
total_len = 4 * num_tracks + MHOD_HEADER_SIZE + MHOD52_BODY_HEADER_SIZE
header = write_mhod_header(52, total_len)
# Body header
body_header = bytearray(MHOD52_BODY_HEADER_SIZE)
struct.pack_into('<I', body_header, 0, sort_type) # sort type
struct.pack_into('<I', body_header, 4, num_tracks) # number of entries
# Remaining 40 bytes are zero padding
# Track indices
indices_data = bytearray(4 * num_tracks)
for i, idx in enumerate(sorted_indices):
struct.pack_into('<I', indices_data, i * 4, idx)
return bytes(header) + bytes(body_header) + bytes(indices_data), jump_entries
def write_mhod_type53(sort_type: int, jump_entries: list[tuple[int, int, int]]) -> bytes:
"""
Write a Type 53 MHOD (library playlist jump table) for one sort category.
Args:
sort_type: Sort category (must match corresponding type 52)
jump_entries: List of (letter, start, count) tuples from write_mhod_type52()
Returns:
Complete MHOD type 53 bytes
"""
num_entries = len(jump_entries)
# Build MHOD type 53 binary data
# Body: sort_type(4) + count(4) + padding(8) + entries(count*12)
total_len = MHOD53_ENTRY_SIZE * num_entries + MHOD_HEADER_SIZE + MHOD53_BODY_HEADER_SIZE
header = write_mhod_header(53, total_len)
# Body header
body_header = bytearray(MHOD53_BODY_HEADER_SIZE)
struct.pack_into('<I', body_header, 0, sort_type) # sort type
struct.pack_into('<I', body_header, 4, num_entries) # number of entries
# 8 bytes zero padding
# Jump table entries: each is letter(u16) + pad(u16) + start(u32) + count(u32)
entries_data = bytearray(MHOD53_ENTRY_SIZE * num_entries)
for i, (letter, start, count) in enumerate(jump_entries):
offset = i * MHOD53_ENTRY_SIZE
struct.pack_into('<H', entries_data, offset, letter) # letter (UTF-16)
struct.pack_into('<H', entries_data, offset + 2, 0) # padding
struct.pack_into('<I', entries_data, offset + 4, start) # start index
struct.pack_into('<I', entries_data, offset + 8, count) # count
return bytes(header) + bytes(body_header) + bytes(entries_data)
def write_library_indices(tracks: list["TrackInfo"], capabilities=None) -> tuple[bytes, int]:
"""
Write all library index MHODs (type 52 + type 53 pairs) for the
master playlist.
Base sort categories (always written):
- Title (0x03), Album (0x04), Artist (0x05), Genre (0x07), Composer (0x12)
Video sort categories (when capabilities.supports_video is True):
- Show (0x1D), Season (0x1E), Episode (0x1F)
Album Artist sort (0x23) is written for all devices with capabilities
(i.e. modern iPods that use the capabilities system).
Args:
tracks: List of all TrackInfo objects
capabilities: Optional DeviceCapabilities for conditional sort types.
Returns:
Tuple of (concatenated MHOD bytes, count of MHODs written)
"""
if not tracks:
return b'', 0
# Build the list of sort types to write
sort_types = list(BASE_SORT_TYPES)
if capabilities is not None:
if capabilities.supports_video:
sort_types.extend(VIDEO_SORT_TYPES)
# Album artist sort for all modern iPods
sort_types.append(SORT_ALBUM_ARTIST)
result = bytearray()
mhod_count = 0
for sort_type in sort_types:
# Write type 52 (sorted index)
mhod52_data, jump_entries = write_mhod_type52(tracks, sort_type)
result.extend(mhod52_data)
mhod_count += 1
# Write type 53 (jump table)
mhod53_data = write_mhod_type53(sort_type, jump_entries)
result.extend(mhod53_data)
mhod_count += 1
return bytes(result), mhod_count
+329
View File
@@ -0,0 +1,329 @@
"""
MHOD Type 50/51 Writer Smart Playlist Preferences & Rules.
Type 50 (SPLPref): Controls live-update, checked-only, and limit settings.
Type 51 (SPLRules/SLst): The actual filter rules that define the smart playlist.
The SLst blob is the ONLY part of the iTunesDB that uses big-endian encoding.
All multi-byte integers within SLst must be written as big-endian, and
string values use UTF-16 BE (not LE like the rest of the database).
Based on libgpod's SPLPref/SPLRules structs in itdb_spl.c / itdb_itunesdb.c
and the parser in iTunesDB_Parser/mhod_parser.py.
"""
import struct
from dataclasses import dataclass, field
from iTunesDB_Shared.mhod_defs import (
MHOD_HEADER_SIZE,
SLST_HEADER_SIZE,
SPL_RULE_DATA_SIZE,
SPL_RULE_HEADER_SIZE,
SPLFT_DATE,
SPLFT_STRING,
SPLPREF_BODY_SIZE,
spl_get_field_type,
write_mhod_header,
)
DATE_RELATIVE_ACTION_IDS = {0x00000200, 0x02000200}
# ────────────────────────────────────────────────────────────
# Data classes
# ────────────────────────────────────────────────────────────
@dataclass
class SmartPlaylistPrefs:
"""Smart playlist preferences (MHOD type 50 / SPLPref).
Mirrors the fields parsed by _parse_mhod50_smart_playlist_data().
"""
live_update: bool = True
check_rules: bool = True
check_limits: bool = False
limit_type: int = 0x03 # 1=minutes, 2=MB, 3=songs, 4=hours, 5=GB
limit_sort: int = 0x02 # 2=random (low byte); high bit 0x80000000 = reverse
limit_value: int = 25
match_checked_only: bool = False
@dataclass
class SmartPlaylistRule:
"""A single smart playlist rule (one entry inside SLst).
field_id and action_id use the raw integer codes from the parser
constants (SPL_FIELD_MAP, SPL_ACTION_MAP).
"""
field_id: int = 0x02 # e.g. 0x02=Song Name, 0x3C=Media Type
action_id: int = 0x01000002 # e.g. 0x01000002 = "contains"
# For STRING rules
string_value: str | None = None
# For non-string rules (INT/DATE/BOOLEAN/PLAYLIST/BINARY_AND)
from_value: int = 0
from_date: int = 0
from_units: int = 0
to_value: int = 0
to_date: int = 0
to_units: int = 0
# Five unknown trailing 32-bit values (preserved for round-trip)
unk052: int = 0
unk056: int = 0
unk060: int = 0
unk064: int = 0
unk068: int = 0
@dataclass
class SmartPlaylistRules:
"""Full smart playlist rules container (MHOD type 51 / SLst).
conjunction: "AND" (match all) or "OR" (match any)
"""
conjunction: str = "AND" # "AND" or "OR"
rules: list[SmartPlaylistRule] = field(default_factory=list)
unk004: int = 0 # SLst header +0x04, usually 0 (preserved for round-trip)
def _signed_i64(value: int) -> int:
value = int(value or 0)
if value >= (1 << 63):
return value - (1 << 64)
return value
def _normalize_relative_date_fields(
from_value: int,
from_date: int,
from_units: int = 0,
) -> tuple[int, int]:
signed_from_value = _signed_i64(from_value)
normalized_from_date = int(from_date or 0)
if normalized_from_date:
normalized_from_date = -abs(normalized_from_date)
elif signed_from_value:
amount = abs(signed_from_value)
units = int(from_units or 0)
if units > 1 and amount >= units and amount % units == 0:
normalized_from_date = -(amount // units)
else:
normalized_from_date = -amount
return 0, normalized_from_date
# ────────────────────────────────────────────────────────────
# MHOD Type 50 — Smart Playlist Preferences
# ────────────────────────────────────────────────────────────
def write_mhod50(prefs: SmartPlaylistPrefs) -> bytes:
"""Write MHOD type 50 (smart playlist preferences / SPLPref).
Returns:
Complete MHOD chunk bytes.
"""
body = bytearray(SPLPREF_BODY_SIZE)
body[0] = 1 if prefs.live_update else 0
body[1] = 1 if prefs.check_rules else 0
body[2] = 1 if prefs.check_limits else 0
body[3] = prefs.limit_type & 0xFF
# limit_sort: low byte at +4, reverse flag at +13
low_byte = prefs.limit_sort & 0xFF
reverse = 1 if (prefs.limit_sort & 0x80000000) else 0
body[4] = low_byte
# 3 bytes padding (5..7) already zero
struct.pack_into('<I', body, 8, prefs.limit_value)
body[12] = 1 if prefs.match_checked_only else 0
body[13] = reverse
# Remaining bytes (14..131) are zero padding
return write_mhod_header(50, MHOD_HEADER_SIZE + SPLPREF_BODY_SIZE) + bytes(body)
# ────────────────────────────────────────────────────────────
# MHOD Type 51 — Smart Playlist Rules (SLst)
# ────────────────────────────────────────────────────────────
def _write_spl_rule(rule: SmartPlaylistRule) -> bytes:
"""Write a single SLst rule entry (big-endian).
Rule layout:
+0x00: field (4 BE)
+0x04: action (4 BE)
+0x08: padding (44 bytes)
+0x34: length (4 BE) byte length of data
+0x38: data (length bytes)
Total = SPL_RULE_HEADER_SIZE + data_length.
"""
ft = spl_get_field_type(rule.field_id)
if ft == SPLFT_STRING and rule.string_value is not None:
# String rule: data = UTF-16 BE string
string_bytes = rule.string_value.encode('utf-16-be')
data_length = len(string_bytes)
data_section = string_bytes
else:
# Non-string: fixed SPL_RULE_DATA_SIZE (68) byte data section
data_length = SPL_RULE_DATA_SIZE
data_section = bytearray(SPL_RULE_DATA_SIZE)
from_value = rule.from_value
from_date = rule.from_date
if ft == SPLFT_DATE and rule.action_id in DATE_RELATIVE_ACTION_IDS:
from_value, from_date = _normalize_relative_date_fields(
from_value,
from_date,
rule.from_units,
)
# from_value, to_value, from_units, to_units use unsigned '>Q' format.
# Mask defensively so legacy in-memory rules with signed values still pack.
_mask = 0xFFFFFFFFFFFFFFFF
struct.pack_into('>Q', data_section, 0x00, from_value & _mask)
struct.pack_into('>q', data_section, 0x08, from_date)
struct.pack_into('>Q', data_section, 0x10, rule.from_units & _mask)
struct.pack_into('>Q', data_section, 0x18, rule.to_value & _mask)
struct.pack_into('>q', data_section, 0x20, rule.to_date)
struct.pack_into('>Q', data_section, 0x28, rule.to_units & _mask)
struct.pack_into('>I', data_section, 0x30, rule.unk052)
struct.pack_into('>I', data_section, 0x34, rule.unk056)
struct.pack_into('>I', data_section, 0x38, rule.unk060)
struct.pack_into('>I', data_section, 0x3C, rule.unk064)
struct.pack_into('>I', data_section, 0x40, rule.unk068)
data_section = bytes(data_section)
# Build rule header
rule_header = bytearray(SPL_RULE_HEADER_SIZE)
struct.pack_into('>I', rule_header, 0x00, rule.field_id)
struct.pack_into('>I', rule_header, 0x04, rule.action_id)
# 44 bytes padding (0x08..0x33) already zero
struct.pack_into('>I', rule_header, 0x34, data_length)
return bytes(rule_header) + data_section
def write_mhod51(rules_data: SmartPlaylistRules) -> bytes:
"""Write MHOD type 51 (smart playlist rules / SLst).
The entire SLst blob is big-endian.
Returns:
Complete MHOD chunk bytes.
"""
# Build SLst header
slst_header = bytearray(SLST_HEADER_SIZE)
slst_header[0:4] = b'SLst'
struct.pack_into('>I', slst_header, 4, rules_data.unk004)
struct.pack_into('>I', slst_header, 8, len(rules_data.rules))
conjunction_val = 1 if rules_data.conjunction.upper() == "OR" else 0
struct.pack_into('>I', slst_header, 12, conjunction_val)
# 120 bytes padding already zero
# Build individual rules
rules_bytes = b''.join(_write_spl_rule(r) for r in rules_data.rules)
slst_body = bytes(slst_header) + rules_bytes
return write_mhod_header(51, MHOD_HEADER_SIZE + len(slst_body)) + slst_body
# ────────────────────────────────────────────────────────────
# MHOD Type 102 — Playlist Settings (opaque blob passthrough)
# ────────────────────────────────────────────────────────────
def write_mhod102(raw_body: bytes) -> bytes:
"""Write MHOD type 102 (playlist settings).
This is an opaque iTunes binary blob. We preserve it verbatim
from the parsed data for round-trip fidelity.
Args:
raw_body: The raw body bytes (everything after the 24-byte header).
Returns:
Complete MHOD chunk bytes.
"""
return write_mhod_header(102, MHOD_HEADER_SIZE + len(raw_body)) + raw_body
# ────────────────────────────────────────────────────────────
# Helpers for building from parsed data (round-trip)
# ────────────────────────────────────────────────────────────
def prefs_from_parsed(parsed: dict) -> SmartPlaylistPrefs:
"""Create SmartPlaylistPrefs from a parsed MHOD type 50 dict.
This is the inverse of _parse_mhod50_smart_playlist_data().
"""
# Parser stores limit_sort as the raw low byte and reverse_sort
# separately. Reconstruct the combined value the writer expects.
limit_sort = parsed.get("limit_sort", 0x02)
if parsed.get("reverse_sort", 0):
limit_sort |= 0x80000000
return SmartPlaylistPrefs(
live_update=parsed.get("live_update", True),
check_rules=parsed.get("check_rules", True),
check_limits=parsed.get("check_limits", False),
limit_type=parsed.get("limit_type", 0x03),
limit_sort=limit_sort,
limit_value=parsed.get("limit_value", 25),
match_checked_only=parsed.get("match_checked_only", False),
)
def rules_from_parsed(parsed: dict) -> SmartPlaylistRules:
"""Create SmartPlaylistRules from a parsed MHOD type 51 dict.
This is the inverse of _parse_mhod51_smart_playlist_rules().
"""
rules = []
for r in parsed.get("rules", []):
field_id = r.get("field_id", 0)
action_id = r.get("action_id", 0)
from_value = r.get("from_value", 0)
from_date = r.get("from_date", 0)
from_units = r.get("from_units", 0)
if spl_get_field_type(field_id) == SPLFT_DATE and action_id in DATE_RELATIVE_ACTION_IDS:
from_value, from_date = _normalize_relative_date_fields(
from_value,
from_date,
from_units,
)
rule = SmartPlaylistRule(
field_id=field_id,
action_id=action_id,
string_value=r.get("string_value"),
from_value=from_value,
from_date=from_date,
from_units=from_units,
to_value=r.get("to_value", 0),
to_date=r.get("to_date", 0),
to_units=r.get("to_units", 0),
unk052=r.get("unk052", 0),
unk056=r.get("unk056", 0),
unk060=r.get("unk060", 0),
unk064=r.get("unk064", 0),
unk068=r.get("unk068", 0),
)
rules.append(rule)
raw_conj = parsed.get("conjunction", "AND")
if isinstance(raw_conj, int):
conj = "OR" if raw_conj == 1 else "AND"
else:
conj = raw_conj
return SmartPlaylistRules(
conjunction=conj,
rules=rules,
unk004=parsed.get("unk004", 0),
)
+434
View File
@@ -0,0 +1,434 @@
"""
MHOD String Writer Write string, podcast URL, and chapter data MHOD chunks.
String MHODs (types 1-14, 18-31, 33-44, 200-204, 300) have:
- Common MHOD header (24 bytes)
- String sub-header (16 bytes): encoding + string_length + unk0x20 + unk0x24
- UTF-16LE encoded string data
Podcast URL MHODs (types 15, 16) have:
- Common MHOD header (24 bytes)
- UTF-8 encoded string directly (NO sub-header)
Chapter Data MHOD (type 17) has:
- Common MHOD header (24 bytes)
- 12-byte preamble (3 × u32 LE)
- Big-endian atom tree: sean chap × N name + hedr
- Stored in iTunesDB, so it is not tied to an AAC/M4A source file
Cross-referenced against:
- iTunesDB_Shared/mhod_defs.py (field definitions and constants)
- iTunesDB_Parser/mhod_parser.py _parse_string_mhod(), _parse_chapter_data()
- libgpod itdb_itunesdb.c: mk_mhod(), itdb_chapterdata_build_chapter_blob_internal()
"""
import struct
from iTunesDB_Shared.constants import (
MHOD_TYPE_ALBUM,
MHOD_TYPE_ALBUM_ARTIST,
MHOD_TYPE_ARTIST,
MHOD_TYPE_CATEGORY,
MHOD_TYPE_CHAPTER_DATA,
MHOD_TYPE_COMMENT,
MHOD_TYPE_COMPOSER,
MHOD_TYPE_DESCRIPTION,
MHOD_TYPE_EPISODE_ID,
MHOD_TYPE_EQ_SETTING,
MHOD_TYPE_FILETYPE,
MHOD_TYPE_GENRE,
MHOD_TYPE_GROUPING,
MHOD_TYPE_KEYWORDS,
MHOD_TYPE_LOCATION,
MHOD_TYPE_LYRICS,
MHOD_TYPE_NETWORK_NAME,
MHOD_TYPE_PODCAST_ENCLOSURE_URL,
MHOD_TYPE_PODCAST_RSS_URL,
MHOD_TYPE_SHOW_LOCALE,
MHOD_TYPE_SHOW_NAME,
MHOD_TYPE_SORT_ALBUM,
MHOD_TYPE_SORT_ALBUM_ARTIST,
MHOD_TYPE_SORT_ARTIST,
MHOD_TYPE_SORT_COMPOSER,
MHOD_TYPE_SORT_NAME,
MHOD_TYPE_SORT_SHOW,
MHOD_TYPE_SUBTITLE,
MHOD_TYPE_TITLE,
)
from iTunesDB_Shared.mhod_defs import (
CHAP_ATOM,
HEDR_ATOM,
HEDR_SIZE,
MHOD_HEADER_SIZE,
MHOD_STRING_SUBHEADER_SIZE,
NAME_ATOM,
SEAN_ATOM,
write_mhod_header,
)
def write_mhod_string(mhod_type: int, value: str,
unk_0x20: int = 1, unk_0x24: int = 0) -> bytes:
"""
Write a string MHOD chunk.
String MHODs have this structure:
- mhod header (24 bytes minimum)
- string data type header (16 bytes)
- UTF-16LE encoded string
Args:
mhod_type: MHOD type (1=title, 2=location, etc.)
value: String value to encode
unk_0x20: Sub-header unknown at offset 0x20 (preserved from parser).
unk_0x24: Sub-header unknown at offset 0x24 (preserved from parser).
Returns:
Complete MHOD chunk as bytes
"""
if not value:
return b''
string_data = value.encode('utf-16-le')
string_len = len(string_data)
total_len = MHOD_HEADER_SIZE + MHOD_STRING_SUBHEADER_SIZE + string_len
header = write_mhod_header(mhod_type, total_len)
# String sub-header: encoding(4) + string_length(4) + unk0x20(4) + unk0x24(4)
# encoding=1 means UTF-16LE
type_header = struct.pack('<IIII', 1, string_len, unk_0x20, unk_0x24)
return header + type_header + string_data
def write_mhod_location(path: str) -> bytes:
"""
Write a location MHOD (type 2) for file path.
iPod paths use colons as separators:
:iPod_Control:Music:F00:ABCD.mp3
Args:
path: iPod-relative path with colon separators
Returns:
Complete MHOD chunk
"""
return write_mhod_string(MHOD_TYPE_LOCATION, path)
def write_mhod_title(title: str) -> bytes:
return write_mhod_string(MHOD_TYPE_TITLE, title)
def write_mhod_artist(artist: str) -> bytes:
return write_mhod_string(MHOD_TYPE_ARTIST, artist)
def write_mhod_album(album: str) -> bytes:
return write_mhod_string(MHOD_TYPE_ALBUM, album)
def write_mhod_genre(genre: str) -> bytes:
return write_mhod_string(MHOD_TYPE_GENRE, genre)
def write_mhod_album_artist(album_artist: str) -> bytes:
return write_mhod_string(MHOD_TYPE_ALBUM_ARTIST, album_artist)
def write_mhod_composer(composer: str) -> bytes:
return write_mhod_string(MHOD_TYPE_COMPOSER, composer)
def write_mhod_comment(comment: str) -> bytes:
return write_mhod_string(MHOD_TYPE_COMMENT, comment)
def write_mhod_filetype(filetype: str) -> bytes:
return write_mhod_string(MHOD_TYPE_FILETYPE, filetype)
def write_mhod_sort_artist(sort_artist: str) -> bytes:
return write_mhod_string(MHOD_TYPE_SORT_ARTIST, sort_artist)
def write_mhod_sort_name(sort_name: str) -> bytes:
return write_mhod_string(MHOD_TYPE_SORT_NAME, sort_name)
def write_mhod_sort_album(sort_album: str) -> bytes:
return write_mhod_string(MHOD_TYPE_SORT_ALBUM, sort_album)
def write_mhod_podcast_url(mhod_type: int, url: str) -> bytes:
"""
Write a podcast URL MHOD (type 15 or 16).
Podcast URL MHODs use a DIFFERENT format from standard string MHODs:
- UTF-8 encoded (NOT UTF-16LE)
- NO type sub-header (string follows directly after the 24-byte header)
- Length = total_length header_length
Per iPodLinux wiki and parser: types 15 (enclosure URL) and 16 (RSS URL)
have no mhod::length field and use UTF-8/ASCII encoding.
Args:
mhod_type: Must be 15 (enclosure URL) or 16 (RSS URL)
url: URL string to encode
Returns:
Complete MHOD chunk as bytes
"""
if not url:
return b''
if mhod_type not in (MHOD_TYPE_PODCAST_ENCLOSURE_URL, MHOD_TYPE_PODCAST_RSS_URL):
raise ValueError(f"write_mhod_podcast_url only supports types 15 and 16, got {mhod_type}")
string_data = url.encode('utf-8')
total_len = MHOD_HEADER_SIZE + len(string_data)
header = write_mhod_header(mhod_type, total_len)
return header + string_data
def write_mhod_chapter_data(
chapters: list[dict],
unk024: int = 0,
unk028: int = 0,
unk032: int = 0,
) -> bytes:
"""Write a chapter data MHOD (type 17).
Chapter data uses big-endian atom tree encoding, matching libgpod's
``itdb_chapterdata_build_chapter_blob_internal()``.
Args:
chapters: List of chapter dicts, each with ``startpos`` (int, ms)
and ``title`` (str).
unk024, unk028, unk032: Preamble unknown fields (preserved from
parser, default 0).
Returns:
Complete MHOD type 17 chunk as bytes, or b'' if chapters is empty.
"""
if not chapters:
return b''
# Build the atom tree body (all big-endian).
atoms = bytearray()
for ch in chapters:
title = ch.get("title", "")
startpos = ch.get("startpos", 0)
title_utf16 = title.encode("utf-16-be")
title_units = len(title_utf16) // 2
# name atom: size(4) + "name"(4) + unk=1(4) + unk=0(4) + unk=0(4) + strlen(2) + string
name_size = 22 + len(title_utf16)
name_atom = struct.pack(">I", name_size)
name_atom += NAME_ATOM
name_atom += struct.pack(">III", 1, 0, 0)
name_atom += struct.pack(">H", title_units)
name_atom += title_utf16
# chap atom: size(4) + "chap"(4) + startpos(4) + children=1(4) + unk=0(4) + name_atom
chap_size = 20 + name_size
chap_atom = struct.pack(">I", chap_size)
chap_atom += CHAP_ATOM
chap_atom += struct.pack(">III", startpos, 1, 0)
chap_atom += name_atom
atoms.extend(chap_atom)
# hedr terminator atom (28 bytes)
hedr_atom = struct.pack(">I", HEDR_SIZE)
hedr_atom += HEDR_ATOM
hedr_atom += struct.pack(">IIIII", 1, 0, 0, 0, 1)
atoms.extend(hedr_atom)
# sean atom header wraps everything
num_children = len(chapters) + 1 # chapters + hedr
sean_size = 20 + len(atoms)
sean_header = struct.pack(">I", sean_size)
sean_header += SEAN_ATOM
sean_header += struct.pack(">III", 1, num_children, 0)
# Preamble (little-endian, 12 bytes)
preamble = struct.pack("<III", unk024, unk028, unk032)
# Complete body = preamble + sean_header + atoms
body = preamble + sean_header + bytes(atoms)
# MHOD header + body
total_length = MHOD_HEADER_SIZE + len(body)
header = write_mhod_header(MHOD_TYPE_CHAPTER_DATA, total_length)
return header + body
def build_chapter_blob(
chapters: list[dict],
unk024: int = 0,
unk028: int = 0,
unk032: int = 0,
) -> bytes:
"""Build the chapter atom blob (no MHOD header) for SQLite Extras.itdb.
Same atom tree format as MHOD type 17 but without the 24-byte MHOD header.
Used by ``SQLiteDB_Writer.extras_writer`` for the ``chapter.data`` BLOB.
Returns:
Raw chapter blob bytes, or b'' if chapters is empty.
"""
full = write_mhod_chapter_data(chapters, unk024, unk028, unk032)
if not full:
return b''
# Strip the MHOD header to get just the atom tree
return full[MHOD_HEADER_SIZE:]
def write_track_mhods(
title: str,
location: str,
artist: str | None = None,
album: str | None = None,
genre: str | None = None,
album_artist: str | None = None,
composer: str | None = None,
comment: str | None = None,
filetype_desc: str | None = None,
sort_artist: str | None = None,
sort_name: str | None = None,
sort_album: str | None = None,
sort_album_artist: str | None = None,
sort_composer: str | None = None,
grouping: str | None = None,
description: str | None = None,
podcast_enclosure_url: str | None = None,
podcast_rss_url: str | None = None,
subtitle: str | None = None,
show_name: str | None = None,
episode_id: str | None = None,
network_name: str | None = None,
keywords: str | None = None,
sort_show: str | None = None,
category: str | None = None,
lyrics: str | None = None,
eq_setting: str | None = None,
show_locale: str | None = None,
chapter_data: dict | None = None,
) -> tuple[bytes, int]:
"""
Write all MHODs for a track.
Args:
title: Track title (required)
location: File path on iPod (required)
artist: Artist name
album: Album name
genre: Genre
album_artist: Album artist (for compilations)
composer: Composer
comment: Comment/notes
filetype_desc: File format description (e.g., "MPEG audio file")
sort_artist: Sort artist name
sort_name: Sort title
sort_album: Sort album name
sort_album_artist: Sort album artist name
sort_composer: Sort composer name
grouping: Grouping tag
description: Track description (type 14)
podcast_enclosure_url: Podcast enclosure URL (type 15, UTF-8, no sub-header)
podcast_rss_url: Podcast RSS feed URL (type 16, UTF-8, no sub-header)
subtitle: Subtitle (type 18)
show_name: TV show name (type 19)
episode_id: Episode ID string (type 20)
network_name: TV network name (type 21)
keywords: Keywords (type 24)
sort_show: Sort show name (type 31)
category: Podcast/audiobook category (type 9)
chapter_data: Chapter data dict with ``chapters`` list (type 17)
Returns:
Tuple of (concatenated MHOD bytes, count of MHODs)
"""
chunks: list[bytes] = []
# Required MHODs
chunks.append(write_mhod_title(title))
chunks.append(write_mhod_location(location))
# Optional string MHODs
if artist:
chunks.append(write_mhod_artist(artist))
if album:
chunks.append(write_mhod_album(album))
if genre:
chunks.append(write_mhod_genre(genre))
if album_artist:
chunks.append(write_mhod_album_artist(album_artist))
if composer:
chunks.append(write_mhod_composer(composer))
if comment:
chunks.append(write_mhod_comment(comment))
if filetype_desc:
chunks.append(write_mhod_filetype(filetype_desc))
if category:
chunks.append(write_mhod_string(MHOD_TYPE_CATEGORY, category))
if description:
chunks.append(write_mhod_string(MHOD_TYPE_DESCRIPTION, description))
if subtitle:
chunks.append(write_mhod_string(MHOD_TYPE_SUBTITLE, subtitle))
if show_name:
chunks.append(write_mhod_string(MHOD_TYPE_SHOW_NAME, show_name))
if episode_id:
chunks.append(write_mhod_string(MHOD_TYPE_EPISODE_ID, episode_id))
if network_name:
chunks.append(write_mhod_string(MHOD_TYPE_NETWORK_NAME, network_name))
if keywords:
chunks.append(write_mhod_string(MHOD_TYPE_KEYWORDS, keywords))
# Sort MHODs
if sort_artist:
chunks.append(write_mhod_sort_artist(sort_artist))
if sort_name:
chunks.append(write_mhod_sort_name(sort_name))
if sort_album:
chunks.append(write_mhod_sort_album(sort_album))
if sort_album_artist:
chunks.append(write_mhod_string(MHOD_TYPE_SORT_ALBUM_ARTIST, sort_album_artist))
if sort_composer:
chunks.append(write_mhod_string(MHOD_TYPE_SORT_COMPOSER, sort_composer))
if sort_show:
chunks.append(write_mhod_string(MHOD_TYPE_SORT_SHOW, sort_show))
if show_locale:
chunks.append(write_mhod_string(MHOD_TYPE_SHOW_LOCALE, show_locale))
if grouping:
chunks.append(write_mhod_string(MHOD_TYPE_GROUPING, grouping))
# Podcast URL MHODs (different format: UTF-8, no sub-header)
if podcast_enclosure_url:
chunks.append(write_mhod_podcast_url(MHOD_TYPE_PODCAST_ENCLOSURE_URL, podcast_enclosure_url))
if podcast_rss_url:
chunks.append(write_mhod_podcast_url(MHOD_TYPE_PODCAST_RSS_URL, podcast_rss_url))
# EQ and lyrics
if eq_setting:
chunks.append(write_mhod_string(MHOD_TYPE_EQ_SETTING, eq_setting))
if lyrics:
chunks.append(write_mhod_string(MHOD_TYPE_LYRICS, lyrics))
# Chapter data (type 17, big-endian atom tree)
if chapter_data and chapter_data.get("chapters"):
chunks.append(write_mhod_chapter_data(
chapters=chapter_data["chapters"],
unk024=chapter_data.get("unk024", 0),
unk028=chapter_data.get("unk028", 0),
unk032=chapter_data.get("unk032", 0),
))
return b''.join(chunks), len(chunks)
+101
View File
@@ -0,0 +1,101 @@
"""MHSD Writer — Write dataset chunks for iTunesDB.
MHSD (dataset) chunks are containers for different types of data.
Each MHSD wraps exactly one child list chunk (mhlt, mhlp, mhla, or mhli).
Header layout (MHSD_HEADER_SIZE = 96 bytes):
+0x00: 'mhsd' magic (4B)
+0x04: header_length (4B)
+0x08: total_length (4B) header + child data
+0x0C: dataset_type (4B):
1 = Track list (mhlt)
2 = Playlist list (mhlp)
3 = Podcast list (mhlp) same content as type 2
4 = Album list (mhla)
5 = Smart playlist list (mhlp)
6 = Empty stub (mhlt with 0 children)
8 = Artist list (mhli with mhii children)
10 = Empty stub (mhlt with 0 children)
Cross-referenced against:
- iTunesDB_Parser/mhsd_parser.py
- libgpod itdb_itunesdb.c: mk_mhsd()
"""
from iTunesDB_Shared.field_base import (
MHLT_HEADER_SIZE,
write_fields,
write_generic_header,
)
from iTunesDB_Shared.mhsd_defs import MHSD_HEADER_SIZE
def write_mhsd(dataset_type: int, child_data: bytes) -> bytes:
"""
Write a MHSD (dataset) chunk.
Args:
dataset_type: Type of dataset
child_data: Child chunk data (mhlt, mhlp, or mhla)
Returns:
Complete MHSD chunk bytes
"""
# Total length = header + child
total_length = MHSD_HEADER_SIZE + len(child_data)
# Build header
header = bytearray(MHSD_HEADER_SIZE)
write_generic_header(header, 0, b'mhsd', MHSD_HEADER_SIZE, total_length)
write_fields(header, 0, 'mhsd', {'dataset_type': dataset_type}, MHSD_HEADER_SIZE)
return bytes(header) + child_data
def write_mhsd_type1(track_list_data: bytes) -> bytes:
"""Write a Type 1 MHSD containing track list."""
return write_mhsd(1, track_list_data)
def write_mhsd_type2(playlist_list_data: bytes) -> bytes:
"""Write a Type 2 MHSD containing playlist list."""
return write_mhsd(2, playlist_list_data)
def write_mhsd_type3(podcast_list_data: bytes) -> bytes:
"""Write a Type 3 MHSD containing podcast list."""
return write_mhsd(3, podcast_list_data)
def write_mhsd_type4(album_list_data: bytes) -> bytes:
"""Write a Type 4 MHSD containing album list."""
return write_mhsd(4, album_list_data)
def write_mhsd_smart_type5(smart_playlist_data: bytes) -> bytes:
"""Write a Type 5 MHSD containing smart playlist list."""
return write_mhsd(5, smart_playlist_data)
def write_mhsd_type8(artist_list_data: bytes) -> bytes:
"""Write a Type 8 MHSD containing artist list (mhli)."""
return write_mhsd(8, artist_list_data)
def write_mhsd_empty_stub(dataset_type: int) -> bytes:
"""Write a stub MHSD containing an empty MHLT (0 children).
Used for types 6 and 10 which libgpod writes as empty track-list
stubs. The child is a minimal MHLT header with count = 0.
Args:
dataset_type: The MHSD type (6 or 10).
Returns:
Complete MHSD + empty MHLT bytes.
"""
# Build an empty MHLT child (92-byte header, 0 tracks)
mhlt = bytearray(MHLT_HEADER_SIZE)
write_generic_header(mhlt, 0, b'mhlt', MHLT_HEADER_SIZE, 0)
return write_mhsd(dataset_type, bytes(mhlt))
+538
View File
@@ -0,0 +1,538 @@
"""
MHYP Writer Write playlist chunks for iTunesDB.
MHYP chunks define playlists. Every iTunesDB MUST have at least one
playlist the Master Playlist (MPL) which references all tracks.
Supports three kinds of playlists:
- Master Playlist (master=True): references all tracks, includes library indices
- Regular playlists: user-created playlists with explicit track lists
- Smart playlists: rule-based playlists with MHOD types 50 (prefs) and 51 (rules)
Header layout (MHYP_HEADER_SIZE = 184 bytes):
+0x00: 'mhyp' magic (4B)
+0x04: header_length (4B)
+0x08: total_length (4B) header + all children
+0x0C: mhod_count (4B)
+0x10: mhip_count (4B)
+0x14: type (1B) + flag1 (1B) + flag2 (1B) + flag3 (1B) master playlist flag
+0x18: timestamp (4B Mac)
+0x1C: playlist_id (8B)
+0x24: unk1 (4B)
+0x28: string_mhod_count (2B)
+0x2A: podcast_flag (2B) 0=normal, 1=podcast playlist (u16, libgpod podcastflag)
+0x2C: sort_order (4B)
+0x3C: db_id_2 (8B) MHBD database ID reference (non-master)
+0x44: playlist_id_copy (8B)
+0x50: mhsd5_type (2B) browsing category for dataset 5
+0x58: timestamp_copy (4B Mac)
Cross-referenced against:
- iTunesDB_Parser/mhyp_parser.py parse_playlist()
- libgpod itdb_itunesdb.c: write_playlist() / mk_mhyp()
- iPodLinux wiki MHYP documentation
"""
import random
import struct
import time
from dataclasses import dataclass, field
from typing import List, Optional, TYPE_CHECKING
if TYPE_CHECKING:
from .mhit_writer import TrackInfo
from iTunesDB_Shared.constants import MHOD_TYPE_TITLE
from iTunesDB_Shared.field_base import write_fields, write_generic_header
from iTunesDB_Shared.mhyp_defs import MHYP_HEADER_SIZE
from iTunesDB_Shared.mhod_defs import (
MHOD_HEADER_SIZE as _MHOD_HEADER_SIZE,
write_mhod_header,
)
from .mhod_writer import write_mhod_string
from .mhip_writer import write_mhip, write_mhip_podcast_group
from .mhod52_writer import write_library_indices
from .mhod_spl_writer import (
SmartPlaylistPrefs,
SmartPlaylistRules,
write_mhod50,
write_mhod51,
write_mhod102,
)
@dataclass
class PlaylistItemMeta:
"""Per-item metadata preserved from parsed MHIP entries for round-trip fidelity.
These fields map directly to MHIP header offsets:
+0x10: podcast_group_flag (4B)
+0x14: group_id (4B) unique MHIP identifier (libgpod: podcastgroupid)
+0x20: podcast_group_ref (4B) references another MHIP's group_id
+0x2C: track_persistent_id (8B) track's db_track_id
+0x3C: mhip_persistent_id (8B) per-track persistent ID
"""
podcast_group_flag: int = 0
group_id: int = 0
podcast_group_ref: int = 0
track_persistent_id: int = 0
mhip_persistent_id: int = 0
@dataclass
class PlaylistInfo:
"""Structured input for writing a playlist to iTunesDB.
Covers regular playlists, smart playlists, and the master playlist.
The master playlist is constructed internally by write_master_playlist()
and does not need a PlaylistInfo.
"""
name: str
track_ids: List[int] = field(default_factory=list)
# Identity
playlist_id: Optional[int] = None # 64-bit; generated if None
master: bool = False # Sets type byte at +0x14 to 1.
# Dataset 2: True for the master playlist only (exactly one).
# Dataset 5: True for ALL built-in categories (Music, Movies, etc.).
# In both cases this controls: (a) the type byte at +0x14,
# (b) whether library indices are generated (only when tracks
# are also provided), and (c) whether db_id_2/playlist_id
# are written at the extended offsets +0x3C/+0x44 (skipped
# when master=True, matching libgpod behaviour).
sortorder: int = 0 # 0=default, 1=manual, 3=title ...
podcast_flag: int = 0 # 0x2A: 0=normal, 1=podcast playlist (u16)
# Smart playlist fields (both must be set for a smart playlist)
smart_prefs: Optional[SmartPlaylistPrefs] = None
smart_rules: Optional[SmartPlaylistRules] = None
# mhsd5Type: browsing category for dataset 5 smart playlists
# (per libgpod: 0=None, 2=Movies, 3=TV Shows, 4=Music, 5=Audiobooks, 6=Ringtones, 7=MovieRentals)
mhsd5_type: int = 0
# Opaque blobs preserved from parsed data for round-trip fidelity
raw_mhod100: Optional[bytes] = None # Playlist prefs (type 100 body)
raw_mhod102: Optional[bytes] = None # Playlist settings (type 102 body)
# Per-MHIP metadata preserved from parsed data for round-trip fidelity.
# When provided, must be the same length as track_ids and in the same order.
item_metadata: Optional[List[PlaylistItemMeta]] = None
@property
def is_smart(self) -> bool:
return self.smart_prefs is not None and self.smart_rules is not None
def generate_playlist_id() -> int:
"""Generate a random 64-bit playlist ID."""
return random.getrandbits(64)
def write_mhyp(
name: str,
track_ids: List[int],
playlist_id: Optional[int] = None,
master: bool = False,
timestamp: Optional[int] = None,
sortorder: int = 0,
podcast_flag: int = 0,
tracks: Optional[List["TrackInfo"]] = None,
db_id_2: int = 0,
smart_prefs: Optional[SmartPlaylistPrefs] = None,
smart_rules: Optional[SmartPlaylistRules] = None,
mhsd5_type: int = 0,
raw_mhod100: Optional[bytes] = None,
raw_mhod102: Optional[bytes] = None,
item_metadata: Optional[List[PlaylistItemMeta]] = None,
capabilities=None,
podcast_grouping: bool = False,
track_album_map: Optional[dict[int, str]] = None,
next_mhip_id_start: int = 1,
) -> bytes:
"""
Write a complete MHYP (playlist) chunk with MHODs and MHIPs.
The structure is:
- MHYP header (184 bytes)
- MHOD title (string)
- MHOD playlist data (type 100 preferences)
- [Smart only] MHOD type 50 (smart playlist prefs)
- [Smart only] MHOD type 51 (smart playlist rules / SLst)
- [Smart only] MHOD type 102 (playlist settings, if provided)
- [Master Playlist only] MHOD type 52/53 pairs (library indices)
- MHIP entries (one per track)
Args:
name: Playlist name
track_ids: List of track IDs to include in this playlist
playlist_id: Playlist ID (generated if not provided)
master: Whether the type byte at +0x14 should be set to 1.
For dataset 2 this means "master playlist" (exactly one).
For dataset 5 this means "built-in system category" (all
categories have master=True). The behavioural effects are:
(a) type byte at +0x14 is written as 1,
(b) library indices are generated IF *tracks* is also
provided (ds5 never passes tracks, so this is safe),
(c) db_id_2 and playlist_id are NOT written at +0x3C/+0x44
(matches libgpod, which zeros these for type=1).
timestamp: Creation timestamp (now if not provided)
sortorder: Sort order (0 = manual)
podcast_flag: 0x2A 0=normal playlist, 1=podcast playlist (u16,
matching libgpod podcastflag).
tracks: List of TrackInfo objects (required for Master Playlist to
generate library index MHODs type 52/53)
db_id_2: Database-wide ID from MHBD offset 0x24. Written at MHYP offset
0x3C for non-master playlists, and used as a validation field.
smart_prefs: Smart playlist preferences (MHOD 50). Both smart_prefs
and smart_rules must be set for a smart playlist.
smart_rules: Smart playlist rules (MHOD 51).
mhsd5_type: Browsing category for dataset 5 smart playlists.
raw_mhod100: If provided, use this raw body for MHOD type 100 instead
of generating a default one.
raw_mhod102: If provided, write an MHOD type 102 with this raw body.
podcast_grouping: When True and this is a podcast playlist, generate
grouped MHIPs (libgpod write_podcast_mhips style) where
episodes are nested under their podcast show by album.
track_album_map: Mapping of track_id album name. Required when
podcast_grouping is True.
next_mhip_id_start: Starting ID for generated MHIP group_id values
(podcast grouping assigns unique IDs to group headers
and child MHIPs).
Returns:
Complete MHYP chunk bytes
"""
if playlist_id is None:
playlist_id = generate_playlist_id()
if timestamp is None:
timestamp = int(time.time())
# Build MHOD for title
mhod_title = write_mhod_string(MHOD_TYPE_TITLE, name)
# Build MHOD for playlist preferences (type 100)
if raw_mhod100 is not None:
mhod_playlist = _write_mhod100_raw(raw_mhod100)
else:
mhod_playlist = write_mhod_playlist_prefs()
# Smart playlist MHODs (type 50 + 51)
mhod_smart = b''
smart_mhod_count = 0
if smart_prefs is not None and smart_rules is not None:
mhod_smart += write_mhod50(smart_prefs)
mhod_smart += write_mhod51(smart_rules)
smart_mhod_count = 2
# Optional MHOD type 102 (playlist settings — opaque iTunes blob)
mhod_settings = b''
settings_count = 0
if raw_mhod102 is not None:
mhod_settings = write_mhod102(raw_mhod102)
settings_count = 1
# Build library index MHODs for master playlist (type 52/53 pairs)
# These are REQUIRED for iPod Classic to build its browsing views
library_indices_data = b''
library_indices_count = 0
if master and tracks:
library_indices_data, library_indices_count = write_library_indices(tracks, capabilities=capabilities)
# Build MHIP entries for each track
mhip_count: int
if podcast_grouping and track_album_map is not None:
# Podcast grouping: group tracks by album (libgpod write_podcast_mhips)
mhip_data, mhip_count = _build_podcast_grouped_mhips(
track_ids, track_album_map, next_mhip_id_start,
)
else:
# Standard flat MHIP list (write_playlist_mhips)
# When item_metadata is provided (round-trip from parsed data), we
# preserve per-MHIP fields: podcastGroupFlag, groupID, podcastGroupRef.
mhips = []
for i, track_id in enumerate(track_ids):
meta = item_metadata[i] if item_metadata and i < len(item_metadata) else None
mhip = write_mhip(
track_id, position=i,
mhip_id=meta.group_id if meta else 0,
podcast_group_flag=meta.podcast_group_flag if meta else 0,
podcast_group_ref=meta.podcast_group_ref if meta else 0,
track_persistent_id=meta.track_persistent_id if meta else 0,
mhip_persistent_id=meta.mhip_persistent_id if meta else 0,
)
mhips.append(mhip)
mhip_data = b''.join(mhips)
mhip_count = len(track_ids)
# Count MHODs (title + playlist prefs + smart + settings + library indices)
mhod_count = 2 + smart_mhod_count + settings_count + library_indices_count
# Total chunk length
total_length = (
MHYP_HEADER_SIZE + len(mhod_title) + len(mhod_playlist) + len(mhod_smart) + len(mhod_settings) + len(library_indices_data) + len(mhip_data)
)
# Build MHYP header
header = bytearray(MHYP_HEADER_SIZE)
write_generic_header(header, 0, b'mhyp', MHYP_HEADER_SIZE, total_length)
# Build values dict for write_fields.
# Timestamps are Unix epoch — write_transform (unix_to_mac) handles conversion.
values: dict[str, int] = {
'mhod_child_count': mhod_count,
'mhip_child_count': mhip_count,
'master_flag': 1 if master else 0,
'timestamp': timestamp,
'playlist_id': playlist_id,
'string_mhod_child_count': 1,
'podcast_flag': podcast_flag,
'sort_order': sortorder,
'timestamp_2': timestamp,
}
# Non-master playlists write db_id_2 and playlist_id at extended offsets.
# For master=True (ds2 master and ds5 built-in categories), these stay
# zeroed — matching libgpod behaviour.
if not master:
values['db_id_2'] = db_id_2
values['playlist_id_2'] = playlist_id
# mhsd5_type — browsing category for dataset 5 smart playlists.
# libgpod writes the same value at +0x50 and +0x52, plus a
# special flag at +0x54 for RINGTONES(6) and MOVIE_RENTALS(7).
if mhsd5_type:
values['mhsd5_type'] = mhsd5_type
values['mhsd5_type_2'] = mhsd5_type
if mhsd5_type in (6, 7):
values['mhsd5_special_flag'] = 0x200
write_fields(header, 0, 'mhyp', values, MHYP_HEADER_SIZE)
return (
bytes(header) + mhod_title + mhod_playlist + mhod_smart + mhod_settings + library_indices_data + mhip_data
)
def _build_podcast_grouped_mhips(
track_ids: List[int],
track_album_map: dict[int, str],
next_id: int,
) -> tuple[bytes, int]:
"""Build podcast-grouped MHIP entries for the type 3 MHSD dataset.
Groups tracks by album name. For each album group, emits:
1. A group header MHIP (``podcast_group_flag=256``, ``track_id=0``,
MHOD type 1 with the album name)
2. One child MHIP per track (``podcast_group_flag=0``,
``group_id_ref`` pointing to the parent group header's ``group_id``,
MHOD type 100 with the child's own unique ``mhip_id`` as position)
This matches libgpod's ``write_podcast_mhips()`` +
``write_one_podcast_group()`` in ``itdb_itunesdb.c``.
Args:
track_ids: Sequential track IDs for the podcast playlist
track_album_map: track_id album name ('' if unknown)
next_id: Starting value for unique MHIP group_id / mhip_id
Returns:
(mhip_bytes, mhip_count) concatenated MHIPs and the total
MHIP count (= number of tracks + number of album groups).
"""
from collections import OrderedDict
# Group tracks by album, preserving insertion order
album_groups: OrderedDict[str, list[int]] = OrderedDict()
for tid in track_ids:
album = track_album_map.get(tid, "")
album_groups.setdefault(album, []).append(tid)
parts: list[bytes] = []
cur_id = next_id
total_mhip_count = 0
for album, tids in album_groups.items():
# Group header MHIP
group_id = cur_id
cur_id += 1
parts.append(write_mhip_podcast_group(album or "Unknown", group_id))
total_mhip_count += 1
# Child MHIPs — one per track in this album group
for tid in tids:
mhip_id = cur_id
cur_id += 1
parts.append(write_mhip(
tid, position=mhip_id,
mhip_id=mhip_id,
podcast_group_flag=0,
podcast_group_ref=group_id,
))
total_mhip_count += 1
return b''.join(parts), total_mhip_count
def write_mhod_playlist_prefs() -> bytes:
"""
Write the playlist preferences MHOD (type 100).
This is a binary blob containing display/sorting preferences.
Based on libgpod's mk_long_mhod_id_playlist().
Total size: 0x288 (648) bytes as written by iTunes.
"""
# libgpod mk_long_mhod_id_playlist() writes exactly 0x288 bytes
# This is critical for proper playlist recognition
total_len = 0x288 # 648 bytes - exactly what libgpod writes
# Build complete MHOD type 100
data = bytearray(total_len)
# Header (24 bytes) — use shared helper, then overlay onto data buffer
hdr = write_mhod_header(100, total_len)
data[:_MHOD_HEADER_SIZE] = hdr
# Body data - based on libgpod mk_long_mhod_id_playlist()
# Offset 0x18 (after header):
struct.pack_into('<I', data, 0x18, 0) # 6 x 0s
struct.pack_into('<I', data, 0x1C, 0)
struct.pack_into('<I', data, 0x20, 0)
struct.pack_into('<I', data, 0x24, 0)
struct.pack_into('<I', data, 0x28, 0)
struct.pack_into('<I', data, 0x2C, 0)
struct.pack_into('<I', data, 0x30, 0x010084) # magic value from libgpod
struct.pack_into('<I', data, 0x34, 0x05) # ?
struct.pack_into('<I', data, 0x38, 0x09) # ?
struct.pack_into('<I', data, 0x3C, 0x03) # ?
struct.pack_into('<I', data, 0x40, 0x120001) # ?
struct.pack_into('<I', data, 0x44, 0) # ?
struct.pack_into('<I', data, 0x48, 0) # ?
struct.pack_into('<I', data, 0x4C, 0x640014) # ?
struct.pack_into('<I', data, 0x50, 0x01) # bool? (visible?)
struct.pack_into('<I', data, 0x54, 0) # 2x0
struct.pack_into('<I', data, 0x58, 0)
struct.pack_into('<I', data, 0x5C, 0x320014) # ?
struct.pack_into('<I', data, 0x60, 0x01) # bool? (visible?)
struct.pack_into('<I', data, 0x64, 0) # 2x0
struct.pack_into('<I', data, 0x68, 0)
struct.pack_into('<I', data, 0x6C, 0x5a0014) # ?
struct.pack_into('<I', data, 0x70, 0x01) # bool? (visible?)
struct.pack_into('<I', data, 0x74, 0) # 2x0
struct.pack_into('<I', data, 0x78, 0)
struct.pack_into('<I', data, 0x7C, 0x500014) # ?
struct.pack_into('<I', data, 0x80, 0x01) # bool? (visible?)
struct.pack_into('<I', data, 0x84, 0) # 2x0
struct.pack_into('<I', data, 0x88, 0)
struct.pack_into('<I', data, 0x8C, 0x7d0015) # ?
struct.pack_into('<I', data, 0x90, 0x01) # bool? (visible?)
# Rest is zeros (padding to 0x288)
return bytes(data)
def _write_mhod100_raw(raw_body: bytes) -> bytes:
"""Write an MHOD type 100 from a raw body blob (round-trip passthrough).
Args:
raw_body: Body bytes (everything after the 24-byte MHOD header).
Returns:
Complete MHOD type 100 chunk.
"""
total_len = _MHOD_HEADER_SIZE + len(raw_body)
return write_mhod_header(100, total_len) + raw_body
def write_playlist(
playlist: "PlaylistInfo",
db_id_2: int = 0,
podcast_grouping: bool = False,
track_album_map: Optional[dict[int, str]] = None,
next_mhip_id_start: int = 1,
) -> bytes:
"""Write a playlist from a PlaylistInfo dataclass.
Handles regular playlists, smart playlists, AND dataset 5 built-in
categories. For dataset 5, PlaylistInfo.master will be True (setting
the type byte at +0x14 to 1) and track_ids will be empty (the iPod
firmware evaluates smart rules at runtime).
The *master playlist* for dataset 2 is NOT written through this
function use write_master_playlist() instead.
Args:
playlist: A PlaylistInfo instance.
db_id_2: Database-wide ID from MHBD offset 0x24.
podcast_grouping: When True and playlist.podcast_flag is set,
generate grouped MHIPs for podcast episodes.
track_album_map: track_id album name (required when
podcast_grouping applies).
next_mhip_id_start: Starting ID for generated MHIP identifiers
during podcast grouping.
Returns:
Complete MHYP chunk bytes.
"""
# Only apply podcast grouping to actual podcast playlists
use_grouping = podcast_grouping and bool(playlist.podcast_flag)
return write_mhyp(
name=playlist.name,
track_ids=playlist.track_ids,
playlist_id=playlist.playlist_id,
master=playlist.master,
sortorder=playlist.sortorder,
podcast_flag=playlist.podcast_flag,
db_id_2=db_id_2,
smart_prefs=playlist.smart_prefs,
smart_rules=playlist.smart_rules,
mhsd5_type=playlist.mhsd5_type,
raw_mhod100=playlist.raw_mhod100,
raw_mhod102=playlist.raw_mhod102,
item_metadata=playlist.item_metadata,
podcast_grouping=use_grouping,
track_album_map=track_album_map,
next_mhip_id_start=next_mhip_id_start,
)
def write_master_playlist(
track_ids: List[int],
db_id_2: int,
name: str = "iPod",
tracks: Optional[List["TrackInfo"]] = None,
capabilities=None,
playlist_id: Optional[int] = None,
) -> bytes:
"""
Write the Master Playlist (MPL).
The master playlist is required and must be the first playlist.
It contains references to ALL tracks in the database.
Args:
track_ids: List of ALL track IDs in the database
name: Playlist name (usually "iPod" or device name)
tracks: List of ALL TrackInfo objects (needed for library indices)
db_id_2: Database-wide ID from MHBD offset 0x24
capabilities: Optional DeviceCapabilities for video sort indices.
Returns:
Complete MHYP chunk for master playlist
"""
# Master playlist MUST have master=True (0x14 field = 1)
# This is how iTunes/iPod identifies the master playlist
return write_mhyp(
name=name,
track_ids=track_ids,
playlist_id=playlist_id,
master=True, # CRITICAL: Master playlist must have type=1
sortorder=5, # Match iTunes default sort order
tracks=tracks,
db_id_2=db_id_2,
capabilities=capabilities,
)
Binary file not shown.
+148
View File
@@ -0,0 +1,148 @@
"""
ipod_device unified iPod device identification & management package.
Re-exports device-identification and model-capability APIs that were
historically spread across multiple legacy modules.
"""
# flake8: noqa: F401
# ── artwork ──────────────────────────────────────────────────────────
from .artwork import (
ARTWORK_FORMATS_BY_ID,
ITHMB_FORMAT_MAP,
ITHMB_SIZE_MAP,
cover_art_format_definitions_for_device,
ithmb_formats_for_device,
photo_formats_for_device,
resolve_cover_art_format_definitions,
resolve_cover_art_format_definitions_for_device,
)
# ── authority ────────────────────────────────────────────────────────
from .authority import (
AUTHORITY_FILENAME,
SOURCE_RANK,
SYSINFO_FIELDS,
cache_sysinfo_extended,
check_authority_coverage,
read_authority,
update_sysinfo,
)
# ── capabilities ─────────────────────────────────────────────────────
from .capabilities import (
ArtworkFormat,
DeviceCapabilities,
capabilities_for_family_gen,
checksum_type_for_family_gen,
cover_art_formats_for_family_gen,
)
from .checksum import (
CHECKSUM_MHBD_SCHEME,
MHBD_SCHEME_TO_CHECKSUM,
ChecksumType,
)
# ── images ───────────────────────────────────────────────────────────
from .images import (
COLOR_MAP,
FAMILY_FALLBACK,
GENERIC_IMAGE,
IMAGE_COLORS,
MODEL_IMAGE,
color_for_image,
image_for_model,
resolve_image_filename,
)
# ── info (device_info) ───────────────────────────────────────────────
from .info import (
DeviceInfo,
clear_current_device,
detect_checksum_type,
enrich,
generate_library_id,
get_current_device,
get_firewire_id,
itdb_write_filename,
read_sysinfo,
resolve_itdb_path,
set_current_device,
)
# ── lookup ───────────────────────────────────────────────────────────
from .lookup import (
extract_model_number,
get_friendly_model_name,
get_model_info,
infer_generation,
lookup_by_serial,
)
# ── models ───────────────────────────────────────────────────────────
from .models import (
IPOD_MODELS,
IPOD_USB_PIDS,
SERIAL_LAST3_TO_MODEL,
USB_PID_TO_MODEL,
)
# ── sysinfo parsing/evidence ─────────────────────────────────────────
from .sysinfo import (
DeviceEvidence,
EvidenceValue,
ParsedSysInfoExtended,
identity_from_sysinfo,
identity_from_sysinfo_extended,
parse_sysinfo_extended,
parse_sysinfo_text,
)
# ── virtual iPods ─────────────────────────────────────────────────────
from .virtual import (
VIRTUAL_IPOD_INFO_FILENAME,
available_virtual_ipod_models,
create_virtual_ipod,
ensure_virtual_itunes_database,
has_virtual_ipod_info,
load_virtual_ipod_info,
virtual_ipod_info_path,
)
# ── checksum ─────────────────────────────────────────────────────────
from .vpd_libusb import (
identify_via_vpd,
)
from .vpd_libusb import (
query_all_ipods as usb_query_all_ipods,
)
# ── vpd_libusb ───────────────────────────────────────────────────────
from .vpd_libusb import (
query_ipod_vpd as usb_query_ipod_vpd,
)
from .vpd_libusb import (
write_sysinfo as usb_write_sysinfo,
)
from .vpd_usb_control import (
query_all_ipod_usb_sysinfo_extended,
query_ipod_usb_sysinfo_extended,
)
try:
from .vpd_linux import query_ipod_vpd_for_path as linux_query_ipod_vpd_for_path
except ImportError:
pass
try:
from .vpd_windows import query_ipod_vpd_for_path as windows_query_ipod_vpd_for_path
except ImportError:
pass
# ── vpd_iokit is macOS-only and raises ImportError on other platforms,
# so we don't import it at package level. Import directly:
# from ipod_device.vpd_iokit import query_ipod_vpd
# ── scanner (GUI/device_scanner) ────────────────────────────────────
from .scanner import identify_ipod_at_path, scan_for_ipods
+184
View File
@@ -0,0 +1,184 @@
"""Artwork lookups backed by the canonical format registry."""
from .artwork_presets import (
ARTWORK_FORMATS_BY_ID,
ArtworkFormat,
)
from .capabilities import capabilities_for_family_gen, cover_art_formats_for_family_gen
ITHMB_FORMAT_MAP = ARTWORK_FORMATS_BY_ID
"""Primary global lookup of ithmb correlation ID -> ``ArtworkFormat``.
Most artwork IDs are globally meaningful. Device-aware code can layer a small
override set on top of this table for the few known conflicts, such as Nano
7G's reinterpretation of ``1013``/``1015``/``1016``.
"""
ITHMB_SIZE_MAP: dict[int, ArtworkFormat] = {}
"""Fallback lookup: byte size -> ``ArtworkFormat``."""
for _af in ITHMB_FORMAT_MAP.values():
_byte_size = _af.row_bytes * _af.height
if _byte_size > 0 and _byte_size not in ITHMB_SIZE_MAP:
ITHMB_SIZE_MAP[_byte_size] = _af
def ithmb_formats_for_device(
family: str,
generation: str,
*,
capacity: str | None = None,
model_number: str | None = None,
) -> dict[int, tuple[int, int]]:
"""Return ``{correlation_id: (width, height)}`` for a device's cover art."""
definitions = cover_art_format_definitions_for_device(
family,
generation,
capacity=capacity,
model_number=model_number,
)
return {fid: (af.width, af.height) for fid, af in definitions.items()}
def _format_dict(formats: tuple[ArtworkFormat, ...]) -> dict[int, ArtworkFormat]:
return {af.format_id: af for af in formats}
def cover_art_format_definitions_for_device(
family: str,
generation: str,
*,
capacity: str | None = None,
model_number: str | None = None,
) -> dict[int, ArtworkFormat]:
"""Return the device's required cover-art definitions.
The normal case is the global registry. Devices with known conflicting IDs
expose a small explicit override set through their capability profile.
"""
caps = capabilities_for_family_gen(
family,
generation or "",
capacity=capacity,
model_number=model_number,
)
if caps is None:
return _format_dict(
cover_art_formats_for_family_gen(
family,
generation,
capacity=capacity,
model_number=model_number,
)
)
if not caps.supports_artwork:
return {}
return _format_dict(caps.cover_art_formats)
def _resolve_observed_format(
format_id: int,
width: int,
height: int,
preferred_defs: dict[int, ArtworkFormat],
) -> ArtworkFormat:
"""Resolve an observed ``id -> dimensions`` using overrides first, then global defaults.
If neither source matches the observed dimensions, fall back to a generic
RGB565 cover-art definition for that observed shape.
"""
for candidate in (
preferred_defs.get(format_id),
ARTWORK_FORMATS_BY_ID.get(format_id),
):
if candidate is None:
continue
if int(candidate.width) == int(width) and int(candidate.height) == int(height):
return candidate
return ArtworkFormat(
int(format_id),
int(width),
int(height),
int(width) * 2,
"RGB565_LE",
"cover",
f"Device artwork format {format_id}",
)
def resolve_cover_art_format_definitions(
family: str = "",
generation: str = "",
*,
capacity: str | None = None,
model_number: str | None = None,
observed_formats: dict[int, tuple[int, int]] | None = None,
) -> dict[int, ArtworkFormat]:
"""Resolve the authoritative cover-art definitions for a device.
``observed_formats`` usually comes from SysInfoExtended or an existing
ArtworkDB. When present, its ID list is authoritative, but each entry still
resolves through device overrides first and the global registry second. Only
unmatched dimensions fall back to a generic inferred definition.
"""
preferred_defs = cover_art_format_definitions_for_device(
family,
generation,
capacity=capacity,
model_number=model_number,
)
if observed_formats:
resolved: dict[int, ArtworkFormat] = {}
for fid, dims in observed_formats.items():
width, height = dims
resolved[int(fid)] = _resolve_observed_format(
int(fid),
int(width),
int(height),
preferred_defs,
)
return resolved
return preferred_defs
def resolve_cover_art_format_definitions_for_device(device) -> dict[int, ArtworkFormat]:
"""Resolve cover-art definitions from a ``DeviceInfo``-like object."""
if device is None:
return {}
return resolve_cover_art_format_definitions(
getattr(device, "model_family", "") or "",
getattr(device, "generation", "") or "",
capacity=getattr(device, "capacity", ""),
model_number=getattr(device, "model_number", ""),
observed_formats=getattr(device, "artwork_formats", None) or None,
)
def photo_formats_for_device(
family: str,
generation: str,
*,
capacity: str | None = None,
model_number: str | None = None,
) -> dict[int, ArtworkFormat]:
"""Return device-specific photo ithmb formats.
This is separate from cover-art formats because iPods keep slide-show/photo
caches in the ``Photos`` hierarchy rather than ``ArtworkDB``. The per-device
formats are sourced from ``DeviceCapabilities.photo_formats``.
"""
caps = capabilities_for_family_gen(
family,
generation or "",
capacity=capacity,
model_number=model_number,
)
formats = caps.photo_formats if caps is not None else ()
if not formats:
return {}
return {af.format_id: af for af in formats}
+119
View File
@@ -0,0 +1,119 @@
"""Canonical ithmb artwork format definitions.
Sources:
- libgpod ``src/itdb_device.c`` fallback artwork tables
- Keith's iPod Photo Reader README (model/prefix cross-checks)
- cyianor/ithmbrdr README (1067 photo payload confirmation)
- local iTunes-authored Nano 7G artwork dump (F1010/F1013/F1015/F1016)
The global registry below is the default source of truth for artwork IDs.
Only a very small number of device families are known to reinterpret IDs,
so those conflicts are modeled as explicit overrides rather than treating
the whole ID space as device-specific.
"""
from dataclasses import dataclass
@dataclass(frozen=True)
class ArtworkFormat:
"""One ithmb artwork format definition."""
format_id: int
width: int
height: int
row_bytes: int
pixel_format: str = "RGB565_LE"
role: str = "cover"
description: str = ""
ARTWORK_FORMATS_BY_ID: dict[int, ArtworkFormat] = {
# iPod Photo / Video era
1005: ArtworkFormat(1005, 80, 80, 160, "RGB565_LE", "photo_thumb", "Nano 7G photo thumbnail"),
1007: ArtworkFormat(1007, 480, 864, 960, "RGB565_LE", "photo_full", "Nano 7G photo full screen"),
1009: ArtworkFormat(1009, 42, 30, 84, "RGB565_LE", "photo_list", "Photo list thumbnail"),
1010: ArtworkFormat(1010, 240, 240, 480, "RGB565_LE", "cover_large", "Nano 7G album art large"),
1013: ArtworkFormat(1013, 220, 176, 440, "RGB565_BE_90", "photo_full", "Photo full screen (rotated)"),
1015: ArtworkFormat(1015, 130, 88, 260, "RGB565_LE", "photo_preview", "Photo/Video preview"),
1016: ArtworkFormat(1016, 140, 140, 280, "RGB565_LE", "cover_large", "Photo album art large"),
1017: ArtworkFormat(1017, 56, 56, 112, "RGB565_LE", "cover_small", "Photo album art small"),
1019: ArtworkFormat(1019, 720, 480, 1440, "UYVY", "tv_out", "Photo/Video NTSC TV output"),
# Compatibility alias preserved from existing Apple databases.
1020: ArtworkFormat(1020, 220, 176, 440, "RGB565_BE_90", "photo_full", "Photo full screen (alt rotated)"),
1023: ArtworkFormat(1023, 176, 132, 352, "RGB565_BE", "photo_full", "Nano full screen"),
1024: ArtworkFormat(1024, 320, 240, 640, "RGB565_LE", "photo_full", "320x240 photo full screen"),
1027: ArtworkFormat(1027, 100, 100, 200, "RGB565_LE", "cover_large", "Nano album art large"),
1028: ArtworkFormat(1028, 100, 100, 200, "RGB565_LE", "cover_small", "Video album art small"),
1029: ArtworkFormat(1029, 200, 200, 400, "RGB565_LE", "cover_large", "Video album art large"),
1031: ArtworkFormat(1031, 42, 42, 84, "RGB565_LE", "cover_small", "Nano album art small"),
1032: ArtworkFormat(1032, 42, 37, 84, "RGB565_LE", "photo_list", "Nano list thumbnail"),
1036: ArtworkFormat(1036, 50, 41, 100, "RGB565_LE", "photo_list", "Video list thumbnail"),
# Classic / later click-wheel iPods
# Compatibility alias preserved from existing Apple databases.
1044: ArtworkFormat(1044, 128, 128, 256, "RGB565_LE", "cover_medium", "Classic album art medium"),
1055: ArtworkFormat(1055, 128, 128, 256, "RGB565_LE", "cover_medium", "Classic album art medium"),
1056: ArtworkFormat(1056, 128, 128, 256, "RGB565_LE", "cover_medium_alt", "128x128 cover art (alternate)"),
1060: ArtworkFormat(1060, 320, 320, 640, "RGB565_LE", "cover_large", "Classic album art large"),
1061: ArtworkFormat(1061, 56, 56, 112, "RGB565_LE", "cover_small", "Classic album art small"),
1066: ArtworkFormat(1066, 64, 64, 128, "RGB565_LE", "photo_thumb", "Classic photo thumbnail"),
1067: ArtworkFormat(1067, 720, 480, 1080, "I420_LE", "tv_out", "Classic TV output (YUV)"),
1068: ArtworkFormat(1068, 128, 128, 256, "RGB565_LE", "cover_medium_alt", "Classic album art medium (alt 2)"),
1071: ArtworkFormat(1071, 240, 240, 480, "RGB565_LE", "cover_large", "Nano 4G album art large"),
1073: ArtworkFormat(1073, 240, 240, 480, "RGB565_LE", "cover_large", "Nano 5G/6G album art large"),
1074: ArtworkFormat(1074, 50, 50, 100, "RGB565_LE", "cover_xsmall", "Nano album art tiny"),
1078: ArtworkFormat(1078, 80, 80, 160, "RGB565_LE", "cover_small", "Nano 4G/5G album art small"),
1079: ArtworkFormat(1079, 80, 80, 160, "RGB565_LE", "photo_thumb", "Nano 4G/5G photo thumbnail"),
1081: ArtworkFormat(1081, 640, 480, 0, "JPEG", "photo_full", "JPEG photo format (experimental/legacy)"),
1083: ArtworkFormat(1083, 240, 320, 480, "RGB565_LE", "photo_full", "Nano 4G photo full screen (portrait)"),
1084: ArtworkFormat(1084, 240, 240, 480, "RGB565_LE", "cover_large_alt", "Nano 4G album art (alt)"),
# Newer iPod-only formats beyond libgpod's older hardcoded tables.
1085: ArtworkFormat(1085, 88, 88, 176, "RGB565_LE", "cover_medium", "Nano 6G album art medium"),
1087: ArtworkFormat(1087, 384, 384, 768, "RGB565_LE", "photo_large", "Nano 5G photo large"),
1089: ArtworkFormat(1089, 58, 58, 116, "RGB565_LE", "cover_small", "Nano 6G album art small"),
1092: ArtworkFormat(1092, 80, 80, 160, "RGB565_LE", "photo_thumb", "Nano 6G photo thumbnail"),
1093: ArtworkFormat(1093, 512, 512, 1024, "RGB565_LE", "photo_full", "Nano 6G photo full screen"),
# Mobile / touch-era formats
2002: ArtworkFormat(2002, 50, 50, 100, "RGB565_BE", "cover_small", "iPod Mobile cover art small"),
2003: ArtworkFormat(2003, 150, 150, 300, "RGB565_BE", "cover_large", "iPod Mobile cover art large"),
3001: ArtworkFormat(3001, 256, 256, 512, "REC_RGB555_LE", "cover_large", "iPod touch cover art large"),
3002: ArtworkFormat(3002, 128, 128, 256, "REC_RGB555_LE", "cover_medium", "iPod touch cover art medium"),
3003: ArtworkFormat(3003, 64, 64, 128, "REC_RGB555_LE", "cover_small", "iPod touch cover art small"),
3005: ArtworkFormat(3005, 320, 320, 640, "RGB555_LE", "cover_xlarge", "iPod touch cover art xlarge"),
}
CLASSIC_COVER_ART_FORMATS = (
ARTWORK_FORMATS_BY_ID[1055],
ARTWORK_FORMATS_BY_ID[1060],
ARTWORK_FORMATS_BY_ID[1061],
ARTWORK_FORMATS_BY_ID[1068],
)
"""Cover-art formats used by click-wheel iPod Classic generations."""
NANO_7G_COVER_ART_OVERRIDES = (
ARTWORK_FORMATS_BY_ID[1010],
ArtworkFormat(1013, 50, 50, 100, "RGB565_LE", "cover_xsmall", "Nano 7G album art tiny"),
ArtworkFormat(1015, 58, 58, 116, "RGB565_LE", "cover_small", "Nano 7G album art small"),
ArtworkFormat(1016, 57, 57, 116, "RGB565_LE", "cover_small_alt", "Nano 7G album art small (aligned)"),
)
"""Known Nano 7G overrides for a few globally-defined artwork IDs."""
# Backward-compatible alias used by capability tables and existing imports.
NANO_7G_COVER_ART_FORMATS = NANO_7G_COVER_ART_OVERRIDES
def artwork_format_candidates() -> tuple[ArtworkFormat, ...]:
"""Return the global registry plus the small set of known override variants."""
candidates = [
*ARTWORK_FORMATS_BY_ID.values(),
*CLASSIC_COVER_ART_FORMATS,
*NANO_7G_COVER_ART_OVERRIDES,
]
unique: dict[tuple[int, int, int, str], ArtworkFormat] = {}
for fmt in candidates:
key = (fmt.format_id, fmt.width, fmt.height, fmt.pixel_format)
unique.setdefault(key, fmt)
return tuple(unique.values())
+671
View File
@@ -0,0 +1,671 @@
"""
iOpenPod SysInfo Authority manages authoritative SysInfo writing.
After device identification is complete, reconciles gathered data with
the existing SysInfo file on the iPod, using per-field provenance tracking
to keep the most reliable value for each field.
The authority file at ``/iPod_Control/Device/iOpenPodSysInfoAuthority``
(JSON) records which data source was used to populate each SysInfo field.
On subsequent runs, if a field's value has changed, the authority
determines whether the new or existing value is more trustworthy.
Source reliability (most least)::
Sure (live hardware):
scsi_vpd / windows_scsi / linux_scsi / usb_vendor > vpd > iokit > ioctl
> device_tree / ioreg / sysfs > wmi
Guesses (files / lookups / derivations):
sysinfo_extended > sysinfo > itunes > serial_lookup
> usb_pid > hashing > unknown
"""
from __future__ import annotations
import hashlib
import json
import logging
import os
from datetime import UTC, datetime
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from .info import DeviceInfo
logger = logging.getLogger(__name__)
# ──────────────────────────────────────────────────────────────────────
# Source reliability ranking — lower index = more reliable
# ──────────────────────────────────────────────────────────────────────
_SOURCE_ORDER: list[str] = [
# ── Sure: live hardware probes ──────────────────────────────────
"scsi_vpd", # Live SCSI INQUIRY VPD plist
"windows_scsi", # Windows SCSI pass-through VPD
"linux_scsi", # Linux SG_IO SCSI pass-through VPD
"usb_vendor", # Live Apple USB vendor-control plist
"vpd", # SCSI Vital Product Data — gold standard
"iokit", # macOS IOKit SCSI (effectively VPD, no unmount)
"ioctl", # Windows direct SCSI inquiry
"device_tree", # Windows PnP device tree (live hardware)
"ioreg", # macOS ioreg (live hardware)
"sysfs", # Linux sysfs (live hardware)
"wmi", # Windows WMI (live hardware query)
# ── Guesses: lookups, derivations, files ────────────────────────
"itunes", # Pre-existing value assumed to be from iTunes
"serial_lookup", # Derived from serial last-3 chars
"usb_pid", # Coarse USB PID mapping
"disk_size", # Live disk-size based capacity estimate
"model_table", # Deterministic inference from known model tuples
"inferred", # Other deterministic inference from known fields
"sysinfo_extended", # SysInfoExtended XML plist — on-disk, stale-prone
"sysinfo", # SysInfo plain text — on-disk, stale-prone
"hashing", # Inferred from hashing scheme
"unknown", # Source not tracked
]
SOURCE_RANK: dict[str, int] = {src: i for i, src in enumerate(_SOURCE_ORDER)}
"""Map source name → rank (lower = more reliable)."""
_WORST_RANK: int = len(_SOURCE_ORDER)
# Anything with rank < _SURE_THRESHOLD is a "sure" (live hardware) source.
# Anything >= is a "guess" (file / lookup / derivation).
_SURE_THRESHOLD: int = SOURCE_RANK["itunes"] # first guess source
# ──────────────────────────────────────────────────────────────────────
# SysInfo key ↔ DeviceInfo field mapping
# ──────────────────────────────────────────────────────────────────────
SYSINFO_FIELDS: list[tuple[str, str]] = [
# ── Core identifiers (written by iTunes / hardware probes) ────────
("pszSerialNumber", "serial"),
("FirewireGuid", "firewire_guid"),
("visibleBuildID", "firmware"),
("BoardHwName", "board"),
("ModelNumStr", "model_number"),
("FamilyID", "family_id"),
("UpdaterFamilyID", "updater_family_id"),
# ── Derived / resolved by iOpenPod for full device granularity ────
# These are deterministically derived from model_number (via
# IPOD_MODELS or serial-last-3 lookup), but caching them in SysInfo
# avoids re-derivation and lets the authority system track provenance.
("ModelFamily", "model_family"),
("Generation", "generation"),
("Capacity", "capacity"),
("Color", "color"),
("USBProductID", "usb_pid"),
]
# Fields whose default DeviceInfo value is a non-empty sentinel that should
# NOT be treated as "already populated" when reading from SysInfo.
# model_family defaults to "iPod" (generic, unresolved).
_SENTINEL_DEFAULTS: dict[str, str] = {
"model_family": "iPod",
}
_DERIVED_SYSINFO_KEYS: frozenset[str] = frozenset({
"ModelFamily",
"Generation",
"Capacity",
"Color",
"USBProductID",
})
# Core identification fields — these drive the "all sure" determination for
# the HIGH authority path. If ALL core fields have sure (live hardware)
# provenance, the expensive hardware and VPD probes are skipped.
#
# Only the essential identification trio is included:
# - Serial number (needed for serial-last-3 exact model resolution)
# - FireWire GUID (needed for database signing)
# - Model number (needed for family/gen/capacity/color derivation)
#
# Other fields (firmware, board) are informational — their absence from
# live hardware probes should NOT force re-probing. Derived fields
# (ModelFamily, Generation, etc.) are excluded because they inherit trust
# from a core field and don't require independent hardware probing.
_CORE_FIELDS: frozenset[str] = frozenset({
"pszSerialNumber",
"FirewireGuid",
"ModelNumStr",
})
AUTHORITY_FILENAME = "iOpenPodSysInfoAuthority"
# ──────────────────────────────────────────────────────────────────────
# Authority coverage check
# ──────────────────────────────────────────────────────────────────────
def check_authority_coverage(
ipod_path: str,
) -> tuple[bool, dict[str, str]]:
"""Check whether the authority file indicates core fields are all tracked.
Returns ``(all_tracked, field_sources)`` where:
* *all_tracked* is ``True`` when every **core** SysInfo field has an
authority entry (i.e., iOpenPod has previously identified this device
and cached the results). The SysInfo is trusted as high-authority
because either (a) it was written by iTunes, or (b) iOpenPod wrote
it after probing hardware. The only things that invalidate trust
are external modification (detected via SHA-256 hashes) or missing
authority file (first run).
* *field_sources* maps DeviceInfo field names to their authority source
strings (e.g. ``{"serial": "vpd", "firewire_guid": "ioctl", ...}``).
If the authority file is missing or empty, returns ``(False, {})`` so
that the caller runs the full probe pipeline.
"""
authority = read_authority(ipod_path)
fields = authority.get("fields", {})
if not fields:
return False, {}
# Tamper detection — if SysInfo/SysInfoExtended were modified externally
# (by iTunes or another tool), we can't trust the cached provenance.
stored_hashes = authority.get("file_hashes", {})
if stored_hashes:
tampered = False
for label, path in [
("SysInfo", _sysinfo_path(ipod_path)),
("SysInfoExtended", _sysinfo_extended_path(ipod_path)),
]:
stored = stored_hashes.get(label)
if stored is not None:
current = _hash_file(path)
if current != stored:
tampered = True
break
if tampered:
logger.info(
"Authority coverage: external modification detected, "
"treating all sources as low-authority",
)
return False, {}
field_sources: dict[str, str] = {}
all_tracked = True
for sysinfo_key, device_field in SYSINFO_FIELDS:
entry = fields.get(sysinfo_key)
if entry is None:
# Field not in authority at all. Only core fields affect
# the "all tracked" flag — missing derived fields are fine
# (they'll be re-derived cheaply from model lookup).
if sysinfo_key in _CORE_FIELDS:
all_tracked = False
continue
source = entry.get("source", "unknown")
field_sources[device_field] = source
return all_tracked, field_sources
# ──────────────────────────────────────────────────────────────────────
# Formatting helpers
# ──────────────────────────────────────────────────────────────────────
def _format_for_sysinfo(sysinfo_key: str, device_value) -> str:
"""Convert a DeviceInfo field value to the SysInfo on-disk format.
Handles both string and non-string field types (e.g. usb_pid is int).
"""
if sysinfo_key == "USBProductID":
# usb_pid is an int on DeviceInfo
if isinstance(device_value, int):
return f"0x{device_value:04X}" if device_value else ""
# String passthrough (shouldn't happen, but safe)
return str(device_value) if device_value else ""
if not device_value:
return ""
device_value = str(device_value)
if sysinfo_key == "FirewireGuid":
clean = device_value
if clean.startswith(("0x", "0X")):
return f"0x{clean[2:].upper()}"
return f"0x{clean.upper()}"
if sysinfo_key == "ModelNumStr":
# SysInfo stores "xA623" — the 'M' prefix becomes 'x'
if device_value.startswith("M"):
return f"x{device_value[1:]}"
return device_value
return device_value
def _normalise_sysinfo_value(sysinfo_key: str, raw_value) -> str:
"""Normalise a raw SysInfo value to a comparable canonical form.
Strips null padding, ``0x`` prefixes, and whitespace so that
comparisons are not tripped up by trivial formatting differences.
"""
val = str(raw_value).strip().rstrip("\x00")
if sysinfo_key == "FirewireGuid":
if val.startswith(("0x", "0X")):
val = val[2:]
return val.upper()
if sysinfo_key == "ModelNumStr":
if val.startswith("x"):
val = "M" + val[1:]
return val.upper().rstrip("\x00")
if sysinfo_key == "USBProductID":
if val.upper().startswith("0X"):
val = val[2:]
return val.upper().lstrip("0") or "0"
return val
# ──────────────────────────────────────────────────────────────────────
# File I/O
# ──────────────────────────────────────────────────────────────────────
def _authority_path(ipod_path: str) -> str:
return os.path.join(
ipod_path, "iPod_Control", "Device", AUTHORITY_FILENAME,
)
def _sysinfo_path(ipod_path: str) -> str:
return os.path.join(ipod_path, "iPod_Control", "Device", "SysInfo")
def _sysinfo_extended_path(ipod_path: str) -> str:
return os.path.join(ipod_path, "iPod_Control", "Device", "SysInfoExtended")
def _hash_file(path: str) -> str | None:
"""Return hex SHA-256 of a file, or ``None`` if the file is missing."""
if not os.path.exists(path):
return None
try:
h = hashlib.sha256()
with open(path, "rb") as f:
for chunk in iter(lambda: f.read(8192), b""):
h.update(chunk)
return h.hexdigest()
except Exception as exc:
logger.warning("Failed to hash %s: %s", path, exc)
return None
def read_authority(ipod_path: str) -> dict:
"""Read the authority file. Returns ``{}`` if missing or corrupt."""
path = _authority_path(ipod_path)
if not os.path.exists(path):
return {}
try:
with open(path, encoding="utf-8") as f:
data = json.load(f)
if isinstance(data, dict):
return data
except Exception as exc:
logger.warning("Authority file read failed: %s", exc)
return {}
def _write_authority(ipod_path: str, authority: dict) -> None:
path = _authority_path(ipod_path)
device_dir = os.path.dirname(path)
os.makedirs(device_dir, exist_ok=True)
try:
with open(path, "w", encoding="utf-8") as f:
json.dump(authority, f, indent=2, ensure_ascii=False)
logger.debug("Wrote authority file to %s", path)
except Exception as exc:
logger.warning("Failed to write authority file: %s", exc)
def _read_sysinfo_raw(ipod_path: str) -> dict[str, str]:
"""Read all key:value pairs from SysInfo, preserving raw values."""
path = _sysinfo_path(ipod_path)
result: dict[str, str] = {}
if not os.path.exists(path):
return result
try:
with open(path, errors="replace") as f:
for line in f:
if ":" in line:
key, val = line.split(":", 1)
result[key.strip()] = val.strip()
except Exception as exc:
logger.warning("SysInfo read failed: %s", exc)
return result
def _write_sysinfo_file(ipod_path: str, fields: dict[str, str]) -> None:
"""Write all fields to the SysInfo file."""
path = _sysinfo_path(ipod_path)
device_dir = os.path.dirname(path)
os.makedirs(device_dir, exist_ok=True)
lines = [f"{k}: {v}" for k, v in fields.items() if v]
try:
with open(path, "w", encoding="utf-8") as f:
f.write("\n".join(lines) + "\n")
logger.info("Wrote SysInfo (%d fields) to %s", len(lines), path)
except Exception as exc:
logger.warning("Failed to write SysInfo: %s", exc)
def _normalise_sysinfo_extended(raw_xml: bytes | str) -> bytes:
"""Return canonical SysInfoExtended plist bytes suitable for caching."""
if isinstance(raw_xml, str):
raw = raw_xml.encode("utf-8", errors="replace")
else:
raw = bytes(raw_xml or b"")
if not raw:
return b""
try:
from .sysinfo import parse_sysinfo_extended
parsed = parse_sysinfo_extended(raw)
if parsed.plist and parsed.raw_xml:
return parsed.raw_xml
except Exception:
pass
for marker in (b"<?xml", b"<plist"):
idx = raw.find(marker)
if idx >= 0:
raw = raw[idx:]
break
raw = raw.strip(b"\x00\r\n\t ")
if raw and b"</plist>" not in raw:
raw += b"\n</dict>\n</plist>"
return raw
def cache_sysinfo_extended(
ipod_path: str,
raw_xml: bytes | str,
*,
source: str = "unknown",
metadata: dict | None = None,
) -> bool:
"""Cache a live SysInfoExtended payload and refresh authority hashes."""
if not ipod_path or not raw_xml:
return False
device_dir = os.path.join(ipod_path, "iPod_Control", "Device")
if not os.path.isdir(device_dir):
return False
data = _normalise_sysinfo_extended(raw_xml)
if not data:
return False
path = _sysinfo_extended_path(ipod_path)
try:
with open(path, "wb") as f:
f.write(data)
except Exception as exc:
logger.warning("Failed to cache SysInfoExtended: %s", exc)
return False
authority = read_authority(ipod_path)
now = datetime.now(UTC).isoformat()
files = authority.setdefault("files", {})
files["SysInfoExtended"] = {
"source": source,
"updated": now,
"bytes": len(data),
}
if metadata:
files["SysInfoExtended"]["metadata"] = {
str(k): v
for k, v in metadata.items()
if isinstance(v, (str, int, float, bool)) and v not in ("", None)
}
authority["version"] = 1
authority["last_updated"] = now
_store_file_hashes(ipod_path, authority)
_write_authority(ipod_path, authority)
logger.debug(
"Cached SysInfoExtended (%d bytes, source=%s) to %s",
len(data),
source,
path,
)
return True
# ──────────────────────────────────────────────────────────────────────
# Tamper detection
# ──────────────────────────────────────────────────────────────────────
def _detect_external_modification(
ipod_path: str,
authority: dict,
fields: dict[str, dict],
now: str,
) -> None:
"""Check whether SysInfo or SysInfoExtended were modified externally.
Compares the stored SHA-256 hashes in the authority file against the
current on-disk files. If either file has been changed (e.g. by
iTunes or another tool), **all** authority source levels are reset to
``"sysinfo"`` because we can no longer trust the provenance the
external software may have overwritten fields with different values.
"""
stored_hashes: dict[str, str] = authority.get("file_hashes", {})
if not stored_hashes:
# First run or no hashes recorded — nothing to compare.
return
tampered: list[str] = []
for label, path in [
("SysInfo", _sysinfo_path(ipod_path)),
("SysInfoExtended", _sysinfo_extended_path(ipod_path)),
]:
stored = stored_hashes.get(label)
if stored is None:
# We never recorded a hash for this file — skip.
continue
current = _hash_file(path)
if current is None:
# File was deleted externally — that counts as a change.
tampered.append(f"{label} (deleted)")
elif current != stored:
tampered.append(label)
if not tampered:
return
logger.warning(
"External modification detected in %s — resetting all "
"authority sources to 'sysinfo'",
", ".join(tampered),
)
for _sysinfo_key, entry in fields.items():
if isinstance(entry, dict) and entry.get("source") != "sysinfo":
entry["source"] = "sysinfo"
entry["updated"] = now
def _store_file_hashes(ipod_path: str, authority: dict) -> None:
"""Compute and store SHA-256 hashes of SysInfo and SysInfoExtended."""
hashes: dict[str, str] = {}
for label, path in [
("SysInfo", _sysinfo_path(ipod_path)),
("SysInfoExtended", _sysinfo_extended_path(ipod_path)),
]:
h = _hash_file(path)
if h is not None:
hashes[label] = h
authority["file_hashes"] = hashes
# ──────────────────────────────────────────────────────────────────────
# Authority-aware SysInfo update
# ──────────────────────────────────────────────────────────────────────
def _rank(source: str) -> int:
"""Lower = more reliable. Unknown sources get worst rank."""
return SOURCE_RANK.get(source, _WORST_RANK)
def _default_existing_source(sysinfo_key: str, old_raw: str) -> str:
"""Return provenance for an existing SysInfo value with no authority entry."""
if not old_raw:
return "unknown"
if sysinfo_key in _DERIVED_SYSINFO_KEYS:
return "sysinfo"
return "itunes"
def update_sysinfo(info: DeviceInfo) -> None:
"""Reconcile gathered DeviceInfo with the on-disk SysInfo.
Called **after** all identification and enrichment is complete.
For each SysInfo-mappable field:
* Missing from SysInfo add it.
* Same value refresh authority timestamp (upgrade source if better).
* Different value keep the one from the more reliable source.
Also writes/updates the ``iOpenPodSysInfoAuthority`` JSON alongside
SysInfo so future runs can make informed decisions.
"""
if not info.path:
return
ipod_path = info.path
device_dir = os.path.join(ipod_path, "iPod_Control", "Device")
if not os.path.isdir(device_dir):
return
existing_sysinfo = _read_sysinfo_raw(ipod_path)
authority = read_authority(ipod_path)
fields = authority.get("fields", {})
now = datetime.now(UTC).isoformat()
# ── Tamper detection: hash SysInfo / SysInfoExtended ──────────
_detect_external_modification(ipod_path, authority, fields, now)
# Start with all existing SysInfo fields so we preserve any we don't map
updated_sysinfo: dict[str, str] = dict(existing_sysinfo)
sysinfo_changed = False
for sysinfo_key, device_field in SYSINFO_FIELDS:
new_value = getattr(info, device_field, "")
# Skip empty / default-sentinel values — these haven't been
# resolved to anything useful yet.
sentinel = _SENTINEL_DEFAULTS.get(device_field)
if sentinel is not None and new_value == sentinel:
continue
if not new_value:
continue
new_source: str = info._field_sources.get(device_field, "unknown")
new_formatted = _format_for_sysinfo(sysinfo_key, new_value)
old_raw = existing_sysinfo.get(sysinfo_key, "")
old_source: str = fields.get(sysinfo_key, {}).get(
"source", _default_existing_source(sysinfo_key, old_raw),
)
# ── Field missing from SysInfo → add it ──────────────────────
if not old_raw:
updated_sysinfo[sysinfo_key] = new_formatted
fields[sysinfo_key] = {
"value": new_formatted,
"source": new_source,
"updated": now,
}
sysinfo_changed = True
logger.debug(
"SysInfo: adding %s = %s (source: %s)",
sysinfo_key, new_formatted, new_source,
)
continue
# ── Compare normalised values ─────────────────────────────────
old_normalised = _normalise_sysinfo_value(sysinfo_key, old_raw)
new_normalised = _normalise_sysinfo_value(sysinfo_key, new_formatted)
if old_normalised == new_normalised:
# Same effective value — upgrade authority source if we're
# more (or equally) reliable, otherwise ensure the existing
# best source is still recorded (so authority coverage check
# sees the field as tracked).
best_source = (
new_source
if _rank(new_source) <= _rank(old_source)
else old_source
)
if sysinfo_key not in fields or _rank(best_source) <= _rank(
fields[sysinfo_key].get("source", "unknown"),
):
fields[sysinfo_key] = {
"value": old_raw, # keep existing formatting
"source": best_source,
"updated": now,
}
continue
# iOpenPod-derived fields are cache material, not immutable iTunes
# facts. If our current resolver produces a different derived label
# after a table/provenance fix, let it refresh stale authority entries
# even when the old cached source had a better historical rank.
if sysinfo_key in _DERIVED_SYSINFO_KEYS:
updated_sysinfo[sysinfo_key] = new_formatted
fields[sysinfo_key] = {
"value": new_formatted,
"source": new_source,
"updated": now,
}
sysinfo_changed = True
logger.debug(
"SysInfo: refreshing derived %s: %r%r (source: %s)",
sysinfo_key,
old_raw,
new_formatted,
new_source,
)
continue
# ── Values differ — use the more reliable source ──────────────
if _rank(new_source) <= _rank(old_source):
# New source is at least as reliable → overwrite
updated_sysinfo[sysinfo_key] = new_formatted
fields[sysinfo_key] = {
"value": new_formatted,
"source": new_source,
"updated": now,
}
sysinfo_changed = True
logger.debug(
"SysInfo: updating %s: %r%r (source %s [rank %d] "
"beats %s [rank %d])",
sysinfo_key, old_raw, new_formatted,
new_source, _rank(new_source),
old_source, _rank(old_source),
)
else:
# Existing value from a more reliable source → keep it
logger.debug(
"SysInfo: keeping %s = %r (source %s [rank %d] beats "
"new %s [rank %d] with %r)",
sysinfo_key, old_raw,
old_source, _rank(old_source),
new_source, _rank(new_source), new_formatted,
)
# ── Persist ───────────────────────────────────────────────────────
if sysinfo_changed:
_write_sysinfo_file(ipod_path, updated_sysinfo)
# Always ensure the authority dict is well-formed before writing.
authority["version"] = 1
authority["fields"] = fields
authority["last_updated"] = now
# Always refresh file hashes so the next run can detect tampering.
_store_file_hashes(ipod_path, authority)
_write_authority(ipod_path, authority)
+641
View File
@@ -0,0 +1,641 @@
"""Device capabilities — per-generation feature map backed by canonical artwork formats.
Sources:
- libgpod ``itdb_device.c`` itdb_device_supports_*() functions,
ipod_info_table, artwork format tables
- libgpod ``itdb_itunesdb.c`` iTunesSD writer, mhbd version handling
- Empirical: iPod Classic 2G, Nano 3G confirmed
This table captures every capability dimension that affects database
writing, artwork generation, or sync behaviour. It is the single
authority for "what does this device support?" questions.
"""
from dataclasses import dataclass
from .artwork_presets import (
ARTWORK_FORMATS_BY_ID,
CLASSIC_COVER_ART_FORMATS,
NANO_7G_COVER_ART_FORMATS,
ArtworkFormat,
)
from .checksum import ChecksumType
@dataclass(frozen=True)
class DeviceCapabilities:
"""Per-generation device capability flags.
Every (family, generation) pair maps to exactly one of these. The
flags drive decisions in the sync engine, iTunesDB writer, and
ArtworkDB writer.
All flags default to the *most common* value so that only deviations
need to be specified in the lookup table.
"""
# ── Database format ────────────────────────────────────────────────
checksum: ChecksumType = ChecksumType.NONE
is_shuffle: bool = False
"""If True, device uses iTunesSD (flat binary) instead of / in addition
to iTunesDB. Shadow DB version determines the iTunesSD format."""
shadow_db_version: int = 0
"""0 = not a shuffle. 1 = iTunesSD v1 (Shuffle 1G/2G, 18-byte header,
558-byte entries, big-endian). 2 = iTunesSD v2 (Shuffle 3G/4G,
bdhs/hths/hphs chunk format, little-endian)."""
supports_compressed_db: bool = False
"""If True, device expects iTunesCDB (zlib-compressed iTunesDB) and will
generate an empty iTunesDB alongside it. Nano 5G/6G/7G only."""
# ── Media type support ─────────────────────────────────────────────
supports_video: bool = False
"""Device can play video files (mediatype & VIDEO != 0)."""
supports_podcast: bool = True
"""Device supports podcast mhsd types (type 3). False only for
very early iPods (1G3G) and iPod Mobile."""
supports_gapless: bool = False
"""Device honours gapless playback fields (pregap, postgap,
samplecount, gapless_data, gapless_track_flag). Introduced with
iPod Video 5.5G (Late 2006)."""
# ── Artwork ────────────────────────────────────────────────────────
supports_artwork: bool = True
"""Device has an ArtworkDB and .ithmb files for album art."""
supports_photo: bool = False
"""Device has additional photo artwork formats (for photo viewer)."""
photo_formats: tuple[ArtworkFormat, ...] = ()
"""Photo/slideshow ithmb formats used by the Photos database pipeline."""
supports_chapter_image: bool = False
"""Device has chapter image artwork formats (for enhanced podcasts)."""
supports_sparse_artwork: bool = False
"""Artwork can be written in sparse mode (Nano 3G+, Classic, Touch)."""
supports_alac: bool = True
"""Device supports Apple Lossless (ALAC) audio playback.
False for iPod 1G3G and Mini 1G (pre-firmware-update era hardware that
received ALAC support only from 4th Gen / Photo / Mini 2G onwards)."""
cover_art_formats: tuple[ArtworkFormat, ...] = ()
"""Supported cover-art thumbnail sizes. Empty means no artwork."""
# ── Storage layout ─────────────────────────────────────────────────
music_dirs: int = 20
"""Number of ``Fxx`` directories under ``iPod_Control/Music/``.
Varies 050 depending on model and storage capacity."""
# ── SQLite database ────────────────────────────────────────────────
uses_sqlite_db: bool = False
"""If True, device uses SQLite databases in
``iTunes Library.itlp/`` instead of (or alongside) binary
iTunesDB/iTunesCDB. The firmware on Nano 6G/7G reads the SQLite
databases and ignores iTunesCDB completely."""
# ── Writer parameters ──────────────────────────────────────────────
db_version: int = 0x30
"""iTunesDB version to write in mhbd header. Older iPods need
lower values (0x0c for Shuffle 1G/2G, 0x13 for pre-Classic)."""
byte_order: str = "le"
"""Byte order for database writing. ``"le"`` for almost all models.
``"be"`` for iPod Mobile (Motorola ROKR/SLVR/RAZR)."""
# ── Screen / display ───────────────────────────────────────────────
has_screen: bool = True
"""Device has a display. Shuffles have no screen."""
# ── Video encoding limits ──────────────────────────────────────────
max_video_width: int = 0
"""Maximum H.264 decode width (pixels). 0 = no video support.
This is the firmware decode ceiling, not the screen resolution
the device downscales to fit its screen."""
max_video_height: int = 0
"""Maximum H.264 decode height (pixels). 0 = no video support."""
max_video_fps: int = 30
"""Maximum frame rate for H.264 decode (fps). All video-capable iPods
support 30 fps; PAL-resolution Nano 7G content is typically 25 fps but
30 fps playback is still supported."""
max_video_bitrate: int = 0
"""Hard bitrate ceiling for H.264 decode (kbps). 0 = no explicit cap
(quality-controlled by CRF only). Non-zero values enforce a -maxrate
flag in ffmpeg.
Nano 3G/4G use Baseline Profile Level 1.3, capped at 768 kbps by spec."""
h264_level: str = "3.0"
"""H.264 Baseline Profile level to target when encoding video.
Most iPods support Level 3.0. iPod Classic supports 3.1.
Nano 3G/4G are limited to Level 1.3 by their hardware decoder."""
# ──────────────────────────────────────────────────────────────────────────
# The master capabilities table
# ──────────────────────────────────────────────────────────────────────────
_FAMILY_GEN_CAPABILITIES: dict[tuple[str, str], DeviceCapabilities] = {
# ── iPod 1G3G: earliest models, no podcast, no gapless ───────────
("iPod", "1st Gen"): DeviceCapabilities(
supports_podcast=False,
supports_artwork=False,
supports_alac=True,
has_screen=True,
music_dirs=20,
db_version=0x13,
),
("iPod", "2nd Gen"): DeviceCapabilities(
supports_podcast=False,
supports_artwork=False,
supports_alac=True,
has_screen=True,
music_dirs=20,
db_version=0x13,
),
("iPod", "3rd Gen"): DeviceCapabilities(
supports_podcast=False,
supports_artwork=False,
supports_alac=True,
has_screen=True,
music_dirs=20,
db_version=0x13,
),
# ── iPod 4G (Click Wheel): first with podcast support ─────────────
("iPod", "4th Gen"): DeviceCapabilities(
supports_artwork=False,
music_dirs=20,
db_version=0x13,
),
# ── iPod U2 Special Edition (4th Gen hardware) ────────────────────
("iPod U2", "4th Gen"): DeviceCapabilities(
supports_artwork=False,
music_dirs=20,
db_version=0x13,
),
# ── iPod Photo (Color Display) ────────────────────────────────────
("iPod Photo", "4th Gen"): DeviceCapabilities(
supports_artwork=True,
supports_photo=True,
photo_formats=(
ARTWORK_FORMATS_BY_ID[1009],
ARTWORK_FORMATS_BY_ID[1013],
ARTWORK_FORMATS_BY_ID[1015],
ARTWORK_FORMATS_BY_ID[1019],
),
cover_art_formats=(
ARTWORK_FORMATS_BY_ID[1017],
ARTWORK_FORMATS_BY_ID[1016],
),
music_dirs=20,
db_version=0x13,
),
# ── iPod Video 5th Gen ────────────────────────────────────────────
("iPod Video", "5th Gen"): DeviceCapabilities(
supports_video=True,
supports_artwork=True,
supports_photo=True,
photo_formats=(
ARTWORK_FORMATS_BY_ID[1036],
ARTWORK_FORMATS_BY_ID[1024],
ARTWORK_FORMATS_BY_ID[1015],
ARTWORK_FORMATS_BY_ID[1019],
),
cover_art_formats=(
ARTWORK_FORMATS_BY_ID[1028],
ARTWORK_FORMATS_BY_ID[1029],
),
music_dirs=20,
db_version=0x19,
max_video_width=640,
max_video_height=480,
),
# ── iPod Video 5.5th Gen — first with gapless playback ───────────
("iPod Video", "5.5th Gen"): DeviceCapabilities(
supports_video=True,
supports_gapless=True,
supports_artwork=True,
supports_photo=True,
photo_formats=(
ARTWORK_FORMATS_BY_ID[1036],
ARTWORK_FORMATS_BY_ID[1024],
ARTWORK_FORMATS_BY_ID[1015],
ARTWORK_FORMATS_BY_ID[1019],
),
cover_art_formats=(
ARTWORK_FORMATS_BY_ID[1028],
ARTWORK_FORMATS_BY_ID[1029],
),
music_dirs=20,
db_version=0x19,
max_video_width=640,
max_video_height=480,
),
# ── iPod Video U2 editions ────────────────────────────────────────
("iPod Video U2", "5th Gen"): DeviceCapabilities(
supports_video=True,
supports_artwork=True,
supports_photo=True,
photo_formats=(
ARTWORK_FORMATS_BY_ID[1036],
ARTWORK_FORMATS_BY_ID[1024],
ARTWORK_FORMATS_BY_ID[1015],
ARTWORK_FORMATS_BY_ID[1019],
),
cover_art_formats=(
ARTWORK_FORMATS_BY_ID[1028],
ARTWORK_FORMATS_BY_ID[1029],
),
music_dirs=20,
db_version=0x19,
max_video_width=640,
max_video_height=480,
),
("iPod Video U2", "5.5th Gen"): DeviceCapabilities(
supports_video=True,
supports_gapless=True,
supports_artwork=True,
supports_photo=True,
photo_formats=(
ARTWORK_FORMATS_BY_ID[1036],
ARTWORK_FORMATS_BY_ID[1024],
ARTWORK_FORMATS_BY_ID[1015],
ARTWORK_FORMATS_BY_ID[1019],
),
cover_art_formats=(
ARTWORK_FORMATS_BY_ID[1028],
ARTWORK_FORMATS_BY_ID[1029],
),
music_dirs=20,
db_version=0x19,
max_video_width=640,
max_video_height=480,
),
# ── iPod Classic (all gens): HASH58, gapless, video ───────────────
("iPod Classic", "1st Gen"): DeviceCapabilities(
checksum=ChecksumType.HASH58,
supports_video=True,
supports_gapless=True,
supports_artwork=True,
supports_photo=True,
photo_formats=(
ARTWORK_FORMATS_BY_ID[1067],
ARTWORK_FORMATS_BY_ID[1024],
ARTWORK_FORMATS_BY_ID[1066],
),
supports_chapter_image=True,
supports_sparse_artwork=True,
cover_art_formats=CLASSIC_COVER_ART_FORMATS,
music_dirs=50,
db_version=0x30,
max_video_width=640,
max_video_height=480,
),
("iPod Classic", "2nd Gen"): DeviceCapabilities(
checksum=ChecksumType.HASH58,
supports_video=True,
supports_gapless=True,
supports_artwork=True,
supports_photo=True,
photo_formats=(
ARTWORK_FORMATS_BY_ID[1067],
ARTWORK_FORMATS_BY_ID[1024],
ARTWORK_FORMATS_BY_ID[1066],
),
supports_chapter_image=True,
supports_sparse_artwork=True,
cover_art_formats=CLASSIC_COVER_ART_FORMATS,
music_dirs=50,
db_version=0x30,
max_video_width=640,
max_video_height=480,
),
("iPod Classic", "3rd Gen"): DeviceCapabilities(
checksum=ChecksumType.HASH58,
supports_video=True,
supports_gapless=True,
supports_artwork=True,
supports_photo=True,
photo_formats=(
ARTWORK_FORMATS_BY_ID[1067],
ARTWORK_FORMATS_BY_ID[1024],
ARTWORK_FORMATS_BY_ID[1066],
),
supports_chapter_image=True,
supports_sparse_artwork=True,
cover_art_formats=CLASSIC_COVER_ART_FORMATS,
music_dirs=50,
db_version=0x30,
max_video_width=640,
max_video_height=480,
),
# ── iPod Mini ─────────────────────────────────────────────────────
("iPod Mini", "1st Gen"): DeviceCapabilities(
supports_artwork=False,
supports_alac=True,
music_dirs=6,
db_version=0x13,
),
("iPod Mini", "2nd Gen"): DeviceCapabilities(
supports_artwork=False,
music_dirs=6,
db_version=0x13,
),
# ── iPod Nano 1G/2G ──────────────────────────────────────────────
("iPod Nano", "1st Gen"): DeviceCapabilities(
supports_artwork=True,
supports_photo=True,
photo_formats=(
ARTWORK_FORMATS_BY_ID[1032],
ARTWORK_FORMATS_BY_ID[1023],
),
cover_art_formats=(
ARTWORK_FORMATS_BY_ID[1031],
ARTWORK_FORMATS_BY_ID[1027],
),
music_dirs=14,
db_version=0x13,
),
("iPod Nano", "2nd Gen"): DeviceCapabilities(
supports_artwork=True,
supports_photo=True,
photo_formats=(
ARTWORK_FORMATS_BY_ID[1032],
ARTWORK_FORMATS_BY_ID[1023],
),
cover_art_formats=(
ARTWORK_FORMATS_BY_ID[1031],
ARTWORK_FORMATS_BY_ID[1027],
),
music_dirs=14,
db_version=0x13,
),
# ── iPod Nano 3G ("Fat"): first Nano with video, HASH58 ──────────
("iPod Nano", "3rd Gen"): DeviceCapabilities(
checksum=ChecksumType.HASH58,
supports_video=True,
supports_gapless=True,
supports_artwork=True,
supports_photo=True,
photo_formats=(
ARTWORK_FORMATS_BY_ID[1067],
ARTWORK_FORMATS_BY_ID[1024],
ARTWORK_FORMATS_BY_ID[1066],
),
supports_sparse_artwork=True,
cover_art_formats=(
ARTWORK_FORMATS_BY_ID[1061],
ARTWORK_FORMATS_BY_ID[1055],
ARTWORK_FORMATS_BY_ID[1068],
ARTWORK_FORMATS_BY_ID[1060],
),
music_dirs=20,
db_version=0x30,
max_video_width=320,
max_video_height=240,
max_video_bitrate=768,
h264_level="1.3",
),
# ── iPod Nano 4G: HASH58 ─────────────────────────────────────────
("iPod Nano", "4th Gen"): DeviceCapabilities(
checksum=ChecksumType.HASH58,
supports_video=True,
supports_gapless=True,
supports_artwork=True,
supports_photo=True,
photo_formats=(
ARTWORK_FORMATS_BY_ID[1024],
ARTWORK_FORMATS_BY_ID[1066],
ARTWORK_FORMATS_BY_ID[1079],
ARTWORK_FORMATS_BY_ID[1083],
),
supports_chapter_image=True,
supports_sparse_artwork=True,
cover_art_formats=(
ARTWORK_FORMATS_BY_ID[1055],
ARTWORK_FORMATS_BY_ID[1068],
ARTWORK_FORMATS_BY_ID[1071],
ARTWORK_FORMATS_BY_ID[1074],
ARTWORK_FORMATS_BY_ID[1078],
ARTWORK_FORMATS_BY_ID[1084],
),
music_dirs=20,
db_version=0x30,
max_video_width=480,
max_video_height=320,
max_video_bitrate=768,
h264_level="1.3",
),
# ── iPod Nano 5G: HASH72, compressed DB + SQLite ─────────────────
("iPod Nano", "5th Gen"): DeviceCapabilities(
checksum=ChecksumType.HASH72,
supports_video=True,
supports_gapless=True,
supports_artwork=True,
supports_photo=True,
photo_formats=(
ARTWORK_FORMATS_BY_ID[1087],
ARTWORK_FORMATS_BY_ID[1079],
ARTWORK_FORMATS_BY_ID[1066],
),
supports_sparse_artwork=True,
supports_compressed_db=True,
uses_sqlite_db=True,
cover_art_formats=(
ARTWORK_FORMATS_BY_ID[1056],
ARTWORK_FORMATS_BY_ID[1078],
ARTWORK_FORMATS_BY_ID[1073],
ARTWORK_FORMATS_BY_ID[1074],
),
music_dirs=14,
db_version=0x30,
max_video_width=640,
max_video_height=480,
),
# ── iPod Nano 6G: HASHAB, no video ───────────────────────────────
("iPod Nano", "6th Gen"): DeviceCapabilities(
checksum=ChecksumType.HASHAB,
supports_video=False,
supports_gapless=True,
supports_artwork=True,
supports_photo=True,
photo_formats=(
ARTWORK_FORMATS_BY_ID[1092],
ARTWORK_FORMATS_BY_ID[1093],
),
supports_sparse_artwork=True,
supports_compressed_db=True,
uses_sqlite_db=True,
cover_art_formats=(
ARTWORK_FORMATS_BY_ID[1073],
ARTWORK_FORMATS_BY_ID[1085],
ARTWORK_FORMATS_BY_ID[1089],
ARTWORK_FORMATS_BY_ID[1074],
),
music_dirs=20,
db_version=0x30,
),
# ── iPod Nano 7G: HASHAB, video returns ──────────────────────────
("iPod Nano", "7th Gen"): DeviceCapabilities(
checksum=ChecksumType.HASHAB,
supports_video=True,
supports_gapless=True,
supports_artwork=True,
supports_photo=True,
photo_formats=(
ARTWORK_FORMATS_BY_ID[1007],
ARTWORK_FORMATS_BY_ID[1005],
),
supports_sparse_artwork=True,
supports_compressed_db=True,
uses_sqlite_db=True,
cover_art_formats=NANO_7G_COVER_ART_FORMATS,
music_dirs=20,
db_version=0x30,
max_video_width=720,
max_video_height=576,
),
# ── iPod Shuffle 1G ──────────────────────────────────────────────
("iPod Shuffle", "1st Gen"): DeviceCapabilities(
is_shuffle=True,
shadow_db_version=1,
supports_podcast=True,
supports_artwork=False,
has_screen=False,
music_dirs=3,
db_version=0x0c,
),
# ── iPod Shuffle 2G ──────────────────────────────────────────────
("iPod Shuffle", "2nd Gen"): DeviceCapabilities(
is_shuffle=True,
shadow_db_version=1,
supports_podcast=True,
supports_artwork=False,
has_screen=False,
music_dirs=3,
db_version=0x13,
),
# ── iPod Shuffle 3G ──────────────────────────────────────────────
("iPod Shuffle", "3rd Gen"): DeviceCapabilities(
is_shuffle=True,
shadow_db_version=2,
supports_podcast=True,
supports_artwork=False,
has_screen=False,
music_dirs=3,
db_version=0x19,
),
# ── iPod Shuffle 4G ──────────────────────────────────────────────
("iPod Shuffle", "4th Gen"): DeviceCapabilities(
is_shuffle=True,
shadow_db_version=2,
supports_podcast=True,
supports_artwork=False,
has_screen=False,
music_dirs=3,
db_version=0x19,
),
}
def capabilities_for_family_gen(
family: str,
generation: str,
*,
capacity: str | None = None,
model_number: str | None = None,
) -> DeviceCapabilities | None:
"""Return the device capabilities for a (family, generation) pair.
If the exact pair is not found but *generation* is empty/unknown,
checks whether all known generations of *family* share identical
capabilities and returns those.
Returns ``None`` if the pair is not in the lookup table and the
family-level fallback is ambiguous.
"""
caps = _FAMILY_GEN_CAPABILITIES.get((family, generation))
if caps is not None:
return caps
if family and not generation:
family_caps = [
c for (f, _g), c in _FAMILY_GEN_CAPABILITIES.items()
if f == family
]
if family_caps and all(c == family_caps[0] for c in family_caps):
return family_caps[0]
return None
def cover_art_formats_for_family_gen(
family: str,
generation: str,
*,
capacity: str | None = None,
model_number: str | None = None,
) -> tuple[ArtworkFormat, ...]:
"""Return cover-art formats for a family/generation pair.
This is intentionally narrower than ``capabilities_for_family_gen``. Some
families have generations with different playback capabilities but the same
ArtworkDB cover formats, so a full capability fallback would be ambiguous
while artwork generation is still safe.
"""
_ = capacity, model_number
caps = _FAMILY_GEN_CAPABILITIES.get((family, generation))
if caps is not None:
return caps.cover_art_formats if caps.supports_artwork else ()
if family and not generation:
family_formats = [
c.cover_art_formats if c.supports_artwork else ()
for (f, _g), c in _FAMILY_GEN_CAPABILITIES.items()
if f == family
]
if family_formats and all(formats == family_formats[0] for formats in family_formats):
return family_formats[0]
return ()
def checksum_type_for_family_gen(
family: str,
generation: str,
) -> ChecksumType | None:
"""Return the checksum type for a (family, generation) pair.
Derives the answer from ``_FAMILY_GEN_CAPABILITIES``. If the exact
(family, generation) pair is not found but *generation* is empty/unknown,
checks whether all known generations of *family* share the same checksum
type and returns it.
Returns ``None`` if the pair is not in the lookup table and the family-
level fallback is ambiguous.
"""
caps = _FAMILY_GEN_CAPABILITIES.get((family, generation))
if caps is not None:
return caps.checksum
if family and not generation:
family_checksums = {
c.checksum
for (f, _g), c in _FAMILY_GEN_CAPABILITIES.items()
if f == family
}
if len(family_checksums) == 1:
return family_checksums.pop()
return None
+41
View File
@@ -0,0 +1,41 @@
"""Checksum type enumeration and MHBD hashing-scheme mappings."""
from enum import IntEnum
class ChecksumType(IntEnum):
"""Checksum types for different iPod generations.
NONE Pre-2007 iPods (1G5G, Photo, Video, Mini, Nano 1G2G, Shuffle)
HASH58 iPod Classic (all gens), Nano 3G, Nano 4G
HASH72 Nano 5G
HASHAB Nano 6G, Nano 7G (white-box AES, via WASM module)
UNSUPPORTED Reserved for any future unsupported scheme
UNKNOWN Device not yet identified
"""
NONE = 0
HASH58 = 1
HASH72 = 2
HASHAB = 3
UNSUPPORTED = 98
UNKNOWN = 99
# ── MHBD hashing scheme ↔ ChecksumType mapping ──────────────────────────
#
# The mhbd header at offset 0x30 stores a 16-bit ``hashing_scheme`` value.
# These constants map between our ``ChecksumType`` enum and the raw wire
# values. Note: HASHAB is enum 3 but wire 4.
CHECKSUM_MHBD_SCHEME: dict[ChecksumType, int] = {
ChecksumType.NONE: 0,
ChecksumType.HASH58: 1,
ChecksumType.HASH72: 2,
ChecksumType.HASHAB: 4,
}
"""Map ``ChecksumType`` → raw ``hashing_scheme`` field in mhbd header."""
MHBD_SCHEME_TO_CHECKSUM: dict[int, ChecksumType] = {
v: k for k, v in CHECKSUM_MHBD_SCHEME.items()
}
"""Map raw ``hashing_scheme`` field in mhbd header → ``ChecksumType``."""
+177
View File
@@ -0,0 +1,177 @@
"""Compact log formatting helpers for device-identification diagnostics."""
from __future__ import annotations
from collections.abc import Iterable, Mapping
from typing import Any
FieldSpec = tuple[str, str]
IDENTITY_FIELDS: tuple[FieldSpec, ...] = (
("model_number", "model"),
("model_family", "family"),
("generation", "gen"),
("capacity", "capacity"),
("color", "color"),
("serial", "serial"),
("firewire_guid", "fwguid"),
("firmware", "fw"),
("usb_vid", "vid"),
("usb_pid", "pid"),
("usb_serial", "usb_serial"),
("scsi_vendor", "scsi_vendor"),
("scsi_product", "scsi_product"),
("scsi_revision", "scsi_rev"),
)
CAPABILITY_FIELDS: tuple[FieldSpec, ...] = (
("family_id", "family_id"),
("updater_family_id", "updater_id"),
("product_type", "product"),
("db_version", "db_version"),
("shadow_db_version", "shadow_db"),
("uses_sqlite_db", "sqlite"),
("supports_sparse_artwork", "sparse_art"),
("max_tracks", "max_tracks"),
("max_file_size_gb", "max_file_gb"),
("max_transfer_speed", "max_transfer"),
("podcasts_supported", "podcasts"),
("voice_memos_supported", "voice_memos"),
("artwork_formats", "art_ids"),
("photo_formats", "photo_ids"),
("chapter_image_formats", "chapter_ids"),
)
SOURCE_FIELDS: tuple[FieldSpec, ...] = (
("serial", "serial"),
("firewire_guid", "fwguid"),
("model_number", "model"),
("model_family", "family"),
("generation", "gen"),
("capacity", "capacity"),
("color", "color"),
("usb_pid", "pid"),
("firmware", "fw"),
)
_HEX_WIDTHS: dict[str, int] = {
"usb_vid": 4,
"usb_pid": 4,
"db_version": 0,
"shadow_db_version": 0,
}
def is_missing(value: Any) -> bool:
return value is None or value == "" or value == b"" or value == {} or value == []
def compact(value: Any, *, max_chars: int = 96) -> str:
text = str(value)
if len(text) <= max_chars:
return text
head = max_chars // 2 - 2
tail = max_chars - head - 3
return f"{text[:head]}...{text[-tail:]}"
def format_value(field: str, value: Any) -> str:
if isinstance(value, bytes):
return f"<{len(value)} bytes>"
if isinstance(value, bool):
return "yes" if value else "no"
if field in _HEX_WIDTHS:
try:
number = int(value)
if not number:
return "0"
width = _HEX_WIDTHS[field]
return f"0x{number:0{width}X}" if width else f"0x{number:X}"
except (TypeError, ValueError):
return compact(value)
if isinstance(value, Mapping):
if field.endswith("_formats") or field in {
"artwork_formats",
"photo_formats",
"chapter_image_formats",
}:
ids = ", ".join(str(k) for k in sorted(value)[:12])
suffix = "..." if len(value) > 12 else ""
return f"{len(value)}[{ids}{suffix}]"
keys = ", ".join(str(k) for k in sorted(value, key=str)[:8])
suffix = "..." if len(value) > 8 else ""
return f"{len(value)} keys[{keys}{suffix}]"
if isinstance(value, Iterable) and not isinstance(value, (str, bytes)):
items = list(value)
shown = ", ".join(str(item) for item in items[:12])
suffix = "..." if len(items) > 12 else ""
return f"{len(items)}[{shown}{suffix}]"
return compact(value)
def format_fields(
data: Mapping[str, Any],
fields: Iterable[FieldSpec] = IDENTITY_FIELDS,
*,
include_false: bool = False,
) -> str:
parts: list[str] = []
for field, label in fields:
if field not in data:
continue
value = data.get(field)
if is_missing(value):
continue
if value == 0 and not isinstance(value, bool):
continue
if value is False and not include_false:
continue
parts.append(f"{label}={format_value(field, value)}")
return ", ".join(parts) if parts else "none"
def format_sources(
sources: Mapping[str, Any] | None,
fields: Iterable[FieldSpec] = SOURCE_FIELDS,
) -> str:
if not sources:
return "none"
parts = []
for field, label in fields:
source = sources.get(field)
if source:
parts.append(f"{label}:{source}")
return ", ".join(parts) if parts else "none"
def format_conflicts(conflicts: Any) -> str:
if not conflicts:
return "none"
if not isinstance(conflicts, list):
return compact(conflicts)
parts: list[str] = []
for conflict in conflicts[:6]:
if isinstance(conflict, Mapping):
field = conflict.get("field", "?")
winner = conflict.get("winner", "")
rejected_source = conflict.get("rejected_source", "")
rejected_value = conflict.get("rejected_value", "")
reason = conflict.get("reason", "")
detail = f"{field}"
if winner:
detail += f" winner={winner}"
if rejected_source or rejected_value:
detail += (
f" rejected={rejected_source or '?'}:"
f"{compact(rejected_value, max_chars=36)}"
)
if reason:
detail += f" reason={compact(reason, max_chars=64)}"
parts.append(detail)
else:
parts.append(compact(conflict, max_chars=96))
if len(conflicts) > 6:
parts.append(f"+{len(conflicts) - 6} more")
return "; ".join(parts)
+495
View File
@@ -0,0 +1,495 @@
"""Read-only diagnostic dump for connected iPods.
Usage:
uv run python -m ipod_device.dump D:\
uv run python -m ipod_device.dump --all
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import sys
from pathlib import Path
from typing import Any
IDENTITY_FIELDS: tuple[str, ...] = (
"serial",
"firewire_guid",
"model_number",
"model_family",
"generation",
"capacity",
"color",
"firmware",
"board",
"family_id",
"updater_family_id",
"product_type",
"usb_vid",
"usb_pid",
"usb_serial",
"scsi_vendor",
"scsi_product",
"scsi_revision",
"connected_bus",
"volume_format",
"db_version",
"shadow_db_version",
"uses_sqlite_db",
"supports_sparse_artwork",
"max_tracks",
"max_file_size_gb",
"max_transfer_speed",
"podcasts_supported",
"voice_memos_supported",
)
def _device_dir(mount_path: str) -> Path:
return Path(mount_path) / "iPod_Control" / "Device"
def _mount_name(mount_path: str) -> str:
if sys.platform == "win32":
drive, _tail = os.path.splitdrive(mount_path)
if drive:
return drive
if mount_path and mount_path[0].isalpha():
return f"{mount_path[0].upper()}:"
return os.path.basename(os.path.normpath(mount_path)) or mount_path
def _normalise_mount_path(mount_path: str) -> str:
if sys.platform == "win32":
stripped = mount_path.strip().strip('"')
if len(stripped) == 2 and stripped[0].isalpha() and stripped[1] == ":":
return stripped + "\\"
return stripped
return mount_path
def _sha256(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def _json_safe(value: Any, *, include_raw: bool = False) -> Any:
if isinstance(value, bytes):
result: dict[str, Any] = {
"bytes": len(value),
"sha256": _sha256(value),
}
if include_raw:
result["text"] = value.decode("utf-8", errors="replace")
return result
if isinstance(value, dict):
return {
str(k): _json_safe(v, include_raw=include_raw)
for k, v in sorted(value.items(), key=lambda item: str(item[0]))
}
if isinstance(value, (list, tuple)):
return [_json_safe(v, include_raw=include_raw) for v in value]
return value
def _normalise_for_compare(field: str, value: Any) -> str:
if value in (None, ""):
return ""
if field == "firewire_guid":
try:
from .sysinfo import normalize_guid
return normalize_guid(value)
except Exception:
pass
if field == "usb_pid":
try:
return f"0x{int(value):04X}"
except Exception:
return str(value).strip().upper()
return str(value).strip()
def _sysinfo_dump(mount_path: str) -> dict[str, Any]:
path = _device_dir(mount_path) / "SysInfo"
if not path.exists():
return {"present": False}
raw = path.read_text(errors="replace")
from .sysinfo import identity_from_sysinfo, parse_sysinfo_text
parsed = parse_sysinfo_text(raw)
return {
"present": True,
"path": str(path),
"sha256": _sha256(raw.encode("utf-8", errors="replace")),
"fields": parsed,
"identity": identity_from_sysinfo(parsed, "sysinfo"),
}
def _sysinfo_extended_dump(
mount_path: str,
*,
include_raw: bool,
) -> dict[str, Any]:
path = _device_dir(mount_path) / "SysInfoExtended"
if not path.exists():
return {"present": False}
raw = path.read_bytes()
from .sysinfo import parse_sysinfo_extended
parsed = parse_sysinfo_extended(raw, source="sysinfo_extended")
result: dict[str, Any] = {
"present": True,
"path": str(path),
"bytes": len(raw),
"sha256": _sha256(raw),
"used_regex_fallback": parsed.used_regex_fallback,
"keys": sorted(parsed.plist.keys()),
"identity": parsed.identity,
"cover_art_formats": parsed.cover_art_formats,
"photo_formats": parsed.photo_formats,
"chapter_image_formats": parsed.chapter_image_formats,
}
if include_raw:
result["raw"] = raw
result["plist"] = parsed.plist
return result
def _identity_from_live_vpd(vpd: dict[str, Any], source: str) -> dict[str, Any]:
from .sysinfo import (
ParsedSysInfoExtended,
identity_from_sysinfo_extended,
parse_sysinfo_extended,
)
raw = vpd.get("vpd_raw_xml")
if raw:
identity = parse_sysinfo_extended(raw, source=source, live=True).identity
else:
parsed = ParsedSysInfoExtended(plist=vpd, source=source, live=True)
identity = identity_from_sysinfo_extended(parsed, source, live=True)
sources = identity.setdefault("_sources", {})
for field in (
"usb_pid",
"usb_vid",
"usb_serial",
"scsi_vendor",
"scsi_product",
"scsi_revision",
"block_device",
):
value = vpd.get(field)
if value not in (None, "", b""):
identity[field] = value
sources[field] = source
return identity
def _live_scsi_dump(
mount_path: str,
final_identity: dict[str, Any],
) -> dict[str, Any]:
try:
if sys.platform == "win32":
from .vpd_windows import query_ipod_vpd_for_path
result = query_ipod_vpd_for_path(
mount_path,
usb_pid=int(final_identity.get("usb_pid") or 0),
serial_filter=str(final_identity.get("firewire_guid") or ""),
)
elif sys.platform == "linux":
from .vpd_linux import query_ipod_vpd_for_path
result = query_ipod_vpd_for_path(
mount_path,
usb_pid=int(final_identity.get("usb_pid") or 0),
serial_filter=str(final_identity.get("firewire_guid") or ""),
)
elif sys.platform == "darwin":
from .vpd_iokit import query_ipod_vpd
result = query_ipod_vpd(
usb_pid=int(final_identity.get("usb_pid") or 0),
serial_filter=str(final_identity.get("firewire_guid") or ""),
)
else:
return {"available": False, "reason": f"unsupported_{sys.platform}"}
if not result:
return {"available": True, "result": None, "error": "no_result"}
source = str(result.get("_source") or {
"win32": "windows_scsi",
"linux": "linux_scsi",
"darwin": "scsi_vpd",
}.get(sys.platform, "scsi_vpd"))
return {
"available": True,
"result": result,
"identity": _identity_from_live_vpd(result, source),
"standard_inquiry": {
"vendor": result.get("scsi_vendor", ""),
"product": result.get("scsi_product", ""),
"revision": result.get("scsi_revision", ""),
},
}
except Exception as exc:
return {"available": True, "result": None, "error": repr(exc)}
def _live_windows_scsi_dump(
mount_path: str,
final_identity: dict[str, Any],
) -> dict[str, Any]:
if sys.platform != "win32":
return {"available": False, "reason": "not_windows"}
return _live_scsi_dump(mount_path, final_identity)
def _live_usb_vendor_dump(
final_identity: dict[str, Any],
) -> dict[str, Any]:
try:
from .usb_backend import backend_diagnostic
from .vpd_usb_control import query_ipod_usb_sysinfo_extended
result = query_ipod_usb_sysinfo_extended(
usb_pid=int(final_identity.get("usb_pid") or 0),
serial_filter=str(final_identity.get("firewire_guid") or ""),
)
if not result:
return {
"available": True,
"result": None,
"error": "no_result",
"backend": backend_diagnostic(),
}
source = str(result.get("_source") or "usb_vendor")
return {
"available": True,
"result": result,
"identity": _identity_from_live_vpd(result, source),
"backend": backend_diagnostic(),
}
except Exception as exc:
return {"available": False, "result": None, "error": repr(exc)}
def _disk_size_gb(mount_path: str) -> float:
try:
import shutil
return round(shutil.disk_usage(mount_path).total / 1e9, 1)
except Exception:
return 0.0
def _final_identity_snapshot(mount_path: str) -> dict[str, Any]:
from .scanner import _probe_filesystem, _probe_hardware, _resolve_model
mount_name = _mount_name(mount_path)
hardware = _probe_hardware(mount_path, mount_name)
filesystem = _probe_filesystem(mount_path)
resolved = _resolve_model(hardware, filesystem, _disk_size_gb(mount_path))
return {
"hardware": hardware,
"filesystem": filesystem,
"resolved": resolved,
}
def _append_identity_evidence(
evidence: dict[str, list[dict[str, Any]]],
source: str,
identity: dict[str, Any] | None,
) -> None:
if not identity:
return
sources = identity.get("_sources", {})
for field in IDENTITY_FIELDS:
value = identity.get(field)
if value in (None, ""):
continue
evidence.setdefault(field, []).append({
"source": sources.get(field, source),
"value": value,
})
def _append_plain_evidence(
evidence: dict[str, list[dict[str, Any]]],
source: str,
data: dict[str, Any] | None,
) -> None:
if not data:
return
sources = data.get("_sources", {})
for field in IDENTITY_FIELDS:
value = data.get(field)
if value in (None, ""):
continue
evidence.setdefault(field, []).append({
"source": sources.get(field, source),
"value": value,
})
def _rejected_conflicts(
final_identity: dict[str, Any],
evidence: dict[str, list[dict[str, Any]]],
) -> list[dict[str, Any]]:
rejected: list[dict[str, Any]] = []
for field, entries in sorted(evidence.items()):
final_value = final_identity.get(field)
final_norm = _normalise_for_compare(field, final_value)
if not final_norm:
continue
for entry in entries:
value_norm = _normalise_for_compare(field, entry.get("value"))
if value_norm and value_norm != final_norm:
rejected.append({
"field": field,
"final_value": final_value,
"rejected_value": entry.get("value"),
"rejected_source": entry.get("source"),
})
return rejected
def dump_device_info(
mount_path: str,
*,
include_raw: bool = False,
probe_usb_vendor: bool = True,
) -> dict[str, Any]:
"""Build a read-only device diagnostic report."""
mount_path = os.path.abspath(_normalise_mount_path(mount_path))
snapshot = _final_identity_snapshot(mount_path)
final_identity = snapshot["resolved"]
sysinfo = _sysinfo_dump(mount_path)
sysinfo_extended = _sysinfo_extended_dump(
mount_path,
include_raw=include_raw,
)
live_scsi = _live_scsi_dump(mount_path, final_identity)
live_windows_scsi = (
live_scsi
if sys.platform == "win32"
else {"available": False, "reason": "not_windows"}
)
live_usb_vendor = (
_live_usb_vendor_dump(final_identity)
if probe_usb_vendor
else {"available": False, "reason": "disabled"}
)
evidence: dict[str, list[dict[str, Any]]] = {}
_append_identity_evidence(evidence, "sysinfo", sysinfo.get("identity"))
_append_identity_evidence(
evidence,
"sysinfo_extended",
sysinfo_extended.get("identity"),
)
_append_plain_evidence(evidence, "hardware", snapshot.get("hardware"))
_append_identity_evidence(
evidence,
"live_scsi",
live_scsi.get("identity"),
)
_append_identity_evidence(
evidence,
"usb_vendor",
live_usb_vendor.get("identity"),
)
usb_details = {
key: snapshot["hardware"].get(key)
for key in (
"usb_vid",
"usb_pid",
"firewire_guid",
"usbstor_instance_id",
"usb_parent_instance_id",
"usb_grandparent_instance_id",
)
if snapshot["hardware"].get(key) not in (None, "")
}
report = {
"mount_path": mount_path,
"mount_name": _mount_name(mount_path),
"sysinfo": sysinfo,
"disk_sysinfo_extended": sysinfo_extended,
"live_scsi_vpd": live_scsi,
"live_windows_scsi_vpd": live_windows_scsi,
"live_usb_vendor": live_usb_vendor,
"standard_inquiry": live_scsi.get("standard_inquiry", {}),
"usb_details": usb_details,
"final_resolved_identity": final_identity,
"resolver_conflicts": final_identity.get("_conflicts", []),
"all_identity_evidence": evidence,
"rejected_conflicting_evidence": _rejected_conflicts(
final_identity,
evidence,
),
}
return _json_safe(report, include_raw=include_raw)
def _default_paths() -> list[str]:
from .scanner import _find_ipod_volumes
return [mount for mount, _display in _find_ipod_volumes()]
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(
description="Dump read-only iPod identity evidence for debugging.",
)
parser.add_argument("paths", nargs="*", help="Mounted iPod path(s)")
parser.add_argument(
"--all",
action="store_true",
help="Dump every mounted volume containing iPod_Control",
)
parser.add_argument(
"--include-raw",
action="store_true",
help="Include raw SysInfoExtended text in the JSON output",
)
parser.add_argument(
"--no-usb-vendor",
action="store_true",
help="Skip the USB vendor-control diagnostic probe",
)
args = parser.parse_args(argv)
paths = list(args.paths)
if args.all or not paths:
paths.extend(path for path in _default_paths() if path not in paths)
if not paths:
print("No iPod volumes found.", file=sys.stderr)
return 1
reports = [
dump_device_info(
path,
include_raw=args.include_raw,
probe_usb_vendor=not args.no_usb_vendor,
)
for path in paths
]
payload: Any = reports[0] if len(reports) == 1 else reports
print(json.dumps(payload, indent=2, ensure_ascii=False))
return 0
if __name__ == "__main__":
raise SystemExit(main())
+1022
View File
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More