Skip to content

Add multiple genres#6367

Merged
snejus merged 31 commits intomasterfrom
add-multiple-genres
Feb 27, 2026
Merged

Add multiple genres#6367
snejus merged 31 commits intomasterfrom
add-multiple-genres

Conversation

@snejus
Copy link
Member

@snejus snejus commented Feb 15, 2026

Add support for a multi-valued genres field

  • Update metadata source plugins to populates genres instead of genre: musicbrainz, beatport, discogs.
  • Remove now redundant separator configuration from lastgenre.

Context

We previously had multiple issues with maintaining both singular and plural fields:

  1. Since both fields write and read the same field in music files, the values in both
    fields must be carefully synchronised, otherwise we see these fields being repeatedly
    retagged / rewritten using commands such as beet write. See related issues
  2. Fixes to sync logic required users manually retagging their libraries, while music
    imported as-is could not be fixed. See Stop perpetually writing mb_artistid, mb_albumartistid and albumtypes fields #5540, for example.

Therefore, this PR replaces a singular genre field by plural genres for good:

  1. We migrate genre -> genres immediately on the first beets invocation
  2. genre field is removed and genres is added
  3. The old genre column in the database is left in place - these values will be ignored
    by beets.
    • If someone migrates and later decides to switch back to using an older version of
      beets, their genre values are still in place.

Migration

  • This PR creates a new DB table migrations(name TEXT, table TEXT)

    • We add an entry when a migration has been fully performed on a specific table
    • Thus we only perform the migration if we don't have an entry for that table
    • Entry is only added when the migration has been performed fully: if someone hits
      CTRL-C during the migration, the migration will continue on the next beets invocation,
      see:
      def migrate_table(self, table: str, *args, **kwargs) -> None:
          """Migrate a specific table."""
          if not self.db.migration_exists(self.name, table):
              self._migrate_data(table, *args, **kwargs)
              self.db.record_migration(self.name, table)
  • Implemented using SQL due to:

    1. Significant speed difference: migrating my 9000 tracks / 2000 albums library:
      • Using our Python implementation: over 11 minutes
      • Using SQL: 2 seconds
    2. Beets seeing only genres field: genre field is only accessible by querying the
      database directly.

Supersedes: #6169

@snejus snejus requested a review from a team as a code owner February 15, 2026 13:52
@codecov
Copy link

codecov bot commented Feb 15, 2026

Codecov Report

❌ Patch coverage is 92.98246% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.32%. Comparing base (dcef1f4) to head (a540a81).
⚠️ Report is 32 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
beetsplug/bpd/__init__.py 0.00% 4 Missing ⚠️
beetsplug/lastgenre/__init__.py 84.61% 3 Missing and 1 partial ⚠️
beets/library/migrations.py 96.36% 1 Missing and 1 partial ⚠️
beets/autotag/hooks.py 90.90% 0 Missing and 1 partial ⚠️
beets/dbcore/db.py 98.43% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6367      +/-   ##
==========================================
+ Coverage   69.18%   69.32%   +0.14%     
==========================================
  Files         140      141       +1     
  Lines       18686    18786     +100     
  Branches     3053     3060       +7     
==========================================
+ Hits        12927    13024      +97     
- Misses       5114     5117       +3     
  Partials      645      645              
Files with missing lines Coverage Δ
beets/dbcore/types.py 96.03% <100.00%> (+0.01%) ⬆️
beets/library/library.py 93.65% <100.00%> (+0.20%) ⬆️
beets/library/models.py 87.10% <100.00%> (ø)
beetsplug/aura.py 57.35% <ø> (ø)
beetsplug/beatport.py 42.97% <100.00%> (-0.83%) ⬇️
beetsplug/discogs/__init__.py 64.30% <100.00%> (-0.57%) ⬇️
beetsplug/fish.py 20.53% <ø> (ø)
beetsplug/smartplaylist.py 75.39% <ø> (ø)
beets/autotag/hooks.py 99.21% <90.90%> (-0.79%) ⬇️
beets/dbcore/db.py 94.41% <98.43%> (+0.26%) ⬆️
... and 3 more
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@snejus snejus force-pushed the add-multiple-genres branch 6 times, most recently from 6f18c92 to 8c0820b Compare February 16, 2026 21:49
Copy link
Member

@JOJ0 JOJ0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

supported separators in docs correct?

I don't have much to say on the sql implementation except that it works when switching back and forth between this branch and current master and it's super-fast 🤩 👍

Copy link
Member

@JOJ0 JOJ0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JOJ0 added a commit to JOJ0/beets that referenced this pull request Feb 19, 2026
@snejus snejus force-pushed the add-multiple-genres branch from 8c0820b to fbf7085 Compare February 21, 2026 10:54
@snejus
Copy link
Member Author

snejus commented Feb 22, 2026

@JOJ0 just waiting for #6387 to be reviewed and merged which will fix the remaining test failures.

Copilot AI review requested due to automatic review settings February 22, 2026 12:16
@snejus snejus force-pushed the add-multiple-genres branch from fbf7085 to a8101c1 Compare February 22, 2026 12:16
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request implements native multi-value genre support in beets by replacing the singular genre field with a plural genres field across the codebase. This addresses long-standing issues with maintaining synchronization between singular and plural fields that caused repeated retagging operations.

Changes:

  • Introduces database migration infrastructure to support automatic data migrations across schema changes
  • Migrates existing genre string values to genres multi-value field, splitting on common separators (, ; /)
  • Updates metadata source plugins (MusicBrainz, Beatport, Discogs, LastGenre) to populate genres as a list rather than genre as a string
  • Removes the separator configuration option from LastGenre plugin since genres are now stored as lists
  • Updates all tests to use genres field instead of genre

Reviewed changes

Copilot reviewed 38 out of 40 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
beets/dbcore/db.py Adds Migration base class and infrastructure for tracking migration state, adds mutate_many for batch operations, adds db_tables cached property for schema introspection
beets/dbcore/types.py Exports MULTI_VALUE_DELIMITER constant for reuse in migrations
beets/library/library.py Registers MultiGenreFieldMigration to run on library initialization
beets/library/migrations.py Implements genre→genres migration logic with automatic separator detection
beets/library/models.py Changes genre field from STRING to genres field with MULTI_VALUE_DSV type in both Item and Album models
beets/autotag/hooks.py Adds deprecation warning for genre parameter in AlbumInfo constructor with automatic conversion to genres list
beetsplug/musicbrainz.py Updates to populate genres list instead of genre string
beetsplug/lastgenre/init.py Refactors to work with genres as lists, removes separator config option, updates type hints
beetsplug/discogs/init.py Updates to populate genres list and simplifies genre/style handling
beetsplug/beatport.py Updates to populate genres list for both releases and tracks
beetsplug/bpd/init.py Maps MPD "Genre" tag type to "genres" field
beetsplug/aura.py Adds mapping for both "genre" and "genres" to "genres" field for compatibility
test/* Updates all tests to use genres field instead of genre, adds migration tests
docs/* Updates documentation to reflect genres field usage and documents migration behavior
setup.cfg Adds follow_untyped_imports = true to mypy configuration

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@snejus
Copy link
Member Author

snejus commented Feb 22, 2026

@JOJ0 given that Copilot also raised a question regarding separators, I've now documented the precedence and the fact that genre is split by the first found separator only.

@snejus snejus force-pushed the add-multiple-genres branch 3 times, most recently from 151468e to 241d335 Compare February 22, 2026 14:50
@snejus snejus changed the base branch from master to fix-handling-multi-valued-fields February 22, 2026 14:50
@snejus snejus force-pushed the add-multiple-genres branch 2 times, most recently from fbe848a to 80d08bc Compare February 22, 2026 14:54
@snejus snejus force-pushed the fix-handling-multi-valued-fields branch from 03114e1 to 54a46bd Compare February 22, 2026 15:57
@snejus snejus force-pushed the add-multiple-genres branch from 80d08bc to ce5214a Compare February 22, 2026 15:58
@snejus snejus force-pushed the add-multiple-genres branch from d0efc62 to a540a81 Compare February 27, 2026 18:36
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 40 out of 42 changed files in this pull request and generated no new comments.

@snejus
Copy link
Member Author

snejus commented Feb 27, 2026

I think this is now in a good place to be merged in, finally! Thanks, @dunkla ❤️

@snejus snejus merged commit 16be1df into master Feb 27, 2026
18 checks passed
@snejus snejus deleted the add-multiple-genres branch February 27, 2026 18:42
@aereaux
Copy link
Contributor

aereaux commented Feb 27, 2026

I'm having a bit of trouble after upgrading to this commit. beet ls -f'$genres' -a doesn't seem to return anything, and beet mbsync doesn't cause any genre changes. It looks like the migration isn't getting run even though it's creating the migration records in the database. According to some print debugging, genre isn't in the fields here, so the migration gets skipped: https://github.com/beetbox/beets/pull/6367/changes#diff-ed0b3efaa45c8c217558e3fcf59827ee9cd1ecf9e2f058b9828c49c95428c85fR62

@JOJ0
Copy link
Member

JOJ0 commented Feb 28, 2026

I'm having a bit of trouble after upgrading to this commit. beet ls -f'$genres' -a doesn't seem to return anything, and beet mbsync doesn't cause any genre changes. It looks like the migration isn't getting run even though it's creating the migration records in the database. According to some print debugging, genre isn't in the fields here, so the migration gets skipped: https://github.com/beetbox/beets/pull/6367/changes#diff-ed0b3efaa45c8c217558e3fcf59827ee9cd1ecf9e2f058b9828c49c95428c85fR62

Hi, can you make a backup copy of your current beets database and then try to reset your database to the pre-migration state? See this post of @snejus: #6367 (comment)

Then for your next report here also add debug logging and post what's happening.

Question: When you did your print tests, was that possibly after the migration was run? Then the genre (singular) field is removed from the codebase, so that check will always prevent (another) migration.

Also some SELECT or actual database schema dumps would help. It all sounds like to me that migration was run already (but maybe didn't do everything it was supposed to do, we can't know without logs). I think resetting and carefully watching what's happening would be good. HTH

@JOJ0
Copy link
Member

JOJ0 commented Feb 28, 2026

Question: When you did your print tests, was that possibly after the migration was run? Then the genre (singular) field is removed from the codebase, so that check will always prevent (another) migration.

I might be mistaken here. Trying hard with the debugger to find out what's going on. I'm not sure anymore what current_fields is. In my just reset testlibrary I don't see genre in the list here and run into the the early return too:

Image

Maybe my breakpoints are too late and everything happened already.

In any case we will need @snejus and I'm sure with some patience we can figure out what's going on. In the meantime I'm investigating more...

@JOJ0
Copy link
Member

JOJ0 commented Feb 28, 2026

@snejus I think I can repro the issue. My last tests were with the add-multiple-genres branch and I'm testing the first time after merging into master. Wild guess is that something happened on the very last rebase before merging?

Resetting, no genres column:

 beets312  dev  ~/git/beets   master  sqlite3 ~/.config/devbeets/library.db "DROP TABLE MIGRATIONS; ALTER TABLE items DROP COLUMN genres; ALTER TABLE albums DROP COLUMN genres"
-- Loading resources from /Users/jojo/.sqliterc
 beets312  dev  ~/git/beets   master  sqlite3 ~/.config/devbeets/library.db "SELECT artist,genre,genres FROM items WHERE genre LIKE '%reggae%';"
-- Loading resources from /Users/jojo/.sqliterc
Error: in prepare, no such column: genres
  SELECT artist,genre,genres FROM items WHERE genre LIKE '%reggae%';
                      ^--- error here
 ✘  beets312  dev  ~/git/beets   master 

Accessing should trigger migration but doesn't or does it incomplete it seems:

 ✘  beets312  dev  ~/git/beets   master  beet -vvv ls bob could you be
user configuration: /Users/jojo/.config/devbeets/config.yaml
data directory: /Users/jojo/.config/devbeets
plugin paths: []
Loading plugins: autobpm, convert, describe, dirfields, discogs, duplicates, edit, fromfilename, importfeeds, info, lastgenre, mbsync, missing, play, playlist, vibenet
lastgenre: Loading whitelist ~/git/home_config/lastgenre_conf/genres.txt
lastgenre: Loading canonicalization tree ~/git/home_config/lastgenre_conf/genres-tree.yaml
Sending event: pluginload
library database: /Users/jojo/.config/devbeets/library.db
library directory: /Users/jojo/Music/devbeets
Sending event: library_opened
Parsed query: AndQuery([OrQuery([SubstringQuery('artist', 'bob', fast=True), SubstringQuery('title', 'bob', fast=True), SubstringQuery('comments', 'bob', fast=True), SubstringQuery('album', 'bob', fast=True), SubstringQuery('albumartist', 'bob', fast=True), SubstringQuery('genres', 'bob', fast=True)]), OrQuery([SubstringQuery('artist', 'could', fast=True), SubstringQuery('title', 'could', fast=True), SubstringQuery('comments', 'could', fast=True), SubstringQuery('album', 'could', fast=True), SubstringQuery('albumartist', 'could', fast=True), SubstringQuery('genres', 'could', fast=True)]), OrQuery([SubstringQuery('artist', 'you', fast=True), SubstringQuery('title', 'you', fast=True), SubstringQuery('comments', 'you', fast=True), SubstringQuery('album', 'you', fast=True), SubstringQuery('albumartist', 'you', fast=True), SubstringQuery('genres', 'you', fast=True)]), OrQuery([SubstringQuery('artist', 'be', fast=True), SubstringQuery('title', 'be', fast=True), SubstringQuery('comments', 'be', fast=True), SubstringQuery('album', 'be', fast=True), SubstringQuery('albumartist', 'be', fast=True), SubstringQuery('genres', 'be', fast=True)])])
Parsed sort: NullSort()
Bob Marley & The Wailers - Legend: The Best of Bob Marley and The Wailers - 03 - Could You Be Loved (1992)  [846 210-2] $genre
Sending event: cli_exit

I have the genres column but it's empty

 beets312  dev  ~/git/beets   master  sqlite3 ~/.config/devbeets/library.db "SELECT artist,genre,genres FROM items WHERE genre LIKE '%reggae%';"
-- Loading resources from /Users/jojo/.sqliterc
artist                               genre                    genres
-----------------------------------  -----------------------  ------
Bobby McFerrin                       Reggae, Jazz
Bennett & Dennis                     reggae; rocksteady; ska
Marshall Williams                    reggae; rocksteady; ska
The Octaves                          reggae; rocksteady; ska
Dr. Ring-Ding & The Senior Allstars  Reggae
Various                              Reggae
Various                              Reggae
Various                              Reggae
Asian Dub Foundation                 Electronic, Reggae
Asian Dub Foundation                 Electronic, Reggae
Asian Dub Foundation                 Electronic, Reggae
Asian Dub Foundation                 Electronic, Reggae

@snejus
Copy link
Member Author

snejus commented Feb 28, 2026

Goddammit, I trusted Copilot too much and made a mistake!

@snejus
Copy link
Member Author

snejus commented Feb 28, 2026

Fix is up in #6401 and going to merge it immediately once tests pass.

Follow instructions in this comment #6367 (comment) before re-running beets again, apologies!

@JOJ0
Copy link
Member

JOJ0 commented Feb 28, 2026

Fix is up in #6401 and going to merge it immediately once tests pass.

Follow instructions in this comment #6367 (comment) before re-running beets again, apologies!

I can confirm that it works now:

Migrating genres for 131 items...
  Migrated 131 items (131/131 processed)...
Migration complete: 131 of 131 items updated

❤️

@aereaux
Copy link
Contributor

aereaux commented Feb 28, 2026

Perfect, that fixes it! Glad I could be your QA!

@aereaux
Copy link
Contributor

aereaux commented Feb 28, 2026

Actually, not totally perfect. When I run mbsync now it seems to be deleting genres from my DB. Even though info.genres is being set in album_info in the musicbrainz plugin.

@snejus
Copy link
Member Author

snejus commented Feb 28, 2026

Could you provide some diff output that you're seeing?

@aereaux
Copy link
Contributor

aereaux commented Feb 28, 2026

From beet mbsync Copland:

Samuel Barber - Barber: Adagio for Strings / Ives: Symphony no. 3 / Copland: Quiet City - Adagio for Strings
  genres:
    - classical
    - orchestral
    - symphony
Aaron Copland - Barber: Adagio for Strings / Ives: Symphony no. 3 / Copland: Quiet City - Quiet City
  genres:
    - classical
    - orchestral
    - symphony
Henry Cowell - Barber: Adagio for Strings / Ives: Symphony no. 3 / Copland: Quiet City - Hymn and Fuguing Tune no. 10 for oboe and strings
  genres:
    - classical
    - orchestral
    - symphony
Paul Creston - Barber: Adagio for Strings / Ives: Symphony no. 3 / Copland: Quiet City - A Rumor
  genres:
    - classical
    - orchestral
    - symphony
Charles Ives - Barber: Adagio for Strings / Ives: Symphony no. 3 / Copland: Quiet City - Symphony no. 3: I. Old Folks Gatherin'
  genres:
    - classical
    - orchestral
    - symphony
Charles Ives - Barber: Adagio for Strings / Ives: Symphony no. 3 / Copland: Quiet City - Symphony no. 3: II. Children's Day
  genres:
    - classical
    - orchestral
    - symphony
Charles Ives - Barber: Adagio for Strings / Ives: Symphony no. 3 / Copland: Quiet City - Symphony no. 3: III. Communion
  genres:
    - classical
    - orchestral
    - symphony

@snejus
Copy link
Member Author

snejus commented Feb 28, 2026

Interesting. Are they kept in place using older version of beets?

@aereaux
Copy link
Contributor

aereaux commented Feb 28, 2026

Yes, going back to version 5f06ce22, they are kept (in the genre, singular, variable).

@snejus
Copy link
Member Author

snejus commented Feb 28, 2026

Would you mind submitting a new issue for this? I will look at it in detail.

@aereaux
Copy link
Contributor

aereaux commented Feb 28, 2026

Created: #6403. I might have some time to look into this later this weekend

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants