Not sure if this is a bug or not but I've been noticing that there are duplicate word entries that are the exact same (kanji, kana, definition, sentences, etc.). It's actually quite common for words to have multiple duplicates show up when using the advanced search feature from what I've noticed.
I've mostly just ignored this up until now but recently the exact same word appeared in my N4 schedule. I learned the word お見舞い a few days prior and yet a duplicate entry appeared as a new word to learn in the exact same schedule.
Please note that the first entry's definition was custom edited by me but was defaulted with the same exact definition as the second entry.
Besides this word, I recall seeing other words that I've already learned show up in different schedules as a new word to learn (despite being a duplicate) over the past couple months.
Is there any way to prevent this from happening in the future? This is just a slight inconvenience but it would be great if the database could be cleaned up to avoid having these duplicate entries show up.
Just a quick suggestion but would it be possible to create a small script that iterates over each entry, filter for duplicates, and remove them if enough fields match? Obviously there's probably more to it to take into consideration but it would be easier than manually checking the data haha
Oh - that's exactly what I did, but apparently my simple script had gaps back then :(. I did an additional check earlier today for n4 and found..I think 2-3 more terms that had duplicates.
Although it's in the past, if my guess is correct, the original cause for the duplicate terms has been fully removed, so I should not have the issue when I rebuild the n3/intermediate lists later this year.
