掲示板 Forums - [Discussion] Better way to handle usage note validation

Hello everyone!

As of today, we have over 60,000!! usage notes (mnemonics, etc.) for words on renshuu, of which just under half are public.

As you may or may not know, I manually validate these notes - I make sure they are SFW, not spam, and not incorrect. When it comes to mnemonics, there is a lot of subjectivity (what works for one person may not work for someone else), but I try to think "will this be helpful for some users"

Anyway, I get anywhere from 50-100 new usage notes a day, 90% of which are mnemonics. I'm finding myself unable to keep up with them, and it's impacting my ability do bug-fixes, new features, etc.

I do not want to completely open up the floodgates and allow anything in, because sometimes the mnemonics are objectively wrong, very inappropriate, hateful, etc. But I'd like a way to hopefully get the community's help in verifying these. I have an idea, but would like your feedback on it.

"Firehose" widget on the dashboard (with something like XP points or kao garden coins built in)

For those who choose this, it would show a list of not-yet-approved mnemonics (non-mnmeonic notes are in the minority, and I can cover those) that the user could do a simple up/down vote on it (with clear rules for what gets voted up and down)

These values would be hidden, but if the same mnemonic gets voted up (for example) 5 times without getting voted down, it would be marked as public for everyone to see.

If it gets voted down 3 times (without getting voted up), it would be marked as hidden.

If it's a mix of ups and downs, it would go to me for verification.

In order to prevent "gaming" the system for whatever reward I give out, I'm thinking of something like this:

1. Points (or whatever) would not be given out immediately, but after a decision was made on each term.

2. If someone is marking things "incorrectly" too much, they'd get warned and possibly removed from the firehose system. This would not be a quick ban or anything, but a way to route out people who are marking everything up (or down, or randomly) in order to get the points.

3. I'd also hide all usernames on submissions, so there is no bias for or against someone based on who they are.

This would take some time to implement, but in the long run, I think it could help balance out my work so I can more effectively help everyone learn more on renshuu.

Thoughts?

I like this idea. Could there also be a skip button in case we feel uncertain, or for whatever reason don't get the mnemonic. Like it seems well written but maybe we don't get the reference?

This sounds like a fantastic idea. Might also be a cool way to learn new words.

I like the idea of the firehose widget, and I like the idea of the controls being as simple as "yes/no/skip" with very clear rules on what counts as yes or no. Unlike ポール, I see the value in mnemonics based on pronunciation distortions, so I wouldn't want there to be that much subjectivity in it.

Behind the scenes, I think it should probably not be 1 person 1 vote. Rather, your vote should be proportional to your mastery of the term in question (plus a small constant). If you're seeing a term for the first time in the firehose, you may not be best qualified to apply the criterion correctly. That way, a few very experienced users can get a mnemonic to the quorum level as fast as a large number of beginners.

As far as rewards go, I think points towards coins would be more valuable than XP. Or maybe even just 好き points, or fractions thereof.

Maybe you could add a report button for usage notes? Or you could make it a requirement to be a certain level in order to make them (like level 50 or something). Maybe even both

How about an automatic chat filter (specifically for usage notes)?

I like the widget idea. Also, maybe removing duplicates or limiting the number to maybe three to five per word (based on the votes) since some terms seem to have a lot of similar mnemonics.

I didn't know you validated all of them yourself Michael, how's that even possible?? Can I have your time management please? For the future, I'll try to add more resources if I add a usage note myself.

Outsourcing mnemonic validation is a good idea. I turned off mnemonics myself bc it became too much and I'm usually only interested in "fun fact" usage notes, it's somehow a small disappointment if you expect an interesting usage note but turns out it's "just" a mnemonic.

Seems like a good idea to me. I think maybe there should be a few options to choose from when you click up and down vote to quickly explain why you made the decision. Something like, if you click the down vote a menu pops up and you can choose NSFW, incorrect information etc. It would help filter out spam, bots and such and it will organize the mnemonics making it easier for you to look at them later. Best of luck on implementatiton.

If it's based on a mispronunciation (like they read く as if it were か) then it's objectively wrong.

Sometimes there's some leeway where the sound is close enough to an English word even if it's not exactly, but sometimes when you read it you can tell the person just mispronounced it in their head and the mnemonic is going to teach the wrong lesson.