Like its "cousins" before it, the number of notes added on sentences is growing (or rather, the rate of sentences is growing) each week.
Here's a little peek into the database table that holds these notes:
Wow, right?
As you may or may not know, all notes attached to words, kanji, and sentences are moderated: this means that no notes are made public immediately. There is the rare spam comment, nsfw/inappropriate material, etc.
In particular, the sentence notes often have "hey, this is wrong" comments which I read, fix (the content), and then block the note so that the notes do not refer to past versions that are no longer accessible. These notes should not exist, though - there is a link on the sentence note page saying "if you are making a correction, please go here". However, I know that people are not maliciously ignoring that, which means I need to do a better job of clarifying it.
That aside, the numbers are growing such that I (and one other long-time user who helps with moderation and responses) easily fall behind in checking these, and some notes it in a "processing" state for weeks - far from ideal.
My goal for the moderation though is a)to give myself a chance to answer questions, and b)to make the comments useful to learners. For example, if you're studying a sentence, have some trouble understanding, and see the comment marker highlighted, I do not want you to think "oh, maybe someone has some info" only to open it up and see a "lol, I totally get the person who wrote this" comment.
So, I am open to ideas to how to adjust the system so the sentence notes can continue to flourish and our awesome community can keep helping one another. One of these big issues is that there is a huge number of sentences, so if someone posts a question, it can be pretty hard for other people who might know the answer to see that question and answer it. In other words, discoverability is tricky. Possible solution is to add a panel on Community in the dashboard to see recent sentence notes, but I don't want to lean on that page as a crutch to make things more visible, otherwise, it might have 10-15 panels to choose from in the future.
Regarding how to improve moderation, maybe I would start by having a larger moderation team with clear responsibilities and later implement some simple way to catch common comments that should not be published. E.g. detect nsfw and "this is wrong" comments. The the user can get a warning like "sorry this comment is not appropriate because..."
Regarding comment visibility, I would like to be notified if someone replies to me. I'm not sure this is happening now.
Regarding better moderation: I would suggest that you choose from the users themselves a few who are eligible and willing to help in moderation, responding and answering questions. You certainly can pick the suitable users judging from their level and also how active they are in the forums and community section.
Regarding useless comments: There should be some kind of a warning system that the "future new" moderators can use when encountering such useless comments as the ones you mentioned. The sender's profile/identity is always under their comment.
I hope that you like my ideas and find them sensible.
I'm in no way competent to fix Japanese sentences, the only issue I had was that the audio was different from the sentence. And there are plenty of such sentences, they should all be reviewed one by one.
Off topic, but what I would love to see added is the ability to choose male/female voice for sentences, much like they have at Bunpro. And of course, more voiced sentences.
Keep up the great work, I love your stance on the AI and multitude of crafted sentences read by real people.
I'm the user that helps with comment moderation. To clarify to everyone how that works, I am just a regular user with the ability see the list of unpublished comments and make them public. I cannot delete or reject comments that are inappropriate (they just remain unpublished until マイコー handles it), and I have no form of contact with the other user to be able to give warnings like a forum moderator would. I'm mostly just a front-line question-answerer.
I do think the best way to give comments visibility would be adding them to the Community tab or another section that users can browse, but the moderation step is definitely still needed and I agree that publishing questions that haven't been answered isn't ideal for the sake of the person asking or any other users with the same question.
At the moderation stage, it would be great to have a way to (publically) flag comments as an unanswered language question when publishing them, and then give users the ability to filter the Community tab comments section by unanswered questions. That way the questions I'm not confident answering can still be published and exposed to other community members who can help, without the comment getting lost in the firehose of other translation and content-related comments. Perhaps unanswered questions could also be sorted by how many likes they have, so users wanting to answer questions can prioritize the ones that the most people want help understanding.
I'm in no way competent to fix Japanese sentences, the only issue I had was that the audio was different from the sentence. And there are plenty of such sentences, they should all be reviewed one by one.
When you get one of those, click the eye button and write a comment that the audio is wrong. Michael takes care of them
I'm the user that helps with comment moderation. To clarify to everyone how that works, I am just a regular user with the ability see the list of unpublished comments and make them public. I cannot delete or reject comments that are inappropriate (they just remain unpublished until マイコー handles it), and I have no form of contact with the other user to be able to give warnings like a forum moderator would. I'm mostly just a front-line question-answerer.
I do think the best way to give comments visibility would be adding them to the Community tab or another section that users can browse, but the moderation step is definitely still needed and I agree that publishing questions that haven't been answered isn't ideal for the sake of the person asking or any other users with the same question.
At the moderation stage, it would be great to have a way to (publically) flag comments as an unanswered language question when publishing them, and then give users the ability to filter the Community tab comments section by unanswered questions. That way the questions I'm not confident answering can still be published and exposed to other community members who can help, without the comment getting lost in the firehose of other translation and content-related comments. Perhaps unanswered questions could also be sorted by how many likes they have, so users wanting to answer questions can prioritize the ones that the most people want help understanding.
Gillianfaith then. I guessed as much. Your dedication is well-appreciated, and I'd like to express my gratitude. I'm sorry for the lack of clarity. I meant you or any future user-moderators should be given the authority to cancel inappropriate comments, which will lighten the burden on Michael's shoulder.
As for your suggestion in the moderation stage, I'm not sure I got what you mean. I'd appreciate if you could explain further...
Speaking from the one(?) note I left: Sometimes a sentence has a piece of grammar or nuance I haven't studied yet (ya particle, a new verb form, yet a new use of the ni particle, etc...) and while I can make an educated guess to what -feels- like is the right answer, I feel like I'm cheating a bit and if I can't quickly learn what it is I feel a bit dejected (like maybe I missed a step?)
Sometimes I am able to figure it out myself by hovering on the words, but often I end up shrugging it ,"i'll find it in a grammar lesson somewhere in the future"
I find it REALLY helpful to see other people's questions to things I didnt understand and that someone kindly clarified for everyone to see. Though, I do feel like sometimes I see a lot of the same type of comments regarding things like "why is ga here? why ni and not x?" (unless they are all the same excercise and I never noticed lol), so perhaps before making a question/note if you select what part of the sentence is that you struggled with it'll prompt you to a clarification made from a previous question and if it did help, you click on it and if enough people found it helpful it gets linked to the sentence/excercise for future users to see...
A similar priority queue could be implemented if enough people feel like a sentence is wrong or not; if you agree with a person's report you click on it and it gets higher priority on the queue. Once it gets clarified , everyone that clicked on it gets notified or something!
Just some random thoughts! Keep up the awesome work, renshuu is great!
this is tricky. You said you wanted to give yourself a chance to answer questions, but the sheer number of them is chocking your ability to moderate. I don't know how popular this app is, but there must be more people coming to it overtime. You will only encounter more and more problems such as these as renshuu continues to grow. I think you need more people whos job will be just to moderate. It's simply a lot of work. But you probably don't want to hire someone.
Then... the best would maybe be making a specific section that will house these sentences. In the community section for example, you can have "sentense related questions" or something. So anybody who wants to access it and give feedback can do so. But then comes the question of motivation, because a lot of us feels like we aren't capable of correcting someone. So, maybe you could devide those sentences into categories based on where they were submited from. N5/4 etc... Then maybe I (who am N4 with grammar) could feel comfortable answering a question of a lower level. If I know where the sentence is from, I can go through my notes to check.
It can be a new tool for learning as well, because by verifing somebodies sentence, you are checking your knowledge too. That's my idea. As always, I hope you find a solution soon. Best of luck.
I think the person above me has the right idea about how sentence feedback and suggestion should be rated by what N-level the user who gave it is. The only problem I see is that the only way to find that out currently is from a self-reported level, which could either be faked or the user may not have taken the JLPT. I feel that a solution to that may be to either create a series of tests that are made to be similar to the JLPT or to have an algorithm that estimates a user’s JLPT level based on the lessons that the user has taken and their overall accuracy. I realize neither of these are perfect solutions, but they seem at least plausible. Another option could be to rank users based off of accepted sentence suggestions, but that’s still a lot of work for moderators and マイコー.
Regarding sentences containing topics that have not been covered by the user, I feel there isn’t an easy way to get around that without revamping the whole sentence bank, so the easiest way to fix that in my opinion would be to have each sentence contain a list of the topics present, and the topics that user has covered so far would be highlighted in blue, while topics not covered would be highlighted in red(I’m assuming there would be a way to do this that doesn’t require human intervention).
These are all just my ideas and opinions , so feel free to take them with a large grain of salt.
Thanks so far for all the ideas! A few points of clarification and extra info to help explain the situation:
1. Corrections to sentences (audio, English, Japanese) are never automated, so anyone can feel free to say "I think this is wrong, I think it should be this" - and it will not reflect badly on you if you're wrong, your account won't be banned, etc.
2. Moderation "teams" are almost impossible, I've found. renshuu is small, and will never be big enough so that I am hiring people to help me with this. Because of that, moderation is going to be purely voluntary, and while there is a ton of people who help out SO much in the forums, in discord, etc., when it comes to more "active" systems, it is very hard to maintain activity (and that's simply the nature of volunteering, I think, and is not a value statement at all on any individual person).
If I do go with a central area for viewing "new questions", I could envision this - there might be a "report content" button there, and like the other places with report content, anything that is reported would get checked by me.
3. I do enjoy answering the questions - it helps me keep my teacher skills honed, I just know that a lot of the questions could be answered by others (sometimes by people with more time, who could give more detailed responses), so I want to make that easier for them.
4. An "FAQ" for common Japanese questions might be useful. Of course, stuff like this is found elsewhere on the internet, but if it was possible within the response field to look up and link to a page somewhere (forums, maybe?) that already has an explanation, it would possibly help me or others having to write down the same thing over and over again. At the same time, a lot of people love renshuu for the feeling of actual humans being here and interacting, and there is something to be said about getting a response that just points them to a different page.
Anyway, I hope to continue to hear your ideas! I am working to get a couple of long-time experimental features out this month, and then I'm diving into this (and a bunch of other community-centric features) in Feb/March.
The forums that are not part of the community dashboard are effectively hidden from the average user, so I'm not sure how much that would help outside of power users :(
The forums that are not part of the community dashboard are effectively hidden from the average user, so I'm not sure how much that would help outside of power users :(
I don't think I'm properly understanding what you are saying. Perhaps its because I access Renshuu via the browser, or perhaps there's a big disconnect between how I think people use the site and how they actually do.
Are you saying that other users don't have access to these forums, or are you saying that they just don't use them, or something else entirely ?
Do most users that participate in posting just use the Japanese Quick help instead ?
Would it be possible to add a "Sentence Talk" forum to the following list of checkboxes ? Would that be undesirable ?
Is it likely that users that don't participate in the forums would participate in discussing sentence notes ?
The surface area that represents what most users use is much smaller than those of you who have been around for years.
Specifically, for that Quick Help panel, it only shows the last 10 posts in all of the selected forums. So, there is a break point of sorts at which if enough traffic is coming through there, most things will fly by before anyone can see/respond to them. 40-50 sentence questions/comments a day is probably sooner than later, which would dwarf the number of total threads being posted on across all forums.
As an aside, making some small visual changes:
In addition to the extra text + kao chan, the note for sending a correction is "outside" of the submission form, so more likely to see it there (I think?)
At the moderation stage, it would be great to have a way to (publically) flag comments as an unanswered language question when publishing them, and then give users the ability to filter the Community tab comments section by unanswered questions. That way the questions I'm not confident answering can still be published and exposed to other community members who can help, without the comment getting lost in the firehose of other translation and content-related comments. Perhaps unanswered questions could also be sorted by how many likes they have, so users wanting to answer questions can prioritize the ones that the most people want help understanding.
So, back when the sentence notes were first added, you could choose from I think three different categories for your note: alternate translation, content note, or question.
There was such a high number of notes that were mismarked that it felt like the category tags cannot be used to reliably filter out things here. It might have been a ui/ux issue, but I do not think that was solely it - I think a lot of users (not just on renshuu, but on the internet in general) will see the text box and avoid other form elements unless they are required.
Now, there are still three categories present (Discussion being the first one), and given that most are questions, this is a good fit.
So, just brainstorming a few things based on what everyone has said.
1. Area for viewing discussion (question) tagged notes that do not have any responses yet. (I'm not sure doing the whole "mark this question as answered" thing is going to be effective). Rank them both on a mix of newness as well as heart count
2. Like other user-content areas, add a Report Content button. These would (like other reports) get highlighted for me. If a comment is switched from public to private, send a note to that user letting them know why.
3. I'm not sure if we need a "this content has not been checked" or similar marker. End goal is that while I will still answer questions when I can (I like to!), it shouldn't be necessary to get a good question or comment in front of others.
So, just brainstorming a few things based on what everyone has said.
1. Area for viewing discussion (question) tagged notes that do not have any responses yet. (I'm not sure doing the whole "mark this question as answered" thing is going to be effective). Rank them both on a mix of newness as well as heart count
2. Like other user-content areas, add a Report Content button. These would (like other reports) get highlighted for me. If a comment is switched from public to private, send a note to that user letting them know why.
3. I'm not sure if we need a "this content has not been checked" or similar marker. End goal is that while I will still answer questions when I can (I like to!), it shouldn't be necessary to get a good question or comment in front of others.
Thank you so much, Michael! Because of your clarifying message, I finally understood what I was asking previously and got no response. Additionally, I was just about to post the same ideas as the points you mentioned last, not realizing that they had already been mentioned.