掲示板 Forums - Fall 2024 Focus: Text Analyzer Improvements

If you include page number and update the url param correctly, back should work as expected to go to previous page (if navigated from). It would be unexpected to go to page 3 from 4, if I'd opened 4 directly or from 1.

For the suggested page tracking (if you end up working on it), could just be a return to (ex) Page 4 under/next to the doc title on the main list

It's not that easy, unfortunately. renshuu has a home-grown SPA (single-page-app) implementation which works, but is not what you'd call *thoroughly robust". So moving back is not just loading the pages (which could be done, if the SPA was not in place, but a ton of JS manipulating the browser history and cache for various speed benefits.

So while I'm pretty confident that it can be done, I'm not quite sure how to handle it.

I'll add some technical notes at the bottom.

As to the missing lightbulb - I'll consider adding that later, but not at the moment. That term list is the same as the term list everywhere else, so I'd need to make special options there to handle the extra icons.

Technical stuff.

So, for most devices, renshuu holds 3 or 4 of the last pages/modals (modals = term lists, grammar pages, and now, text analyzer reader) in a small cache. When a new page is loaded, the url is rewritten by JS, and then the page is loaded via ajax and put in place. Press back, and it rolls the window's history back one, which triggers the cache system to not actually reload the previous page, but to toss the cached page back up (unless the cache runs out).

Here's the problem: consider a 5 page reader.

When you open it, the page's url will be /text/text_id/pg_number, or as an example, /text/1000/0

If you go the next page, it is /text/1000/1, /text/1000/2, and so on.

If you were to use the back button, this would normally work fine - it would grab the last cached copy, which is the previous page. Easy!

However, let's say the cache has three pages on it:

/text/1000/0

/text/1000/1

/text/1000/2

If you then press the x in the top right, you'd want to clear out those 3 entries, and return to whatever was before it (the main TA page).

I originally tried something like this: (pseudo-code)

while( last cache entry is a reader page ) {

//add code to prevent the page from actually showing the cache for all the intermediary pages

window.location.back();

}

You may think this would work, as it would clean off all the cached pages, and clear the urls out of the browser's history (which you cannot do manually - you can either replace the current one, or add onto it)

However, window.location.back() is asynchronous! AHHHH. So that loop would run 1000 times before the 1st one finishes, and the browser would (and did) crash.

The main issue with all of this is that you cannot easily manipulate the browser history when going backwards - you must use the window.location.back().

Tricky.

I'd like to see the "always use kana when the original text does" option expanded to have a second, not mutually exclusive option: "keep usage of rare kanji intact where the original text used them". (Presumably it wouldn't do this if my settings had rare kanji spellings intact, but I got tired of learning them for words where it was only used some 0.00001% of the time when I enabled it to see and learn cases where it's used 5~20% of the time. (I'm excited to see the per-word toggle for that option! since you did mention you were working on that))

Thank you for everything, as always!!

An alternative would be a pair of range inputs or dual sliders, with min/max and a checkbox for property (occurrences/ percentage/rank)

I tend to prefer custom filtering inline, especially since a download would already make filtering easyish in a spreadsheet tool, if you plan to do something in bulk offline with it.

Although, very small pagination is a regular limitation I encounter on the site. To me, if anything that is easy to hit the pagination limit had an <all> option or similar in the page dropdown, I'd be very happy.

My 2 cents. For frequency I'm used to just an integer "occurs xx or more times", but as long as its clear what percentage means (percent of what? Total words in the passage?) that's fine too.

I like integers because it's easy to visualize and to exclude words that only occur a handful of times.

Although I had this planned as a feature later on, I was able to link the recent downtime on renshuu to some "bad" processing on text analyzer files. Nothing on the security end, but there is a small chunk of code that can get out of control, and I'm looking to pull it out.

I will most likely make this change in the next day or two, so any bugs can be reported here.

Currently, you can see that when you submit a text for analysis, it gives you back and estimated word count fairly quickly, before processing is done. This is due to a second text parser that is quick but less accurate. It's used for the temporary word count, and more importantly, breaking the text into (roughly) 1000 "word" chunks for easier processing and display in the reader.

This one is purely number-based (as to where it cuts off each chunk), so it can come right in the middle of a sentence.

Instead, I'd like it to make clean breaks at the end of lines. After thinking on it, I think I can get both the server issue and this feature nailed down in one go, but as always, it may adjust the way in which the text is parsed and displayed.

I'm not sure if this was suggested before, but when using "work with this passage" tool it I think would be quite useful to have an option to sort the list by frequency (times in text)

The frequency sort/filter functionality will definitely be added, but I do not believe I'll be able to get it done during this round of updates.