Sorting table with non-english characters



  • Hello, I am having trouble getting a custom sorting going on the sortable tables.
    I set the local to Turkish, and loaded my data, but the sorting doesn’t function properly.

    Instead of sorting the non-english characters in place, it pushes them to the end of the list.

    I tried to use custom-sort hoping to change anything without luck.
    Any help appreciated,
    Thanks in advance!



  • Just wanted to share an update on the issue, since I can’t fix it myself.
    In Turkish there are non-english characters such as Ç Ö Ğ Ü Ğ Ş İ
    The sortabla table in Turkish should sort a list as follows
    Name:

    • Can
    • Çetin
    • Onur
    • Ömer
    • Samsun
    • Şakir
    • Uzun
    • Üzüm

    But it sorts it as:

    • Can
    • Onur
    • Samsun
    • Uzun
    • Çetin
    • Ömer
    • Şakir
    • Üzüm

    I found some sorting functions but wasn’t successful in implementing it on custom sortable table. Can anybody please help out?

    Thank you very much.



  • Hey Everybody, I came to find a weird ass solution for the situation. Wanted to share it here as well for anybody who experiences similar issues.
    I figured out that the problem was happening because of how Firestore sorts stuff rather than quasar.

    So i ended up sorting by a calculated property that changed strings based on their character:
    For example ‘Can’ became ‘Caan’ and ‘Çetin’ became ‘Cbetin’ so the table didn’t have an issue sorting.
    It is a shitty workaround though…



  • @uzunu said in Sorting table with non-english characters:

    So i ended up sorting by a calculated property that changed strings based on their character:
    For example ‘Can’ became ‘Caan’ and ‘Çetin’ became ‘Cbetin’ so the table didn’t have an issue sorting.
    It is a shitty workaround though…

    It’s quite ingenious in his own style of reinventing the wheel 🙂 Anyway, do try to use localeCompare API instead:

    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/localeCompare

    Here is tutorial:

    https://www.techonthenet.com/js/string_localecompare.php

    It should do the trick considering proper configuration (the sensitivity is tricky).

    Anyway your current method is essentially a unicode normalization. It is standardized even:
    https://unicode.org/reports/tr15/

    More here:
    https://en.wikipedia.org/wiki/Unicode_equivalence

    In javascript normalization is available here:

    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/normalize

    so for example string ąćę could be converted into ace.

    Hope this helps.



  • Thank you for the reply. I did try localeCompare before reinventing the wheel:)

    The problem was how Firestore sorts by default, for which I have no way of doing a localeCompare unless you are getting the whole array, which wasn’t the case for me.

    Think of a collection of 190 documents that I’m paginating 20 by 20. The alphabet goes like a,b,c,ç,d,e,… but firestore when you query with sort, it queries as v,y,z,ç… which is why I had to do the wheel invention 🙂

    Here is where I got inspired from
    https://stackoverflow.com/questions/48366384/firestore-how-can-i-correctly-order-by-string-with-accent-characters



  • @uzunu said in Sorting table with non-english characters:

    The problem was how Firestore sorts by default, for which I have no way of doing a localeCompare unless you are getting the whole array, which wasn’t the case for me.

    Think of a collection of 190 documents that I’m paginating 20 by 20. The alphabet goes like a,b,c,ç,d,e,… but firestore when you query with sort, it queries as v,y,z,ç… which is why I had to do the wheel invention 🙂

    well, thats why I do prefer to use some “real” 🙂 database on backend - you’ll never know what a surprise will you get with such an exotic datastores.

    If you are using q-table and get data from API/store, then it seems, then you need to load those data and sort them before display with your own sort order using proper locale. There is a method sort(col) => void 0 in QTable API:

    https://quasar.dev/vue-components/table#QTable-API

    43ffd4bf-158c-4591-a25b-4cd45aa5600c-image.png



  • One other problem that happens when I load an array-table with more than 100 items in it is that the page starts to get unresponsive and sluggish. I don’t understand how in the year 2020 web browsers struggle to load 200 items from a database to memories of gigabytes. I understand this might be happening for a bunch of reasons, but I still fail to figure out how to not get the browser to slow down.
    On some pages there are things that get loaded from database in arrays of objects, and I am assuming that when user navigates away from the page they get deleted from the memory.

    I read about memory leaks, and to be honest I might not be understanding the concept as well as one should have. But it would be great to know of a solution to at least make sure the things loaded in the database don’t get compunded as the user uses the app.

    o read about on destroy, but don’t think the components are destroyed when user navigates to another route with a different component.
    Any suggestions?
    I know that this became unrelated to the original subject, just wanted to ask since you seem to know a lot more than me and I like your snarky comments:)



  • @uzunu said in Sorting table with non-english characters:

    One other problem that happens when I load an array-table with more than 100 items in it is that the page starts to get unresponsive and sluggish. I don’t understand how in the year 2020 web browsers struggle to load 200 items from a database to memories of gigabytes. I understand this might be happening for a bunch of reasons, but I still fail to figure out how to not get the browser to slow down.

    well, your default code is running in one thread with all the rest of the browser code - DOM parsing, drawing, updating, event handling. If you want “fast”, you need to use web workers. Something like this (there are many of such solutions):
    https://forum.quasar-framework.org/topic/1695/web-workers-made-easy

    Anyhow, if you process 100-200 items, then I would strongly argue, that it is something in your code. Possibly you are making blocking calls (the most probable cause), ignore asynchronous code where it should give control to the browser or something like this. In modern browsers, in 2020, I assume, that the problems should be visible at about 2000 generic items on one page. If you have problems with less, then it is very (much very) probably your code.

    On some pages there are things that get loaded from database in arrays of objects, and I am assuming that when user navigates away from the page they get deleted from the memory.

    you need to check this in developer tools. The tooling is quite ok in 2020. You need to REALLY learn how to use that, it’s beneficial. For example how to connect debugger, how to profile, how to record sessions, how to analyze EVERY single piece of information the developer tools give you. EVERY single number has its own meaning and helps you somehow.

    I read about memory leaks, and to be honest I might not be understanding the concept as well as one should have. But it would be great to know of a solution to at least make sure the things loaded in the database don’t get compunded as the user uses the app.

    Memory leaks are quite easy to detect - just use developer tools, run your program and it will just show you where problems are.

    o read about on destroy, but don’t think the components are destroyed when user navigates to another route with a different component.
    Any suggestions?

    It’s possible - use developer tools to find those objects, check the garbage collector, check what takes the most time during updates (I assume you do not load/process data in visual update handlers?).

    I know that this became unrelated to the original subject, just wanted to ask since you seem to know a lot more than me and I like your snarky comments:)

    You expext a lot from me - snarkiness is an art and needs a lots of love and quite precise timing. It’s not for sale or showing off on random request! 🙂

    Yeah, from my observations, the time to really grasp all those questions is about two years. You just need to write a lot of code, check everything and use available tools. Everybody thinks javascript and frameworks and browsers are easy. Well, let me agree to disagree.

    Ach, and about data processing there’s always local data processing such as indexeddb for example:
    https://dexie.org/

    … and many more - for amusement you can check this daily:
    https://libraries.io/search?q=vue&sort=latest_release_published_at



  • Thank you for the time you spent to write this reply. I appreciate it and it’s very useful. I only use developer tools and the vue extension to make sure all arrays got populated and whatnot.
    The sluggishness I get is when I load 200 items on a q-table

    Also when I open firebase database admin; if I have a document that has an array of a thousand items it gets very very unresponsive (i think by design they want to discourage people from doing that) but just telling it because I don’t think it’s not just my code.

    I will definitely learn more about web workers, how to do garbage collection and will try to figure out how to detect memory leaks. The only resource I came across was the task manager in chrome to check it with this example:
    https://codepen.io/freeman-g/pen/qobpxo
    but i am not there yet, to understand it i guess…
    My takeaway from your message is learn these:
    “For example how to connect debugger, how to profile, how to record sessions, how to analyze EVERY single piece of information the developer tools give you. EVERY single number has its own meaning and helps you somehow.”
    And those I will do sir.
    Thanks for your time, and I would agree to disagree on snarkiness being an art, it’s a way of life for some, it’s a hide out for others, and it’s just fun for some, I think for you it sounds like it’s more in the fun category.
    Thanks



  • @uzunu said in Sorting table with non-english characters:

    The only resource I came across was the task manager in chrome to check it with this example:
    https://codepen.io/freeman-g/pen/qobpxo

    Strange. There’s hundreds of valuable resources available for each experience level. Try this:

    https://www.google.com/search?lr=&hl=en&as_qdr=all&q=chrome+developer+memory+leak+detection

    Well, I don’t know all the categories of snarkiness in the world, yet for clarifying, my cultural background identifies this term with Monty Python art. Something like this:

    https://books.google.nl/books?id=olmCDwAAQBAJ&pg=PT7&lpg=PT7&dq=monty+python+snarkiness

    Ach, and let me just emphasize - slow browser (as in sluggish visual user experience) - the primary thing to check is broken asynchronous code. The visual updates can not be bound to a costly IO/cpu handler in a synchronous manner. As I think of this, that is exactly my main reason for loving Vue and reactivity as a systems architecture. Traditional event based systems (in browsers) are very easily prone to sneaky long computational functions. With reactivity, you just concentrate on simple, proper, asynchronous data preparing/transforming/loading and the observable pattern will do the rest in the most optimal way for DOM nodes. This optimal solution strongly depends on: proper keying of virtual DOM nodes (it thrives with cache), proper asynchronous “callback/await” dance (web workers are heaven), and of course on not abusing what’s in Vue<->browser mechanics (don’t touch the DOM, easy with CSS, prefer prerendering - in fact everything with “pre” prefix is good - prefetch, preload, prequeue, …). From the q-table perspective, yes, this component has a LOT of optimization potential, BUT, as I said, proper user code should give you the smooth user experience you want, even for hundeds/few thousand items. Keep testing, start with the really minimal and hard coded JSON 2000 items and just see if it is ok, then add some visual effects for columns, rows, then add loading, then user behaviour etc. At some change point, you will feel the difference, and you will have your “why?”.



  • I love how Vue solves bunch of problems in such a smooth way, and figuring it out more is very exciting for me. Thank you for your comments. We might have similar definitions of snarky.
    10bfaf839e170054411ad4551f84dc43.jpg
    I’ll go get some shrubbery. Have a great day and thank you for your help and guidance.


Log in to reply