[Solved] Datatable search performance
-
I have a datatable and around 23K rows. Page size is 50, and rendering time good.
My problem: search is very slow (compared to other datatable components from different provider). Is there any way to improve search performance? -
web worker? something like:
https://github.com/hbi99/defiant.js
it gives at least x10 speedup in json search.
(I assume you have 23K rows in memory not on the server)
-
you should consider doing proper pagination and backend search… 23k rows dont really belong into the browser…:)
-
What Max is saying is that you should consider distributing your concerns and leveraging the things that different parts of your stack are good at. The browser is good for interacting with some data. The server is good for crunching huge databases. Instead of dumping 50,000 records to the browser (which is also a traffic hit, btw) you should use a request and response approach to get what you want from the data storage.
You could also avoid JSON entirely and make sure that your objects and arrays are densely packed (as in no holes).
-
Thanks for your responses. Right now, I don’t have any problems renering the datatable with 23K records, and data I get from server is compressed, so it is pretty fast. Plus, I need this feature for a controlled scenario in my intranet.
So, basically, I am force to find a way to make search faster.As qyloxe correctly assummed , data is in memory, so his suggestion might be a good way to handle search, but I had no luck making it work correctly to search string values in all my columns. I need exactly the same functionality as integrated search… I just need it to be faster… I tried the same amount of records in a third party grid component, and search is a lot faster… I need to find out how their integrated search works, and port it to quasar database search…
This is the structure of a typical array (I want to search a string value in all of my columns, and return result to datatable…
[{“Account_Invoice_ID”:“490248”,“Account_Invoice_Type”:1,“Account_Invoice_Date”:“2018-08-13T00:00:00.000Z”,“Account_Invoice_DueDate”:“2018-08-14T00:00:00.000Z”,“Account_Journal_ID”:3,“Partner_ID”:820,“Partner_RUC”:“1790017478001”,“Partner_Name”:“3M ECUADOR C.A.”,“Partner_Name2”:“3M ECUADOR C.A.”,“Account_Invoice_Status_ID”:3,“Account_Payment_Term_ID”:2,“Account_Invoice_Comments”:“fdgdfgdfgfd fd\nfdg\ndfg\ndf\ngdf\ngdf\ngfdgfdg\ndf\ngfd\ng”,“Creation_User”:“LUIS”,“Creation_Date”:“2018-08-13T13:36:46.883Z”,“Last_User”:“LUIS”,“Last_Date”:“2018-08-13T13:36:46.883Z”,“Company_ID”:1,“SRI_Tipo_Comprobante_ID”:"04 ",“Document_Number”:“304-323-000000066”,“Document_Authorization”:“88888888888888888888”,“Retention_Number”:null,“Retention_Authorization”:“0”,“Retention_Move_ID”:“809151”,“Amount_Untaxed”:5,“Amount_Tax”:0.6,“Amount_Total”:5.6,“Amount_Paid”:0.58,“Amount_Unpaid”:5.02,“Account_Move_ID”:“809150”,“Account_Move_Date”:“2018-08-13T00:00:00.000Z”,“Allow_Edit”:true}] -
Really? Full text search? Like “RUC” is what the user types and then it should iterate through 30 fields across 23000 records and filter for the ones it finds? Well, I have some bad news for you.
If any single value in your array contains a null (guess what - I found one in your row sample) then the javascript runtime will downgrade your array to HOLEY_ELEMENTS status, which is slower than a room full of rocks. The problem is you don’t understand your own data and jump to the conclusion that the framework is the problem.
I am really sorry to tell you this, but your data model is the MAIN ingredient - over which the framework cannot and should not ever have any control over. If you haven’t read the javascript spec, then now is the time. Seriously.
But I highly doubt you will bother to read it, so to give you something that will marginally improve your day, here is a blog entry and a video: https://v8project.blogspot.com/2017/09/elements-kinds-in-v8.html
Good luck.
-
So, based in all your comments, I changed my mind and start using server-side… It looks promising and performance is a lot better (searching and initial render also, because fewer records are sent)…
Now, I need to make some changes to my SQL procedures, but already found a lot of examples out there…Thanks for all your help!
