Mike Downey had once said, “I was even asked by one team of developers how they could solve a performance problem they encounter when loading 700,000 complex data records into the Flash player via Remoting! OMG!”, at the MAX India 2004. Well, I was also there and I tried to reply with a rather surprised notion of, “Why so much record at a time?”.
The instant thought that came into my mind was break it up. Anyway, as we are talking on a superficial manner, I was unable to help the developer totally solve it. Today, while I was discussing a project with one of my developer friend, the same topic came up again. Why? but why do we need to get that much of record into an application? And we are talking of Flash here, where it is best left to take in things in pieces. And with remoting (the developer told us that he was using remoting), for instance, CFMX remoting have the inbuilt paging option for your recordset, so why not use it? If there is no way to solve the situation in that manner, then I am really doubtful if the application have been architectured well in the first place.
If that team of developer is observing this, you can comment on the same and we can discuss and may be I or other reader would be able to help you.
Don't like it? There are lots of published articles, pick a random one.
Brajeshwar posted this article
on Mon, Mar 29th, 2004 at 4:18 pm
Categorized under Technology and has the following tags
Comments Post Yours
There are 6 responses so far. You can follow any responses to this entry through the RSS feed. You can skip to the end and leave a response. Pinging is currently not allowed.
Two examples come to mind (I was not one of the original people to mention this):
1) I had a project a year ago, FP6, where we wanted to load all the majority of the Universities/Colleges in the United States into a combo box so that 1 could be selected. FP6 didn’t like that many (10k+). I tried building code that would allow you to enter text into a text box and as the 3rd character was entered, the 3 characters were sent to the CFMX back end (SQL 2000), a select where like ‘%var%’ was done, and the records came back. Worked great for most search criteria, but for 3 terms it still sent back too many records (University, College, State), and hence someone using the application had to type a less common term in.
2) Now, a peer has a need to load a combo box (html) with the employees of a large corporation (60k+ users). 60k is taking quite a while to load. It would be nice if there was a way for someone to start entering the name and it to populate the combo box on the fly. Obviously, populating after 1 character is typed doesn’t help much, you may still have 5-10k records. However, 2nd & 3rd characters may be ok and cut the list to 1-2k.
Either way, html and Flash Player do not easily handle more than a few hundred rows. Likewise, if you use a Grid control in Flash, with many columns, it becomes unwieldy quickly.
As far as I’m concerned - this ultimately stems from a UI design problem. The user should never have to wade through 700,000 records, 60,000 records or even 5,000 records (even if they are alphabetized). In these cases there needs to be an intuitive data drill-down interface so the user can be guided into zeroing in on what they are looking for in the first place.
I have found there are always better ways to load data when you think there isn’t. For example in your #1 why couldn’t you have some type of drill down (like Sean mentions) Why would you want them to find their university in a large combobox? You have a couple options. One is to provide a state combo box which would return a combobox with the universities of that state. Or a second option would be to have them type the name and just validate it on the server (if your app was more to browse colleges why would you have a form anyway you should drill down by city/state/type/etc)
Anyway this was a good post and Brajeshwar makes a good point.
At my work our application has to work with very large data sets. We deal with them by logically breaking our data up into smaller chunks and then cashing the remaining pieces on the server for fast retrieval.
Like Sean said even 5000 records is too much. So if you find yourself dealing with that many records at one time on the client, you probably need to rethink your design.
what ever happened to search fields?
i like all the ideas here. and if a group of flash developers were presented with the problem then i’m sure they could figure out a workaround like some already have. but not taking into account the time it takes to download shouldn’t the application be able to handle this? i talk to java users and they do not believe that flash can handle the load. before flash can break out of the animation title and into the application circle it has to be robust. it has to work out and go to the gym. then it gets big muscles and the girls too would come after it. “oh flash you are so strong, giggle”
Post yours