My last two posts have been about dramatically speeding up how you serve large, complicated data sets to your users. As described above, serving the data was now quick, but rendering the data in the browser took a full 6 minutes; I timed it, repeatedly. For reference, this was done on a current generation MBP, so the CPU is no slouch (i7 4770HQ @ 2.2ghz). The issue here is that multiple MB of HTML were being served up in a report to the browser as a DataTable. I detailed the caching process in my earlier post, but the gist is that each
<tr>...</tr> worth of data is saved in a solr cache for each individual row of the report. These rows are styled, with each individual
<td>...</td> also having styling. That adds a significant amount of processing time to display data to the user.
Clusterize solves this exact problem. Check out the demo on their page to see it in action. The basic mechanism is that it takes an array of data (as table rows, each row is one element) and only renders the ones you can see at any given time. Absolutely perfect for our needs, since no matter how big a user’s screen is, they’ll only be displaying a tiny fraction, 0.1% absolute max, of the report at a given time. Perfect for our needs.
Setting up our app to use clusterize will involve the following steps:
Don’t forget to add them to your precompile list in
initializers/assets.rb if needed.
This model exists solely to retrieve the prerendered contract data as an array from solr given a solr search on
# app/models/admin/reports/contracts.rb class Admin::Reports::Contracts def contract_data(solr_search) contract_rows =  solr_search.hits.each do |hit| contract_rows << hit.stored(:mar_partial) end contract_rows end end
# config/routes.rb namespace :admin do namespace :reports do resources :contracts, only: [:index] end end
# app/controllers/admin/reports/contracts_controller.rb class Admin::Reports::ContractsController < Admin::BaseController def index search = DealJacket.solr_search do with(:contract_status, params[:contract_status]) if params[:contract_status].present? #If no filter, include all paginate(per_page: Contract.count) end render json: Admin::Reports::Contracts.new.contract_data(search) end end
This simply takes a search param to scope down contracts (if provided) and renders the json data out. Try visiting the route in your browser, if you see a screen full of raw json text, it’s working as planned.
Coffee/JS is definitely my week point, so I’m sure this code can be improved, but it works great. What this does is the following:
windowfor later usage
The main points here are the following:
#contentArea.clusterize-content, this tells Clusterize where to manage the displayed rows
All together, this loads the page with static table info (that “Loading Data…” row), and pulls your table rows into a new Clusterize object. As I mentioned, before clusterize, start to finish on a fast computer it took at least 6 minutes (that’s 360 seconds) to load and render the data so that it was usable. Now, it’s done consistently under 2 seconds, every single time. Hooking this all together made displaying a backend optimized report 180x faster.
I’ll call that a good accomplishment for the day.