Hi everybody. Today we will be speaking about Webix ways to optimize server-client communication in case of huge data. Those of you who are happy to fetch long datasets at once, please raise your hands!
Just as I’ve expected, nobody’s here. Everybody has gone to dynamic loading which is implemented very easily with Webix data components.
view:"datatable", url:"mydata.php", datafetch:100
});
Initially, a limited number of records (e.g. 100) is loaded, while further requests are triggered by scrolling or paging automatically. Guys, isn’t it nice?
Still, such a pattern requires quite a specific format of a server response – {data:[ /*array of records*/], pos:0, total_count:100 } – and you may ask yourself: “So what should I do, if I cannot tune the response or I don’t know the total length of server-side data?” Indeed, there are lots of web services that do not provide such information.
The article below shows the solution that relies totally on Webix public API and can be built by any of you.
For such cases Webix offers proxy objects that contain custom logic of data loading. The package already includes a number of proxies that serve different purposes, e.g. enable RESTful requests, connect to GraphQL, etc.
In the same way we can create a proxy that implements dynamic loading of an “unlimited” number of records without the need to modify server response.
A proxy object must have $proxy parameter set to true and functions that trigger on loading and saving. Its name is prefixed to the loading/saving url or both. Basically, it looks like this:
webix.proxy.myProxy = {
$proxy:true,
load:function(view, params){ ..},
save:function(view, update, dp){ .. }
};
//use proxy
webix.ui({
view:"datatable",
datafetch:100,
url:"myProxy->mydata.php"
});
You can read more about proxy tuning in the documentation.
Here our goal is to redefine the loading pattern. The widget we work with should:
- load initial data, say, 50 records;
- each time user scrolls till the end of the dataset fetch a new portion from server.
Let’s implement this using Webix library and PHP backend.
Loading initial data
We are tuning data loading, so we need to define the load() method. It adds the count parameter to the loading url to pass the number of requested records to server.
webix.ui({
view:"datatable",
datafetch:50, //the number of records per portion
url:"idata->./data_dyn.php"
});
//proxy configuration
webix.proxy.idata = {
$proxy:true,
load:function(view, callback){
var url = this.source;
url += (url.indexOf("?") == -1 ) ? "?": "&";
url += "count="+(view.config.datafetch || 0);
//url will look like "./data_dyn.php?count=50"
return webix.ajax(url).then(webix.bind(function(data) {
data = data.json();
this._checkLoadNext(data);
return data;
}, this));
},
_checkLoadNext:function(data){
if(!data.length)
this._dontLoadNext = true;
}
};
In addition to returning the fetched data to the widget, we provide the code to check whether the server still returns data. If no data has come – finally, it should come to an end – further requests are senseless, so we set _dontLoadNext flag to true (see its usage below).
Dynamic loading while scrolling
We need to listen to scrolling events in the widget and catch the moment when the dataset is fully scrolled to load the new data:
$proxy:true,
load:function(view, callback){
this._attachHandlers(view);
...
},
_attachHandlers:function(view){
var proxy = this;
if(view.config.columns)
view.attachEvent("onScrollY", function(){ proxy._loadNext(this); });
else
view.attachEvent("onAfterScroll", function(){ proxy._loadNext(this); });
//attach handlers once
this._attachHandlers = function(){};
},
_loadNext:function(view){
var contentScroll = view.getScrollState().y + view.$view.clientHeight;
var node = view.getItemNode(view.getLastId());
var height = view.config.rowHeight || view.type.height;
if(node && contentScroll >= node.offsetTop+height && !this._dontLoadNext){
view.loadNext(view.config.datafetch, view.count()+1);
}
}
};
All scrollable data management widgets have the onAfterScroll event while the Datatable features a specific pair of them, the onScrollY and onScrollX, depending on the scrolling direction. We need to check it before attaching the handler to make our proxy compatible with all Webix data widgets.
The handler function checks whether the last data item is visible at the moment and, if so, calls the loadNext() method that – watch the hands! – passes the number of records and offset position to widget’s load() method as a third argument. And this method is modified by our proxy.
So, all we need to do is to tune the proxy’s load() to process these parameters:
this._attachHandlers(view);
var url = this.source;
url += (url.indexOf("?") == -1 ) ? "?": "&";
var count = params?params.count:view.config.datafetch || 0;
var start = params?params.start:0;
//url will look like "../data.php?count=50&start=51"
url += "count="+count;
url += start?"&start="+start:"";
return webix.ajax(url).then(webix.bind(function(data) {
data = data.json();
this._checkLoadNext(data);
return data;
}, this));
}
Basically that’s all with the client side. Your server script should process the count and start parameters for data queries, and you’ll get the needed response.
You can check Data Loading documentation page.
As you can see, Webix proxies are the way to a better world where everyone can explore their developer creativity without sticking to this or that tool.
As a home task, I suggest you to connect the above proxy to paging so that nothing can stop you from loading dynamically. Feel free to ask questions in the comments. See you!
