Suggested updates to records

[jerrybrown5]

To all,
I broke down some important store record updates that I am using.


This code allows records to handle events. It also allows you to specify a rowObj inside the meta data to facilitate row level objects without having to rewrite the readers for each different object type.


Ext.apply(Ext.data.Record.prototype, Ext.util.Observable.prototype);
Ext.data.Record.create=Ext.data.Record.create.interceptResult(function(recordtype, fields, meta){

var tableDef;
if (meta && meta.rowObj && meta.rowObj.prototype ){
Ext.apply(recordtype.prototype, meta.rowObj.prototype);
}

var returns=Ext.extend(recordtype, {
constructor: function(data, id) {

this.addEvents( "update", "remove" );
recordtype.superclass.constructor.apply(this, arguments);

if (!this.id){ this.id = (id || id === 0) ? id : ++Ext.data.Record.AUTO_ID };
if (!this.data){ this.data = data; }

}
});

return returns;

});
The following code makes sure that the readers pass in the metaData when they create a new record.

Ext.data.XmlReader.superclass.constructor=Ext.data.JsonReader.superclass.constructor=Ext.data.DataReader = function(meta, recordType){

this.meta = meta;
this.recordType = Ext.isArray(recordType) ?
Ext.data.Record.create(recordType, this.meta) : recordType;

return;
};

Ext.data.JsonReader.prototype.read=function(response){
var json = response.responseText;
var o = eval("("+json+")");
if(!o) {
throw {message: "JsonReader.read: Json object not found"};
}
if(o.metaData){
delete this.ef;
this.meta = o.metaData;
this.recordType = Ext.data.Record.create(o.metaData.fields, this.meta);
this.onMetaChange(this.meta, this.recordType, o);
}
return this.readRecords(o);
}
This code is not clean because it doesn't check to see if the data has changed, but it does fire the update event on the row. This facilitates easier synchronized db updates.

Ext.data.Record.prototype.set=Ext.data.Record.prototype.set.createSequence(function(name, value){

this.fireEvent('update', this, name, value);

});
Now that we have row level functions, this adds something that can use it...templates on the DataView.


Ext.DataView.prototype.prepareData = function(data, index, record){

if (record.events){
var dataCopy={'_record':record};
for(var key in data){
dataCopy[key]=data[key];
}
return dataCopy;
}
return data;
};
A required function to help with these overrides

Function.prototype.interceptResult = function(fcn, scope){
if(typeof fcn != "function"){
return this;
}
var method = this;
var interception=function() {
var retval = method.apply(this || window, arguments);
var callArgs = Array.prototype.slice.call(arguments, 0);
var args=[retval].concat(callArgs);
var newRetval=fcn.apply(scope || this || window, args);
return newRetval;
};
if (this.prototype){
Ext.apply(interception.prototype, this.prototype)
if (this.superclass){ interception.superclass=this.superclass; }
if (this.override){ interception.override=this.override; }
}
return interception;
};
Alright, I hope these updates work as well for everyone else as much as they helped me.

Best regards,
Jerry Brown

[jack.slocum]

This goes against the general principal of the design of the record implementation. The design considers that "Record"s are lightweight objects that can be replaced at any time. In a store, the same record can be added and remove over and over (e.g. with paging) and subscribing to events on an individual record object would then be useless. The Model (or Store) though will be perpetual and can provide proxying for all record related events. This is why it is used for that purpose.

Also worth noting is that by having the events on records, instead of on the store, you lose the ability to listen for additions and removal - two critical events.

You could define events in both places, but I think this just adds another level of complexity (listen in 2 spots instead of 1), processing (more events to fire) and a heavier implementation (observable is heavier than the lightweight record implementation).

I have been using a similar pattern for server-side Java development for quite some time (singleton Models with events, lightweight data objects) and have had great success. Whatever issue you are bumping up against, there is a solution available. ;)

[jerrybrown5]

Jack,
You make a good point about the events on the records. It was purely a convenience method. The 2nd purpose of the updates was to extend an object onto the record prototype. This allows methods to be called directly from it or even from an XTemplate from within dataView that is referencing it. The only requirement to do this was to make sure that whenever a record was created the metaData was passed as a parameter. (The metaData contains the object prototype.) I have used this method with nice success so farm but what are you thoughts on this one?

Best regards,
Jerry Brown

[jack.slocum]

interceptResult looks useful, but I wonder if there might be a better way to do it without wrapping the original function. I'm not sure what you are doing there. Can you explain a little further?

[jerrybrown5]

interceptResult looks useful, but I wonder if there might be a better way to do it without wrapping the original function. I'm not sure what you are doing there. Can you explain a little further?

The purpose of interceptResult is a function prototype method when you need to override the output of a function. The last if portion of it I do not have a need for in production. I coded it purely to make a simple backwards extend-esque feature.

Also, perhaps the code below should use an extend command, but the effect of it is the same. It allows records to take on prototype functions that would typically contain business logic. Functions at this level allow great manipulation to be embedded. For example I have a template function that gets embedded across all of my recordtype objects. This particular function coupled the prepareData override allows me to call record level templates from within the primary drop down box's template very easy; however, this is only one quick example. I have taken this logic and concept much further with great success by integrating it with a form/template repository that is organized by table objects that represent real tables in my database server.


var tableDef;
if (meta && meta.rowObj && meta.rowObj.prototype ){
Ext.apply(recordtype.prototype, meta.rowObj.prototype);
}

[jclawson]

I have a situation where the destroy-all pattern in the store doesn't really work. Granted we are really pushing beyond what Ext was made to do... but hear me out on this. We implemented this with modified version of store that merges record changes instead of replacing them.

We implemented a CRUD proxy to allow creating, reading, updating, and deleting records in batch. You can send a single request that does any/all of the crud actions on any number of records you choose.

The server side code always returns the latest data for each record to update the client side record with. The issue here is that you might not perform the action on ALL records. So, you will only get back data for a handful of records that are in the store.

We cannot return all the data. In many cases the data on the server has over 20,000 records and the user only needs to see a handful at a time (maybe 20). What they see is based on where they are navigating in a tree panel. (Tree expansions load records on-demand)

The code depends on the reader to read the JSON encoded data from the CRUDProxy. This means Store.loadRecords will be called. The standard implementation of loadRecords simply removes all of the records from store, and adds them back. I modified loadRecords to optionally merge the records in with what is currently existing in the store. It simply tries to match up the id fields and updates that data from the new record to the existing record. If it cannot find an existing record it adds it.

Also, we have sub-stores. Stores that exist within a record field. This means that you can be watching for events on a substore--- ie have a grid displaying a sub-store. If you were to simply replace the record, these observers would be invalidated with the overwritten sub-store.

What we have done here, I think, is beyond what you envisioned for store but is absolutely necessary for our complex application. Data exists in many different places in the same interface. We needed a way to synchronize all the changes of this data and the new store allows us to do things like: Tree's with multiple node types based on store's with all node manipulations controlled by store operations. Joining stores similar to SQL joins-- bringing data from one store over to another based on some relationship. CRUD actions on stores and sub-Stores and sub-sub-sub-Stores and easily saving and processing the data on the server side in an incredibly efficient manner. We can also do something like... Reloading the data on an individual record. This is incredibly useful for us as nearly all the data in our interface is based on records.

I have also implemented a form of an Observable record. We do not use this type of record in all cases. We only use it when we need the Observable functionality. We have tabs based on records that get opened. It is quite nice to be able to watch for update events on the record rather than the store and check if the update event on the store is for this record. Additionally when update events fire on the record, I pass over the modified object so that an observer can see what fields were modified. (This turns out to be quite useful) <- perhaps you can implement this on the store update,edit events?

Thats my 2 cents. As I said the merge functionality is optional. Our store works just like the Ext.data.Store unless you use the CRUDProxy or you specifically call loadRecords with the merge:true flag. I would like to see this in Ext if only so I don't have to support it as new Ext versions come out.

[jack.slocum]

My suggestion for this is why not update the existing record, rather than replacing it? e.g., I have a block of code in an next version of Ext app that looks something like this:


Ext.data.StoreProxy = {
add : function(storeId, rawData){
var store = Ext.StoreMgr.get(storeId);
var r = store.reader.readRecords(this.fixData(rawData));
store.add(r.records);
},

load : function(storeId, rawData){
var store = Ext.StoreMgr.get(storeId);
store.loadData(this.fixData(rawData));
},

insert : function(storeId, index, rawData){
var store = Ext.StoreMgr.get(storeId);
var r = store.reader.readRecords(this.fixData(rawData));
store.insert(index, r.records);
},

remove : function(storeId, rawData){
var store = Ext.StoreMgr.get(storeId);
var r = store.reader.readRecords(this.fixData(rawData));
if(r.records){
var o = r.records[0];
var existing = store.getById(o.id);
if(existing){
store.remove(existing);
}
}
},

clear : function(storeId){
Ext.StoreMgr.get(storeId).removeAll();
},

update : function(storeId, rawData){
var store = Ext.StoreMgr.get(storeId);
var r = store.reader.readRecords(this.fixData(rawData));
if(r.records){
var o = r.records[0];
var existing = store.getById(o.id);
if(existing){
for(var k in o.data){
existing.data[k] = o.data[k];
}
existing.commit();
}
}
},

fixData : function(data){
return Ext.isArray(data) ? data : [data];
}
};

Ext.Direct.on({
'storeadd' : function(e){
this.add(e.target, e.data);
},

'storeload' : function(e){
this.load(e.target, e.data);
},

'storeremove' : function(e){
this.remove(e.target, e.data);
},

'storeupdate' : function(e){
this.update(e.target, e.data);
},

'storeinsert' : function(e){
this.insert(e.target, e.index, e.data);
},

'storeclear' : function(e){
this.clear(e.target);
},
scope: Ext.data.StoreProxy
});


This is streaming updates to the store from Ext.Direct server-side events and applying the updates, deletes, etc on the fly. I guess what I am trying to express, is you don't always have to "load" the store, there are manipulation functions on the store which can be used.

[jclawson]

By merge I meant "update the existing record rather than replacing it." The way you have done it could be modified to work in my case.

I think I am doing something a little different than you. I think you are migrating changes from the server to the client? I am migrating changes from the client, to the server. What I have done allows you to:

You have an editable grid. You can add, edit, update, and read (reload record data) records. When you add a record it immediately adds it to the store so it is visible in the grid, and can be edited later. The record.id of the newly added, un-saved record is set to an auto-decrementing id < 0. When you save the store, the CRUDProxy gets a the list of records that are created, read, updated, and deleted. It sends all this data to the server. The server returns the latest record data (in the same order as it was posted) back to the CRUDProxy. The CRUDProxy is able to match up the created records that currently exist in the store to the data the server sent back. It passes the existing records to loadRecords as an option so the merge functionality can match them up and update the record.id's to the server generated id's.

If I had to do it over I would probably put the functionality into a special HTTPProxy like you have and had my CRUDProxy extend that. I will admit that this functionality was hacked together... and then hacked some more... and then hacked more yet again... and then again to support an ever increasing need to synchronize data around our app.

Given the amount of data we have, it becomes more efficient to have an observable record rather than 100 different listeners on store trying to figure out if they care about an update event that was fired.

[jclawson]

Ah... I just remembered why I didn't implement it in the proxy. The proxy isn't aware of the store so I can't really access the store from it. I suppose I could have coupled them together but this tight coupling kindof goes against the design of HttpProxy. So... by implementing the functionality in loadRecords I maintained the design integrity of Ext which... perhaps... is arguably a bad design decision.

Perhaps proxy should be aware of store.

Would you ever have the same proxy attached to more than one different component... with a good reason to do so? I can't think of any good reasons off-hand... so maybe proxy should be aware of the component it is "attached" to.

Thoughts?

[jerrybrown5]

Brian or Jack,
Congrats on the 2.1 release. The website redesign and new samples look great.

Is it possible to add the store's/reader's meta information as the second parameter when creating a new record? It shouldn't slow anything down when not used but this is the key information that you need if you want to have a more intellient generic override of the Record.create method.

This would spare me the painful --and ugly-- task of having to override all of the readRecords functions on every new update. Thanks for your consideration.

Regards,
Jerry

[jerrybrown5]

Jack and developers?
bump?! Are you feeling it for this last minor request?
Thanks in advance.
Jerry

[jack.slocum]

The field data is available on every record as record.fields. ;)