[Syncing] Syncing technologies
The objective of this group is to talk about various syncing techniques ....
as for me personally, I first met with syncing with IOS, and I have to say, that I was hooked! It was really nice to watch, connecting even month later from my notebook, to sync properly what was missing .... However, later on, I found some limitation to such aproach. I don't want to talk about the speed - that could be corrected by using different technique for local storage, e.g. RebDB, but about following two problems:
- red-icons problem - untill particular rambo ticked is resolved, rebol's windows time-zone can't be in any way be regarded as being correct. and it does not depend if it is the bug of the OS, or rebol somehow ommits summertime adjusting flag of OS (if there is any under Windows)
- the problem of IOS syncing is, that it can't work in sync-and-forget mode - simply put, once you delete your file locally, there is no simple solution of how to tell server, that it is still ok
- timestamp based syncing - that is another point - is it really a good idea? If it works, then it works, but not sure about its reliability ....
I would like to post your opinion, links to various sync mechanism we could study, etc. IIRC Kaj talked about Groove techniques, so if it could be reposted, I would be glad ...
mine links are:
rsync - file syncing protocol - http://samba.anu.edu.au/rsync
http://www.opensync.org- synchronisation framework, although imo it solves higher level, e.g. connection to certain target apps via plug-ins etc.
or even better - http://wiki.mozilla.org/Calendar:Home_Page , there is more syncing chapters ....
one Czech product - http://www.mobiledit.com- quite popular, provides SDK etc.
what is to be said the best file-sync for Mac - ChronoSync - http://www.econtechnologies.com
there also seems to be one standard - SyncML - http://www.syncml.org
Because there were some licensing issues with SyncML, it was taken over by http://www.openmobilealliance.orgor so ... it seems to me that OMa DS and DM replace SyncML or so ...
maybe wrong info - SyncML was maybe a bit proprietary, now it seems that oma is developing it as a standard ....
so - enough food for thought, gona read some papers/discussions now ....
Currently my intention is not to sync with various devices. It is simply kind of attempt to improve on IOS syncing, allowing sync-and-forget, or even not syncing (typical remote/server only access) and to improve/support off-line working reblets ...
I would like to have answered: 1) what technique to use for "timestamping" - do we continue with timestamps against one central time, or do we use hashes, or sequencing numbers, or mixture of mentioned techniqueues? We might look how others do it ... 2) better support for possible conflicts - imagine following scenario - you have some reblet, e.g. Contacts - one person starts editing it, then another person starts editing it too. First one syncs (saves changes), then second one does the same - changes of first ones are lost. How to aproach this - introduce some kind of resources locking? (not real locking, but we are message based, so could be queued) It could work as follows - person 1 wants to edit some record. Edit button posts request-for-lock. Lock is assigned. I other person tries to edit, it will not obtain lock. We could even introduce protocol level support, so that the person is informed, who has the lock, and how long. The trouble usually comes, when person goes off-line after the edit started - we need to remove dead-locks, so by default, I would lock for 20 minutes e.g. and the lock would have to be renewed, if person 1 wants to work longer with the given document ... another scenario is, when you actually start editing something which might require locking, but you start already being off-line. We could create lock-request, just not synced yet. Once you go on-line, you simply check seqno, if the lock is possible, and the given record not modified. But what if it was modified in the meantime? e.g. you might be working with stock system and someone else in the office sells few units, for which you may start writing offer for to another customer. Tough scenario - would like to know your opinion. Maybe some things simply need to be done on-line only? 3) I needed small file-sync scenario - could use IOS, but IOS can't sync and "forget". Simply idea is to have different kind of syncing techniques, so e.g. for file transfer you have dir to sync, if correctly synced, log it, forget it, delete it on client and or server (or not, it depends) as for IOS, local storage could be encrypted (or not), imported into RebDB (no single better solution so far introduced for rebol), sync-per-record or record-set could remain (record=document). RebDB on Serve would speed things up significantly too ... another possibility is to think outside the IOS terms, in more general way - simply thinking about world of objects, being in various states, with various life-time around internet and on-line or off-line devices. I think that maybe we could find some simpler solution than SyncML and the likes ... another point - such techniques should be transport independent, so I would not like to hear that it needs this or that ;-) So, anyone?
We have 3) solved in BEER.
Are willing to license it?
1) Hashing is the only way.
2) You either implement locks or smart merging.
Yes. Syncronization of files for multiple clients.
Everybody gets the same files. It is a comercial product.
yes, I might be, if I decide to go for the app ... I will post our company requirements in oportunities group, maybe today evening. Our company failed with big Sap system for file transfer. I wanted to create something upon which business could be based. I will not be selfish - I better publish what could be a good business imo, than doing nothing myself due to lack of free time or the lack of my coding knowledges ...
I have got competition here though, guys try to write it in delphi - it looks ugly and sucks (they would take years to even aproach IOS level), yet they refused to even look into IOS, which would be better fit for our company right fromt he start ...
I see. They could test our tool immediately.
But we be aware if you are looking for all of IOS functionality. We don't have that.
no, absolutly not ... the thing is, we need e-signatures, required by our law :-( ... maybe some command line util could be used? openssl? dunno :-(
e-signatures. No problem. What scheme.
Jaime - is that your commercial app? How different is it from your Beer plus FT profile?
what scheme? We use PKCS7 certificates - qualified certificates ....
I might try to better earn some money, then finding free time and struggle implementing it myself. I could become CZ implementator, if I like the product :-)
If it works, I would LOVE to sell it to my own company I work for - to beat those Sap complex solutions :-)
BEER+FT is just the framework. Not an Application. For example you need to code the gui, path monitoring, filesets management, guid, plus others.
well, if multilanguage, I am in and we can talk privately ...
back in mins ...
In theory, timestamp syncing can be done by asking the server for its current time or time zone, comparing against your own, and then adjusting your timestamps for comparison by the difference. I'm not sure how well this would handle latency issues (we should look at time protocols), but it would handle the client's computer clock being wrong. This would have to be done at the beginning of each syncing session, incase the client moves time zones or otherwise adjusts their clock between sessions.
Better yet would be for both sides to adjust to UTC for comparison purposes.
By "for comparison purposes", I mean don't change the file date, just subtract the diff between you and UTC to get the adjusted file date in memory and then compare against the adjusted file date from the server.
one problem is summertime and windows.
What is the problem with summertime ? When switching to daylight saving, the time zone changes. Eg ours goes from +12 to +13.
the problems is reported in rambo ticket, will find it for you - it can report two zones +1 or + 2 here, and it is a problem - it depends if you skip the time the time is switched on, or if you go thru it ...
it may be bug in Windows, but it needs to be sorted out. With Ladislav we even wrapped some Win32 funcs I found in rebol.exe, but watched nothing obvious ...
Basically the filedates reported by windows are not adjusted to summertime. so after summertime-change the older files are one hour off. sometimes. depends on things like partition-types and dont know what else.