r3wp [groups: 83 posts: 189283]
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search

World: r3wp

[Syncing] Syncing technologies

The objective of this group is to talk about various syncing techniques 
as for me personally, I first met with syncing with IOS, and I have 
to say, that I was hooked! It was really nice to watch, connecting 
even month later from my notebook, to sync properly what was missing 
.... However, later on, I found some limitation to such aproach. 
I don't want to talk about the speed - that could be corrected by 
using different technique for local storage, e.g. RebDB, but about 
following two problems:
- red-icons problem - untill particular rambo ticked is resolved, 
rebol's windows time-zone can't be in any way be regarded as being 
correct. and it does not depend if it is the bug of the OS, or rebol 
somehow ommits summertime adjusting flag of OS (if there is any under 
- the problem of IOS syncing is, that it can't work in sync-and-forget 
mode - simply put, once you delete your file locally, there is no 
simple solution of how to tell server, that it is still ok
- timestamp based syncing - that is another point - is it really 
a good idea? If it works, then it works, but not sure about its reliability 
I would like to post your opinion, links to various sync mechanism 
we could study, etc. IIRC Kaj talked about Groove techniques, so 
if it could be reposted, I would be glad ...
mine links are:
rsync - file syncing protocol - http://samba.anu.edu.au/rsync
http://www.opensync.org- synchronisation framework, although imo 
it solves higher level, e.g. connection to certain target apps via 
plug-ins etc.
or even better - http://wiki.mozilla.org/Calendar:Home_Page , there 
is more syncing chapters ....
one Czech product - http://www.mobiledit.com- quite popular, provides 
SDK etc.
what is to be said the best file-sync for Mac - ChronoSync - http://www.econtechnologies.com
there also seems to be one standard - SyncML - http://www.syncml.org
Because there were some licensing issues with SyncML, it was taken 
over by http://www.openmobilealliance.orgor so ... it seems to me 
that OMa DS and DM replace SyncML or so ...
maybe wrong info - SyncML was maybe a bit proprietary, now it seems 
that oma is developing it as a standard ....
so - enough food for thought, gona read some papers/discussions now 
Currently my intention is not to sync with various devices. It is 
simply kind of attempt to improve on IOS syncing, allowing sync-and-forget, 
or even not syncing (typical remote/server only access) and to improve/support 
off-line working reblets ...
I would like to have answered:

1) what technique to use for "timestamping" - do we continue with 
timestamps against one central time, or do we use hashes, or sequencing 
numbers, or mixture of mentioned techniqueues? We might look how 
others do it ...

2) better support for possible conflicts - imagine following scenario 
- you have some reblet, e.g. Contacts - one person starts editing 
it, then another person starts editing it too. First one syncs (saves 
changes), then second one does the same - changes of first ones are 
lost. How to aproach this - introduce some kind of resources locking? 
(not real locking, but we are message based, so could be queued)

It could work as follows - person 1 wants to edit some record. Edit 
button posts request-for-lock. Lock is assigned. I other person tries 
to edit, it will not obtain lock. We could even introduce protocol 
level support, so that the person is informed, who has the lock, 
and how long. The trouble usually comes, when person goes off-line 
after the edit started - we need to remove dead-locks, so by default, 
I would lock for 20 minutes e.g. and the lock would have to be renewed, 
if person 1 wants to work longer with the given document ...

another scenario is, when you actually start editing something which 
might require locking, but you start already being off-line. We could 
create lock-request, just not synced yet. Once you go on-line, you 
simply check seqno, if the lock is possible, and the given record 
not modified. But what if it was modified in the meantime? e.g. you 
might be working with stock system and someone else in the office 
sells few units, for which you may start writing offer for to another 
customer. Tough scenario - would like to know your opinion. Maybe 
some things simply need to be done on-line only?

3) I needed small file-sync scenario - could use IOS, but IOS can't 
sync and "forget". Simply idea is to have different kind of syncing 
techniques, so e.g. for file transfer you have dir to sync, if correctly 
synced, log it, forget it, delete it on client and or server (or 
not, it depends)

as for IOS, local storage could be encrypted (or not), imported into 
RebDB (no single better solution so far introduced for rebol), sync-per-record 
or record-set could remain (record=document). RebDB on Serve would 
speed things up significantly too ...

another possibility is to think outside the IOS terms, in more general 
way - simply thinking about world of objects, being in various states, 
with various life-time around internet and on-line or off-line devices. 
I think that maybe we could find some simpler solution than SyncML 
and the likes ...

another point - such techniques should be transport independent, 
so I would not like to hear that it needs this or that ;-)

So, anyone?
We have 3) solved in BEER.
Are willing to license it?
1) Hashing is the only way.
2) You either implement locks or smart merging.
Yes. Syncronization of files for multiple clients.
Everybody gets the same files. It is a comercial product.
yes, I might be, if I decide to go for the app ... I will post our 
company requirements in oportunities group, maybe today evening. 
Our company failed with big Sap system for file transfer. I wanted 
to create something upon which business could be based. I will not 
be selfish - I better publish what could be a good business imo, 
than doing nothing myself due to lack of free time or the lack of 
my coding knowledges ...
I have got competition here though, guys try to write it in delphi 
- it looks ugly and sucks (they would take years to even aproach 
IOS level), yet they refused to even look into IOS, which would be 
better fit for our company right fromt he start ...
I see. They could test our tool immediately.
But we be aware if you are looking for all of IOS functionality. 
We don't have that.
no, absolutly not ... the thing is, we need e-signatures, required 
by our law :-( ... maybe some command line util could be used? openssl? 
dunno :-(
e-signatures. No problem. What scheme.
Jaime - is that your commercial app? How different is it from your 
Beer plus FT profile?
what scheme? We use PKCS7 certificates - qualified certificates ....
I might try to better earn some money, then finding free time and 
struggle implementing it myself. I could become CZ implementator, 
if I like the product :-)
If it works, I would LOVE to sell it to my own company I work for 
- to beat those Sap complex solutions :-)
BEER+FT is just the framework. Not an Application. For example you 
need to code the gui, path monitoring, filesets management, guid, 
plus others.
well, if multilanguage, I am in and we can talk privately ...
back in mins ...
In theory, timestamp syncing can be done by asking the server for 
its current time or time zone, comparing against your own, and then 
adjusting your timestamps for comparison by the difference. I'm not 
sure how well this would handle latency issues (we should look at 
time protocols), but it would handle the client's computer clock 
being wrong. This would have to be done at the beginning of each 
syncing session, incase the client moves time zones or otherwise 
adjusts their clock between sessions.
Better yet would be for both sides to adjust to UTC for comparison 
By "for comparison purposes", I mean don't change the file date, 
just subtract the diff between you and UTC to get the adjusted file 
date in memory and then compare against the adjusted file date from 
the server.
one problem is summertime and windows.
What is the problem with summertime ?  When switching to daylight 
saving, the time zone changes.  Eg ours goes from +12 to +13.
the problems is reported in rambo ticket, will find it for you - 
it can report two zones +1 or + 2 here, and it is a problem - it 
depends if you skip the time the time is switched on, or if you go 
thru it ...
it may be bug in Windows, but it needs to be sorted out. With Ladislav 
we even wrapped some Win32 funcs I found in rebol.exe, but watched 
nothing obvious ...
Basically the filedates reported by windows are not adjusted to summertime. 
so after summertime-change the older files are one hour off. sometimes. 
depends on things like partition-types and dont know what else.