Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

Threading continued

 [1/17] from: maarten:vrijheid at: 29-Jan-2004 17:05


Hi, I have got some code and some questions. I have a working version of 'eval a function that takes a block of code and evaluates it like 'do. Big deal? Well... it responds to a 'rest word, giving you the restof the block to be evaluated and the value of the last evaluated expression. So: eval [ 10 + 10 join "a" "b" rest 20 print "howdy" ]
>> [ "ab" [ 20 print "howdy"]]
Note that the second item (the rest of the code) is the position in the original code, so a ' head on that gives you the original code back. Anyway.... this interruptable evaluator run on a block with 10.000 now invocations 3.5 times slower than do. That's slow, but probably fast enough for most things. And Moore's law and all... On top of such an eval function you can easily build a co-op threading engine. Instead of doing a script you would 'eval it, making it four times slower but concurrent. And it would be easy to add all sort of utility threads out-of-the box for newbies, like a dns reader thread that returns the value to a function on completion etc. Question: is this a thing any of you want despite the performance hit of 400% ? It is easy for me to get there, but I need to know if there is anybody who'd like to see it available. Thanks, Maarten<

 [2/17] from: maximo:meteorstudios at: 29-Jan-2004 11:37


good question, can you guarantee that one crashing thread (just an evaluation error) can never stop execution of the other threads, and allow some sort of signals to warn you that a thread has dumped? Allowing you to respawn it? -MAx --- You can either be part of the problem or part of the solution, but in the end, being part of the problem is much more fun.

 [3/17] from: maarten:vrijheid at: 29-Jan-2004 18:41


Yes, I can do all that (although it is good somebody reminds me of it ;-))) But is it worth the effort? --Maarten Maxim Olivier-Adlhoch wrote:

 [4/17] from: maximo:meteorstudios at: 29-Jan-2004 13:31


I think I know exactly how you feel. as an example, liquid, one of my brainchilds took a lot of time to mature (and its about to undergo a new round of refinemnet). It exists, and was shared and is available to anyone who asks for it (its now in slim format, just not yet released)... liquid is really usefull, if you "get" it. Yet was it worth the "release" effort... I still don't know... I didn't get much feedback. I don't even know if anyone else is using it... probably not. so as for your tool. as much as I'd like to say to python users that I can thread executions of scripts (and do it better than them, cause they can't remotely kill threads), at this point, I wonder how many tools really need that functionality. Who writes servers in rebol... not many of us... How many people even use threads in python... not that whole lot either. I've had to, they (threads) are usefull, but with rebol's smaller user base... even if 1% of us used(needs) it, you'd still have a handfull of users, and many would use it just to see if it works... and if it goes 400% slower... not sure that's fit for server usage in all circumstances... maybe someone who wants to create a networked front-end for RebDB might want to introduce threading... but will it help go faster in the end, if all threads run on one cpu (even in a multi-cpu machine...). Tough question indeed. I hope you get people which can give you a definite yes, giving you some push towards putting time on it. It looks like a fun challenge you want to put yourself against. :-) cheers! -MAx --- You can either be part of the problem or part of the solution, but in the end, being part of the problem is much more fun.

 [5/17] from: jvargas:whywire at: 29-Jan-2004 13:45


I think it is worth. It will help with some applications that are doing a lot of different things all the time. It is kind of nice to invoke a thread let it run in the background a yield back to the gui and only waiting for completion of the thread. Don't know about performance though. If it is too slow people may choose a different approach to their code. Cheers, Jaime -- The best way to predict the future is to invent it -- Steve Jobs On Jan 29, 2004, at 12:41 PM, Maarten Koopmans wrote:

 [6/17] from: gchiu:compkarori at: 29-Jan-2004 20:04


Maarten Koopmans wrote.. apparently on 29-Jan-2004/18:41:39+1:00
>Yes, I can do all that (although it is good somebody reminds me of it ;-))) > >But is it worth the effort?
That is up to you to decide, but if you do do it, I would certainly use it. -- Graham Chiu http://www.compkarori.com/cerebrus<

 [7/17] from: chrismorency:videotron:ca at: 29-Jan-2004 14:56


> Jaime Vargas wrote: > -- The best way to predict the future is to invent it -- Steve Jobs
Sorry if this off-topic and seems like neat-picking, but I have way too much respect for inovators in computer history to let this through ;) (Those do not include (jobs/gates/linus)). Amongs said innovators: Alan Kay and Carl Sassenrath. So in order to restore historic fact, the quote is not by Steve Jobs, it's from Alan Kay, member of the Xerox Palo Alto Research Center (PARC) research team responsible for : Network computing as we know it today. GUI and Mouse interactivity as we know it today. Smalltalk and OOP as we know it today. The story of how Jobs and/or Gates were inspired by PARC can be read at http://www.smalltalk.org/alankay.html Best Chris

 [8/17] from: andreas:bolka:gmx at: 29-Jan-2004 21:48


Thursday, January 29, 2004, 5:05:11 PM, Maarten wrote:
> I have got some code and some questions. I have a working version of > 'eval a function that takes a block of code and evaluates it like > 'do.
Great! (And you have an interesting timing Maarten ... but that's another story ;)
> Question: is this a thing any of you want despite the performance > hit of 400% ?
You should know my answer: YES!
> It is easy for me to get there, but I need to know if there is > anybody who'd like to see it available.
Yep, I'd definitely like to see it available. If you decide to not release it, a personal mail would be fine too ... :) -- Best regards, Andreas

 [9/17] from: andreas:bolka:gmx at: 29-Jan-2004 21:50


Thursday, January 29, 2004, 7:31:50 PM, Maxim wrote:
> Who writes servers in rebol... not many of us...
I do. Although REBOL tries to punish everyone who does. So maybe that's another reason why not many write servers in REBOL - it's simply missing crucial features for some scenarios. -- Best regards, Andreas

 [10/17] from: jvargas:whywire at: 29-Jan-2004 15:58


Thanks Chris for your correction, it great to have well informed people in this list! Cheers, Jaime -- The best way to predict the future is to invent it -- Alan Kay On Jan 29, 2004, at 2:56 PM, Christian Morency wrote:

 [11/17] from: jason:cunliffe:verizon at: 29-Jan-2004 16:44


> Question: is this a thing any of you want despite the performance hit of > 400% ?
I think the answer should be YES. But I honestly do not really understand what is at stake here. I am not a server developer. But like many am a server user... So what does this do for Rebol that it is missing now? Can anyone please explain what this proposed threading solution would do to improve speed of using Vanilla? http://www.langreiter.com/space/Vanilla Or for creating an XML socket server like Unity? http://www.moock.org/unity/ Can you briefly describe any use-cases where this threading would make a difference . thanks very much - Jason

 [12/17] from: atruter:labyrinth:au at: 30-Jan-2004 9:54


Hi Maarten,
> Question: is this a thing any of you want despite the performance hit of > 400% ?
In my case, it depends on "where" this performance hit is. Take the following example: do [ init-func big-loop-func end-func ] Now, if 'do takes 0.001s to interate through these calls and your 'eval func takes 0.004s that's OK. If 'big-loop-func takes 10s to evaluate, but the use of 'eval blows this out to 40s then we have a problem. So I suppose my question is, once 'eval hits a func call does it effect how long it takes to evaluate that function in and of itself?
> It is easy for me to get there, but I need to know if there is anybody > who'd like to see it available.
Depends on answer to above, but I've got a couple of interesting things I could do with this (multi-threaded listener, background IO writers, etc). Regards, Ashley<

 [13/17] from: maximo:meteorstudios at: 29-Jan-2004 17:58


> -----Original Message----- > From: Jason Cunliffe [mailto:[jason--cunliffe--verizon--net]]
<<quoted lines omitted: 4>>
> would make a > difference .
one example for threading useage is that you can crunch information WHILE you are also waiting on a port. this basically allows your server to be up all the time, although he is busy with any matter of processes. when creating a render queue or a transaction-based system, you can tell the system to do things, and he won't do them right away, but you can still send the commands and receive confirmations that your processes have been done. then if you need information, another thread, can lookup the processing queue and verify if there is yet some processing to be done on that item. if there is then it either processes it right away in its own thread, if possible, before returning the new and improved data. this doesn't keep the other thread from continuing to crunch the other items in the list while they are not needed. if you didn't have threads, then you'd have to wait for the current processing to finish before your server even responds to any new request. threading allows your server to be more responsive, altough it also slows down each of those responses as they process simultaneously. As maarteen explained, if threading is implemented virtually, then the cost of tracking/handling the threads means that you remove some processing cycles from each of the threads themselves. HTH! Maarten, I am not an expert, but would it be possible, with the design you have, to throttle some threads, so that the server can allocate more processing where it really needs it? like giving less time to a thread which handles communications and more to another which does heavy number crunching ... or the opposite, if server querry/response is more important than the rate at which your processing queue actually flushes itself? -MAx PS, Maarten, if you DO decide to tackle this threading project... I WILL be checking it out, as I know some uses I'd make of it.

 [14/17] from: joel:neely:fedex at: 29-Jan-2004 17:09


Hi, Maarten, Maarten Koopmans wrote:
> ... it responds to a 'rest word, giving you the restof the > block to be evaluated and the value of the last evaluated expression. > > So: eval [ 10 + 10 join "a" "b" rest 20 print "howdy" ] > >> [ "ab" [ 20 print "howdy"]] > > Note that the second item (the rest of the code) is the position in the > original code, so a ' head on that gives you the original code back. >
Sort of like iterating DO/NEXT with an additional check for REST ?
> Question: is this a thing any of you want despite the performance hit of > 400% ? >
I pondered doing something with DO/NEXT a while back, but gave up in frustration (I hope you're smarter and/or more patient than I ;-) as I want to be able to write expressions/functions more sophisticated than a single block, but still have interruptability. For example: eval [ foreach line read/lines %somefile [ print line rest ] ] Or interruptableFunction: func [aLine [string!]] [ print aLine doSomethingInteresting aLine rest ] eval [ foreach line read/lines %someOtherFile [ interruptableFunction line ] ] You said "takes a block" so I inferred that things such as the above (nested blocks, including function evaluation) are currently out of reach. Am I too pessimistic? If I have to do all the work of managing e.g. loops and other "control structures", including functions, I think I'd probably want to just go ahead and evert the code into a task object (per the article on REBOLforces) with a STEP method that does some meaningful (but not over long) piece of work and then saves state and returns control. -jn- -- ---------------------------------------------------------------------- Joel Neely joelDOTneelyATfedexDOTcom 901-263-4446 Enron Accountingg in a Nutshell: 1c=$0.01=($0.10)**2=(10c)**2=100c=$1

 [15/17] from: gerardcote:sympatico:ca at: 29-Jan-2004 23:48


Hello Max and MArteen, What a great news, If I can share some thought with both of you about the fact that producing something of really great value - without knowing if the number of real users truly justifies the effort - I would say YES for the following reasons : 1. All the GREAT HIGH QUALITY tools that only a handful of HIGH LEVEL REBOL programmers were and are able to produce here merit a great RESPECT for the accomplishment and for PLUS VALUE they give to the rest of us current programmers (no less good but generally otherwise inclined) who want to learn from a practical POV about such things that would be left inaccessible for most of us if you were not there to show us the WHAT and HOW-TO carry out these wonderful things. 2. Second it is not evident to SELL those evolved tools to current not so low-level programmers if they don't have the necessary skills to potentialize your work or they don't have any need for them in the immediate or worse yet if they don't see what can be gained from them in their current day-to-day tasks. Will the ratio (Elapsed development time vs Learning curve time) be in favor or not for using these tools ? This is generally the question you have to help your customers (that is other REBOL programmers) answer to before they become converted to your tools. Nobody wants to waste time to learn something he is not confident if it will be useful to him. 3. I want to help you deploy the usage of your tools but I myself need more time to learn the ABC's of using them and then help others to get a smaller learning curve before they can use them too. For example I'm sure Rugby can help me to remotely execute some script located on my server and I will need this feature in some scripts since I don't want to leave my sources in the hands of my customers - and for now I have no sufficient money to invest into the many toolkits RT is selling - but I don't know what else I can do for now. Wiil your product fit my needs and will it be hard to use. I'll try to get some DOC about it into the Library very soon. I will study it a bit and I'll see but for sure I'd like to see some DEMOS of the many uses someone can realize with it. And for Max the same is true - although this time I read the DOC - with Slim and other products you put online (Liquid being one of them that I would like to use instead of directly using /View). But this requires time and some basic REBOL knowledge and sometimes advanced CS concepts too to cope with all this stuff. Don't despair there will always be some EARS and EYES that listen, look and even wait for the delivery of your products ... I'm one of those. 4. Last but not least : Do it for yourself first - If you get some personal satisfaction to deliver a real software engineering wonder. Why bother if others want it too. Let's do it and may be after some time somebody will even pay for your services. I don't dream here. I'm serious. Think for your own benefits first and then others should come after yourself. Don't lose sight of this however : it the product is not perfectly finished and is not accompanied by some decent-reasonable doc, it will never gain much of the momentum and popularity it really should deserve. So I'm a fierce supporter of your products and be proud of them - even if they are not for the moment very popular. Keep up the great work guys . And just for terminating, Marteen I am curious about the way I could expect myself to use your EVAL in collaboration with the actual /View platform as a new way to eliminate some concurrency limits in a teaching/learning/simulation perspective. Here is more detail. I actually work trying to hack Carl S.'s easy-vid script to let /View start the execution of another script using /Core or /View (using do, launch and CALL with /View 1.3). I got many problems the most important being REBOL blocks itself while the other script is running and the console is closed upon termination while I would like to keep it open for hands-on by the student following the lessons displayed simultaneously in the first REBOL instance. Do you think you would permit me to do this using your new tool ? I am almost sure the answer is YES but I would nevertheless get your final advice on this. ----- Original Message ----- From: "Maxim Olivier-Adlhoch" <[maximo--meteorstudios--com]> To: <[rebol-list--rebol--com]> Sent: Thursday, January 29, 2004 1:31 PM Subject: [REBOL] Re: Threading continued
> I think I know exactly how you feel. > > as an example, liquid, one of my brainchilds took a lot of time to mature (and its about to undergo a new round of refinemnet).
It exists, and was shared and is available to anyone who asks for it (its now in slim format, just not yet released)...
> liquid is really usefull, if you "get" it. Yet was it worth the "release" effort... > > I still don't know... I didn't get much feedback. I don't even know if anyone else is using it... probably not. > > so as for your tool. as much as I'd like to say to python users that I can thread executions of scripts (and do it better than
them, cause they can't remotely kill threads), at this point, I wonder how many tools really need that functionality.
> Who writes servers in rebol... not many of us... > > How many people even use threads in python... not that whole lot either. > > I've had to, they (threads) are usefull, but with rebol's smaller user base... even if 1% of us used(needs) it, you'd still have
a handfull of users, and many would use it just to see if it works... and if it goes 400% slower... not sure that's fit for server usage in all circumstances... maybe someone who wants to create a networked front-end for RebDB might want to introduce threading... but will it help go faster in the end, if all threads run on one cpu (even in a multi-cpu machine...).

 [16/17] from: gerardcote:sympatico:ca at: 29-Jan-2004 23:56


Hi Marteen, Not only I'm interested but I really think that such a tool has an inestimable value for pedagogical apps as those I am trying to put together from varying existing material. Immediately one of the magnificent tools that everybody would have liked to be in touch with when he began learning and programming REBOL - and I named a debugger/simulator - could be to some extent partially created using such a tool. It would be easier to get some intermediate breakpoints and some step by step with words value spying than it is for now with many simple evaluations and only our beginner's heart remember how this could have been of invaluable help ... I also answer your question inside the comment I did to the other Quote form Max. Thanks for the great job, Gerard

 [17/17] from: maarten:vrijheid at: 30-Jan-2004 8:38


Hi Joel,
>Sort of like iterating DO/NEXT with an additional check for REST ? >
Yes.
>I pondered doing something with DO/NEXT a while back, but gave up in >frustration (I hope you're smarter and/or more patient than I ;-) as
<<quoted lines omitted: 20>>
>(nested blocks, including function evaluation) are currently out of >reach. Am I too pessimistic?
Yes and no. Right now, if rest is returned as value of a subexpression, this works nicely. All three code pieces work, but I because I am patching all looping and conditional constructs (to set all kinds of hidden flags for rest and eval). More patient I guess :-)
>If I have to do all the work of managing e.g. loops and other "control >structures", including functions, I think I'd probably want to just go >ahead and evert the code into a task object (per the article on >REBOLforces) with a STEP method that does some meaningful (but not over >long) piece of work and then saves state and returns control. >
I agree. Transparancy is the way to go. Once I get some stuff in a decent state I'll need others to test it and see if it works "naturally" --Maarten<

Notes
  • Quoted lines have been omitted from some messages.
    View the message alone to see the lines that have been omitted