Threading continued
[1/17] from: maarten:vrijheid at: 29-Jan-2004 17:05
Hi,
I have got some code and some questions. I have a working version of
'eval a function that takes a block of code and evaluates it like 'do.
Big deal? Well... it responds to a 'rest word, giving you the restof the
block to be evaluated and the value of the last evaluated expression.
So: eval [ 10 + 10 join "a" "b" rest 20 print "howdy" ]
>> [ "ab" [ 20 print "howdy"]]
Note that the second item (the rest of the code) is the position in the
original code, so a ' head on that gives you the original code back.
Anyway.... this interruptable evaluator run on a block with 10.000 now
invocations 3.5 times slower than do. That's slow, but probably fast
enough for most things. And Moore's law and all...
On top of such an eval function you can easily build a co-op threading
engine. Instead of doing a script you would 'eval it, making it four
times slower but concurrent.
And it would be easy to add all sort of utility threads out-of-the box
for newbies, like a dns reader thread that returns the value to a
function on completion etc.
Question: is this a thing any of you want despite the performance hit of
400% ?
It is easy for me to get there, but I need to know if there is anybody
who'd like to see it available.
Thanks,
Maarten<
[2/17] from: maximo:meteorstudios at: 29-Jan-2004 11:37
good question,
can you guarantee that one crashing thread (just an evaluation error) can never stop
execution of the other threads, and allow some sort of signals to warn you that a thread
has dumped?
Allowing you to respawn it?
-MAx
---
You can either be part of the problem or part of the solution, but in the end, being
part of the problem is much more fun.
[3/17] from: maarten:vrijheid at: 29-Jan-2004 18:41
Yes, I can do all that (although it is good somebody reminds me of it ;-)))
But is it worth the effort?
--Maarten
Maxim Olivier-Adlhoch wrote:
[4/17] from: maximo:meteorstudios at: 29-Jan-2004 13:31
I think I know exactly how you feel.
as an example, liquid, one of my brainchilds took a lot of time to mature (and its about
to undergo a new round of refinemnet). It exists, and was shared and is available to
anyone who asks for it (its now in slim format, just not yet released)...
liquid is really usefull, if you "get" it. Yet was it worth the "release" effort...
I still don't know... I didn't get much feedback. I don't even know if anyone else is
using it... probably not.
so as for your tool. as much as I'd like to say to python users that I can thread executions
of scripts (and do it better than them, cause they can't remotely kill threads), at this
point, I wonder how many tools really need that functionality.
Who writes servers in rebol... not many of us...
How many people even use threads in python... not that whole lot either.
I've had to, they (threads) are usefull, but with rebol's smaller user base... even if
1% of us used(needs) it, you'd still have a handfull of users, and many would use it
just to see if it works... and if it goes 400% slower... not sure that's fit for server
usage in all circumstances... maybe someone who wants to create a networked front-end
for RebDB might want to introduce threading... but will it help go faster in the end,
if all threads run on one cpu (even in a multi-cpu machine...).
Tough question indeed.
I hope you get people which can give you a definite yes, giving you some push towards
putting time on it. It looks like a fun challenge you want to put yourself against.
:-)
cheers!
-MAx
---
You can either be part of the problem or part of the solution, but in the end, being
part of the problem is much more fun.
[5/17] from: jvargas:whywire at: 29-Jan-2004 13:45
I think it is worth. It will help with some applications that are
doing a lot
of different things all the time. It is kind of nice to invoke a thread
let
it run in the background a yield back to the gui and only waiting for
completion of the thread. Don't know about performance though. If
it is too slow people may choose a different approach to their code.
Cheers, Jaime
-- The best way to predict the future is to invent it -- Steve Jobs
On Jan 29, 2004, at 12:41 PM, Maarten Koopmans wrote:
[6/17] from: gchiu:compkarori at: 29-Jan-2004 20:04
Maarten Koopmans wrote.. apparently on 29-Jan-2004/18:41:39+1:00
>Yes, I can do all that (although it is good somebody reminds me of it ;-)))
>
>But is it worth the effort?
That is up to you to decide, but if you do do it, I would certainly use it.
--
Graham Chiu
http://www.compkarori.com/cerebrus<
[7/17] from: chrismorency:videotron:ca at: 29-Jan-2004 14:56
> Jaime Vargas wrote:
> -- The best way to predict the future is to invent it -- Steve Jobs
Sorry if this off-topic and seems like neat-picking, but I have way too
much respect for inovators in computer history to let this through ;)
(Those do not include (jobs/gates/linus)). Amongs said innovators: Alan
Kay and Carl Sassenrath.
So in order to restore historic fact, the quote is not by Steve Jobs,
it's from Alan Kay, member of the Xerox Palo Alto Research Center (PARC)
research team responsible for :
Network computing as we know it today.
GUI and Mouse interactivity as we know it today.
Smalltalk and OOP as we know it today.
The story of how Jobs and/or Gates were inspired by PARC can be read at
http://www.smalltalk.org/alankay.html
Best
Chris
[8/17] from: andreas:bolka:gmx at: 29-Jan-2004 21:48
Thursday, January 29, 2004, 5:05:11 PM, Maarten wrote:
> I have got some code and some questions. I have a working version of
> 'eval a function that takes a block of code and evaluates it like
> 'do.
Great! (And you have an interesting timing Maarten ... but that's
another story ;)
> Question: is this a thing any of you want despite the performance
> hit of 400% ?
You should know my answer: YES!
> It is easy for me to get there, but I need to know if there is
> anybody who'd like to see it available.
Yep, I'd definitely like to see it available. If you decide to not
release it, a personal mail would be fine too ... :)
--
Best regards,
Andreas
[9/17] from: andreas:bolka:gmx at: 29-Jan-2004 21:50
Thursday, January 29, 2004, 7:31:50 PM, Maxim wrote:
> Who writes servers in rebol... not many of us...
I do. Although REBOL tries to punish everyone who does. So maybe
that's another reason why not many write servers in REBOL - it's
simply missing crucial features for some scenarios.
--
Best regards,
Andreas
[10/17] from: jvargas:whywire at: 29-Jan-2004 15:58
Thanks Chris for your correction, it great to have well
informed people in this list!
Cheers, Jaime
-- The best way to predict the future is to invent it -- Alan Kay
On Jan 29, 2004, at 2:56 PM, Christian Morency wrote:
[11/17] from: jason:cunliffe:verizon at: 29-Jan-2004 16:44
> Question: is this a thing any of you want despite the performance hit of
> 400% ?
I think the answer should be YES.
But I honestly do not really understand what is at stake here.
I am not a server developer. But like many am a server user...
So what does this do for Rebol that it is missing now?
Can anyone please explain what this proposed threading solution would do to
improve speed of using Vanilla?
http://www.langreiter.com/space/Vanilla
Or for creating an XML socket server like Unity?
http://www.moock.org/unity/
Can you briefly describe any use-cases where this threading would make a
difference .
thanks very much
- Jason
[12/17] from: atruter:labyrinth:au at: 30-Jan-2004 9:54
Hi Maarten,
> Question: is this a thing any of you want despite the performance hit of
> 400% ?
In my case, it depends on "where" this performance hit is. Take the
following example:
do [
init-func
big-loop-func
end-func
]
Now, if 'do takes 0.001s to interate through these calls and your 'eval
func takes 0.004s that's OK. If 'big-loop-func takes 10s to evaluate, but
the use of 'eval blows this out to 40s then we have a problem. So I
suppose my question is, once 'eval hits a func call does it effect how
long it takes to evaluate that function in and of itself?
> It is easy for me to get there, but I need to know if there is anybody
> who'd like to see it available.
Depends on answer to above, but I've got a couple of interesting things I
could do with this (multi-threaded listener, background IO writers, etc).
Regards,
Ashley<
[13/17] from: maximo:meteorstudios at: 29-Jan-2004 17:58
> -----Original Message-----
> From: Jason Cunliffe [mailto:[jason--cunliffe--verizon--net]]
<<quoted lines omitted: 4>>
> would make a
> difference .
one example for threading useage is that you can crunch information WHILE you are also
waiting on a port.
this basically allows your server to be up all the time, although he is busy with any
matter of processes.
when creating a render queue or a transaction-based system, you can tell the system to
do things, and he won't do them right away, but you can still send the commands and receive
confirmations that your processes have been done.
then if you need information, another thread, can lookup the processing queue and verify
if there is yet some processing to be done on that item. if there is then it either
processes it right away in its own thread, if possible, before returning the new and
improved data. this doesn't keep the other thread from continuing to crunch the other
items in the list while they are not needed.
if you didn't have threads, then you'd have to wait for the current processing to finish
before your server even responds to any new request.
threading allows your server to be more responsive, altough it also slows down each of
those responses as they process simultaneously. As maarteen explained, if threading
is implemented virtually, then the cost of tracking/handling the threads means that you
remove some processing cycles from each of the threads themselves.
HTH!
Maarten, I am not an expert, but would it be possible, with the design you have, to
throttle some threads, so that the server can allocate more processing where it really
needs it? like giving less time to a thread which handles communications and more to
another which does heavy number crunching ... or the opposite, if server querry/response
is more important than the rate at which your processing queue actually flushes itself?
-MAx
PS, Maarten, if you DO decide to tackle this threading project... I WILL be checking
it out, as I know some uses I'd make of it.
[14/17] from: joel:neely:fedex at: 29-Jan-2004 17:09
Hi, Maarten,
Maarten Koopmans wrote:
> ... it responds to a 'rest word, giving you the restof the
> block to be evaluated and the value of the last evaluated expression.
>
> So: eval [ 10 + 10 join "a" "b" rest 20 print "howdy" ]
> >> [ "ab" [ 20 print "howdy"]]
>
> Note that the second item (the rest of the code) is the position in the
> original code, so a ' head on that gives you the original code back.
>
Sort of like iterating DO/NEXT with an additional check for REST ?
> Question: is this a thing any of you want despite the performance hit of
> 400% ?
>
I pondered doing something with DO/NEXT a while back, but gave up in
frustration (I hope you're smarter and/or more patient than I ;-) as
I want to be able to write expressions/functions more sophisticated
than a single block, but still have interruptability. For example:
eval [
foreach line read/lines %somefile [
print line
rest
]
]
Or
interruptableFunction: func [aLine [string!]] [
print aLine
doSomethingInteresting aLine
rest
]
eval [
foreach line read/lines %someOtherFile [
interruptableFunction line
]
]
You said "takes a block" so I inferred that things such as the above
(nested blocks, including function evaluation) are currently out of
reach. Am I too pessimistic?
If I have to do all the work of managing e.g. loops and other "control
structures", including functions, I think I'd probably want to just go
ahead and evert the code into a task object (per the article on
REBOLforces) with a STEP method that does some meaningful (but not over
long) piece of work and then saves state and returns control.
-jn-
--
----------------------------------------------------------------------
Joel Neely joelDOTneelyATfedexDOTcom 901-263-4446
Enron Accountingg in a Nutshell: 1c=$0.01=($0.10)**2=(10c)**2=100c=$1
[15/17] from: gerardcote:sympatico:ca at: 29-Jan-2004 23:48
Hello Max and MArteen,
What a great news,
If I can share some thought with both of you about the fact that producing something
of really great value - without knowing if the
number of real users truly justifies the effort - I would say YES for the following reasons
:
1. All the GREAT HIGH QUALITY tools that only a handful of HIGH LEVEL REBOL programmers
were and are able to produce here merit a
great RESPECT for the accomplishment and for PLUS VALUE they give to the rest of us current
programmers (no less good but generally
otherwise inclined) who want to learn from a practical POV about such things that would
be left inaccessible for most of us if you
were not there to show us the WHAT and HOW-TO carry out these wonderful things.
2. Second it is not evident to SELL those evolved tools to current not so low-level programmers
if they don't have the necessary
skills to potentialize your work or they don't have any need for them in the immediate
or worse yet if they don't see what can be
gained from them in their current day-to-day tasks. Will the ratio (Elapsed development
time vs Learning curve time) be in favor or
not for using these tools ?
This is generally the question you have to help your customers (that is other REBOL programmers)
answer to before they become
converted to your tools. Nobody wants to waste time to learn something he is not confident
if it will be useful to him.
3. I want to help you deploy the usage of your tools but I myself need more time to learn
the ABC's of using them and then help
others to get a smaller learning curve before they can use them too.
For example I'm sure Rugby can help me to remotely execute some script located on my
server and I will need this feature in some
scripts since I don't want to leave my sources in the hands of my customers - and for
now I have no sufficient money to invest into
the many toolkits RT is selling - but I don't know what else I can do for now. Wiil your
product fit my needs and will it be hard to
use. I'll try to get some DOC about it into the Library very soon. I will study it a
bit and I'll see but for sure I'd like to see
some DEMOS of the many uses someone can realize with it.
And for Max the same is true - although this time I read the DOC - with Slim and other
products you put online (Liquid being one of
them that I would like to use instead of directly using /View). But this requires time
and some basic REBOL knowledge and sometimes
advanced CS concepts too to cope with all this stuff.
Don't despair there will always be some EARS and EYES that listen, look and even wait
for the delivery of your products ... I'm one
of those.
4. Last but not least : Do it for yourself first - If you get some personal satisfaction
to deliver a real software engineering
wonder. Why bother if others want it too. Let's do it and may be after some time somebody
will even pay for your services. I don't
dream here. I'm serious. Think for your own benefits first and then others should come
after yourself. Don't lose sight of this
however : it the product is not perfectly finished and is not accompanied by some decent-reasonable
doc, it will never gain much of
the momentum and popularity it really should deserve.
So I'm a fierce supporter of your products and be proud of them - even if they are not
for the moment very popular. Keep up the
great work guys .
And just for terminating, Marteen I am curious about the way I could expect myself to
use your EVAL in collaboration with the actual
/View platform as a new way to eliminate some concurrency limits in a teaching/learning/simulation
perspective. Here is more detail.
I actually work trying to hack Carl S.'s easy-vid script to let /View start the execution
of another script using /Core or /View
(using do, launch and CALL with /View 1.3). I got many problems the most important being
REBOL blocks itself while the other script
is running and the console is closed upon termination while I would like to keep it open
for hands-on by the student following the
lessons displayed simultaneously in the first REBOL instance. Do you think you would
permit me to do this using your new tool ? I am
almost sure the answer is YES but I would nevertheless get your final advice on this.
----- Original Message -----
From: "Maxim Olivier-Adlhoch" <[maximo--meteorstudios--com]>
To: <[rebol-list--rebol--com]>
Sent: Thursday, January 29, 2004 1:31 PM
Subject: [REBOL] Re: Threading continued
> I think I know exactly how you feel.
>
> as an example, liquid, one of my brainchilds took a lot of time to mature (and its
about to undergo a new round of refinemnet).
It exists, and was shared and is available to anyone who asks for it (its now in slim
format, just not yet released)...
> liquid is really usefull, if you "get" it. Yet was it worth the "release" effort...
>
> I still don't know... I didn't get much feedback. I don't even know if anyone else
is using it... probably not.
>
> so as for your tool. as much as I'd like to say to python users that I can thread
executions of scripts (and do it better than
them, cause they can't remotely kill threads), at this point, I wonder how many tools
really need that functionality.
> Who writes servers in rebol... not many of us...
>
> How many people even use threads in python... not that whole lot either.
>
> I've had to, they (threads) are usefull, but with rebol's smaller user base... even
if 1% of us used(needs) it, you'd still have
a handfull of users, and many would use it just to see if it works... and if it goes
400% slower... not sure that's fit for server
usage in all circumstances... maybe someone who wants to create a networked front-end
for RebDB might want to introduce threading...
but will it help go faster in the end, if all threads run on one cpu (even in a multi-cpu
machine...).
[16/17] from: gerardcote:sympatico:ca at: 29-Jan-2004 23:56
Hi Marteen,
Not only I'm interested but I really think that such a tool has an inestimable value
for pedagogical apps as those I am trying to
put together from varying existing material.
Immediately one of the magnificent tools that everybody would have liked to be in touch
with when he began learning and programming
REBOL - and I named a debugger/simulator - could be to some extent partially created
using such a tool. It would be easier to get
some intermediate breakpoints and some step by step with words value spying than it is
for now with many simple evaluations and only
our beginner's heart remember how this could have been of invaluable help ...
I also answer your question inside the comment I did to the other Quote form Max.
Thanks for the great job,
Gerard
[17/17] from: maarten:vrijheid at: 30-Jan-2004 8:38
Hi Joel,
>Sort of like iterating DO/NEXT with an additional check for REST ?
>
Yes.
>I pondered doing something with DO/NEXT a while back, but gave up in
>frustration (I hope you're smarter and/or more patient than I ;-) as
<<quoted lines omitted: 20>>
>(nested blocks, including function evaluation) are currently out of
>reach. Am I too pessimistic?
Yes and no. Right now, if rest is returned as value of a subexpression,
this works nicely. All three code pieces work, but I because I am patching
all looping and conditional constructs (to set all kinds of hidden flags
for rest and eval).
More patient I guess :-)
>If I have to do all the work of managing e.g. loops and other "control
>structures", including functions, I think I'd probably want to just go
>ahead and evert the code into a task object (per the article on
>REBOLforces) with a STEP method that does some meaningful (but not over
>long) piece of work and then saves state and returns control.
>
I agree. Transparancy is the way to go. Once I get some stuff in a
decent state I'll need others to test it and see if it works "naturally"
--Maarten<
Notes
- Quoted lines have been omitted from some messages.
View the message alone to see the lines that have been omitted